channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev&y=18&m=7&d=15

Sunday, 15 July 2018

00:00:00 --- log: started osdev/18.07.15
00:00:20 <klys> absolute123, I'm just trying to parse why you're talking about the biggest confusion
00:00:41 --- quit: Lowl3v3l (Ping timeout: 244 seconds)
00:01:22 <bcos_> doug16k: Was thinking, maybe, slap together a macro that does "clear PGE flag in CR4, then set it again" (to nuke TLBs) and sprinkle that everywhere (in case non-determism is caused by stale TLBs)\
00:01:52 <absolute123> klys: gpu pointers! it is easy to broadcast from 256 regs with the same base around different cu's, but when you have compiled more warps than there are regs, they have different bases, i dunno how it concatenates the array from different bases than
00:02:07 <absolute123> miaow authors are also confused about this
00:02:15 <absolute123> *then
00:03:13 <klys> absolute123, mebby you should talk on a video dev channel. <sigh>
00:03:24 <doug16k> bcos_, I read your mind just now. I put a total TLB flush there. didn't help
00:03:39 <doug16k> including global pages - toggles CR4.PGE
00:04:57 <doug16k> ah wait
00:05:03 <doug16k> I only tlb flushed the fastpath
00:05:05 <doug16k> 1 sec
00:05:20 --- join: nortega (~nortega@gateway/tor-sasl/deathsbreed) joined #osdev
00:05:46 <doug16k> now I completely flush tlb as I check/add each paging level and before returning...
00:07:06 <doug16k> dude... doesn't crash!
00:07:16 <doug16k> tried 6x
00:07:24 <bcos_> That's good (maybe!) :-)
00:07:30 <bcos_> ..do it 6 more times!
00:08:31 <bcos_> The sad part is that (if TLB nukages fixes it) you'll have to go through all your paging code with a magnifying glass
00:09:14 <doug16k> worked 10x in a row
00:09:40 <doug16k> maybe I just need one global nuke early on
00:09:58 <doug16k> I remember recommending that to klange the other day. I should follow my own advice, lol
00:10:09 --- quit: jakogut_ (Quit: jakogut_)
00:10:32 <absolute123> klys: banned there as well as on ##verilog , however i could make an easy patch against miaow to make it work, it would take the next wid, and use a number of regs from there too, it does not currently work in miaow
00:10:47 <absolute123> have to go
00:10:53 --- quit: absolute123 (Quit: Leaving)
00:14:37 <doug16k> that explains why it only fails on real hardware. I doubt TCG implements the ridiculous number of TLB entries that are present on a real CPU
00:15:37 <doug16k> real hardware and KVM*
00:15:58 <bcos_> Would imply that you're not invalidating TLB (or one of the higher level caches) properly somewhere - global TLB nuke is just hiding symptoms temporarily
00:16:10 <doug16k> oh I know it isn't a solution
00:16:14 <doug16k> huge clue though
00:16:18 * bcos_ nods
00:17:37 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-128-113.002.203.pools.vodafone-ip.de) joined #osdev
00:18:03 <doug16k> means the code is right, as stated, but there's a subtle issue with invalidation. makes it even harder to find in a way - there isn't necessarily a wrong line, only a missing line :)
00:18:48 <doug16k> thankfully it is occurring so early that there isn't that much code coverage
00:22:03 <klys> as I understand it, the tlb caches the descriptor. so, if you've been using ds: with foreign code, and then reloaded ds: with the same parameters, there may be some chance of using those parameters. except, not in protected/long mode.
00:22:16 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
00:22:53 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
00:26:17 --- join: Bjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
00:26:53 --- quit: Belxjander (Read error: Connection reset by peer)
00:26:53 --- nick: Bjander -> Belxjander
00:27:14 --- quit: ljc (Remote host closed the connection)
00:28:23 <doug16k> klys, tlb caches entries from the page tables, so it doesn't have to chase 4 page table entries for every memory access
00:29:02 <klys> thanks
00:29:07 <doug16k> but yeah, if the bootloader had global mappings that were freed and you didn't invalidate the tlb, they'd leak over and the cpu would skip reading the page tables for that range if you updated them
00:29:57 <doug16k> I do totally flush the tlb already early on. thankfully I'm not as dumb as I thought I was a few minutes ago :D
00:30:22 <doug16k> thankfully/unfortunately. if I were that dumb, flushing might have been fix!
00:30:41 --- quit: Belxjander (Client Quit)
00:31:28 <klys> can you easily invalidate tlb with a memory access?
00:31:44 <doug16k> there are 3 ways:
00:31:52 <klys> I think there's an invlpg instruction
00:32:07 <doug16k> the invlpg instruction, toggling CR4.PGE, and invpcid instruction
00:32:23 <doug16k> last one might not exist on all processors
00:32:32 <doug16k> 2nd one might not either, on totally ancient 32 bit processors
00:32:35 <klys> righ, as cr3 might not
00:32:39 <klys> cr4
00:32:54 <klys> right, as cr4 might not.
00:33:02 <doug16k> on processors that old, simply writing cr3 flushes the TLB altogether
00:33:04 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
00:33:08 --- quit: drakonis (Read error: Connection reset by peer)
00:33:46 <doug16k> ...even if you write the value it already had
00:34:12 <klys> so, why do you use the cr4 method?
00:34:25 <doug16k> it flushes global mappings. writing cr3 doesn't
00:34:50 <doug16k> that's what global means. means never flush, unless you *really* flush with cr4.PGE toggle or invpcid instruction
00:35:12 <doug16k> CR4.PGE = Page Global Enable
00:35:34 <doug16k> turning off PGE makes the CPU never cache global mappings. turning it off then on flushes them all then reenables them
00:36:22 <doug16k> on processors that support PCID, you can use the invpcid instruction to do it more elegantly
00:36:37 <klys> hmm ok
00:37:47 --- join: ljc (~ljc@unaffiliated/ljc) joined #osdev
00:39:00 <doug16k> page table entries have a bit to make a mapping global. general idea is to make the kernel mappings global, since the kernel is always mapped in and doesn't change because you switched processes
00:39:32 <doug16k> now that intel has made a disastrous mess of things with meltdown, life isn't that easy anymore
00:40:02 <klys> I have been working on this data structure for a while, it's supposed to help me allocate regions of physical memory (without paging.) now I have a bunch of notes on how I'm doing it.
00:41:44 --- join: zeus1 (~zeus@197.239.32.36) joined #osdev
00:44:49 <klys> http://show.ing.me/cmm3.txt <-- this is all my notes from friday and saturday
00:44:53 <bslsk05> ​show.ing.me <no title>
00:46:28 <klys> the from_top_mm routine is kind of generic though it's my best algorithm yet for dealing with this data structure
00:48:16 <klys> it has a kind of analogue to paging in that you can calculate the address of memory referred to by its position in the [ 16 ] arrays.
00:50:50 <klys> the array rid[ pid ][ 4 ] points to a trunk, branch, twig, and leaf, for each pid present (16-bit pids), and free memory (pid 0).
00:51:38 <klys> from those you should have a doubly linked list of all used leaves, or branches, for example.
00:52:00 <klys> or in the case of free memory, all free leaves or branches, etc.
00:52:52 --- join: m3nt4L (~asvos@2a02:587:a01d:4100:3285:a9ff:fe8f:665d) joined #osdev
00:53:03 <klys> the mutual exclusion (BUSY) only takes place within the various members arrays (or end[] in the cae of leaves.)
00:53:10 --- quit: graphene (Remote host closed the connection)
00:54:09 <klys> case*
00:54:41 --- join: graphene (~graphene@46.101.134.251) joined #osdev
00:55:09 --- quit: Philpax_ (Quit: Leaving)
00:55:10 <NightBlade> seems pretty cool
00:55:14 <klys> the levels referred to in the comment at the top of from_top_mm describe 0=page 1=leaf...5=trunk
00:55:29 <klys> yeah
00:55:36 --- join: manny (~manny@ec2-52-36-117-99.us-west-2.compute.amazonaws.com) joined #osdev
00:56:02 <NightBlade> im a bit too tired to take it all in but i read half of the text document
00:56:16 <klys> that's enough to understand it i'm sure
00:56:24 <NightBlade> yeah
00:57:47 <klys> so to begin looking for leaf 2 in a newly allocated process/resident, one would call from_top_mm( page0+0x1000, pid, 0, 0, 5, NULL );
00:58:37 <klys> it would return a struct ret_top_mm, which you don't see because this is all pseudocode anyway for now.
00:59:23 --- join: w41 (~w41@unaffiliated/w41) joined #osdev
00:59:33 <klys> the information is what location did it find, what array members are locked, and what level did it return at.
01:00:55 --- join: Lowl3v3l1 (~Lowl3v3l@dslb-002-203-152-028.002.203.pools.vodafone-ip.de) joined #osdev
01:01:08 --- join: Asu (~sdelang@37.20.90.92.rev.sfr.net) joined #osdev
01:01:11 <klys> it should then be possible to allocate local mm structures for substructure within the level you have locked.
01:02:20 <klys> the thing about the list's prev and next pointers is that you don't really have access until you lock it via the array, from the top.
01:02:31 --- quit: Lowl3v3l (Ping timeout: 260 seconds)
01:03:59 <klys> and this should provide a fair level of mutual exclusion for anybody trying to add or take away from a resident structure.
01:04:02 <doug16k> found it!!! I had 4 huge 1GB pages physical mapping the low 4GB to bootstrap paging and didn't adequately flush them
01:04:17 <klys> ah, doug16k, congrats.
01:04:18 <doug16k> later the code maps some 4KB stuff in there. boom
01:05:09 <doug16k> didn't adequately flush them after freeing it*
01:05:33 <klys> i suppose I can see how that would land in a tlb
01:05:52 --- join: asymptotically (~asymptoti@gateway/tor-sasl/asymptotically) joined #osdev
01:06:38 <doug16k> I sort of have stuff to handle huge pages, but it's not thoroughly tested
01:07:39 <klys> if you care to look, I've been explaining how my algorithm allocates within 32bits' 4 GiB physical address space.
01:07:50 <doug16k> yeah I looked :)
01:08:16 <doug16k> don't follow it very well
01:08:24 --- quit: ljc (Ping timeout: 265 seconds)
01:09:17 <klys> the function called "fun" in there is supposed to make some use of from_top_mm()
01:11:19 <klys> one thing that it doesn't have yet is the stack structure where I can just push() and pop() like in python
01:11:33 <klys> should be easy enough
01:13:03 <klys> the throttle is set to where it'll look for whole chunks or sub-chunks depending on whether it's closer to the larger or smaller relevant size.
01:13:19 --- quit: Lowl3v3l1 (Ping timeout: 264 seconds)
01:14:08 <klys> it's supposed to use first-fit all around. this is a page-level allocator, so it never allocates anything smaller than 0x1000.
01:14:35 <doug16k> yeah, push down stack is basically push) stack[--stack_ptr] = value pop) value = stack[stack_ptr++], and initialize stack_ptr = countof(stack) at the beginning
01:15:02 <klys> right, that's very simple and would work.
01:15:38 --- quit: zeus1 (Read error: Connection reset by peer)
01:16:04 <klys> if you see res.dir there, that's "at what level did it return?" and you'll see it probably returned a locked leaf or higher
01:17:01 <klys> whereas the program specified "at what level should it return" as 0. that's because it wants pages.
01:17:53 <klys> and the locked structure level variable was 0 because it shouldn't lock anything over a page.
01:18:34 <klys> imean
01:19:17 <klys> and the lok variable was 0 because it shouldn't lock anything over a page. the locked structure level variable is default at 5. that permits access from the trunk.
01:20:02 <klys> access from the trunk is the only way to guarantee you can match the very end of the list.
01:22:19 <klys> o.t.o.h. I have a sub-page allocator which is working, albeit somewhat slow, using a bitmap at 0xe38 within the page.
01:25:43 <klys> perhaps it would be more interesting if I shared working code, though this stuff is pretty new so, thought to get some feedback.
01:27:03 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-190-085.002.203.pools.vodafone-ip.de) joined #osdev
01:28:04 <doug16k> klys, I suggest documenting roughly what each block of stuff is doing, since the names of the variables don't make it very obvious
01:28:32 --- join: Jari-- (~vai@85-76-45-170-nat.elisa-mobile.fi) joined #osdev
01:28:38 <doug16k> either make the variable names a bit more telling, or briefly describe why that bit is doing what it does
01:29:00 <klys> thanks yeah, the functions toward the bottom weren't finished so they could be doing anything pretty much
01:29:17 --- quit: Jari-- (Remote host closed the connection)
01:30:08 <doug16k> no idea what "rid" or "struc" or "var" or "dir" or "create structs" or "lsl" or "lok" or "trunk1" or ... mean
01:30:28 <klys> rid is the 1 MiB array of resident id's
01:30:33 --- join: hmmmm (~sdfgsf@pool-72-79-166-141.sctnpa.east.verizon.net) joined #osdev
01:31:25 <klys> lock is supposed to cmpxchg with struc and make sure it's not busy via yielding to other threads in the mm process/process group.
01:31:50 --- quit: Lowl3v3l (Ping timeout: 268 seconds)
01:32:06 <klys> dir is "at what level did it return."
01:32:35 <graphitemaster> so intel is buying an asic company
01:33:00 <klys> "create structs" is just meaning to indicate you would have to create additional substructure because ( dir > sir ) was true
01:33:37 --- quit: spare (Quit: leaving)
01:33:39 <klys> lsl was the locked structure level for interfacing with from_top_mm().
01:34:22 <klys> graphitemaster, also intel recently bought altera. what do you suppose they could be up to?
01:35:34 <graphitemaster> yeah, this is the 3rd asic company they have bought in the past 6 months
01:35:39 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
01:36:07 --- quit: nortega (Quit: Vivu lante, vivu feliĉe!)
01:36:48 --- join: blackandblue (~batdownh@gateway/tor-sasl/blackandblue) joined #osdev
01:39:28 --- join: hmmmmm (~sdfgsf@pool-72-79-163-252.sctnpa.east.verizon.net) joined #osdev
01:39:31 --- quit: hmmmmm (Remote host closed the connection)
01:39:47 --- quit: m3nt4L (Remote host closed the connection)
01:42:15 --- quit: graphene (Remote host closed the connection)
01:42:19 --- quit: hmmmm (Ping timeout: 268 seconds)
01:43:49 --- join: graphene (~graphene@46.101.134.251) joined #osdev
01:45:18 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
01:46:59 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
01:47:23 --- join: absolute123 (~vodka@185.192.189.222) joined #osdev
01:48:08 --- join: zeus1 (~zeus@154.224.162.124) joined #osdev
01:49:13 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-152-214.002.203.pools.vodafone-ip.de) joined #osdev
01:49:20 --- quit: manny (Quit: ZNC - http://znc.in)
01:51:11 --- join: manny (~manny@ec2-52-36-117-99.us-west-2.compute.amazonaws.com) joined #osdev
01:53:12 --- quit: manny (Client Quit)
01:53:56 --- join: Kimundi_ (~Kimundi@i577A9E65.versanet.de) joined #osdev
01:54:53 --- quit: Goplat (Remote host closed the connection)
01:57:43 --- quit: comero (Quit: Connection closed for inactivity)
02:00:50 --- join: manny (~manny@ec2-52-36-117-99.us-west-2.compute.amazonaws.com) joined #osdev
02:10:38 --- join: Philpax (~Philpax@n175-34-175-169.sun1.vic.optusnet.com.au) joined #osdev
02:12:30 <absolute123> https://github.com/aras-p/glsl-optimizer/blob/master/src/glsl/ast_to_hir.cpp line 855
02:12:38 <bslsk05> ​github.com: glsl-optimizer/ast_to_hir.cpp at master · aras-p/glsl-optimizer · GitHub
02:12:47 <doug16k> interesting, intel masks IRQs for one instruction after sti. I wonder why. maybe to allow code that does sti ; cli to let at least one IRQ in?
02:13:25 <Gynvael> wasn't it to allow iret or sth?
02:14:04 <doug16k> it doesn't make sense for iret - it's going to pop flags from the stack frame
02:14:10 <Gynvael> or am I confusing sti and cli as usual..
02:14:11 <doug16k> sti before iret is totally futile
02:16:29 <Gynvael> fair enough
02:16:59 <bcos_> doug16k: I'd be tempted to assume it's a pipelining issue - e.g. STI doesn't retire until next instruction has begun
02:17:42 --- quit: hunterlabs (Ping timeout: 256 seconds)
02:23:32 --- join: hunterlabs (Elite20801@gateway/shell/elitebnc/x-hvhwckxblahikmvr) joined #osdev
02:24:49 --- quit: w41 (Ping timeout: 240 seconds)
02:28:43 --- quit: absolute123 (Ping timeout: 256 seconds)
02:28:51 --- join: SopaXT (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
02:30:06 --- join: CheckDavid (uid14990@gateway/web/irccloud.com/x-vqrmpsmnbuiaygjb) joined #osdev
02:32:21 --- quit: SopaXorzTaker (Ping timeout: 260 seconds)
02:32:48 --- nick: SopaXT -> SopaXorzTaker
02:38:26 --- join: SopaXT (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
02:38:46 --- quit: SopaXorzTaker (Disconnected by services)
02:38:50 --- nick: SopaXT -> SopaXorzTaker
02:50:07 --- quit: asymptotically (Quit: Leaving)
02:54:21 --- join: absolute123 (~vodka@185.192.189.222) joined #osdev
02:55:00 <absolute123> have any of you nearby understand what i was talking about?
02:55:37 <graphitemaster> absolute123, float x[]; float b[] = { 1 }; x = b; makes x[1] ?
02:56:20 <absolute123> nah, you myst dereference
02:56:23 <absolute123> must
02:56:50 <absolute123> when you want to alias to another variable and change it's contents
02:57:24 --- join: ljc (~ljc@unaffiliated/ljc) joined #osdev
02:58:10 <absolute123> graphitemaster: but arrays and pointers are equivalent on gpu actually, they work bit differently there
02:58:52 <absolute123> also x[1]
02:59:04 <absolute123> is not the value for 1 , but dimension, i.e index
02:59:38 <absolute123> but on gpu , eah this thing is legal, what you wrote
03:01:29 <absolute123> also x[]=a; a=1;
03:01:36 <absolute123> this is whole array operation
03:02:13 --- quit: ljc (Client Quit)
03:02:32 <absolute123> also x[2]=a; a=1; this too
03:02:48 <absolute123> also x[]=b[2]; b=1; this too
03:05:09 <absolute123> err b[2]={1,2}; or something
03:06:05 <absolute123> anyways, graphitemaster: when gpu is in lsu_select state, it copies the opcode into every CU, it takes the variable which was dereferenced from the alu_state dest obviously
03:06:36 <absolute123> then finds out that this is whole array operation cause previously those private regs had been written
03:06:54 <absolute123> and runs on the whole array, but broadcast only the dest reg to other CU's
03:08:15 <graphitemaster> neat, have to go now tho
03:11:34 <Ameisen> bcos_ - I'd use WSL if I could access the APIs without having to use the rest of the subsystem fully
03:11:48 <Ameisen> doug16k - yes, the shell spawns a LOT, which slows things down on NT
03:11:51 <Ameisen> which I'm working
03:11:56 <Ameisen> ... on. Hit enter by mistake.
03:12:36 --- quit: immibis (Ping timeout: 260 seconds)
03:24:00 --- join: AIOP (ad102383@gateway/web/freenode/ip.173.16.35.131) joined #osdev
03:24:58 <AIOP> Would you guys be mad if I were to use an old tape recorder and a custom program to load in a custom operating system over my 1.5mb flash drive?
03:26:27 --- join: svk (~svk@p200300876E754F00B0F18B4A3FDE65F9.dip0.t-ipconnect.de) joined #osdev
03:27:08 --- join: isaac_ (~isaac@host81-129-159-113.range81-129.btcentralplus.com) joined #osdev
03:27:16 --- nick: isaac_ -> isaacwoods
03:28:38 --- quit: bauen1 (Remote host closed the connection)
03:34:18 <doug16k> AIOP, I would freak!
03:35:26 <doug16k> if you use an FFT and decode 8 or 16 serial bitstreams at different audio frequencies, it would be okay
03:38:12 <bcos_> AIOP: Real programmers just use a microphone...
03:38:38 <AIOP> *screams out a basic program*
03:39:26 --- quit: SopaXorzTaker (Remote host closed the connection)
03:40:00 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
03:46:40 <doug16k> one christmas I was visiting family and one of them was interested in CW and morse code and stuff. for a laugh I made him a 1024 point spectrum analyzer that decoded morse code -> https://github.com/doug65536/FastSpectrumAnalyzer/blob/master/fft.h
03:46:43 <bslsk05> ​github.com: FastSpectrumAnalyzer/fft.h at master · doug65536/FastSpectrumAnalyzer · GitHub
03:50:05 --- quit: Lowl3v3l (Ping timeout: 244 seconds)
03:50:49 --- join: asymptotically (~asymptoti@gateway/tor-sasl/asymptotically) joined #osdev
03:51:38 --- join: sortie (~sortie@static-5-186-55-44.ip.fibianet.dk) joined #osdev
03:52:20 --- join: ljc (~ljc@unaffiliated/ljc) joined #osdev
03:52:43 --- quit: [Brain] (Ping timeout: 240 seconds)
03:54:17 --- quit: kanzure (Ping timeout: 268 seconds)
03:55:47 <doug16k> guess I didn't push the morse code commits. oh well :D
04:05:55 <clever> AIOP: i have heard of old tv shows transmitting code with a flashing square in the corner of the show, and then you just tape a photo-resistor to it and download the program
04:06:42 <AIOP> lol
04:07:05 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-152-214.002.203.pools.vodafone-ip.de) joined #osdev
04:14:46 --- join: glauxosdever (~alex@athedsl-4564601.home.otenet.gr) joined #osdev
04:15:49 --- quit: nwm (Ping timeout: 240 seconds)
04:17:28 --- quit: Lowl3v3l (Ping timeout: 244 seconds)
04:18:58 --- join: Kimundi__ (~Kimundi@i577A9BE6.versanet.de) joined #osdev
04:20:43 --- join: zenix_2k2 (~zenix_2k2@42.112.227.117) joined #osdev
04:20:51 --- join: CrystalMath (~coderain@reactos/developer/theflash) joined #osdev
04:22:36 --- quit: Kimundi_ (Ping timeout: 260 seconds)
04:30:20 <absolute123> graphitemaster: just minor comment when the dimension thing is fixed I mean replaced by value, in GLSL it may not be legal, but in hw, your line is legal ontop of physical regs
04:31:27 <sortie> absolute123: Hi, how are you?
04:32:13 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-195-170.002.203.pools.vodafone-ip.de) joined #osdev
04:32:45 <zenix_2k2> guys one nooby question, what "file" ACTUALLY is ? i mean it not just a movable icon on a GUI right ? it has to be that my computer or OS understands
04:32:57 <zenix_2k2> just wondering
04:33:25 <sortie> zenix_2k2: http://www.nobius.org/dbg/practical-file-system-design.pdf ← Here's a good book on the topic
04:33:28 <absolute123> round about fine, i am here either on weekends or in the evenings sometimes during the week only though
04:34:15 <sortie> zenix_2k2: A file is essentially a named sequence of bytes (values from 0 to 255 inclusive, 256 different values). It resides in a file system, as described in that book.
04:34:21 <Lowl3v3l> zenix_2k2: are we talking about homework grade stuff here? I just want to know what kind of answer youd want. though sorties suggestion is a good one
04:34:58 <zenix_2k2> Lowl3v3l: nya, i am just wondering... but let's me look at what sortie suggested first before saying more
04:35:44 <sortie> zenix_2k2: A file is really a basic thing, nothing complicated. But it takes experience to understand just how non-magical they are.
04:35:55 <sortie> (of course filesystems can be complicated)
04:36:57 --- join: kanzure (~kanzure@unaffiliated/kanzure) joined #osdev
04:37:23 <zenix_2k2> sortie: does that book also explain how can i "name" a sequence of bytes ?
04:37:31 <sortie> Exactly
04:38:53 <Lowl3v3l> zenix_2k2: for one practical example of an ancient and bad file system there's a wikipedia article on the design of the fat file systems that roughly sketches that one with pictures
04:38:53 <sortie> zenix_2k2: Essentially each file corresponds to exactly one 'inode', which describes the file (who owns it, last modified, its size, and where on the disk its contents are located). Directories are a list of file names and inode. Directories are also inodes, where the file contents are that list of files inside the directories.
04:39:00 <zenix_2k2> sortie: and talking about file-system, is it the thing that make my Operating system understand which file is file ? ( since you know there are a lot of same-named files in one system )
04:39:21 <zenix_2k2> and i am not asking for its full definition yet
04:39:35 <zenix_2k2> i am just asking if one of its functionality is that or not
04:39:44 <sortie> zenix_2k2: The filesystem has a default directory which is the root of the filesystem. The path of a file, is then the path you take from that root directory, following the right file name inside each directory, until you find the file (inode) you wanted. That's how files are named.
04:39:48 --- quit: CheckDavid (Quit: Connection closed for inactivity)
04:41:14 <sortie> The structure of the filesystem (directories and files) is what creates the file paths everything has.
04:41:55 <sortie> Everything corresponds to an inode, which has an unique number, so you know it's that exact thing. However, you can only open files by their path.
04:42:29 <sortie> zenix_2k2: That said, I think you should read the book.
04:45:52 <zenix_2k2> sortie: ok one final question, does that book you pasted me also explains how Operating system detects one file instead of mistakenly detecting another one ? cause you know there are lots of same-named files in one system as i said above
04:46:13 <CrystalMath> oh, zenix_2k2, you're here too
04:46:32 <CrystalMath> you could look at a filesystem as a key-value database
04:46:45 <Lowl3v3l> zenix_2k2: the name is completely irrelevant
04:47:46 <Lowl3v3l> zenix_2k2: the name is literally just a help for you poor human
04:47:57 <bcos_> zenix_2k2: Is the string "foo/bar/hello.txt" the same as the string "far/boo/hello.txt"?
04:48:21 <CrystalMath> zenix_2k2: if you go a little bit under the hood you'll see that the filesystem is really a collection of nodes
04:48:43 <CrystalMath> that is, there's some backing storage system for this key-value database
04:49:20 <zenix_2k2> CrystalMath: yea i am here since i thought this would be a nicer place to ask that question
04:50:59 <zenix_2k2> bcos_: no but i mean let's say i have 2 files both named "foo.txt", one of each is put in another directory then how can my OS detect if ? and yea, since directories are also a type of files, i think all of them has like be like in one place ?
04:52:46 <zenix_2k2> HHHmmm, am i being too abstract ?
04:55:47 <bcos_> zenix_2k2: The thing is that the whole path is used. For example, a program might open the file "hello.txt", but somewhere there's a "current working directory" that gets prepended (so it might become "/foo/bar/hello.txt"), and the OS (virtual file system?) actually gets asked to open "/foo/bar/hello.txt" (and knows which file because there's only one "hello.txt" in that directory)
05:00:37 <zenix_2k2> bcos_: yea but that is also what i was wondering, if that is what you mean then it means directory can contains other files, but technically directory is also a file and "file contains files" ?
05:04:25 <bcos_> A file and a directory are both a named set of bytes (the data). For a file the data is the file's data. For a directory the data is a list of what is in the directory
05:06:24 <bcos_> You could say that file and directory are siblings that inherit from the same "named set of bytes" parent
05:09:15 <zenix_2k2> yea i could tell, but probably i should look at that book
05:11:02 <sortie> zenix_2k2: That's called directories.
05:11:29 <sortie> zenix_2k2: Two files with the same names, being different, because they are in different directories, that's because they are in different directories. I explained that above.
05:15:18 <zenix_2k2> oh right yea, didn't see that above :P
05:16:14 --- join: angel0xff (~zzz@158-58-227-127.sf.ddns.bulsat.com) joined #osdev
05:17:59 --- quit: ljc (Quit: ayy)
05:18:15 --- quit: absolute123 (Quit: Leaving)
05:20:08 --- join: zhiayang_ (~zhiayang@175.156.221.216) joined #osdev
05:20:31 --- quit: zhiayang (Ping timeout: 260 seconds)
05:32:24 --- quit: SopaXorzTaker (Remote host closed the connection)
05:32:58 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
05:33:16 --- quit: zenix_2k2 (Quit: Leaving)
05:39:24 --- join: prokbird (~golu@117.205.137.232) joined #osdev
05:39:24 --- join: promach_ (~promach@bb219-74-174-136.singnet.com.sg) joined #osdev
05:39:47 --- quit: prokbird (Client Quit)
05:40:29 --- quit: SopaXorzTaker (Read error: Connection reset by peer)
05:40:53 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
05:41:11 --- join: general (~golu@117.205.137.232) joined #osdev
05:41:18 --- part: general left #osdev
05:47:15 --- join: Shikadi (~Shikadi@cpe-98-10-34-205.rochester.res.rr.com) joined #osdev
06:01:10 --- quit: graphene (Remote host closed the connection)
06:02:43 --- join: graphene (~graphene@46.101.134.251) joined #osdev
06:08:10 --- quit: adam4813 (Quit: Connection closed for inactivity)
06:11:14 * graphitemaster is so confused
06:11:28 <graphitemaster> I have a function when called from main, all it does is read two fields out of a struct and print them
06:12:12 <graphitemaster> when I make that function instead return the struct, then read the fields from main and print them
06:12:21 <graphitemaster> only one of them contains the value that should be there and the other is always 0
06:12:32 <AIOP> show code
06:12:36 <graphitemaster> at all optimization levels, on both clang and gcc
06:12:40 <graphitemaster> it's not exactly easy to isolate
06:12:59 <graphitemaster> I must be just very tired because I don't really get it
06:13:00 <AIOP> print your addresses
06:14:20 <graphitemaster> right now I have a function, which gets passed two pointers to ints, the function then reads values out of a struct, storing them in those pointers passed, this, when called from main works find
06:14:34 <graphitemaster> if I give it two integers, pass them by address, then print the result out, it's fine
06:15:01 * graphitemaster valgrinds it
06:15:09 <graphitemaster> I think I have stack problems
06:16:52 <graphitemaster> ah, actually I see the issue
06:17:16 --- quit: AIOP (Ping timeout: 252 seconds)
06:17:48 <graphitemaster> I need to delay this
06:18:16 <palk> graphene: GCC's calling convention for when a function returns a struct is a little bit convoluted
06:18:21 <palk> graphitemaster: *
06:19:32 <palk> If memory serves, the caller is supposed to allocate space for the struct, and pass a pointer to the struct as a hidden first argument to the function
06:20:10 --- part: akasei left #osdev
06:20:11 <palk> the function in turn hides the struct pointer when returning via the `RET IMM` form
06:20:34 <palk> this behavior can very easily appear to be a "stack problem"
06:25:35 --- quit: zeus1 (Ping timeout: 256 seconds)
06:26:44 --- quit: ybyourmom (Quit: Lost terminal)
06:34:54 --- quit: angel0xff (Ping timeout: 244 seconds)
06:36:43 --- join: JusticeEX (~justiceex@pool-98-113-143-43.nycmny.fios.verizon.net) joined #osdev
06:37:08 --- join: tacco| (~tacco@i59F52C3F.versanet.de) joined #osdev
06:41:19 --- join: angel0xff (~zzz@158-58-227-127.sf.ddns.bulsat.com) joined #osdev
06:42:11 --- join: absolute123 (~vodka@185.192.189.222) joined #osdev
06:57:48 --- join: User__ (~User@46.228.182.159) joined #osdev
06:59:14 --- quit: User__ (Remote host closed the connection)
06:59:34 --- join: w17t (~w17t@unaffiliated/w17t) joined #osdev
07:09:14 --- join: zeus1 (~zeus@154.224.162.124) joined #osdev
07:14:58 --- join: SopaXT (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
07:16:57 --- quit: SopaXorzTaker (Ping timeout: 240 seconds)
07:19:43 --- quit: sysfault_ (Ping timeout: 240 seconds)
07:21:30 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
07:21:41 --- quit: pie__ (Ping timeout: 260 seconds)
07:21:55 --- quit: SopaXT (Ping timeout: 244 seconds)
07:23:12 --- quit: w17t (Quit: Leaving)
07:23:33 --- join: w17t (~w17t@unaffiliated/w17t) joined #osdev
07:40:57 --- quit: Lowl3v3l (Ping timeout: 256 seconds)
07:44:14 --- join: sysfault (~exalted@pool-108-53-210-20.nwrknj.fios.verizon.net) joined #osdev
07:47:22 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
07:49:49 --- quit: sysfault (Ping timeout: 244 seconds)
07:50:06 --- join: sysfault (~exalted@pool-108-53-210-20.nwrknj.fios.verizon.net) joined #osdev
07:56:40 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-152-178.002.203.pools.vodafone-ip.de) joined #osdev
08:04:05 --- join: Desetude (~Desetude@unaffiliated/desetude) joined #osdev
08:04:06 --- join: bm371613 (~bartek@89-64-30-31.dynamic.chello.pl) joined #osdev
08:04:47 <Desetude> Hey all, I am currently doing a survey on Operating Systems for a project at college. Any responses would be appreciated: https://goo.gl/forms/QaUffNFUL1t3Ihj92
08:04:48 <bslsk05> ​goo.gl: Operating Systems
08:05:23 <Desetude> (before anyone says this isn't to do with osdev, this is research for how I would make my os different to available ones)
08:08:28 <glauxosdever> "What would you change about your choice of OS?"
08:08:43 <glauxosdever> Did you mean: "OS of choice"?
08:08:55 <Desetude> Yeah I rephrased that after I realised that sounds stupid
08:09:18 <Desetude> I meant what do you dislike about your OS, and what would you change
08:09:44 <glauxosdever> Oh, ok. I have some complaints anyway
08:09:49 --- quit: Philpax (Ping timeout: 240 seconds)
08:11:33 <glauxosdever> Done
08:11:47 <glauxosdever> Are there any results?
08:12:42 <sortie> What operating system (OS) do you mainly use on your desktop computer? *
08:12:50 <sortie> Almost sounds like phishing
08:13:00 <sortie> Desetude: Don't forget the BSDs, Plan 9, Haiku, etc.
08:13:03 <sortie> Solaris
08:13:12 <glauxosdever> Also, Sortix :p
08:13:18 <glauxosdever> jk
08:13:21 <sortie> Nobody uses that glauxosdever
08:13:27 <glauxosdever> I know, I know :-)
08:14:03 <Desetude> glauxosdever: yeah yours came through, I also posted it in a programming discord which I kind of regret because I got a load of stupid responses
08:14:30 <sortie> Desetude: Answered it, but you're not quite asking the right questions :)
08:14:48 <Desetude> sortie: I was thinking that most likely a very small amount of people use those so it's easier for that to just be in other
08:14:54 <Desetude> sortie: what should I be asked? :P
08:15:00 <sortie> Desetude: Go research differences between BSD and Linux, and why BSD is better, even if not always. It'll teach you a lot.
08:15:20 <Desetude> be asking*
08:15:21 <sortie> Desetude: Hello. Welcome to #osdev. The place where unusual people reside.
08:15:58 <sortie> Yes I do use Linux Mint, but I do play a bunch with BSDs and the like
08:16:27 --- join: [Brain] (~brain@brainwave.brainbox.cc) joined #osdev
08:16:39 <sortie> Desetude: Open ended questions like "what would your change about your OS" are so big it's hard for people to answer unless they already have a cached response
08:16:53 <sortie> It's not a fair question to ask in terms of time required to answer it well, and you won't get too many good answers
08:16:58 <sortie> More specific questions are better
08:17:20 <glauxosdever> ^ Good point. I today got angry at GTK having memory leaks, so the answer was cached
08:17:41 <glauxosdever> BTW, anything other than GTK to possibly try?
08:18:36 <glauxosdever> Something that's not very complex (GTK is), something supports a core OpenGL context?
08:18:53 <glauxosdever> I know, I'm being to specific now, so you would need some cached answers
08:19:01 <glauxosdever> Actually, disregard the question
08:19:43 <sortie> Desetude: Look at presentations like https://www.openbsd.org/events.html to see what the alternative OS crowd is working on
08:19:46 <bslsk05> ​www.openbsd.org: OpenBSD: Events and Papers
08:20:59 <Desetude> sortie: Thanks ill check that out
08:21:46 <glauxosdever> The contradicting points of sortie and me are that if you are too generic responses may need to be cached, but also if you are too specific responses may need to be cached
08:21:49 <glauxosdever> So..
08:22:52 <sortie> I'm mostly saying that Desetude has picked an interesting topic, but Desetude doesn't know enough about the topic, and a form like that isn't telling much about the topic
08:23:44 <glauxosdever> Another interesting question would be "do you prefer monolithic, hybrid or micro kernels?"
08:23:45 <Desetude> sortie: Well I need to do some primary research for the project and thought a survey like that would be the best thing to do
08:24:20 <glauxosdever> Also "what do you think of various IPC strategies"?
08:25:02 <glauxosdever> Let's brainstorm questions! :p
08:25:31 <Desetude> glauxosdever: Yeah I was thinking about having more specific OS-dev related questions but I don't know if I'd get enough data from it as not many people will be able to answer those
08:25:53 <glauxosdever> It depends who you ask :-)
08:26:17 <glauxosdever> I suggest you do an updated form and link to it in here
08:27:52 <absolute123> well does some of you think that BSD is better than Linux? never used it, but driver wise it is not as well supported as Linux is perhaps i am thinking to myself probably?
08:28:11 <glauxosdever> Desetude: Maybe google forms allow you to set a list to be checked in order of preference?
08:28:41 <glauxosdever> Indeed Linux is better supported. And that's because it has more users
08:28:53 <Desetude> glauxosdever: It doesnt look like it
08:28:54 <sortie> Desetude: What is the project?
08:28:59 <sortie> Desetude: You want to make an OS?
08:29:13 <Desetude> sortie: Well I have started it a bit but it's really basic at this point
08:29:17 <sortie> absolute123: BSD is doing alright with drivers. This isn't 2004.
08:29:33 <Desetude> sortie: https://github.com/desetude/hafos
08:29:34 <bslsk05> ​Desetude/Hafos - Basic Operating System Created in C (0 forks/0 watchers)
08:29:53 <sortie> absolute123: BSD is about reliability and quality engineering. It's about a whole system that fits together. Security. Etc. Linux is chaos.
08:30:03 <sortie> Linux gets the job done, but not well. That's an overall difference.
08:30:23 <sortie> Desetude: Is this an university project with a timeframe and key accomplishments it's supposed to have?
08:30:31 --- quit: zeus1 (Ping timeout: 260 seconds)
08:31:03 --- quit: Lowl3v3l (Ping timeout: 248 seconds)
08:31:04 <Desetude> sortie: Well it's sixth form college, but yes
08:31:20 <Desetude> sortie: My timeline is very scuffed at this point though and I'm doing the research I was meant to do at the start quite late into it
08:31:50 <glauxosdever> Are these statistics for design considerations for your OS?
08:31:50 <sortie> Desetude: Honestly if this is your first OS and this is a school project, I recommend just doing an OS and seeing what you learn along the way.
08:31:56 <glauxosdever> If so, I'm also interested
08:32:11 <sortie> Desetude: There's so much to learn here that you'll probably not get to the interesting research parts for at least a year
08:32:36 <sortie> I mean, it's still valuable to learn all this stuff
08:32:55 --- join: CheckDavid (uid14990@gateway/web/irccloud.com/x-rhiyhnqohohojtlg) joined #osdev
08:33:08 <sortie> If you're trying to do something novel, it's a lot of work to get that far, especially as you seem to be basing it on a tried and true design
08:33:17 <Desetude> glauxosdever: That is what it is meant to be for yes
08:33:34 <glauxosdever> Cool
08:33:48 <absolute123> yeah i know couple of locals using it, but was not aware about the status, playstation uses BSD under the hood right
08:34:04 <sortie> Desetude: Honestly Linux and BSD to an lack quality and consistency. If you do a POSIX system that's clean, secure, consistent, documented, reliable, and nothing surprising and fancy, then you've already done something novel.
08:34:04 <Desetude> glauxosdever: but Im a bit too for in to make big design changes now :P (since I don't have much time left to finish it)
08:34:27 <glauxosdever> You can restart it later and do it better!
08:34:32 <glauxosdever> If you want ofc
08:34:47 <Desetude> sortie: Well for now I'm just trying to go as far as possible with an OS and see where it takes me
08:34:53 <glauxosdever> For brainstorming questions, see #osdev-brainstorm
08:35:03 <sortie> Desetude: That's somewhat incompatible with it being a formal project.
08:35:27 <Desetude> glauxosdever: Ye I'm definitely interested in starting it again but I'll have to do that after this project
08:35:59 <sortie> Desetude: Here's some resources of mine that may help you
08:36:24 <sortie> Desetude: https://wiki.osdev.org/Meaty_Skeleton ← This is my example OS. You may already have seen it since some of your structure look a little like it.
08:36:26 <bslsk05> ​wiki.osdev.org: Meaty Skeleton - OSDev Wiki
08:36:43 <absolute123> anyhow i am off to watching soccer now, would chat with graphitemaster and doug16k and rain1 about a possible article to come through, well graphics drivers can be ported to BSD if they have other drivers allready done
08:36:49 <sortie> Desetude: https://gitlab.com/sortie/myos/ is the continuation with more code. I recommend looking at it, it does things a bit more cleanly.
08:36:51 <bslsk05> ​gitlab.com: Jonas Termansen / Myos · GitLab
08:37:32 <sortie> Desetude: https://sortix.org/ is my main project that's very far, far enough to be quite interesting even if it's unfinished and has technical debt.
08:37:32 <bslsk05> ​sortix.org: The Sortix Operating System
08:37:59 <sortie> I think looking into my code might be interesting to you
08:38:25 <sortie> Desetude: It basically has the BSD model of a base system that fits well together, and ports of much third party software on top.
08:39:12 <rain1> have a good day absolute123
08:39:18 <Desetude> sortie: Well my goal for the project will be something more concrete then see how far I can get but I can decide that later on
08:39:38 <Desetude> sortie: Ill definitely check that out thanks
08:41:30 --- quit: SopaXorzTaker (Read error: Connection reset by peer)
08:42:38 <Desetude> sortie: tbh I have come to realise that creating an OS has a very low work to impressiveness ratio for the people that will be marking my project
08:43:02 <Desetude> so in hindsight I probably should have chosen this :P
08:43:09 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
08:46:29 <sortie> Desetude: Likewise the people grading the project are unaware of the increasing curve of difficulty
08:46:32 <rain1> I couldn't figure out how to copy files in and out of sortix
08:46:39 <sortie> “Ah, so it has a command line? Can you dad networking?”
08:46:48 <sortie> rain1: You can't. It's secure that way.
08:47:05 --- quit: drsn0w (Ping timeout: 265 seconds)
08:47:10 <sortie> rain1: https://sortix.org/man/man7/serial-transfer.7.html is a solution using the serial line
08:47:10 <bslsk05> ​sortix.org: serial-transfer(7)
08:47:28 <sortie> rain1: Currently merging networking, so you can sftp data in and out
08:47:37 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-152-178.002.203.pools.vodafone-ip.de) joined #osdev
08:49:48 <Desetude> sortie: Looks like everything I do is very similiar to myos
08:53:58 <absolute123> on my laptop i have booted over sftp or whatever that netboot was pxaboot or similar, it was a bit tricky to setup, but i found some laptops having buggy firmware in this regard
08:55:24 <absolute123> pxe maybe better off
08:56:04 <absolute123> pxa was something other, as those earlier intel embedded arm chips, i had one of those too, but it did not decode video in hw
08:56:35 <sortie> Are you thinking of tftp?
08:56:49 <absolute123> ah yeah, well that was it , sorry
08:57:02 <absolute123> but what is sftp though?
08:57:38 <sortie> https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol
08:57:39 <bslsk05> ​en.wikipedia.org: SSH File Transfer Protocol - Wikipedia
08:57:44 <sortie> lmgtfy
08:57:47 <absolute123> aight, yeah just found it
08:57:56 <absolute123> secure ftp, aka ftp over ssh
08:58:05 <rain1> wait sftp is ssh ftp? I had no idea
08:58:18 <rain1> I assumed it was closer to TLS ftp
08:59:37 <absolute123> both would end up using TLS libs probably anyways
09:01:46 <rain1> I'm interested in this new elliptic curve crypto 4Q
09:02:07 <rain1> it splits the math into 4 parts which can be done in parallel. so it's about 2x faster than ed25519
09:02:25 <rain1> although the math is pretty advanced,i can't really understand it all :/
09:03:22 * geist yawns
09:03:23 <absolute123> i do some gardening work in netherlands, and i am in between of purchasing some cheap FPGA finally or is it better just to drink and party instead :)?
09:03:59 <rain1> geist that comes across really rude
09:04:14 <geist> eh?
09:04:29 <rain1> *feigns ignorance*
09:04:36 * geist shrugs
09:04:39 <rain1> cool dude
09:04:47 <geist> too early to think clearly anyway
09:06:30 --- join: drsn0w (~drsn0w@unaffiliated/drsn0w) joined #osdev
09:07:06 <sortie> Yo geist
09:07:34 <geist> hola
09:08:12 <sortie> I have three weeks of vacation now :)
09:08:20 <sortie> Finally getting a little bit of osdev done
09:08:27 <geist> oh yay
09:08:41 <geist> im on my last day of a little 2 week vacation
09:08:55 <geist> been good too, haven't checked any email since
09:08:59 <sortie> Yay
09:09:12 <sortie> How much vacation do you get over there?
09:10:36 <sortie> I've been messing a bit with libc headers
09:10:53 <sortie> Turns out I had some bad namespace pollution going on, lots of my code not working on other libcs
09:10:58 <sortie> Well a bit
09:11:14 <sortie> Nice to be back to good old trivialities like that
09:11:26 <sortie> I looked a bit into virtio. It seems straightforward and nice.
09:11:40 <sortie> I want to implement it so I can get my stuff running on compute engine
09:13:29 <absolute123> what hw does this compute engine runs on, graphics cards?
09:13:41 <absolute123> *run
09:13:58 <absolute123> prolly gpu or fpga right, as openmp would allow
09:15:20 <sortie> absolute123: lmgtfy https://en.wikipedia.org/wiki/Google_Compute_Engine
09:15:21 <bslsk05> ​en.wikipedia.org: Google Compute Engine - Wikipedia
09:15:46 <absolute123> well, i found my ideas functioning by the hw code appearance, slight testing to be done and that is it, i can explain in the article about most bits of hw, but verilog may be tricky for most of others
09:15:51 <absolute123> hmm, reading...
09:16:29 <graphitemaster> yall vacation taking slackers I tell ya, wipper snappers
09:16:54 <graphitemaster> here I'm working on a weekend while also trying to run a company
09:17:24 <graphitemaster> fucking unbelievable
09:17:26 <absolute123> well that seems similar to virtualgl does not it, it's a networked virtual machine that sends only pixels back to render
09:18:10 --- quit: graphene (Remote host closed the connection)
09:18:17 <sortie> absolute123: This is the cloud..
09:18:34 <sortie> It's not a GPU platform, though GPUs are available IIRC
09:19:41 --- join: graphene (~graphene@46.101.134.251) joined #osdev
09:20:23 <geist> graphitemaster: yeah well this is the first vacation i've taken all year. needed it
09:20:31 <geist> i'm bad about it, i keep letting my vacation hours max out
09:21:40 <absolute123> sortie: ok, i understood..but the idea is something similar even though it is offloaded to cpu's or other accelerators than gpu's
09:21:57 <absolute123> GPUs i should say again and CPUs
09:22:23 <sortie> It's really more of a data center
09:22:36 <sortie> It can be used for tons of purposes
09:23:31 --- quit: angel0xff (Ping timeout: 264 seconds)
09:23:57 <absolute123> yeah, since it fires up the whole of a VM instead of only a lib, and gets back the picture pixels of a framebuffer
09:24:39 <graphitemaster> geist, pft, some would say working for Google is a vacation
09:24:42 --- quit: isaacwoods (Quit: isaacwoods)
09:24:54 <sortie> graphitemaster: That's not how vacation works
09:25:36 <geist> yeah, i am almost offended
09:25:39 <graphitemaster> Google is so culturally diverse and the offices are so big you can visit every country, learn every culture and eat every type of food
09:25:46 <graphitemaster> why the hell do you need to go anywhere
09:25:55 <geist> yes but that does not lend to you getting a break and rest
09:26:01 <geist> to recharge so you can operate at 100%
09:26:12 <graphitemaster> yeah
09:26:14 <geist> if you just continually work, even if it's a nice environment, it will eventually drag you down
09:26:27 <graphitemaster> I usually take an entire month off once a year, it's December
09:27:09 <geist> in fact i think the google offices eing nice and whatnot actually can be detrimental in some ways since it lends itself to not vacationing as much as you should
09:27:48 <absolute123> graphitemaster: i see you were close to understanding again, being one of the few or one of the first to do it once again. however i in august probably will try to elaborate all the details how the performant code works on gpus
09:30:13 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
09:33:00 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
09:33:29 <absolute123> Aaron Lefohn, actually did that research earlier, i put the link of it https://developer.nvidia.com/gpugems/GPUGems2/gpugems2_chapter33.html i think he also knows how to do the stuff, but just has not posted any code, cause it appears he works on nvidia, but i know how to do all of that too
09:33:29 <bslsk05> ​developer.nvidia.com: GPU Gems | NVIDIA Developer
09:36:20 --- quit: mavhq (Read error: Connection reset by peer)
09:37:32 --- join: hmmmm (~sdfgsf@pool-72-79-164-172.sctnpa.east.verizon.net) joined #osdev
09:37:34 --- join: mavhq (~quassel@cpc77319-basf12-2-0-cust433.12-3.cable.virginm.net) joined #osdev
09:42:52 --- join: jakogut_ (~jakogut@162.251.69.147) joined #osdev
09:50:10 --- join: nicht (~nicht@179.186.125.214.dynamic.adsl.gvt.net.br) joined #osdev
09:50:11 --- quit: nicht (Max SendQ exceeded)
09:50:32 --- join: xenos1984 (~xenos1984@22-164-191-90.dyn.estpak.ee) joined #osdev
09:51:06 --- quit: Desetude (Quit: WeeChat 2.1)
09:53:30 <absolute123> indians have landed two hunks of appearingly good looking code, x86 core, miaow -- the last one has a testbench which is compatible with mram too, mram is going to rule the world, it seems pretty clear, it will make most hw vintagely stale, which dominated before
09:55:07 * geist is starting to guess that absolute123 is our crazy estonian friend
09:55:23 <absolute123> everywhere where we look actually hindus are very much interested in the hw beforehand anything else
09:55:33 <absolute123> geist: you are the crazy one
09:55:41 <geist> like a fox
09:56:38 <absolute123> miaow/src/verilog/tb/issue/issue_tb.v is the file
09:56:43 --- join: MrOnlineCoder (~MrOnlineC@195.225.231.218) joined #osdev
09:57:01 <absolute123> geist: well your other violators from estonia aka your heros, they get killed soon
09:57:14 * geist raises an eyebrow
10:01:33 --- quit: absolute123 (Remote host closed the connection)
10:05:12 --- join: Goplat (~Goplat@reactos/developer/Goplat) joined #osdev
10:06:12 --- join: zeus1 (~zeus@154.224.162.124) joined #osdev
10:08:22 --- join: dennis95 (~dennis@mue-88-130-61-065.dsl.tropolys.de) joined #osdev
10:11:18 --- quit: S_Gautam (Quit: Connection closed for inactivity)
10:11:36 --- join: shams (~golu@117.205.137.137) joined #osdev
10:12:21 --- part: shams left #osdev
10:14:26 --- quit: graphene (Remote host closed the connection)
10:15:55 --- join: graphene (~graphene@46.101.134.251) joined #osdev
10:18:27 --- join: grumble1 (~grumble@freenode/staff/grumble) joined #osdev
10:18:35 --- quit: grumble (Read error: Connection reset by peer)
10:18:43 --- nick: grumble1 -> grumbe
10:18:44 --- nick: grumbe -> grumble
10:22:16 --- quit: bm371613 (Quit: Konversation terminated!)
10:23:23 --- quit: SopaXorzTaker (Remote host closed the connection)
10:23:47 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
10:35:09 --- quit: graphene (Remote host closed the connection)
10:36:57 --- join: graphene (~graphene@46.101.134.251) joined #osdev
10:43:36 --- join: isd (~isd@pool-71-174-32-198.bstnma.east.verizon.net) joined #osdev
10:43:58 --- quit: graphene (Remote host closed the connection)
10:45:24 --- join: graphene (~graphene@46.101.134.251) joined #osdev
10:49:57 --- quit: geist (Quit: leaving)
10:51:49 --- join: geist (~geist@tkgeisel.com) joined #osdev
11:00:21 --- quit: systmkor (Quit: ZNC - http://znc.in)
11:04:51 --- join: baschdel (~baschdel@2a01:5c0:1c:6201:bb0:858:a281:6ed8) joined #osdev
11:05:10 <baschdel> Hello
11:06:52 <sortie> Hello baschdel
11:07:11 --- join: isaac_ (~isaac@host81-129-159-113.range81-129.btcentralplus.com) joined #osdev
11:07:21 --- nick: isaac_ -> isaacwoods
11:07:42 --- quit: baschdel (Remote host closed the connection)
11:08:00 --- join: baschdel (~baschdel@2a01:5c0:1c:6201:bb0:858:a281:6ed8) joined #osdev
11:10:12 --- quit: asymptotically (Remote host closed the connection)
11:10:30 --- join: asymptotically (~asymptoti@gateway/tor-sasl/asymptotically) joined #osdev
11:11:04 --- quit: ^Inuyasha^ (Ping timeout: 256 seconds)
11:12:36 --- quit: CheckDavid (Quit: Connection closed for inactivity)
11:12:37 --- join: ^Inuyasha^ (~quassel@vmi86808.contabo.host) joined #osdev
11:14:30 <baschdel> Does somebody know a simple database that can be used to store some short binary data (16B) in predefined memory regions (My ideas: a fat like filesystem,Lua tables (Beacause why not),a slow and boring lookup table).
11:15:18 <baschdel> Not sure about the lua tables (at all) ...
11:18:15 <mischief> have you heard of berkeley database
11:18:56 <baschdel> Not yet
11:24:24 <baschdel> Looks like a monser
11:24:31 <baschdel> *monster
11:25:38 <mischief> i don't mean the oracle version
11:25:43 <mischief> i mean e.g. http://man.openbsd.org/man3/dbopen.3
11:25:44 <bslsk05> ​man.openbsd.org: dbopen(3) - OpenBSD manual pages
11:28:15 <baschdel> Do you have a download link for the sourcecode?
11:29:31 <mischief> the above manual is for openbsd's copy; you can download openbsd from openbsd.org.
11:29:59 <mischief> see https://en.wikipedia.org/wiki/Dbm
11:30:00 <bslsk05> ​en.wikipedia.org: dbm - Wikipedia
11:33:00 <baschdel> Thank you
11:42:48 --- quit: mavhq (Read error: Connection reset by peer)
11:44:03 --- join: mavhq (~quassel@cpc77319-basf12-2-0-cust433.12-3.cable.virginm.net) joined #osdev
11:44:11 --- quit: promach_ (Ping timeout: 260 seconds)
12:01:43 --- quit: manzerbredes (Ping timeout: 240 seconds)
12:09:21 --- join: lachlan_s (uid265665@gateway/web/irccloud.com/x-iccstayoqozsbvro) joined #osdev
12:09:23 <lachlan_s> How's everyone doing?
12:09:30 <lachlan_s> It's been a while since I logged into this channel.
12:10:20 --- join: spare (~user@unaffiliated/spareproject) joined #osdev
12:14:05 --- quit: [WaterMotion] (Quit: crappy power cut, again.)
12:15:15 --- join: [DeadInMotion] (~while@unaffiliated/awaxx/x-0928682) joined #osdev
12:20:43 --- join: m3nt4L (~asvos@2a02:587:a01d:4100:3285:a9ff:fe8f:665d) joined #osdev
12:24:11 --- join: kryptonunited (~batdownh@gateway/tor-sasl/blackandblue) joined #osdev
12:24:19 --- quit: graphene (Read error: Connection reset by peer)
12:25:48 --- join: graphene (~graphene@46.101.134.251) joined #osdev
12:26:35 --- quit: blackandblue (Ping timeout: 250 seconds)
12:26:39 --- join: drakonis (~drakonis@unaffiliated/drakonis) joined #osdev
12:27:26 <doug16k> lol, when you do alt-printscreen (sysrq) in qemu, you get alt-down, alt-up, alt-down, alt-up, alt-down, sysrq-down, sysrq-up, alt-up, alt-down, alt-up
12:29:30 <doug16k> at least they got the down and up counts to match though
12:30:43 --- quit: [Brain] (Ping timeout: 240 seconds)
12:31:45 --- quit: m3nt4L (Remote host closed the connection)
12:32:06 <doug16k> funnier still, when you hold the initial alt-down, you get that, then when you press sysrq nothing happens, and when you release the alt you get the rest in a burst
12:32:54 <doug16k> I suppose they are doing some low level desktop input thing to capture such an obscure key and I'm lucky sysrq even works
12:33:46 <doug16k> ^^ oh I didn't mention, in qemu
12:33:48 --- quit: baschdel (Ping timeout: 256 seconds)
12:33:54 <doug16k> oh I did actually :)
12:34:57 --- quit: X-Scale (Ping timeout: 240 seconds)
12:36:59 <doug16k> lachlan_s, doing well thanks
12:37:40 --- quit: drakonis (Remote host closed the connection)
12:37:41 --- quit: kryptonunited (Quit: Leaving)
12:39:43 <sortie> Yo lachlan_s
12:41:03 --- join: drakonis (~drakonis@unaffiliated/drakonis) joined #osdev
12:41:54 --- quit: jer64 ()
12:42:52 --- quit: mavhq (Read error: Connection reset by peer)
12:44:05 --- join: mavhq (~quassel@cpc77319-basf12-2-0-cust433.12-3.cable.virginm.net) joined #osdev
12:47:48 --- quit: SopaXorzTaker (Remote host closed the connection)
12:48:03 --- quit: frolv (Ping timeout: 255 seconds)
12:49:32 --- join: bauen1 (~bauen1@ipbcc18c77.dynamic.kabel-deutschland.de) joined #osdev
12:50:18 <doug16k> regressions are fascinating. it's amazing how unused code gradually has subtle issues introduced into it
12:50:49 <doug16k> got my i8042 code working perfect again
12:51:41 <doug16k> I subtly broke it when I refactored it to work with usb keyboards
12:52:31 <doug16k> I had to split out a bit of the virtual key state machine / shift state stuff
12:52:55 <doug16k> I could have sworn it was okay when I first did it though :)
12:59:18 --- quit: ^Inuyasha^ (Ping timeout: 256 seconds)
13:01:40 --- join: ^Inuyasha^ (~quassel@vmi86808.contabo.host) joined #osdev
13:08:31 --- quit: Lowl3v3l (Ping timeout: 264 seconds)
13:10:11 --- quit: apetresc (Ping timeout: 256 seconds)
13:12:12 --- join: apetresc (~apetresc@CPEbc4dfb720b93-CMbc4dfb720b90.cpe.net.cable.rogers.com) joined #osdev
13:12:54 --- quit: ^Inuyasha^ (Ping timeout: 256 seconds)
13:15:25 --- join: ^Inuyasha^ (~quassel@vmi86808.contabo.host) joined #osdev
13:15:27 --- quit: zeus1 (Ping timeout: 268 seconds)
13:15:35 --- join: stee3 (~junk@66.252.139.92) joined #osdev
13:19:06 --- join: m_t (~m_t@p5DDA3524.dip0.t-ipconnect.de) joined #osdev
13:19:09 --- quit: stee (Ping timeout: 268 seconds)
13:24:01 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-190-048.002.203.pools.vodafone-ip.de) joined #osdev
13:24:55 --- quit: isd (Quit: Leaving.)
13:24:56 --- quit: w17t (Quit: Leaving)
13:27:19 --- quit: smeso (Disconnected by services)
13:27:20 --- join: smeso (~smeso@unaffiliated/smeso) joined #osdev
13:27:31 --- join: light2yellow (~l2y@217.30.64.102) joined #osdev
13:28:55 --- quit: Lowl3v3l (Ping timeout: 264 seconds)
13:33:22 --- quit: bluezinc (Quit: Do not go gentle into that goodnight)
13:34:53 --- join: Tazmain (~Tazmain@unaffiliated/tazmain) joined #osdev
13:41:26 --- quit: sandeepkr (Ping timeout: 260 seconds)
13:42:46 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-190-048.002.203.pools.vodafone-ip.de) joined #osdev
13:43:38 --- join: sandeepkr (~sandeepkr@ec2-52-29-251-54.eu-central-1.compute.amazonaws.com) joined #osdev
13:49:33 --- join: X-Scale (~ARM@83.223.225.237) joined #osdev
13:52:11 <mischief> somehow i think my gpu operating at 68 c is a bad thing
13:54:16 --- join: immibis (~chatzilla@222-155-163-212-fibre.sparkbb.co.nz) joined #osdev
13:58:09 --- quit: svk (Quit: Leaving)
13:58:13 --- quit: light2yellow (Quit: light2yellow)
13:58:50 --- join: patv (ac61a3ea@gateway/web/freenode/ip.172.97.163.234) joined #osdev
13:59:04 <sortie> I like to boil water with my CPU
13:59:54 <glauxosdever> Heating people in winter when rendering! :-)
14:00:12 <glauxosdever> (It has been done BTW, too lazy to find the link now)
14:01:52 <glauxosdever> gtg, goodnight!
14:01:54 --- quit: glauxosdever (Quit: leaving)
14:12:21 <mischief> it is actually very warm here
14:12:48 <mischief> i'm not sure my gpu was made to withstand active use in this weather
14:13:19 <asymptotically> my gpu is designed to idle at 65, it's probably okay :)
14:13:44 --- join: freakazoid0223 (~frkazoid@pool-108-52-244-197.phlapa.fios.verizon.net) joined #osdev
14:14:33 <patv> 65?! proper units or freedom units?
14:17:34 <clever> "2018-07-15 18:17:07 bedroom temp: 26.38c(79.47f), kitchen: 27.75c(81.95f), living room: 27.06c(80.71f), outdoor: 24.06c(75.31f), server: 29.06c(84.31f) VCC: over 4.5 volts portb: 00000000"
14:17:49 <clever> patv: i gave up with silly americans, and just wrote my program to output both :P
14:19:28 <patv> clever: handy
14:19:46 <doug16k> air conditioning man, air conditioning
14:20:20 <clever> doug16k: canada
14:20:37 <asymptotically> patv: celsius lol. the fan on it only comes on when there's some load on the gpu
14:20:55 <doug16k> yes
14:21:13 <asymptotically> i think it's meant to work like this, or maybe it's just broken :)
14:23:17 <doug16k> my GTX 1060 idles at 56C and 59C when gaming. sucky idle temp, impressive load temp
14:23:55 <mischief> i have to operate the gpu fans at 6500 rpm to maintain 68 c
14:24:06 <doug16k> my temp display is funny. right now my cpu is 33C and my practically idle gpu is 56C
14:24:06 <mischief> i think my recent hangs are from overheating
14:25:13 <doug16k> mischief, you need to get one of those compressed air cans and blow it out
14:25:42 <doug16k> even better, take off the enclosure and blow it out directly
14:26:00 <mischief> it's not even that old or full of stuff.
14:26:07 <mischief> this locale is simply warm
14:26:58 <mischief> completely idle and clocked down the gpu is 40 c
14:27:13 <mischief> its just when used, it can't disappate heat fast enough i guess
14:27:14 --- quit: graphene (Remote host closed the connection)
14:28:01 --- quit: asymptotically (Quit: Leaving)
14:28:44 --- join: graphene (~graphene@46.101.134.251) joined #osdev
14:30:53 <patv> see that's more like it at 40c idle, not 65c
14:31:22 <doug16k> working on virtio base. there's only one place to put the queue_desc, queue_avail, and queue_used in the virtio_pci_common_cfg structure. you can only have one virtqueue?
14:34:46 --- quit: sortie (Quit: Leaving)
14:35:09 <mischief> doug16k: depends on the device
14:35:14 <mischief> some have multiple queues
14:36:07 <doug16k> how though? there's only one place to put the physaddr of the 3 things in the virtio_pci_common_cfg
14:36:39 <doug16k> wording elsewhere implies that multiple are possible, but I don't see how
14:36:48 --- join: adam4813 (uid52180@gateway/web/irccloud.com/x-ykobptykdpqrkesa) joined #osdev
14:38:57 --- quit: Lowl3v3l (Ping timeout: 265 seconds)
14:40:03 <doug16k> "Each device can have zero or more virtqueues 1." ... then footnote 1 says " For
14:40:03 <doug16k> example, the simplest network device has one virtqueue for transmit and one for receive."
14:40:29 <doug16k> so mischief is definitely correct. It all made sense until I saw the 3 fields in virtio_common_cfg :(
14:41:32 <mischief> it's been a while since i actually looked at virtio
14:41:39 <mischief> but i did write a virtio ethernet driver once
14:42:29 <mischief> §4.1.5.1.4 Virtqueue Configuration
14:43:08 <doug16k> oh! it's bank switched!
14:43:20 <doug16k> you select one queue at a time and set the fields
14:43:24 <doug16k> thanks!
14:43:26 <mischief> yep
14:43:31 --- quit: dennis95 (Quit: Leaving)
14:44:46 <mischief> doug16k: see http://code.9front.org/hg/plan9front/file/a1433cb28946/sys/src/9/pc/ethervirtio.c
14:44:50 <bslsk05> ​code.9front.org: hg/plan9front:sys/src/9/pc/ethervirtio.c
14:44:55 <mischief> search for 4.1.5.1.4
14:48:31 --- quit: MrOnlineCoder (Read error: Connection reset by peer)
14:50:21 <doug16k> makes sense
14:50:39 --- quit: graphene (Remote host closed the connection)
14:51:01 <doug16k> clever to make them banked like that. almost all devices just have a big array of separate fields, switching like that was not in my mind at all
14:52:11 --- join: graphene (~graphene@46.101.134.251) joined #osdev
14:53:57 --- quit: isaacwoods (Quit: isaacwoods)
14:54:44 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-190-048.002.203.pools.vodafone-ip.de) joined #osdev
14:58:49 --- join: CheckDavid (uid14990@gateway/web/irccloud.com/x-csglexkajjnjzsag) joined #osdev
15:00:42 <doug16k> they hardcoded all the virtqueue stuff into the ethernet driver eh? I'm making mine a shared base that factors out the common virtio functionality, so each virtio driver can avoid duplicating all that
15:02:58 --- quit: drakonis (Remote host closed the connection)
15:03:35 <doug16k> should make it almost trivial to implement each virtio driver
15:05:47 <geist> yah, that's what i did when i wrote it
15:05:51 --- quit: Asu (Remote host closed the connection)
15:05:55 <geist> actually works fairly nicely, the layering is pretty clear
15:07:57 --- quit: vaibhav|gone (Ping timeout: 244 seconds)
15:08:29 --- join: drakonis (~drakonis@unaffiliated/drakonis) joined #osdev
15:09:35 <doug16k> I guess I need a fence after switching banks right? to guarantee no load speculated past the store that selects a different queue?
15:09:44 <doug16k> full barrier I mean
15:11:29 <doug16k> hmm, actually, I'd have to map it no-cache. that alone should guarantee sequential consistency eh?
15:12:03 <doug16k> geist, does arm serialize accesses to no-cache pages?
15:14:01 <geist> there's about 16 variants of no cache, but if you mean the full deal yes
15:14:06 <geist> strongly-ordered
15:14:24 <geist> device memory which is how you usually map registers you should do a barrier, yes
15:14:41 <geist> it'll serialize access within that class, but not necessarily relative to other accesses
15:14:44 <geist> or accesses on another bank
15:15:11 --- quit: Kimundi__ (Ping timeout: 244 seconds)
15:15:13 <geist> usual pattern is do all the memory business you want, barrier, then go do register bits, barrier
15:15:38 <doug16k> thanks
15:17:07 <geist> note that virtio generally needs none of this since it doesn't eixst in real hardware
15:17:34 <geist> and there's no need to be ordered relative to the VM
15:18:02 <doug16k> ah... but with hardware virtualization speculations won't sneak around?
15:18:19 <doug16k> I guess it wouldn't vmexit until retirement eh?
15:18:54 <doug16k> good point, thanks
15:20:23 <doug16k> yeah the MMIO would be a nested page fault and it would do a total pipeline flush
15:25:13 <lachlan_s> How's everyone's projects going?
15:25:35 --- join: Kimundi__ (~Kimundi@i577A9BE6.versanet.de) joined #osdev
15:25:37 --- join: frolv (~frolv@CPE00fc8d4905d3-CM00fc8d4905d0.cpe.net.cable.rogers.com) joined #osdev
15:26:10 <doug16k> great. fixed a nasty bug yesterday. making solid progress toward virtio-gpu :)
15:26:21 <lachlan_s> Wow, very nice
15:46:46 --- join: tanner00 (2d389a96@gateway/web/freenode/ip.45.56.154.150) joined #osdev
15:53:21 <Sjors> I'm in the middle of debugging why ASAN triggers a strange error on my OS
15:54:18 <Sjors> I ported ASAN to it in the past couple of weeks
15:54:59 <Sjors> that's been working very well, but now it's finding a heap buffer overflow in a weird place
15:58:50 <patv> I'm in the middle of adding SMP support, its going greadoorly
15:58:59 <Sjors> patv: oh nice!
15:59:04 <Sjors> patv: how far is the rest of your OS?
15:59:13 <Sjors> patv: like, at what point are you adding SMP?
15:59:54 --- quit: Tazmain (Quit: Leaving)
16:00:05 --- join: ljc (~ljc@unaffiliated/ljc) joined #osdev
16:00:53 <patv> Sjors: Not super far, got the standard paging/multitasking/ext2 things, as well as the start of a GUI framework
16:03:57 --- join: absolute123 (~vodka@185.192.189.222) joined #osdev
16:04:27 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
16:04:46 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
16:07:04 <absolute123> can't sleep when such a monkey calls me crazy, geist let's be honest you have not done any reasonable work in your whole life, you o not know what real work is! and let's get the other thing straight, you are not a clear neither in the morning nor in the evening , i have not seen you doing better during the evenings, so no! Usually genetical handycaps can not do more then pump brumous and telling others how hard work they do!
16:07:20 <absolute123> *do
16:07:47 --- quit: absolute123 (Client Quit)
16:08:56 <Sjors> patv: ah, that seems like a good spot to do SMP
16:09:01 <patv> Anywho... Sjors is your OS on github?
16:09:13 <Sjors> patv: I have a lot more running already, but pretty afraid that things will fail horribly once I add SMP :D
16:09:21 <Sjors> patv: yeah: https://github.com/sgielen/cosix/
16:09:22 <bslsk05> ​sgielen/cosix - A fully capability-based operating system (2 forks/11 watchers)
16:09:36 <patv> Sjors: yeah that was my thinking, I'm spending the rest of the summer getting the foundation (smp, 64bit, scheduler) solid before progressing
16:09:40 <geist> yah, gotta do it pretty soon
16:09:48 <geist> a very good idea
16:10:08 <Sjors> my scheduler is still really simple, too: it just does a round robin through its running threads list
16:10:25 <Sjors> but threads are pretty good at blocking themselves when waiting for something
16:10:39 <Sjors> so usually, most of them will be blocking, and the rest get their fair share pretty much automatically
16:10:56 <Sjors> but there's no prioritization or priority inheritance of any kind
16:11:17 <patv> I started off that way, then started to refine it. I was surprised at how drastic the response time could be just by adding priorities
16:11:24 <Sjors> I'm sure, yeah
16:11:26 <geist> yah but that's easy to fix. have a separate list of only threads that are ready to run
16:11:37 <Sjors> geist: yeah I have that
16:11:45 <geist> ah, that's not what you described there
16:11:55 <geist> oh i see, 'running threads list' never mind!
16:12:08 <Sjors> ;)
16:12:31 <Sjors> patv: you can get away with pretty horrible performance as long as the thread the user is communicating with responds quickly, I guess
16:13:31 <Sjors> ok, from reading through ASAN's code, I've just figured out that it's correct at aborting my thread with an error
16:14:00 <Sjors> I initially thought the error was a false positive, because the dump it did seemed to imply that the read it interrupted was actually OK
16:14:12 <Sjors> but it's right
16:14:17 --- quit: MDude (Ping timeout: 268 seconds)
16:14:25 <Sjors> I am learning a *lot* about ASAN's internals this way
16:14:47 <Sjors> lo and behold, it isn't actually magic
16:23:28 --- quit: ljc (Quit: ayy)
16:29:00 <klys> what's ASAN
16:30:58 --- join: tadni (~tadni@24-182-175-184.dhcp.stls.mo.charter.com) joined #osdev
16:31:20 --- join: tadni_ (~tadni@24-182-175-184.dhcp.stls.mo.charter.com) joined #osdev
16:31:40 --- join: MDude (~MDude@pa-67-234-83-197.dhcp.embarqhsd.net) joined #osdev
16:32:52 --- join: chao-tic (~chao@203.97.21.86) joined #osdev
16:34:52 --- join: dallpickle (~Mutter@2605:a601:5e4:d900:12:2c7d:2911:895f) joined #osdev
16:37:50 <Sjors> klys: ah, Address Sanitizer
16:38:16 <klys> is it for tcp/ip
16:38:23 <Sjors> klys: it's C/C++ compiler instrumentation developed by Google, which surrounds every memory access with some code that checks whether the access is legal
16:38:39 <klys> oh ok
16:39:04 <Sjors> klys: then if the access is illegal, it aborts the application and gives a pretty clear overview of the address that was read, why it was illegal, whether it was close to anything that would be legal, where it was allocated, perhaps where it was freed, etc
16:39:15 <Sjors> which helps immensely to find and fix such issues
16:39:58 <Sjors> it's a bit like valgrind, but much faster, and also better at finding some specific issues
16:40:50 <klys> thanks sjors.
16:41:13 --- quit: dallpickle (Remote host closed the connection)
16:41:18 <Sjors> if you write C/C++ sometimes, I'd very much recommend it
16:41:38 <doug16k> Sjors, you added ASAN support in your kernel? how much stuff did you need to do to get it working?
16:41:47 <Sjors> doug16k: no, in the userland only for now
16:41:52 <klange> i also like ubsan
16:41:54 <Sjors> doug16k: but I think it would be possible to implement it in the kernel, too
16:42:25 <Sjors> doug16k: essentially, the most important thing is that you have to place shadow memory somewhere, and teach ASAN where to find it, both in the compiler instrumentation and in the compiler-rt runtime library
16:42:52 <Sjors> doug16k: and then you have to intercept the calls to the allocator and your "libc" functions to the ASAN wrappers, so they can make the necessary changes
16:43:19 --- quit: xenos1984 (Quit: Leaving.)
16:43:57 <klange> can I do that by making an asan version of libc.so?
16:44:09 <Sjors> yeah I've been wondering about that, too
16:44:13 <doug16k> shadow memory is a 2nd copy of the address space that is initially identical to the real one?
16:44:22 <Sjors> I think so, although the current ASAN code wouldn't really help in creating that libc.so
16:44:40 <Sjors> doug16k: somewhat; it uses one byte to represent a configurable amount of bytes in the normal address space
16:44:46 <doug16k> ah
16:44:57 <Sjors> doug16k: by default, its "shadow scale" is 3, meaning every bit represents a byte in normal address space (1 << 3 = 8)
16:45:23 <Sjors> this is how ASAN is actually useful on i386 as well
16:46:07 <Sjors> the shadow range falls somewhere in the middle of the normal memory range, so memory ranges below it are mapped into its lower half, and memory ranges above it are mapped into its upper half
16:46:50 <Sjors> the per-memory-access instrumentation is really simple and fast, just a few instructions, and only if it detects something weird it calls into the runtime library to figure out what went wrong
16:47:01 <Sjors> here's a description: https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm
16:47:03 <bslsk05> ​github.com: AddressSanitizerAlgorithm · google/sanitizers Wiki · GitHub
16:47:17 <doug16k> so it keeps track of what hasn't been initialized and errors if you read something you didn't write, or access freed memory, and that kind of thing?
16:47:46 <Sjors> not initialized, allocated
16:48:04 <Sjors> it won't find use of uninitialized bytes, although it technically might, in some cases
16:48:45 <Sjors> it allocates the shadow memory in one go, assuming the OS will perform lazy initialization of the pages
16:49:06 <doug16k> sounds really doable
16:49:28 <doug16k> I love bug detection tooling
16:49:40 <Sjors> by default, that shadow memory will contain all zeroes, of course, which is the value for unpoisoned memory, so it won't trigger an error
16:49:58 <Sjors> this is why ASAN doesn't find strlen(0x12345) to be initially bad, although it will catch the SIGSEGV and report everything it knows then
16:50:31 <Sjors> but if you malloc() something, it will make sure the returned range is unpoisoned, and some memory around it is poisoned (this is called redzoning)
16:50:39 <Sjors> so if you read to the left or right of an allocation, it'll know
16:50:54 <Sjors> if you malloc(4), it will set one byte of shadow memory to 0x4, meaning "you can read the first bytes of this allocation"
16:50:57 <Sjors> 4 bytes*
16:51:00 <doug16k> guards are great
16:51:03 <Sjors> yep
16:51:12 <Sjors> and in this case, the guards are in separate memory, which is pretty nice
16:51:31 <Sjors> if you free(malloc(4)), that memory will be set to a specific shadow memory byte for "this used to be an allocation"
16:51:34 <Sjors> so it'll be able to report double frees
16:51:43 <Sjors> then it'll keep that memory quarantained for quite a while
16:52:09 <Sjors> it'll save the first free() call somewhere else, so it'll even be able to tell you when the first free() was
16:52:19 <Sjors> it's a really strong tool
16:52:40 <Sjors> it even works with stack memory, too
16:53:02 <Sjors> it adds a guard range around all globals, all stack variables, etc, so it will be able to detect buffer overflow on globals/stack
16:53:28 <Sjors> (in that sense, you could use it as a security tool, too)
16:53:41 <Sjors> (if you accept the performance loss, which is significant, but not as large as valgrind's)
16:57:30 <doug16k> I noticed fsanitize=kernel-address is there too. neat
17:00:29 --- quit: atk (Quit: Well this is unexpected.)
17:00:44 --- join: atk (~Arch-TK@ircpuzzles/staff/Arch-TK) joined #osdev
17:01:07 <doug16k> all you do is implement kasan_mem_to_shadow, __asan_load*(addr), and __asan_store*(addr)) ?? wow
17:03:04 <doug16k> * means a version for 1, 2, 4, 8, and 16 byte operands
17:04:23 <doug16k> and of course you'd have to augment your page allocation and heap allocations to mark ranges live or dead
17:05:10 <doug16k> Sjors, thanks :)
17:05:37 <Sjors> oh, wow, it's that easy?
17:05:53 <doug16k> looks like it -> https://www.kernel.org/doc/html/v4.10/dev-tools/kasan.html
17:05:53 <bslsk05> ​www.kernel.org: The Kernel Address Sanitizer (KASAN) — The Linux Kernel documentation
17:05:55 <Sjors> that's even simpler than I expected
17:06:00 <Sjors> nice!
17:06:13 <Sjors> doug16k: I'll expect a 6-page report on its implementation in your kernel on my desk in a couple of days
17:06:19 <doug16k> lol
17:07:24 <patv> and don't forget the TPS cover letter, you got the memo right?
17:07:48 <doug16k> yes
17:08:42 <klange> Sjors: A couple of days? That sounds almost realistic. You're supposed to say "tomorrow".
17:09:41 --- quit: Lowl3v3l (Ping timeout: 260 seconds)
17:09:51 <Sjors> klange: tomorrow, then, that'd be greaaaat *sips coffee*
17:10:00 <doug16k> klange, don't worry, he'll make sure I get so many meeting invites that I have no chance of finishing it
17:10:47 <Sjors> here I find myself annotating libc++ with printf debugging statements to understand why ASAN thinks some vector buffer access is overflowing
17:11:00 <Sjors> vector<_Tp, _Allocator>::__swap_out_circular_buffer(__split_buffer<value_type, allocator_type&>& __v)
17:11:12 --- quit: drakonis (Remote host closed the connection)
17:11:30 <klange> I have no idea what to work on, but I don't want to work on th at.
17:11:35 --- quit: spare (Ping timeout: 248 seconds)
17:11:48 --- mode: ChanServ set -o klange
17:12:03 <Sjors> I've seen so many bugs that I know it's unwise to suspect libc++ of a bug
17:12:09 <Sjors> or the compiler, although it does happen
17:12:21 <Sjors> but now I'm forced to choose between libc++ or ASAN as the more likely location of a bug
17:12:30 <Sjors> it's for real, dawg
17:12:47 --- join: promach_ (~promach@bb219-74-174-136.singnet.com.sg) joined #osdev
17:13:42 <Sjors> we should have an OSDev conference someday
17:13:50 <klange> it has been talked about so many times
17:14:10 --- join: drakonis (~drakonis@unaffiliated/drakonis) joined #osdev
17:14:17 <patv> I'd be down for that, if it were conveniently located within 100km
17:14:31 <Sjors> we probably have some people who work on a campus with a well connected airport in the neighbourhood
17:14:43 <klange> We'd need to find a venue, probably have a little indiegogo fundraiser to lock down the money to rent it out, and confirm a half dozen talks.
17:15:23 <Sjors> many people who work on campuses can get rooms on the cheap in the weekend for example
17:15:23 <klange> There's a big question of where - due to immigration concerns over the past few years, the US seems out. Somewhere in Europe is most likely. Our best bet may be to negotiate something with an existing tech conference.
17:15:34 <Sjors> ah yeah that sounds interesting
17:15:52 <Sjors> I know some people who organize the Dutch Unix User Group meetings who might be interested
17:16:13 <klange> I think if we can get enough guaranteed attendees with talks, we could approach someone with a larger event to ensure an audience.
17:16:17 <Sjors> but that's a somewhat luxurious conference, and already pretty busy, so could be too much for them
17:16:29 <Sjors> I'll talk to some of them about it
17:16:41 <Sjors> and with luxurious I mean expensive
17:16:46 <Sjors> might rule some people out
17:18:44 <klange> As much as I'd love to hold something here in Tokyo...
17:19:12 <klange> My, uh, associated venues are not exactly the right kind for such an event - and I think Tokyo is too problematically located for just about everyone.
17:19:41 <Sjors> yeah, I'm afraid so
17:20:25 <Sjors> I may have some short lines to computer science in Amsterdam, which is very well connected to airports around the world
17:20:35 --- join: palk_ (~palk@unaffiliated/palk) joined #osdev
17:20:49 <Sjors> and perhaps I can get some scientists there interested in sponsoring the location in the weekends
17:20:55 <Sjors> I'll ask around!
17:21:38 --- join: jack_rabbit (~jack_rabb@c-73-211-181-224.hsd1.il.comcast.net) joined #osdev
17:23:11 --- join: ybyourmom (~ybintell@vampire.ertos.nicta.com.au) joined #osdev
17:23:43 --- quit: palk (Ping timeout: 240 seconds)
17:23:55 <klange> The other option is to get a corporate sponsor with a venue; one that might have some topics of their own to talk about at such an event... *coughs loudly in the general direction of geist*
17:24:33 * ybyourmom hacks up some phlegm at geist
17:24:42 <drakonis> cough cough google
17:24:50 <drakonis> get the linux foundation fyi
17:25:19 <drakonis> they're a bridge between corporate sponsors and linux
17:25:26 <klys> I'm not sure what kind of websites you
17:25:42 <klys> 're insterested in announcing this on,
17:26:06 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-128-059.002.203.pools.vodafone-ip.de) joined #osdev
17:26:16 <klys> I would suggest consorting with various linux users groups
17:26:16 * ybyourmom releases a whooping cough at geordi
17:26:25 <drakonis> their focus isn't just linux btw
17:26:33 <drakonis> they also sponsor other OSes
17:27:07 * ybyourmom falls over in geist 's lap coughing with TB
17:27:11 <klys> can you think of some other local groups that may be interested?
17:27:44 <Sjors> klange: do you know where geist works? in what city?
17:28:00 <klange> I would love to suggest sigops, but they only seem to care about highly academic distribution systems topics these days...
17:28:06 <klange> distributed*
17:28:19 <Sjors> drakonis, klys: linux foundation / linux user groups is interesting, I wonder if they would be interested in talking about non-Linux, too
17:28:26 <Sjors> klange: I'll ask them, too!
17:28:33 <klange> Sjors: geist is in the cold northern land amongst the Microsofts, Amazons, and Valves.
17:28:50 <drakonis> the valves are in washington tho
17:28:58 <drakonis> they left seattle a while ago
17:29:09 <Sjors> klange: ah, so U.S.?
17:29:12 <drakonis> the foundation sponsors other two OS projects i think
17:29:15 <drakonis> zephyr and something else
17:29:15 <Sjors> I first thought you said Volvos
17:29:26 <klys> I wouldn't really recommend the ACM, if that's still around; as if they are really interested in computing, they seem more like a college fraternity.
17:30:02 <Sjors> I just thought about Nokia, who also used to sponsor Qt and KDE, but
17:30:10 <Sjors> it's probably painful to talk about OSes to anyone from Nokia
17:30:11 <klange> drakonis: Bellevue ~= Seattle, and Microsoft is just north of there.
17:30:31 <drakonis> they're not in bellevue anymore
17:30:41 <drakonis> oh wait
17:30:45 <drakonis> that's where they went
17:30:52 <drakonis> lmao i am a confused person
17:31:40 <klange> And I believe Google's got an office in Kirland, which is just next to Redmond.
17:32:03 <Sjors> klange, patv: anyway, I guess I'll start calling around and have a report on your desk by tomorrow, complete with TPS cover page
17:33:00 <patv> Sjors: That'd be great, mmk
17:33:11 <Sjors> :D
17:34:10 --- quit: chao-tic (Ping timeout: 244 seconds)
17:38:40 <drakonis> sounds good
17:39:54 <klys> a few organizations local to me include: https://www.meetup.com/Lehi-Developers-User-Group/ https://plug.org/ https://linuxclub.cs.byu.edu/
17:39:56 <bslsk05> ​www.meetup.com: Lehi Developers User Group (Lehi, UT) | Meetup
17:39:57 <bslsk05> ​plug.org: PLUG: Provo Linux User Group — Provo, Utah, USA
17:39:59 <drakonis> fosdem fyi
17:39:59 <bslsk05> ​linuxclub.cs.byu.edu: BYU Linux Club – Help new and experienced Linux users gain marketable administration skills.
17:40:11 <drakonis> they're expanding their focus
17:40:17 <drakonis> look up fosdem x
17:40:43 <drakonis> FOSDEMx
17:40:44 <drakonis> rather
17:40:47 <drakonis> https://x.fosdem.org/0/
17:40:48 <bslsk05> ​x.fosdem.org: FOSDEMx Edition 0
17:42:11 <drakonis> the first one was about python
17:43:10 <drakonis> perhaps the next one could be operating systems
17:45:31 --- quit: tacco| ()
17:45:43 --- join: AverageJ0e (~joe@ip72-222-140-99.ph.ph.cox.net) joined #osdev
17:45:59 <Sjors> I visited FOSDEM for quite some years, but lately it's been too crowded
17:46:21 <Sjors> all the interesting rooms are fully filled, and there almost definitely won't be room for another OS development room
17:46:24 <doug16k> this is what I had to define to get my kernel to link with -fsanitize=kernel-address -> https://gist.github.com/doug65536/a86375f07a7a5bfc9c606fefc35e8afe
17:46:25 <Sjors> but who knows
17:46:25 <bslsk05> ​gist.github.com: asan.cc · GitHub
17:47:31 <drakonis> thus you want fosdemx
17:47:42 <drakonis> its a single day single track event
17:47:57 <drakonis> "
17:47:57 <drakonis> FOSDEMx is a small scale spin-off of FOSDEM, combining a workshop track geared towards students and anyone interested in the topic, with a main track for a broader audience."
17:48:57 <klys> drakonis, would you call osdev a single-track event, tho?
17:49:01 <drakonis> hmm
17:49:07 <drakonis> i couldn't
17:49:10 <drakonis> it lacks nuance
17:49:21 <drakonis> its too extensive to be an single track event
17:50:42 <klys> I'm sure we could demonstrate various operating systems, just that we would need to make solid connections to OS theory in front of CS stunents.
17:50:51 <klys> students*
17:52:15 <drakonis> certainly
17:52:20 <drakonis> but it is the cost of hosting fosdemx
17:53:21 <geist> wat.
17:53:42 <drakonis> needing to connect the speaker's work to theory
17:53:44 <Sjors> I've added FOSDEMx to the list of people to contact about this
17:53:55 <drakonis> FOSDEMx = FOSDEM
17:53:55 <klange> run away, geist is here!
17:54:02 <drakonis> RUN TO THE HILLS
17:54:06 <Sjors> if they don't think it's a good fit, perhaps they'll have some good advice
17:54:07 <drakonis> THE GHASTLY MAN IS HERE
17:54:23 <klys> geist, scroll to :13
17:54:25 <geist> wait, what'd i do this time?
17:54:50 <geist> didn't do anything, though i've been replacing my network gear at home with ubiquiti stuff all day
17:54:57 <geist> lovely hardware, recommended if you can afford it
17:55:28 <Sjors> absolutely agree
17:55:47 <Sjors> recently replaced all network hardware at my employer with it, really easy to setup and it works great
17:56:06 <geist> my aging airports are not really getting updates anymore, and i saw a friend set up their house with it
17:56:13 <geist> was impressed with the configurability and ui and whatnot
17:56:21 <geist> so just got a pair of APs, been happy thus far
17:56:44 <Sjors> we have a pretty small setup, only four APs and a USG, but it should scale to a much larger setup
17:57:08 <geist> yah my house is just big enough that it needs two APs on both ends to have really good solid network
17:57:16 <klys> sjors, I have my doubts that geist ever reads his scroll buffer...
17:57:16 <geist> and then as soon as you have two aps you really want em to interop well
17:57:27 <geist> i did, but i'm not sure what you want me to do about it
17:57:36 <geist> as in there was some blabbing about some stuff a while ago
17:57:44 <Sjors> ah it's ok, it wasn't too relevant I think
17:57:44 <geist> and then got off topic and then, shrug
17:57:52 <klys> oh, well that's enough for now then
17:57:59 <geist> yes, i live in seattle. and there's a lot of companies here
17:58:04 <geist> it's a nice place to live
17:58:13 <Sjors> we were discussing an OSdev conference and somebody suggested Google could be interested in sponsoring it, as it would have topics of its own as well
17:58:18 <geist> ah
17:58:19 <klange> geist: There was some discussion on holding an OSdev conference/meetup.
17:58:23 <geist> right
17:58:30 <klys> are you aware of a convsntion center in your community
17:58:44 <geist> where emeraldcon is
17:58:46 <klange> (It was also suggested that the US might not be a good place to hold such a conference, due to immigration concerns)
17:58:55 <geist> cosplay requirement
17:59:04 --- quit: lachlan_s (Quit: Connection closed for inactivity)
17:59:05 <klange> we can all cosplay OS-tans
17:59:07 <geist> everyone has to dress up as their os-tan of choice
17:59:08 <Sjors> heh, combine it
17:59:24 <geist> oh wow, that reminds me. we need a fuchsia-tan
17:59:30 <geist> or zircon-tan
17:59:54 <geist> though there's already a Zircon character on land of the lustrous
18:00:35 <geist> though for whatever reason they have orange hair
18:01:01 <klange> My OS actually has an OS-tan, and I have a t-shirt somewhere.
18:02:01 <klange> It's just a generic character in the school outfit from とある科学の超電磁砲
18:02:35 <geist> needs a unicorn horn
18:03:21 --- quit: Lowl3v3l (Ping timeout: 260 seconds)
18:03:21 <klys> https://www.google.com/maps/place/Washington+State+Convention+Center/@47.6114557,-122.3326792,146m/data=!3m1!1e3!4m5!3m4!1s0x54906ab5090f3cc7:0x38d32c762126bc69!8m2!3d47.6115429!4d-122.3326296
18:03:22 <bslsk05> ​www.google.com: Google Maps
18:03:39 <geist> yep, also home to PAX west
18:05:15 <klys> geist, are you familiar with some local linux users groups?
18:05:37 <klange> I don't see why we're still talking about Seattle, the US a non-starter for too many people.
18:05:46 <geist> i am not, and i'm probably not that popular with them
18:06:08 <klys> klange, did yo usuggest the netherlands or something?
18:06:41 <klange> I only said Europe generically and noted Tokyo would likely be unsuitable for most people.
18:07:16 <geist> it's warm today too
18:07:36 <drakonis> klange, nice anime weeaboo
18:07:41 <drakonis> (its actually not bad)
18:07:54 <drakonis> surprisingly good
18:08:19 <klys> https://media.mnn.com/assets/images/2016/11/authagraph-world-map-1.jpg.838x0_q80.jpg
18:09:22 <klys> I wonder if anyone here is actually from europe.
18:09:45 <klange> A significant portion of this IRC's userbase is in Europe.
18:10:13 <klys> you mean like, freenode
18:10:24 <klange> No, I mean this channel.
18:10:27 <ybyourmom> Ugh, europe, yuck
18:10:31 <klys> o.o
18:10:44 <ybyourmom> https://www.youtube.com/watch?v=UfTYptEDlpw
18:10:45 <bslsk05> ​'Europe is a garbage continent' by J B (00:00:52)
18:11:31 <klys> i guess bring it up again in 12 hours or so
18:11:31 --- quit: AverageJ0e (Ping timeout: 264 seconds)
18:15:00 --- quit: m_t (Quit: Leaving)
18:15:02 <geist> it's definitely a bit late there now
18:18:15 <klys> you're about the same distance from london as you are from tokyo.
18:18:26 <Sjors> <geist> it's definitely a bit late there now
18:18:31 <Sjors> can confirm. am in netherlands, 3:18 a.m.
18:18:50 <Sjors> klys: I guess a big part of organizing this would be to see who would be willing to travel where
18:19:16 <klys> sjors, and who is interesting out of the bunch
18:19:16 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-128-089.002.203.pools.vodafone-ip.de) joined #osdev
18:19:52 <Sjors> I'd be more comfortable organizing it in Europe, myself, because then it's practical for me to fly there beforehand to meet people and stuff
18:21:11 <klys> osdev is likely done ar various extremes of personal income.
18:22:45 <klange> Sjors: go to bed
18:23:04 <Sjors> klange: good point
18:23:05 <Sjors> ttyl!
18:23:32 --- quit: tanner00 (Quit: Page closed)
18:25:13 --- quit: promach_ (Quit: WeeChat 2.1)
18:26:48 <grange_c> can confirm too, french here, nearly ~3:30am, still awake
18:27:51 <klange> I guess I'm one to judge... I was up until 6am, got maybe two solid hours of sleep, and now I'm at work!
18:27:59 <klange> club life is hard life
18:28:09 <geist> le club
18:34:21 --- quit: adam4813 (Quit: Connection closed for inactivity)
18:40:33 <klys> klange is about 10k miles from paris and chicago, and about 8k miles from abu dhabi and san francisco.
18:40:51 <klange> sounds about right yeah
18:41:05 <klys> :)
18:41:09 <klange> not that I would ever want to go to abu dhabi, and not that I particularly enjoy going to san francisco
18:41:20 <klange> I'll be in San Francisco next week, for one day.
18:41:41 <klys> what brings you to SF ?
18:41:52 <klange> Technically a layover on the way to Minneapolis to see family.
18:42:02 <klys> ah, family, gr8
18:42:03 <klange> But it was intentionally planned to be a 13 hour layover.
18:42:18 <klange> So that I can try to empty some stuff out of a storage unit and possibly move whatever's left to a smaller one.
18:43:57 --- quit: robert_ (Ping timeout: 256 seconds)
18:44:43 <klange> Unfortunately, it wasn't perfectly ideal - despite an excellent layover from roughly 11am to midnight on a weekday, the shiping company I wanted to use to just empty the whole thing was not available for pick up that day.
18:45:03 <klange> So instead I'll probably end up shipping a bunch of separate boxes by FedEx.
18:45:19 <klys> they ship that much overseas? I wonder what company that is.
18:45:45 <klange> The company I was looking at is Yamato, Japanese shipping company - they offer moving services to and from Japan.
18:46:00 <klys> cool
18:46:11 <klange> Yeah, unfortunate it didn't work out.
19:00:09 <klys> klange, wasn't sure if you were referring to this when you said "the US a non-starter for too many people." https://cdn.images.express.co.uk/img/dynamic/139/590x/secondary/Trump-ban-982361.jpg
19:00:42 <klange> Not a major factor, I don't think.
19:01:26 <klange> There was a spell of perfectly ordinary people from allied western nations being denied ESTAs for tech conferences over the past couple of years.
19:04:49 --- quit: Kimundi__ (Ping timeout: 240 seconds)
19:06:50 --- join: chao-tic (~chao@203.97.21.86) joined #osdev
19:08:18 --- quit: CheckDavid (Quit: Connection closed for inactivity)
19:13:25 <drakonis> https://www.cs.vu.nl/~herbertb/papers/osreview2008-2.pdf
19:21:16 --- join: zeus1 (~zeus@197.239.37.104) joined #osdev
19:21:20 <klys> so it creates pipes in honor of filtered network traffic
19:24:58 --- join: Matviy (49dff905@gateway/web/freenode/ip.73.223.249.5) joined #osdev
19:25:01 --- quit: JusticeEX (Ping timeout: 260 seconds)
19:26:43 <Matviy> How would an OS know what IRQ some interrupt arrived from? AFAIK the PIC translates the IRQ into a vector, which gets read via I/O port by the kernel. Since the vector can be programmed to be literally anything, how does the OS even know about IRQ?
19:26:56 <Matviy> (on an x86 PC)
19:27:13 <klys> you usually set the irqs to different vectors.
19:28:48 <Matviy> Yeah so say the PIC was programmed to send vector 200 to the CPU for IRQ 10. All the Kernel sees is vector 200, and it simply runs IDT[200], knows nothing about IRQ 10 itself.
19:29:44 <klys> usually the system that programmed the pic is the same system that also handles the vector.
19:45:35 --- quit: drakonis (Remote host closed the connection)
19:50:41 --- quit: zeus1 (Ping timeout: 260 seconds)
19:54:49 --- quit: Lowl3v3l (Ping timeout: 240 seconds)
19:57:18 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
19:57:37 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
20:06:53 <doug16k> Matviy, your OS has to keep track of it
20:08:00 <doug16k> I have a 256 byte lookup table. I look at the vector number, read that byte of the array, bang, I know which IRQ it is
20:08:48 <doug16k> purely for the convenience of the driver. drivers that have multiple IRQs like to take irq - first_irq to get an offset and act on it accordingly
20:09:20 --- join: Lowl3v3l (~Lowl3v3l@dslb-002-203-195-110.002.203.pools.vodafone-ip.de) joined #osdev
20:09:54 <doug16k> I also have a reverse lookup table. you can ask "which vector is this irq", lookup that index, bang, I have the vector
20:10:50 <Matviy> KK got it, it just wanted to make sure that there wasn't some detail I was missing.
20:16:24 <Matviy> I have a book that says "There are three ways to select a line for an IRQ-configurable device:" and it mentions using jumpers on the device, reading/writing config space...etc
20:16:33 <doug16k> I also have a lookup table to instantaneously find which IOAPIC a given IRQ is. the majority of interrupts are MSI or MSIx though if you have good pci code
20:16:41 <Matviy> How is this possible? Isn't each device wired to the PIC with a dedicated IRQ line?
20:17:01 <doug16k> no
20:17:24 <klys> it used to be that way
20:17:30 <doug16k> physically PCIe will just be a packet on a dedicated link, but PCIe tries to be very PCI like
20:18:05 <doug16k> if you mean ancient, each PCI card has 4 possible IRQ lines it can drive. A B C or D
20:18:23 <klys> until the line drivers were replaced with the plugplay/edid mechanics
20:18:26 <doug16k> but PCI also has a vastly superior way of sending IRQs: MSI
20:18:48 <klys> and that was eventually replaced with the apic
20:18:50 <doug16k> you can just pick a vector, and that device's IRQ is that vector
20:19:14 <doug16k> PCI can also have a device use 1, 2, 4, 8, or 16 vectors
20:19:15 <Matviy> So is there even a PIC anymore? Does the "IRQ message" that the device sends to the the CPU with the IRQ# appear in the OS and the OS turns it into a vector?
20:19:43 <doug16k> PCIx can have tons of vectors per device, and they can all be arbitrary, no need to be contiguous, and each can be routed to any CPU
20:20:11 <doug16k> Matviy, from the programmer perspective, it looks like the PIC is there, but it is an illusion
20:20:27 <doug16k> it's integrated into a gigantic complex PCIe thing now
20:22:28 <klys> the apic is usable to devs who can already do things like lsusb and lspci.
20:22:37 --- quit: patv (Ping timeout: 252 seconds)
20:22:45 <Matviy> Yeah i wish the book I'm reading didn't jump from "How to master-slave two 8259A PIC's together" to "Here's how to assign each device it's own IRQ using config space".
20:23:11 <doug16k> Matviy, do you want to support SMP?
20:23:31 <klys> symmetric multiprocessing
20:23:56 <Matviy> I'm just trying to understand Linux, but that involves SMP AFAIK so yeah
20:24:17 <klys> have you done make menuconfig?
20:24:35 <doug16k> oh, I thought you were implementing something
20:24:52 <Matviy> klys, i did but I didn't change anything.
20:25:31 <klys> smp is the way to use multiple cpu cores.
20:26:27 <klys> you can enable it with CONFIG_SMP=y
20:31:16 <klys> the basic fundamentals you should understand about linux and kernels include pc architecture: segmentation, paging, assembly, and more generic things: memory management, virtual file system, C, device drivers (i/o, irq, and dma), booting, and char/block devices sockets and pipes.
20:35:01 <Matviy> Thanks I'm working on most of those right now, got a bit stuck in the IRQ/interrupts/exceptions section
20:36:50 <klys> linux can also use the PIC if you specify noapic on the kernel command line.
20:37:24 <doug16k> PIC is ancient, PCI INT A/B/C/D are old and should be avoided, IOAPIC is half decent, MSI is great and reasonably modern, MSIx is excellent and modern
20:38:12 <klys> msi is message signalled interrupts
20:42:19 <klys> doug16k, would you know anything about irq13 using the coprocessor/fpu ?
20:42:40 <doug16k> yes. in ancient times when the FPU was external it signalled an exception through IRQ 13
20:43:16 <klys> 486 and prior
20:43:24 <doug16k> in this century, you setup CR0 to use a normal exception vector and not signal external hardware
20:44:37 <doug16k> see CR0.NE bit in intel manual. when it is 1, modern mode is enabled and it raises FPU exceptions internally
20:45:08 <doug16k> I don't think 0 even does anything anymore, but I'm not sure, and don't really care to be honest :)
20:45:18 <doug16k> set it to 1 and forget it
20:45:53 <klys> would you suppos it electrically feasible for a CPU to use another CPU of the same MHz rating as a coprocessor?
20:46:29 <doug16k> today? no. there's no bus like then
20:46:53 <doug16k> the way that the cpu connects to the motherboard is completely different
20:47:03 <doug16k> drastically different
20:47:34 <doug16k> oh you don't mean an FPU?
20:47:35 <klys> in order to get a different brand of processor in there one may be back at the 486 level (150 MHz or so)
20:47:58 <doug16k> well of course there is SMP
20:48:28 <doug16k> you can get motherboards with multiple sockets and they all work together
20:48:40 <doug16k> you need special server-grade CPUs for that capability too
20:49:02 <klys> what does symmetric mean?
20:49:09 <doug16k> means all the same
20:49:24 <doug16k> asymmetric means they can be different
20:50:03 <klys> well I think I want different cpus with differing instruction sets, and doubt it would be possible with even a pentium-200
20:50:27 <doug16k> that's called heterogeneous computing
20:50:39 <doug16k> a CPU and an entirely different GPU is an example
20:51:29 <klys> yeah except heterogenous access to the same DRAM
20:51:35 --- join: AverageJ0e (~joe@ip72-222-140-99.ph.ph.cox.net) joined #osdev
20:51:44 <doug16k> a GPU can bus master access the same RAM
20:51:56 <klys> it's slower though
20:52:30 <ybyourmom> bus master access is necessarily slower than the CPU?
20:52:47 <doug16k> it can be crazy fast
20:53:01 <doug16k> 31.5GB/sec
20:53:17 <klys> it has to go through the bus, so you're not at "northbridge" speeds
20:53:23 <doug16k> https://en.wikipedia.org/wiki/PCI_Express
20:53:24 <bslsk05> ​en.wikipedia.org: PCI Express - Wikipedia
20:53:31 <doug16k> klys, there is no such thing as a bus anymore
20:53:48 <klys> yeah it sounds like a real mess.
20:53:52 <ybyourmom> ???
20:54:19 <doug16k> PCIe has a ton of individual point to point links
20:54:29 <doug16k> they don't take turns doing something one at a time (a bus)
20:55:12 <ybyourmom> As far as I know, what doug16k is correct; why would bus master be necessarily slower than CPU access, other than the fact that the RAM chip is located near the CPU?
20:56:01 <klys> so right, I can picture, like with an fpga, making a heterogenous bus within the vhdl so that it has multiple cores
20:56:30 <doug16k> the memory controller is integrated into the CPU, as is the PCIe root complex. when a device does DMA it requests it from the CPU through its PCIe links and the cpu's memory controller is used
20:57:00 <doug16k> there is also a crossbar - one device can DMA to/from another device
20:57:53 <doug16k> klys, your argument is based on simply screaming through memory. that is not all there is to it
20:58:18 <klys> explain.
20:58:49 <doug16k> devices can prefetch ahead of what they are processing. a device doesn't sit there hammering DMA continuously
20:59:04 <doug16k> so there may not necessarily be any slowdown
20:59:13 --- quit: nj0rd (Ping timeout: 240 seconds)
20:59:20 <doug16k> assuming it starts a transfer far enough ahead of time that it is done by the time it needs it
21:00:05 <doug16k> is it perfect? no. that's why devices have a ton of onboard memory, so they can do the majority of their work on their own local memory
21:00:14 <klys> a cpu attached to a dma bus running an operating system would have to hammer dma constantly.
21:00:36 <doug16k> maybe a fantasy CPU that has zero cache
21:01:26 <doug16k> caches are about 95% effective for most workloads. only about 1 to 5% of memory accesses miss cache typically
21:02:14 <klys> caching would be a problem if you had heterogenous cpus
21:02:23 <doug16k> particularly because CPUs have predictors that predict what is going to be needed in the near future and they start bringing it in automagically
21:02:56 <doug16k> so by the time the code gets to that address, it was brought in already in the meantime
21:03:17 <ybyourmom> and that's how you get speculative execution of instructions with committed side effects and whooooaaa Meltdown+Spectre
21:03:33 <doug16k> that's not meltdown or spectre though
21:03:43 <doug16k> it's just a prefetcher
21:03:54 <ybyourmom> I know - I was talking about speculative execution
21:04:01 <ybyourmom> Go on, don't mind me
21:04:09 <doug16k> meltdown is intel being dumbass enough to do a memory access before checking TLB permission bits
21:05:12 <klys> meltdown was about the time it took to segv accessing predictions and the readable timestamps attached to that.
21:05:13 <doug16k> the permission bits are right there. they cheat to shave a couple of cycles off a miss
21:05:21 --- quit: Lowl3v3l (Ping timeout: 260 seconds)
21:05:42 <geist> yeah, but that's probaby fairly significant
21:06:13 <geist> the complexity of the architecture works against them, since it's hard to store all the perm bits in the L1 cache tag
21:06:23 <geist> even ARM doesn't do it most of the time, i was digging into it
21:06:41 <doug16k> they have to at retirement, so they must be there
21:06:45 --- join: xenos1984 (~xenos1984@22-164-191-90.dyn.estpak.ee) joined #osdev
21:06:59 <geist> nope
21:07:22 <doug16k> no they magically #PF by divining whether it is a supervisor page from nothing?
21:07:33 <geist> no, they do the TLB lookup on the side
21:07:38 <geist> but not after prefetching it
21:07:56 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
21:08:03 <geist> it allows those two ops to proceed side by side, just not retire it until all the boxes are checked
21:08:06 <doug16k> meltdown works when the TLB is warmed up though
21:08:46 <geist> perhaps, but it doesn't matter much, it really requires the L1 cache be warmed up
21:08:52 <Ameisen> I want to know who named these
21:09:06 <Ameisen> is there like a position somewhere for 'Senior Security Flaw Nomenclature Engineer'?
21:09:18 <geist> even if the TLB is fully up to date, it's possible they go ahead and use the L1 anyway, and only do the TLB check in case the path is taken
21:09:23 <ybyourmom> CVS-344343
21:09:25 <geist> and that may be where they get in trouble vs intel
21:09:29 <geist> er vs AMD
21:09:46 <geist> what's unclear is precisely what AMD does to avoid the problem, since Ryzen is about as sophisticated in this department
21:09:50 --- join: zeus1 (~zeus@197.239.5.26) joined #osdev
21:10:04 <geist> could be perhaps ryzen doesn't speculate until the TLB check is done
21:10:30 <geist> *or* they store the ful perm bits in the L1 tag, but that seems fairly complicated since you now have to do L1 cache shootdowns on perm bit changes
21:10:51 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
21:11:00 <doug16k> full? it would only need to have the supervisor bit to handle the security issue
21:11:08 <geist> and i'm not sure it's reasonable to fully flatten all the perms that affect a page into the L1 tag anyway. U/S bit is easy, but what about segmentation?
21:11:36 <geist> technically that affects the behavior as well, and one needs to validate against all the segment limit nonsense
21:12:04 <doug16k> it has to do virtual->linear to know where to lookup in the cache at all
21:12:04 <geist> though i suppose that doesn't need to be flattened into the L1
21:12:44 <geist> I suppose. anyway, i looked into the L1 tags on most of the arm cores and was surprised to see they dont store the U/S bit equivalent either
21:13:07 <geist> even the fairly sophisticated ones like cortex-a72, a73
21:13:20 --- join: nj0rd (~nj0rd@200116b845173b00f806cd096ea28cea.dip.versatel-1u1.de) joined #osdev
21:13:27 --- quit: Belxjander (Excess Flood)
21:13:30 <geist> though iirc, a75 is actually meltdown suceptible. so it may be that a72/a73 just doesn't speculate quite as far to get into trouble, but a75 does
21:13:49 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
21:13:51 <doug16k> the exact specifics aren't that important. what's important is a totally ridiculous security hole was created by them saving a bit of latency on a miss that is slow anyway
21:14:10 <geist> sure, except that AMD isn't suceptable. I wonder why
21:14:30 <doug16k> they must be checking it in the TLB. how else can they do it?
21:14:44 <doug16k> at least just the supervisor bit
21:14:48 <geist> i dont know. that's what i wonder. i dont think just assuming it does it one way is a good answer
21:15:02 <geist> perhaps that's what they do, but would be nice to know definitively instead of just throwing it out there
21:15:18 <doug16k> it's possible that it invalidates the cache line when it sees that the prefetch shouldn't have been allowed
21:15:51 * geist nods
21:16:02 <geist> so far i haven't found anything that indicates someone has dove into it
21:16:16 <doug16k> that'd be best of both worlds: crazy aggressive latency reduction, no information leakage
21:16:32 <geist> and everyone i ask that knows the details of that stuff stares blankly at me when i mention AMD
21:16:44 <geist> ie 'AMD who? what's their market share? 0'
21:16:46 <geist> sheesh
21:17:19 <geist> doubleplus so when i mention high end AMD cores
21:17:20 <geist> ARM
21:17:27 <ybyourmom> jeez louise
21:18:35 <doug16k> geist, did you guys add mitigation for meltdown yet?
21:18:49 <geist> negative
21:18:58 <hgoel> with how frequently these bugs are being found, I've started to lose track of them...
21:19:00 <ybyourmom> Nice, my kernel is still in good company then
21:19:16 <doug16k> I haven't even lifted a finger to mitigate it in my project yet
21:19:34 <hgoel> pretty much just overlooking adding mitigations for any of them, since they can be added later when necessary
21:19:35 <ybyourmom> I haven't even fired a synapse
21:19:42 <geist> it's on the roadmap, but i'd like to really really understand what the future holds there before diving in
21:20:09 <geist> my main question is is it possible to just go all in and mitigate it forever more by simply never mapping the kernel and user space at the same time
21:20:15 <hgoel> not like someone's gonna expect to takeover any computers by attacking my kernel just yet
21:20:21 <ybyourmom> geist: I mean, information can be leaked lol, the future holds people using the exploit to access information they shouldn't be privy to
21:20:24 <geist> which i think is possible with our design, since there's very little copyin/copyout in the kernel
21:20:40 <geist> which would be a much more radical mitigation
21:21:02 <geist> trouble is it would suck on AMD until they get PCID. would be fine on ARM, but then ARM is largely not suceptable to meltdown
21:21:20 <hgoel> or perhaps just not keeping the information that needs to be kept private mapped?
21:21:22 --- quit: hmmmm (Quit: Leaving)
21:21:25 <doug16k> how many PCIDs does KAISER use? just two?
21:21:31 <geist> per process
21:21:40 <doug16k> oh it fully uses PCID then?
21:21:43 --- join: hmmmm (~sdfgsf@pool-72-79-164-172.sctnpa.east.verizon.net) joined #osdev
21:21:43 <geist> one with and one without the kernel mapped simultaneously
21:21:45 <hgoel> so the code is still there, but not most of the data
21:21:49 <geist> i think so. yes
21:22:02 <geist> hgoel: that's basically what the KPTI stuff does
21:22:12 <geist> it's just easier on a design like linux to do it all at once
21:22:14 <ybyourmom> Isn't it still one per process, but the kernel has its own dedicated ASID?
21:22:29 <doug16k> why not let it be one PCID for kernel and one for the current process, and flush as usual when process switching? that's what I was considering doing in my project
21:22:54 <geist> there's some dumb reason, but i honestly forgot
21:23:04 <doug16k> the flush would be just the user PCID
21:23:06 <geist> and the implementation details of the ARM version of it are close enough to be pretty confusing
21:23:09 <hgoel> ah I see, I wasn't aware
21:23:20 <hgoel> this convo is literally my first time hearing of this
21:24:44 <doug16k> I started adding PCID a while back, until I realized it makes shootdown a bit of a nightmare. I backed out
21:24:46 <geist> doug16k: oh yeah i think i remember
21:25:01 <geist> you need two because what you're actually interested in is not the kernel by itself (that's probably a dedicated PCID for kernel threads)
21:25:26 <geist> it's 2 PCIDs per process for the combination of: user space + trampoline; user space + kernel
21:25:48 <Ameisen> I _removed_ the mitigations from the linux kernel in this process.
21:25:50 <Ameisen> So I did the opposite.
21:25:51 <geist> so when you're running user code you use the first one, when you're in the kernel on behalf of that process you use the second
21:25:55 <Ameisen> project*
21:26:49 <doug16k> ah
21:26:59 <geist> so you can see it's not super ideal: if user space touches a page and then makes a syscall to do some IO on that page, when the kernel switches to the Other PCID, it end sup having to TLB miss it all over again under a new PCID
21:27:31 --- quit: freakazoid0223 (Ping timeout: 260 seconds)
21:27:37 <doug16k> and when you shootdown you shootdown two PCIDs
21:27:43 <geist> yah
21:27:49 <geist> for user space stuff that is
21:27:54 <doug16k> and if the kernel shoots down kernel space, it what, shoots down the entire TLB?
21:27:58 <doug16k> or every user+kernel PCID?
21:28:11 <geist> probably? it already kind of did before anyway, with global pages
21:28:27 <geist> it'd basically need to do a mega shootdown for kernel modifications (or some complex thing to try to mitigate it)
21:28:47 <doug16k> ya
21:28:54 <geist> well mega shootdown of that page, not necessarily dumping the entire TLB
21:29:25 <doug16k> mega shootdown = doing a series of invpcid instructions in a loop?
21:29:27 <geist> actually now that i think about it, this alone is probably why you can't cache U/S stuff in the L1 cache tags
21:29:38 <geist> if you switch PCIDs you dont dump the L1, but now the perm bits are different
21:29:48 <geist> you'd have to dump the L1 cache as well when doing a PCID swap
21:30:00 <geist> or really ust a TLB swap. you dont want to have to dump L1 for that
21:31:22 <doug16k> in my kernel, if I shootdown kernel space I nuke the TLB on every CPU. can I do better?
21:32:02 <doug16k> at first I did a thing where it had queues and it tried to be specific. perf was awful
21:32:03 <geist> there are probably some 31337 hax you can do
21:32:11 --- quit: return0e ()
21:32:13 <geist> like, for example, only shotodown the kernel TLB on cores that are not idle
21:32:16 <ybyourmom> you can go scorched earth by also flushing the caches too
21:32:29 <geist> and if they're idle, just mark something that says flush on next wakeup
21:32:43 <doug16k> geist, yeah I have a sequence number thing I increment
21:33:03 <geist> when does it check?
21:33:07 <doug16k> per cpu keeps track of what sequence it has, and there's a global one
21:33:15 --- join: spare (~user@unaffiliated/spareproject) joined #osdev
21:33:18 <geist> and it checks as it comes out of the parked idle state?
21:34:24 <geist> of course that itself causes a cache black hole on the global counter
21:34:49 --- quit: zeus1 (Ping timeout: 240 seconds)
21:35:04 <doug16k> yeah
21:35:20 <doug16k> I forget what I do now. that was years ago now
21:35:38 <doug16k> I think it has something to do with lazy PF but that doesn't make a lot of sense anymore
21:36:14 <geist> so i was thinking if we just always do a 100% context switch to the kernel process, which is also mapped at 0
21:36:26 <geist> andl ean on ASID/PCID for that, it works the way it was designed
21:36:53 <geist> no funny business, the trampoline just slams to the new PCID, and kernel gets its own
21:37:06 <geist> now. the problem is its hard to access user space pages
21:37:10 <geist> you have to do it indirectly always
21:38:09 <klys> so I guess the latest cpu with an exposed bus is an intel core2 (LGA 771 or 775) or an amd athlon (socket A).
21:38:19 <geist> so the question is can you make that fast enough, perhaps with some LRU style page lookup caches to try to keep the translations hot
21:38:23 <doug16k> why not pairs of PCIDs, kernel and kernel+user ?
21:38:47 <doug16k> then when there is a flurry of transitions at least it gets both PCIDs warmed up and it's good
21:39:04 <geist> not sure i understand how that helps
21:39:24 <geist> what about when user is running? does it have a 3rd PCID?
21:39:46 <geist> the whole point is that user needs to run without kernel mapped, so it's a given that you have some sort of PCID for user + trampoline
21:39:47 <doug16k> oops I meant user and user+kernel
21:39:59 <geist> yes that's what KPTI does
21:40:28 <doug16k> yes but that solves what you said just now? then you just access user directly
21:40:39 <geist> one of the big downsides is each process combination has it's own PCID for kernel. so there's effectively no global pages anymore
21:40:53 <geist> and each context switch between processes dumps the entire kernel TLB state by switching to a new PCID
21:41:26 <geist> if kernel runs in its own address space it gets a single PCID, say 1, that is the same everywhere
21:41:29 <geist> so you dont pay that penalty
21:41:41 <geist> question is is that a big enough win, given the complexity of it all
21:42:06 <doug16k> TLB miss isn't bad though. it's about 15 cycles or so if the page tables are in the cache
21:42:11 <geist> it's a given that global pages are Right Out now
21:42:33 <geist> page walker cache is dumped too, remember
21:42:52 <geist> so it probably has to do a complete 4 level walk through the L1
21:42:55 <doug16k> what page walker cache?
21:43:13 <geist> oh boy. you need to really grok the page walker cache
21:43:21 <geist> on x86 it's Mostly Hidden, but very important
21:43:36 <doug16k> why would that be dumped?
21:43:42 <geist> on AMD there's a feature bit to expose it so you can more direclty micromanage it, but i dunno if any does it
21:43:45 <doug16k> page walks are all physical addresses
21:44:07 <geist> because it's stored in the TLB itself
21:44:11 <doug16k> the physical addresses don't suddenly become invalid
21:44:17 <geist> and thus tagged with the same PCID
21:44:29 <doug16k> ah I see what you mean
21:44:31 <doug16k> the levels
21:44:32 <geist> f course they do, you're switching PCIDs, by reloading CR3
21:45:11 <doug16k> so it doesn't have to chase 4 PTEs over and over. gotcha
21:45:20 <geist> so from what i remember on intel cpus it says quite specifically that any invlpgs always implicitly invalidates the page walker cache for that page (i believe)
21:45:40 <geist> which is why you've never had to deal with it
21:46:00 <geist> AMD has a feature bit you can set that turns that off, and then requires that you manually invalidate page walker cache entries
21:46:14 <geist> on ARM it's directy exposed like that, and you have to do it from the start
21:46:51 <geist> it generally only matters when you unmap a page table, if you didn't invalidate it you'd end up with stale lookups to a freed page
21:47:01 <doug16k> idk man. if you totally turned off global pages and totally flushed at every switch, I don't think it would be that bad
21:47:10 <geist> and i had a gnarly bug about a year ago on ARM where i didn't do something right and precisely this happened in an SMp environment
21:47:13 <doug16k> x86 page walks are beast
21:47:26 <geist> yes but are they beast because the page walker cache helps immensely
21:47:36 <geist> or because of L1
21:48:05 <geist> really, i guess the way to test is simply disable globla pages and observe the perforance penalty
21:48:11 --- quit: CrystalMath (Quit: Support Free Software - https://www.fsf.org/)
21:48:15 <doug16k> that would be fascinating, yes
21:48:47 --- join: zeus1 (~zeus@h12cb.n1.ips.mtn.co.ug) joined #osdev
21:50:12 <doug16k> one crap but effective mitigation for intel would be to simply never have any global bit set and invalidate the entire TLB when returning to user mode code, right?
21:50:20 <klys> can you verify: "In long mode, all segment offsets are ignored, except for the FS and GS segments." ?
21:50:29 <doug16k> klys, correct
21:50:56 <klys> except I thought that's how linux corrected for meltdown
21:51:14 <klys> was to not expose kernel addres space
21:51:57 <klys> address space uses a separate CR3 value per process now?
21:52:06 <doug16k> yes but it's more complicated than what I said just now if the cpu has PCID. they try to be more efficient using PCID
21:52:39 <geist> actually i'm trying to remember why the PCID thing actually helps anyway, except maybe the L1 entries are tagged with the PCID
21:52:59 <doug16k> geist, yeah I believe they are tagged
21:53:24 <geist> so yeah dumping the entire TLB doesn't help, because if the L1 still has active entries it can still do the meltdown stuff
21:53:24 <doug16k> seems pretty wide though, 12 bits
21:53:40 <klys> doug16k was that yes directed at my question?
21:53:59 <doug16k> geist, yeah you'd have to switch to a different root page table
21:54:01 <geist> it's really wide on arm, 16 bits
21:54:14 <doug16k> klys, tes
21:54:15 <doug16k> yes
21:54:29 <geist> and depending on how the stage 2 translations (for the hypervisor stuff) L1 entries end up being 16:16 tagged for both ASID:VMID
21:54:31 <klys> ok, that's some fast typing
21:56:31 <doug16k> I'm a reasonably competent typist. I have a rule: you're not allowed to look at the keyboard, ever. over the years it improves your typing :)
21:57:03 <klys> why would such a key feature of segmentation have been removed? was it considered too intrusive, perhaps
21:57:16 <doug16k> klys, because it sucks and nobody uses it
21:58:14 <doug16k> segments are a crappy hack when your offsets can't reach the whole address space
21:58:25 <klys> well that means all data used as code or in cs: has to be linked to the same address space
21:58:28 <doug16k> for example, real mode. 1MB space, 64KB max offset
21:58:39 <doug16k> or 286, 16MB space, 64KB offset
21:58:59 <doug16k> total pain. good riddance to segmentation
21:59:16 <doug16k> if you've ever had to do big real mode programs, you'd know how much it sucks
22:00:04 <klys> I well suppose that I considered segmentation to be intrusive early on and pehaps it led to me working on real-mode currently.
22:02:03 <klys> I suppose you've figured on ways to solve all those problems with paging, though.
22:02:28 <doug16k> example: my bootloader has about 40KB of code. it loads at 0x7c00. I put the code that relocates it to 0x1000 at the end to avoid the relocation code being overwritten during the move...
22:02:46 <doug16k> guess what? it ended up past 0xFFFF....
22:02:57 <klys> in real mode?
22:03:01 <doug16k> yes
22:03:08 <doug16k> so I had to fudge cs to a higher base, and fudge the offset down
22:03:17 <doug16k> because segments are garbage
22:03:29 <klys> yeah because 0x10000 > 0xffff
22:04:55 <doug16k> 0x7c00 + ~40KB > 0x10000
22:05:04 <doug16k> can't call it
22:05:14 <doug16k> have to fudge cs and call fudged offset
22:05:18 <klys> oh you were loading it into video and rom address space then
22:05:23 <doug16k> no
22:05:34 <klys> right no
22:05:48 <doug16k> you're thinking 0xa0000... 0x10000 can't be reached when cs is 0
22:06:02 <klys> 46 isn't that much
22:06:11 <doug16k> it loads at 0x7c00
22:06:22 <doug16k> so that's almost 32KB lost right there
22:06:41 <doug16k> it's 31KB
22:07:05 <doug16k> 33KB is the highest you can branch to without changing cs
22:07:13 <klys> yeah you wrapped a 16-bit segment
22:08:48 <doug16k> so I needed this hack -> https://github.com/doug65536/dgos/blob/master/boot/pxe-bios.S#L26
22:08:49 <bslsk05> ​github.com: dgos/pxe-bios.S at master · doug65536/dgos · GitHub
22:09:13 <doug16k> bumps cs forward and fudges offset down into 64KB range
22:10:09 <klys> so you could load code into a 64-bit segment.
22:10:13 <klys> 16
22:10:20 <klys> so you could load code into a 16-bit segment.
22:10:30 <doug16k> it loads it at 0x7c00...
22:10:38 <doug16k> so there's 31KB of range out the window
22:10:39 <klys> that's understandable
22:10:47 --- quit: CWiz (Ping timeout: 256 seconds)
22:11:01 <doug16k> so I have 33KB left
22:11:23 <doug16k> the code that relocates itself has to be at the end so it won't overwrite itself when moving the code down to 0x1000
22:11:27 <klys> have you ever disassembled a dos bootsector? the math can get a little tricky
22:12:16 <doug16k> my project includes an MBR and chainload bootsector and network booting for EFI or BIOS
22:12:42 <klys> I have before...it's kind of weird thinking in CHS mode.
22:12:47 <doug16k> the only thing left to add is GPT partition support and it'll support everything
22:13:38 <klys> I've thought about GPT this past week and wondered if it would be worth replacing.
22:13:42 <doug16k> CHS is ancient crap
22:13:44 <doug16k> use LBA
22:14:41 <doug16k> someone tell the dumbass that made sector numbers start at 1 that he sucks
22:14:49 <klys> I have some preliminary notes on replacing GPT
22:14:54 <klys> will paste
22:15:40 <doug16k> how you going to do that? the bios won't mindread your format
22:15:53 <klys> https://paste.debian.net/1033873/
22:15:54 <doug16k> it has to be something supported by the firmware
22:15:55 <bslsk05> ​paste.debian.net: debian Pastezone
22:16:55 --- quit: zeus1 (Ping timeout: 264 seconds)
22:17:26 <klys> I predict in the future folks will have EFI load custom compatibility support modules.
22:17:45 <doug16k> from where?
22:18:00 <doug16k> from the EFI boot partition that uses a standard partition format
22:18:20 <klys> so, the gpt structures will be overwritten and written back.
22:18:30 --- quit: chao-tic (Quit: WeeChat 1.0.1)
22:18:38 <doug16k> how? your code isn't running yet when you turn it on
22:18:57 <klys> when booting, the new partition layout will have to write EFI back to the disk.
22:19:24 <klys> amd ot
22:19:44 <klys> and it's anybody's game what was there first
22:19:59 <klys> it just has to be present in the firmware of your board.
22:22:23 <doug16k> I'd rather make something that works on a normal computer myself
22:22:40 <klys> yeah, those are just thoughts that came up
22:25:32 <doug16k> 4 more functions to finish implementing and adjustments to my page allocator and heap and I should have kernel address sanitizer working
22:25:32 --- quit: jakogut_ (Quit: jakogut_)
22:30:36 --- join: zeus1 (~zeus@154.227.89.155) joined #osdev
22:39:22 --- join: antkit (uid256318@gateway/web/irccloud.com/x-odcifxdlntfefqls) joined #osdev
22:42:02 --- join: S_Gautam (~sgautam@59.182.248.230) joined #osdev
22:42:16 --- quit: S_Gautam (Changing host)
22:42:16 --- join: S_Gautam (~sgautam@unaffiliated/gautams) joined #osdev
22:47:03 --- quit: zeus1 (Ping timeout: 248 seconds)
22:48:08 --- join: manzerbredes (~loic@2a01cb0885e31500cfc30bb534d5bc37.ipv6.abo.wanadoo.fr) joined #osdev
22:49:47 --- join: S_Gautam_ (~sgautam@59.182.254.66) joined #osdev
22:50:36 --- join: zeus1 (~zeus@h12cb.n1.ips.mtn.co.ug) joined #osdev
22:52:19 --- quit: S_Gautam (Ping timeout: 240 seconds)
22:54:20 --- quit: S_Gautam_ (Changing host)
22:54:20 --- join: S_Gautam_ (~sgautam@unaffiliated/gautams) joined #osdev
22:54:35 --- nick: S_Gautam_ -> S_Gautam
22:58:44 --- join: flacks (flacks@184.91.69.131) joined #osdev
23:01:52 --- join: nwm (~nwm@d162-156-46-158.bchsia.telus.net) joined #osdev
23:03:12 <doug16k> kernel ASAN is working!
23:04:19 --- quit: zeus1 (Ping timeout: 264 seconds)
23:10:04 --- join: grouse (~grouse@83-233-9-2.cust.bredband2.com) joined #osdev
23:12:10 --- join: CWiz (~cipherwiz@216.21.169.52) joined #osdev
23:12:26 --- quit: manzerbredes (Ping timeout: 256 seconds)
23:20:43 --- quit: froggey (Ping timeout: 240 seconds)
23:22:48 --- join: froggey (~froggey@unaffiliated/froggey) joined #osdev
23:27:19 --- quit: S_Gautam (Remote host closed the connection)
23:33:08 --- join: Lowl3v3l (~Lowl3v3l@ulbp2362.ulb.uni-jena.de) joined #osdev
23:49:31 --- join: manzerbredes (~loic@myriads-lg.irisa.fr) joined #osdev
23:54:01 --- quit: Lowl3v3l (Remote host closed the connection)
23:59:00 --- quit: [DeadInMotion] (Quit: There are paranoid crossroads and shipwreck alleys.)
23:59:37 --- join: zeus1 (~zeus@62.56.248.65) joined #osdev
23:59:43 --- join: [Basix] (~while@62-210-114-224.rev.poneytelecom.eu) joined #osdev
23:59:50 --- quit: [Basix] (Changing host)
23:59:50 --- join: [Basix] (~while@unaffiliated/awaxx/x-0928682) joined #osdev
23:59:59 --- log: ended osdev/18.07.15