channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev&y=18&m=9&d=15

Saturday, 15 September 2018

12:15:11 <ohhithere> bcos_: that makes sense, but i still dont understand the part where why would the (software) thread that was HLTed gets cpu time again (e.g. the next instruction after HLT gets executed).
12:16:06 <zid> By it resuming.
12:16:15 <zid> The interrupt handler will ret to it.
12:16:33 <zid> interrupt handler doesn't care if you were in the midde of a hlt or an add
12:16:40 <zid> it's jumping back to you regardless
12:17:08 <zid> It can't even tell unless it goes out of its way to grab your instruction pointer off the stack, dereference it, disassemble the instruction there, and check if it was hlt
12:19:20 <ohhithere> but that doesnt make any sense, becuase there are hundreds(?) of other software threads that needs cpu time, the scheduler will take care of dividing the cpu to each and every thread unless it asked to sleep or for that matter wait for an event.
12:19:38 <zid> You just defeated your own argument
12:19:54 <zid> It didn't ask to sleep and is not waiting for an event, and the scheduler will guaruntee you cpu time
12:20:25 <bcos_> ohhithere: Normally threads are not allowed to use HLT - they tell scheduler "Hey, I need to wait" and the scheduler switches to a differen thread. It's only when the scheduler has no other threads to run that the scheduler (and nothing else) uses HLT
12:22:33 <ohhithere> so from the OS prespective this thread is not sleeping or waiting, thats why it will get executed? and thats the only reason this thread gets cpu time? and that cpu time will be quick because it will still HLT right? the next instruction won't get executed, correct?
12:23:06 <heat> HLT is the cpu equivalent of yield
12:23:13 <zid> halt is 'execute an infinite loop internally until an interrupt stops it' with fancy bells and whistles
12:23:27 <zid> (like power saving)
12:27:03 <zid> You're essentially saying for(;;); should never get cpu time, but not saying /why/ not
12:29:39 <ohhithere> i thought that the cpu's HLT instruction handler would just proceed to other tasks instead of wasting time HLTing, therefore, this hlting thread is "virtually" waiting forever.
12:29:57 <klange> The CPU has no idea what a task is (TSS aside)
12:30:14 <zid> it doesn't have any 'other tasks'
12:30:29 <zid> Its sole task is to dereference eip, execute what it gets, and increment eip
12:31:01 <zid> If you then design some software which responds to interrupts and changes eip to execute different blocks of code in a round robin.. that's on you
12:31:05 <zid> the cpu doesn't know you're doing that
12:31:29 <Shockk> if I understand it correctly, for(;;); would do a jmp on every instruction, forever, whereas hlt would cease instruction execution until the PIC raises an interrupt, right?
12:32:02 <zid> You can think of it just disabling the next instruction fetch
12:32:10 <zid> it gets wedged until the interrupt pin sets EIP to something else
12:32:12 <klange> HLT is "Do nothing, efficiently, until the next interrupt".
12:33:55 <ohhithere> i think i get it now the OS is still scheduling this thread, and everything it does is HLT untill it gets swapped out by the scheduler timer or an unmasked interrupts arrives, correct?
12:34:52 <bcos_> No
12:35:20 <bcos_> "everything it does is HLT"?
12:36:06 <zid> I should be checking my bootloader to kernel interface because I removed a parameter but.. eh..
12:36:11 <heat> well yes
12:36:15 <zid> it seems to work.. so it's fine right? :P
12:36:29 <heat> it does just stay HLTed until an external interrupt arrives
12:36:30 <bcos_> Normally (with CPL=3 and IOPL=0) if a thread tries to execute HLT it causes a general protection fualt; then the kernel stabs it with a rusty pitchfork for a while and wipes the executable off of the hard drive for being malware
12:37:08 <ohhithere> lol
12:37:34 <zid> go_long((u32)pml4, e->e_entry); wtf is an e_entry
12:38:05 <zid> oh, it's _start
12:38:35 <heat> its the elf entry point
12:38:38 <zid> yep!
12:41:05 * zid completely fails to understand his old assembly
12:41:20 <Shockk> I've got all my os assembly commented in columns 81-120
12:41:26 <zid> comment mine please
12:41:44 <heat> how bad can it be?
12:41:48 <zid> https://github.com/zid/boros/blob/master/boot/long.asm What is in esi at the start and why does it get or'd onto rax on 42
12:41:49 <bslsk05> ​github.com: boros/long.asm at master · zid/boros · GitHub
12:42:22 <Shockk> zid: this is what my comments are like https://github.com/shockkolate/shockk-os/blob/master/boot/bootsector.asm
12:42:24 <bslsk05> ​github.com: shockk-os/bootsector.asm at master · shockkolate/shockk-os · GitHub
12:42:32 <zid> okay so apparently main's proto is u64 free_page, whatever that was supposed to mean.. I think that can disappear
12:42:53 <zid> I should be passing through the new phys allocator garbage I think
12:43:30 <ohhithere> i get it now, if for some reason the next instruction after HLT ever gets executed it was because there was an unmasked interrupt in its cpu time slice.
12:43:31 <ohhithere> d
12:45:15 <ohhithere> thank you guys for the time & patience answering my questions :) bcos_ heat zid klange
12:46:07 <klange> on the topic of hlt in user code, I ran across a library that was all C++ except for one assembly call to hlt that made no damn sense
12:46:23 <zid> aha I think I have it, e_entry will be an offset, which is why I am adding it to virtual kernel base
12:46:33 <heat> intentional crash maybe?
12:46:43 <klange> no, and hlt does not crash on linux
12:46:48 <zid> xchg bx, bx for life
12:47:06 <heat> it doesn't crash?
12:47:09 <zid> Well you can tell they were mental because they wrote a library in C+
12:47:27 <ohhithere> lol
12:48:38 <klange> this function was clearly labeled as intending to yield
12:49:16 <klange> I was in the process of getting my company's software stack running on ARM. A few little snags here and there, but this one stupid line of assembly was the funniest.
12:50:03 <ohhithere> another question regarding this topic of scheduling and task-switching: whats the deal with MWAIT instruction when you want to get notified when a specific memory gets written to for example, at the time of the write, the MWAIT thread should wake up, by who? cpu, right?
12:50:51 <heat> yes
12:51:28 <ohhithere> so the CPU performs a context-switch?
12:51:44 <ohhithere> at the time of the write
12:52:13 <ohhithere> essentially blocking the thread that was writing?
01:02:29 <zid> bcos_: I've just realized something slightly ugly, I can't remove the 'next' element from the page I return from the physical allocator because it won't be mapped anywhere.
01:02:43 <zid> so it's going to have 4 bytes of garbage in it at the start
01:03:29 * zid checks if he definitely didn't set up his page tables to have a window into all physical memory
01:03:52 <heat> if you're on x86_64 you might as well
01:05:02 <zid> Doesn't seem like I did that
01:05:50 <zid> How disgusting would it be to keep a single 4k mapping around that zero_page(); uses to wipe pages
01:06:04 <zid> and I just swap which physical page it points to a lot
01:07:13 <heat> absolutely disgusting
01:07:16 <heat> just map it all
01:07:24 <zid> fiine :(
01:07:28 <zid> does bochs support 2G pages? :D
01:07:39 <heat> you mean 1G?
01:07:43 <zid> oh, 1G
01:07:46 <zid> 2M, 1G, right
01:07:49 <heat> No idea
01:07:59 <ohhithere> zid: out of curiosity, what are you developing with bochs?
01:08:05 <zid> an.. OS?
01:08:06 <heat> QEMU doesn't when it's not KVM afaik
01:08:34 <heat> I keep a page directory handy for when 1GB pages are not supported
01:08:54 <zid> I can just map it all with 2M pages then I guess
01:09:02 <heat> since I only bootstrap the first gigabyte at first
01:09:29 <ohhithere> zid: just like that? what for?
01:09:29 <zid> I'm only bootstrapping everything under 4G because it's all the bootloader can address
01:09:29 <heat> then when I'm able grab some memory from the memory map I just map 512GB
01:09:48 <zid> technically it might be less if there are holes, I only support 1 memory range atm :P
01:24:45 <klys> mm, energy was out for an hour or so
01:31:01 <zid> I mean.. I could just wipe things at mmap() time..
01:32:03 <zid> That seems lazier, done
01:32:42 <zid> I'm going to need a virtual mapping to grab the ethernet frames out the buffers (Which is mostly why I wrote this allocator) so I can just mmap it in and wipe it in mmap
01:35:31 <heat> wiping all the memory at boot time is madness
01:35:43 <zid> I am not
01:36:49 <zid> but my memory is in a free-list and has been dirtied by evil ->next pointers, so the choice is either that physical allocations are all dirty, I map all of physical memory so that I can arbitrarily set the ->next back to 0 to undirty them, or I only undirty things in mmap() where it is actually mapped
01:37:35 <heat> I zero memory if requested, right after the allocation
01:37:51 <heat> since it's just a simple memset PHYS_BASE + page away
01:38:01 <zid> yea I don't have a PHYS_BASE :P
01:43:30 <Shockk> hmm heat I still don't quite get the relationship between fread and read
01:43:58 <Shockk> er, fopen and open* I mean
01:44:03 <zid> fopen is the C interface
01:44:09 <zid> open is the linux interface
01:44:42 <heat> your FILE* is an opaque struct
01:44:48 <Shockk> right
01:44:58 <heat> basically you need it to describe a FILE handle + all the buffering
01:45:15 <klange> wait where did linux come from here
01:45:36 <Shockk> I wasn't going to correct it but yes it's the posix interface
01:45:51 <heat> if you stick to C11 you'll basically need to store the fd(if on posix, HANDLE in windows)
01:46:28 <heat> if you implement POSIX 2008 you'll need fmemopen which is adds a FILE interface for memory(which needs callbacks in order to be done)
01:46:41 <Shockk> heat: I might have misunderstood, but what I'm confused about is that you said earlier that the FILE shouldn't explicitly contain the fd
01:47:09 <zid> because yo can infer it, presumably
01:47:17 <Shockk> but in that case, if I'm calling open from fopen, how would I determine the fd corresponding to that FILE, in order to call open?
01:47:30 <heat> well, you can keep the fd in the FILE
01:47:53 <heat> you do need to know if it's a valid stream for fileno(maybe store -1 in there in case it's a fmemopen)
01:48:27 <heat> (also, you don't need any fds in order to call fopen(or open))
01:48:58 <Shockk> er oh I'm confusing the various functions
01:48:58 <Shockk> lol
01:49:01 <Shockk> sorry
01:49:01 <klange> i would like to my own libc for reference, but it's pretty bad, so I won't
01:49:16 <Shockk> in my head I'm thinking of the syntax for read, for some reason
01:49:21 <klange> link to*
01:49:30 <heat> sadly there's no fopenat
01:49:58 <klange> you could openat and fdopen in posix at least
01:50:41 <heat> yeah but the whole point of using FILE is that it's portable/easy to understand
01:50:51 <Shockk> oh right
01:51:00 <klange> well, and the buffering and fprintf/fscanf interfaces...
01:51:43 <klange> but i'll take your point, there should be an fopenat that takes a DIR*... you should propose it for whatever comes after C18 :)
01:53:34 <Shockk> hmm so let me just think out loud here, I know that there are many ways to design an OS,
01:54:05 <Shockk> so if I implement `open` by requesting an fd from the kernel, and fopen calls open to get that fd,
01:54:37 <heat> excuse me
01:54:43 <heat> but openat is the superior open
01:54:50 <Shockk> hmm
01:55:21 <Shockk> wait are you suggesting open would call openat?
01:55:28 <zid> that seems typical shockkk
01:55:32 <heat> that's how you do that these days
01:55:34 <Shockk> oh right
01:55:40 <Shockk> that kind of changes my perspective
01:55:41 <heat> open(...) = openat(AT_FDCWD, ...)
01:56:06 <Shockk> so the open fds are tracked by userspace in that case?
01:56:20 <heat> they're always tracked by the kernel
01:56:29 <Shockk> oh, hm
01:56:44 <heat> all the C library does is to keep a FILE struct with an fd
01:57:27 <Shockk> oh *wait*
01:57:36 <Shockk> I think I completely misunderstood what openat does
01:57:58 <Shockk> I was assuming from its prototype that it would open a file, into the specified fd
01:58:04 <heat> no
01:58:10 <Shockk> lol
01:58:13 <heat> that's open(at) + dup2
01:58:32 <klange> openat is unrelated to the problem at hand and heat is really only causing more confusion by talking about it :)
01:59:05 * heat needs to stop talking about the glorious openat
01:59:07 <heat> :)
01:59:17 <Shockk> is the first argument to openat just an fd to a directory?
01:59:21 <heat> yes
01:59:32 <Shockk> okay that makes infinitely more sense now
01:59:43 <heat> think of it as temporarily replacing the working directory just for that call
01:59:48 <Shockk> right
02:01:46 <Shockk> my knowledge of this stuff is lacking in so many areas but I really enjoy learning it
02:04:24 <Shockk> hm so heat, if I were to keep an array of FILE*, indexed by fd, am I right in thinking this should be in userspace?
02:04:36 <klange> you shouldn't do that
02:04:38 <Shockk> oh
02:05:28 <Shockk> why is that?
02:05:41 <klange> because it gets the association between the two backwards
02:06:55 <Shockk> hmm.. let me try and re-understand
02:08:43 <Shockk> correct me if I get anything wrong here
02:08:44 <Shockk> so my kernel should allocate an fd and return it to `open` (or `openat` or `open_klange` or whatever I'm using), and then that returns the fd back to `fopen`, which should allocate a FILE* and return it
02:08:49 <zid> if you're thinking about my silly trick, don't
02:10:23 <Shockk> so when the process is cleaning up for exit, its fds are closed/deallocated/whatever by the kernel
02:10:50 <Shockk> and would I be right in thinking that the memory allocated by FILE* will either be freed when calling fclose(), or at process exit?
02:11:39 <heat> yes
02:11:52 <Shockk> if this is the right thinking (correct me if it's not), then FILE* being opaque makes it quite nice, because it means you can't `close` a file descriptor that was opened with `fopen`
02:11:56 <Shockk> and vice versa
02:12:11 <heat> its not
02:12:21 <Shockk> not the right thinking?
02:12:28 <heat> in POSIX you can get the fd by using fileno()
02:12:38 <heat> not the right thinking
02:13:08 <Shockk> oh okay
02:13:50 <heat> fclose possibly can fail if you close the fd and then do fclose()
02:13:53 <Shockk> so I could fopen() then fileno(), and then close() that fileno which would leave the FILE* hanging around until the end of the process? or, what would hap-
02:13:54 <Shockk> ah right
02:13:59 <Shockk> that's exactly what I was about to ask
02:14:40 <heat> in any case, you just free the file struct
02:14:59 <heat> the stream is specified by POSIX as undefined
02:15:07 <heat> so it might as well not exist
02:16:00 <heat> note that you need to keep track of which streams are open
02:16:18 <heat> since fflush(NULL) flushes every open output stream
02:16:26 <Shockk> ah yes I remember reading that
02:16:31 <heat> (and I believe that you need to do it on program exit)
02:16:33 <Shockk> that sounds fun, I didn't realize that was possible
02:16:36 <Shockk> oh interesting
02:16:42 <Shockk> that I didn't know
02:16:53 <zid> You might want to read the C spec for these functions
02:16:59 <zid> and the POSIX spec, they're different
02:17:13 <zid> C requires you to seek between reads and writes, but posix does not, being the main one I can think of
02:17:30 <Shockk> oh, huh
02:17:55 <heat> generally you just want to comply to POSIX
02:18:05 <zid> I bit myself with that once writing a program on linux and testing it there, and not having it work on windows
02:18:19 <klys> would you do this on a regular linux system? fclose( fileno( f ) ); free( f ); // ???
02:18:33 <heat> that doesn't work
02:18:40 <zid> Shockk: Are you implementing the libc or the kernel or both?
02:18:41 <heat> fclose doesn't work with ints
02:18:48 <Shockk> wouldn't that just be close, not fclose @ klys?
02:18:50 <heat> also, you're not freeing everything
02:18:57 <klys> oh right
02:19:00 <heat> and you're not flushing anything as well
02:19:03 <Shockk> zid: yes
02:19:08 <zid> ..thanks
02:19:09 <klys> would you do this on a regular linux system? close( fileno( f ) ); free( f ); // ???
02:19:12 <Shockk> zid: :D
02:19:16 <Shockk> yes as in all of the above
02:19:20 <heat> klys, see above
02:19:27 <zid> okay then, I'd just do them entirely seperately then and stop thinking about how they interact :P
02:19:30 <heat> it's possible but stupid
02:19:51 <zid> Unless you're doing your own syscalls and not stealing linux's or whatever
02:20:03 <Shockk> I'm doing my own
02:20:26 <heat> and I don't believe fopen guarantees that fopen returns a malloc'd pointer
02:20:26 <zid> ah okay and you've not written either side before?
02:20:52 <zid> a FILE * could quite happily be an integer cast to a pointer and just directly represent an fd
02:20:57 <zid> if your OS has the same semantics as C/posix
02:21:07 <zid> malloc isn't going to be happy there
02:21:21 <heat> except it would be missing lots of stdio
02:21:51 <klange> the key parts of stdio are the buffering that happens within userspace using potentially user-supplied buffers
02:21:55 <Shockk> FILE needs to keep track of buffer and position though as well, right?
02:22:23 <bcos_> Potentially, yes
02:22:30 <heat> note that even though FILE is opaque, it doesn't stop people from trying to peek there
02:22:42 <heat> so some programs break when the libc isn't glibc
02:22:54 <Shockk> ..that sounds very fun
02:22:58 <Shockk> I won't be porting anything that does that
02:23:12 <heat> https://git.musl-libc.org/cgit/musl/tree/src/internal/stdio_impl.h#n21
02:23:13 <bslsk05> ​git.musl-libc.org: stdio_impl.h\internal\src - musl - musl - an implementation of the standard library for Linux-based systems
02:23:30 <heat> that's musl's glibc compatible FILE struct
02:25:30 <Shockk> just saw what you said about not guaranteed to return a malloc'd pointer
02:25:44 <Shockk> I was going to point that out, I don't see anything like that in the spec
02:25:56 <Shockk> so freeing it may be undefined behaviour
02:26:04 <heat> it obviously is
02:26:30 <Shockk> oh, well, I meant it will be UB if it's not a malloc'd pointer, but I guess that *is* UB in itself
02:27:13 <Shockk> I'm really tired and should go, thanks for all the advice and answers and stuff
02:27:23 <zid> did you write my free() yet
02:27:29 <Shockk> yes
02:27:29 <heat> yea I need to go too
02:27:47 * zid waiting for oven to heat up for his cheapo pizza
02:28:18 <Shockk> free() { __asm__ ("hlt"); }
02:28:23 <Shockk> perfect
02:29:12 <Shockk> if you measure timing, it'll look like it's actually doing something, when it's actually just waiting for an interrupt
02:32:23 <klange> Shockk: honestly... i suggest you just go read some implementations and see how they work; glibc, musl, some bsd libcs, sortix's libc...
02:32:29 <klange> don't read mine though
02:32:32 <klange> it's garbage
02:33:28 <zid> or mine, it doesn't exist
03:14:42 <zid> nice, I hit a not very solveable catch-22, I knew one would show up eventually
03:15:17 <zid> bcos_: My bitmap has no virtual address and if I give 4k pages away to the mmap code for page table entries it'll want to write to the bitmap :)
03:18:48 <zid> I guess there's a ghetto solution of an entirely custom codepath to map the bitmap in which rescans the page tables after it has done to bitmap mark the pages it uses
03:18:55 <zid> that doesn't seem ideal though
06:22:45 <mobile_c> is there any way to force an atexit to be called AFTER all other atexits have been called
06:23:30 <klys> are you privy to the list of them?
06:23:48 <mobile_c> ?
06:24:01 <bcos_> Aren't they done in reverse order?
06:24:13 <bcos_> e.g. whatever gets added first is called last
06:24:36 <mobile_c> bcos_: apparently not o.o
06:25:34 <bcos_> http://www.cplusplus.com/reference/cstdlib/atexit/
06:25:34 <bslsk05> ​www.cplusplus.com: atexit - C++ Reference
06:25:47 <bcos_> "If more than one atexit function has been specified by different calls to this function, they are all executed in reverse order as a stack (i.e. the last function specified is the first to be executed at exit)."
06:26:08 <zid> bcos_: I said a thing earlier, idk if you saw it or not
06:27:09 <bcos_> zid: Just preallocate a few "reserved for mmapping bitmap only" pages
06:27:42 <zid> bcos_: And put them where, take them from where, etc
06:27:47 <zid> It's probably less code to do it my dumb way
06:28:12 * zid doesn't like weird exceptions
06:28:30 <bcos_> Could put them in .bss; like "uint64_t sparePage1, sparePage2;"
06:28:49 <zid> it's going to scale with the size of the bitmap though
06:28:57 <zid> which scales with total memory installed
06:29:02 <mobile_c> well i have my garbage collector as executing its atexit first but somehow its atexit is caled before the other atexits making the atexits crash cus the garbage collector is shutdown
06:29:10 <zid> Although I guess technically it caps at 4GB with my current code
06:29:52 <zid> so like.. 2054 pages is the max or something? heh
06:30:03 <zid> oh forgot to divide by 8 I guess, so like 50 idk
06:30:13 <bcos_> zid: Your bitmap covers all physical memory (not just a 512 bits when a 2MiB page gets split up)?
06:30:45 <zid> currently it covers the region the e820 had the kernel in
06:30:45 <bcos_> In that case, probably easier to preallocate and map the whole thing
06:31:02 <zid> I plan to do all regions not just that one, but my current layout has no holes
06:31:05 <zid> so I just went the easy route
06:31:26 <zid> but if I pre-allocated a bitmap for 4GB of ram that seems excessive if you only have 64MB
06:31:57 <bcos_> You can't scan through E820 and find out max. physical address first?
06:32:25 <zid> I suppose, but if I am goign to all that trouble I think I might as well just have the bootloader pre-map the bitmap properly
06:32:44 <zid> it has some pagetable code and is under an identity map so it's easy to write at that point
06:33:19 <zid> I could just pick a random address out of a hat, or stick it after the kernel or something
06:33:42 <zid> It could have an entire page directory to itself or something I'm only using 3 so far
06:34:01 <zid> I might just do that, and I could extend it to 'all available memory' later on just fine from there
06:34:25 <zid> so it's just pml4->e[PHYS_BITMAP] or whatever
06:35:38 <zid> atm I just have a recursive in the last slot, kernel in the middle slot, and one 'reserved' below half way for userspace 'at some point, I could just give it slot 509, map it in the bootloader and call it done, I wouldn't even need to give the kernel the address of the unsigned char *bitmap then
06:36:02 <zid> It can just add a constant address instead
06:37:02 <zid> Okay this seems workable unless you can think of a reason it won't work
06:39:22 <mobile_c> i ded a test and they ARE called in the order they are specified
06:39:29 <mobile_c> i did a test and they ARE called in the order they are specified
06:39:45 <mobile_c> atexit(a); atexit(b); // calls a then b
06:40:13 <mobile_c> thst isnt suposed to happen right?
06:40:24 <mobile_c> its supposed to call b then a right?
06:42:02 <Mutabah> According to the C99 standard I have open, yes.
06:42:17 <mobile_c> ;-;
06:42:37 <Mutabah> That said, why are you wanting this defined order?
06:42:50 <Mutabah> You mentioned a GC, does that GC's cleanup have side-effects?
06:43:16 <mobile_c> cus i need to ensure that my GC is ALWAYS ran LAST including after all atexits
06:44:12 <mobile_c> what if i call an atexit inside an atexit?
06:44:34 <bcos_> So that your GC can free memory slowly and inefficiently just before kernel destroys the virtual address space (and frees memory fast an efficiently)?
06:44:43 <klange> https://i.imgur.com/nnt7OC5.png *shrug*
06:45:08 <mobile_c> fk it doesnt work ;-;
06:45:49 <mobile_c> registering an atexit inside of an atexit doesnt work ;-;
06:46:01 <klange> yeah that sounds like a bad idea
06:46:30 <mobile_c> bcos_: cus ios has no garbage collection
06:47:31 <bcos_> But..
06:47:32 <mobile_c> so when a program exits its memory IS NOT freed and thus crashes the app
06:47:49 <bcos_> WHen not just not bother calling GC immediately before kernel frees everything anyway?
06:47:49 <klange> I am pretty sure you are wrong about something here.
06:47:50 <Mutabah> ... what?
06:48:11 <Mutabah> Or apple have a strange policy in place to prevent memory leaks
06:48:49 <mobile_c> bcos_: cus the kernel DOES NOT free the memory
06:49:12 <bcos_> Program exits and ..crashes the app? Is this like, one or more programs running inside of a program?
06:49:16 <graphitemaster> I can assure you that on iOS when an app closes it's memory does get freed, the problem is on iOS apps are never closed unless force-stopped
06:50:03 <bcos_> Erm, that sounds silly, but that would also mean "exit()" wouldn't be called either
06:50:07 <klange> That's not strictly true, but you don't have much control over when you get "actually closed".
06:50:17 <mobile_c> idfk if the kernel itself actually manages the compilers memory or not
06:50:42 <graphitemaster> klange, oh you do, you can make apps that actually terminate, it's just against the apple guidelines
06:50:53 <klange> I'm well aware.
06:51:25 <bcos_> Heh - sounds like a unique solution to the halting problem..
06:51:28 <graphitemaster> anyways atexit handlers are stupid
06:51:38 <bcos_> "thou shalt never terminate!"
06:51:48 <mobile_c> lol
06:53:11 <graphitemaster> atexit handlers are for doing work that would otherwise not be done even in the event of an exit, so memory management inside an atexit is a bug, closing a FILE in an atexit is a bug (because exit is required close all open stdio streams), closing a socket is a bug (the kernel handles most of that too), the only thing you should be really focused about is things the OS can't do for you or the libc doesn't already do for you
06:53:31 <mobile_c> this is my situation: app compiles program> executes program> does whatever to get back to where it was before the app got compiled> nothing from the execution is cleaned up
06:53:48 <mobile_c> before the program got compiled*
06:53:51 <Mutabah> Wait, your app is like VM host?
06:55:12 <graphitemaster> when your app executes a program, is your program sandboxing and doing the executing or is it just asking the OS to execute some binary?
06:55:21 <graphitemaster> for the latter it doesn't matter, the OS will clean it up
06:55:39 <graphitemaster> for the former, yeah all memory management is on you
06:56:02 <mobile_c> the problem is, because the app itself does not clean up any memory the compiled/ran programs always leak whatever memory that is malloced but has not been freed by the user, wich leads to, depending on how big the leak, crashing instantly after exit or eventually crashing after executing say 96 times
06:56:13 <bcos_> for the former, the guest program exiting has nothing to do with the host terminating
06:56:19 <mobile_c> so the*
06:56:56 <Mutabah> Sounds like you need to properly sandbox the child program
06:57:14 <graphitemaster> no, it sounds like he doesn't know what he is doing and he's chasing a symptom of some other problem
06:57:20 <Mutabah> Either in its own process, or with some sort of tracing allocator
06:57:23 <bcos_> What is the child program written in?
06:57:24 <graphitemaster> because a child program terminating correctly cannot leak memory
06:58:27 <mobile_c> i dont own the app so i dont have access to the apps source code
06:58:44 <zid> Unless on windows :P
06:58:53 <mobile_c> so i cannot fix the app itself so it correctly manages the memory of executed programs
06:58:56 <bcos_> So it's some kind of java byte-code, or .NET/CIL, or..?
06:59:18 <graphitemaster> pretty sure this has nothing to do with osdev anyways
06:59:45 <zid> I ignored him a week ago in a different channel, he's obviously got issues
07:00:10 <zid> Trying to optimize his tinfoil hat so he can speak to dragons type
07:00:19 <mobile_c> lol
07:00:48 * bcos_ is thinking it's something like "jave byte-code calls exit()" so java virtual machine does "exit()" when it shouldn't
07:01:18 <graphitemaster> the problem with the question is it's not even formed like a question, it's mixing terminology, it shows a severe lack of understanding of how operating systems execute programs and the pipeline to get from compiled code to binary and execute the program
07:02:08 <graphitemaster> there's so many things wrong everywhere in between that you can't even tell
07:02:22 <mobile_c> anyway, could it be possible to implement atexit itself so when it is called, will execute all registered functions in reverse order as per the atexit man page
07:02:46 <mobile_c> instead of executing them in the order they are registered
07:03:01 <graphitemaster> yeah, implement your own atexit, then register it with atexit
07:03:09 <mobile_c> ok
07:03:19 <graphitemaster> only have one atexit that calls your atexit with your own stuff
07:03:52 <bcos_> ..and call it "at_atexit_exit()" :-)
07:04:00 <mobile_c> should be easy enough to implement
07:04:15 <klange> write your own os
07:04:20 <klange> so you can solve all these problems
07:05:13 <mobile_c> except it needs to be executed in reverse, so i dont think i can use a linked list for that
07:05:23 <klange> why not
07:06:03 <graphitemaster> void (*aeh[32])(); int aei = 0; void my_atexit(void (*h)()) { aeh[aei++] = h; } void aehw() { do { aeh[aei--](); } while(aei); } atexit(aehw);
07:06:35 <graphitemaster> C standard only requires you register up to 32 at exit handlers anyways
07:06:38 <mobile_c> cus it will be dequed in the order it is qued
07:06:40 <graphitemaster> anymore is implementation defined
07:06:53 <graphitemaster> this should cover you, no linked list needed
07:07:11 <graphitemaster> just an array of function pointers and bump up to add bump in reverse to call them
07:07:50 <graphitemaster> aeh = at exit handler, aei = at exit index, aehw = at exit handler wrapper
07:08:03 <graphitemaster> like my terse naming scheme? I do lel
07:08:45 <mobile_c> graphitemaster: ill try that...
07:12:24 <graphitemaster> e.g https://ideone.com/k4ftau
07:12:24 <bslsk05> ​ideone.com: Ideone.com - k4ftau - Online C Compiler & Debugging Tool
07:12:40 <graphitemaster> if you're smart you'll use a lock around it
07:12:46 <klange> locks are for squares
07:14:13 <klange> (should also make sure you avoid the obvious buffer overflow if too many calls to atexit are, and an atexit implementation should return `int`, 0 for success, non-zero for failure)
07:17:57 <mobile_c> i get Null pointer access
07:18:06 <mobile_c> at aeh[aei--]();
07:20:46 <mobile_c> http://rextester.com/PJDVQ53497
07:20:47 <bslsk05> ​rextester.com: Atexith, C (gcc) - rextester
07:27:50 * klange quietly sneaks atexit() into toaru-nih's libc
07:30:30 * zid quietly sneaks a kipper into klange's shoes
07:31:04 <klange> Smoke me a kipper, I'll be back for breakfast.
07:31:25 <graphitemaster> mobile_c, try sometimes https://ideone.com/qlnxMT
07:31:26 <bslsk05> ​ideone.com: Ideone.com - qlnxMT - Online C Compiler & Debugging Tool
07:31:44 <graphitemaster> this one supports the atexit argument and also is thread safe
07:31:54 <graphitemaster> and handles the error when there is too many registered
07:33:19 <zid> thread safe? showoff
07:34:00 <graphitemaster> it's kind of tricky to be thread safe here
07:34:10 <graphitemaster> can't rely on the count
07:34:16 <graphitemaster> for the for loop
07:34:25 <graphitemaster> since it can change when one of the handlers registers with atexit
11:06:33 <ohhithere> j
11:44:10 <mischief> has anyone read this? http://pages.cs.wisc.edu/~remzi/OSTEP/
11:44:11 <bslsk05> ​pages.cs.wisc.edu: Operating Systems: Three Easy Pieces
12:46:25 <n3t> mischief: I've read a few chapters. IMO it's quite good and doesn't seem hard.
01:36:06 <klange> *sigh* what should I work on next... https://i.imgur.com/U0pV9C5.png
01:50:24 <mischief> plan 9 compat
01:50:49 <klange> no, go away
01:50:55 <mischief> .theo
01:50:55 <glenda> That is not true.
01:50:55 * klange pushes mischief out the door
01:52:58 <winsoff> Is the plan 9 os book still a good read?
01:53:59 <winsoff> Is it even a book?
01:54:36 <mischief> what even is a book
01:55:22 <klange> there's a book?
02:13:55 <klys> screen/tmux
02:15:06 <klys> or css
02:24:32 <klange> I'm assuming that was an answer to my earlier not-actually-a-question "what should I work on next..." in which case... 1) not really caring about ports at the moment, and 2) css would have been a great thing to work on... in my last project... a year ago... but I don't even have an HTTP parser at the moment, so, no.
02:24:55 <klange> and I'd rather write my own multiplexer screen/tmux replacement than port of them anyway
02:26:06 <klys> myeah the world needs a new terminal/tty multiplexer
02:26:36 <klange> the world totally needs a new mediocre one-man linux clone with a full suite of its own replacement software
02:26:39 <klange> so fuck you
02:27:27 <klys> toaru's not gnunix
02:28:10 <klange> I'm not in the hobby of making useful software. That's what I do at work.
02:28:48 <klys> how about a three-paned or multi-paned file browser
02:58:54 <sham1> A GUI Norton Commander clone
03:00:36 <klys> looks like norton commander is only a two-paned solution.
03:08:28 <asymptotically> klange: what useful software do you make at work?
05:23:33 <rakesh4545> hello!
05:26:45 <rakesh4545> I want to learn how to link two object files with gnu linker. Do you know some tutorial?
05:30:26 <mischief> try: man ld
05:31:36 <zid> If you think you're ready for osdev when that is your question you're asking, I have very bad news
05:35:14 <rakesh4545> I am sorry. I was trying to link a bootloader written in assembly and a c kernel object files. The assembly must jump to say 0x8000 where the c kernel should be loaded. But for some reason the pmemsave in qemu shows series of 0x00000 at 0x8000 mem addr.
05:35:47 <zid> who is supposed to have loaded it to 8000?
05:37:36 <rakesh4545> ld -o kernel.bin -Ttext 0x8000 kernel.o --oformat binary I have tried with this command
05:37:46 <zid> who is supposed to have loaded it to 8000?
05:38:01 <rakesh4545> linker loader
05:38:15 <zid> the linker on your computer is supposed to change the memory of another computer?
05:39:09 <rakesh4545> I thought it is equivalent to [org 0x8000] ? I don't know.
05:39:18 <zid> I didn't ask where the kernel *expects* to be in memory
05:39:22 <zid> I asked who *PUT* it there in memory
05:40:35 <rakesh4545> linker I think.
05:40:38 <zid> what linker
05:40:43 <zid> you wrote a linker?
05:41:06 <rakesh4545> gnu's linker
05:41:12 <zid> I turn the computer on, your bootloader runs
05:41:34 <rakesh4545> yes.
05:41:37 <zid> then another magic copy of linux and gnu ld gets ran in the background, copies the kernel to 0x8000, then your bootloader jumps to 0x8000
05:41:39 <zid> that's what you're telling me
05:42:41 <rakesh4545> oh i forgot to mention. I am concatenating the two binary files as mentioned in https://stackoverflow.com/questions/27051471/call-c-kernel-from-assembly-bootloader
05:42:43 <bslsk05> ​stackoverflow.com: Call C kernel from assembly bootloader - Stack Overflow
05:42:59 <zid> How are you booting, bios? grub? uefi?
05:43:18 <zid> I am assuming bios
05:43:32 <rakesh4545> yes
05:43:45 <zid> okay then so how does bios booting work, in your own words
05:43:56 <zid> how does it know it's a bootloader, what does it do with it
05:45:01 <rakesh4545> bios looks for 0xaa55 in first sector and then loads it in the memory at 0x7c00
05:45:08 <zid> how long is one sector
05:45:19 <rakesh4545> 512 bytes
05:45:26 <zid> so what is 7c00 + 512 bytes
05:45:32 <sortie> 'long' depends on whether it's a 32-bit or 64-bit system
05:46:10 <zid> sortie: if you're windows it does :P
05:46:17 <zid> or rather, doesn't
05:46:40 <zid> addition has stumped him apparently
05:47:53 <rakesh4545> 7c00 + 5x16^2 +1x16^1+ 2x16^0 am i wrong? I dont have pen or paper to calculate that
05:48:01 <zid> calculate? lol
05:48:04 <zid> calc.exe
05:48:13 <zid> 7E00
05:48:21 <rakesh4545> ok
05:48:25 <zid> 256 is 0x100, 4096 is 0x1000
05:48:40 <rakesh4545> nice
05:48:44 <zid> Okay so the bios copies data to 7C00-7DFF
05:48:57 <zid> So the question remains, *who loaded the kernel to 0x8000?*
05:50:02 <rakesh4545> I wonder who!
05:50:15 <zid> So why are you surprised there's no kernel there?
05:50:26 <zid> bios didn't do it, you didn't do it, 1+1 = HRR I AM LOST
05:51:06 <rakesh4545> So how did this person https://stackoverflow.com/questions/27051471/call-c-kernel-from-assembly-bootloader here have the kernel?
05:51:16 <zid> top person didn't, 2nd person wrote code to do it
05:51:20 <zid> but you didn't even bother to read it
05:51:27 <zid> he literally wrote a function called load_kernel
05:55:16 <rakesh4545> his load_kernel is actually jumping at KERNEL_ADDRESS defaulting to 0x100000
05:55:47 <zid> It is not.
05:56:13 <zid> It contains precisely 0 jumps
05:57:22 <rakesh4545> its jmp KERNEL_ADDRESS right there and then he is using ld -Ttext 0x100000 -o kernel.elf loader.o main.o command
05:57:38 <zid> Okay just lie to me, idk
05:57:47 <zid> I'll just stop talking to you, easily done
05:57:52 <rakesh4545> ok sorry forgive me
05:57:56 <rakesh4545> please
05:58:09 <rakesh4545> please please!!
05:58:40 <rakesh4545> don't stop talking with me.
06:00:38 <rakesh4545> I want to create my own bootloader and learn and not use GRUB.
06:28:06 <rakesh4545> How should I load my C kernel from assembly bootloader? I have already set up the GDT and wants to call my C kernel?
06:36:09 <dhoelzer> Have you loaded the kernel into ram?
06:36:21 <dhoelzer> If so, jmp to it
06:37:57 <geist> step 1, put your kernel in the box
06:38:22 <geist> well, maybe step 0, cut a hole in the box
06:59:37 <graphitemaster> geist, if you put your kernel in a bag, you can pop it in a microwave
06:59:44 <sham1> Don't they teach how to call C functions from assembly anymore
06:59:46 * sham1 shms
06:59:50 <sham1> smhs*
07:03:58 <sham1> What has the world come to
07:04:17 <glauxosdever> We had C classes. The amount of (important) things that were left untouched is astonishing
07:04:34 <graphitemaster> to be fair, in C you don't actually interface with assembly
07:04:38 <graphitemaster> like ever
07:04:45 <graphitemaster> only in OSDEV really
07:04:51 <graphitemaster> and libc for system calls
07:04:57 <glauxosdever> And embedded programming too
07:05:07 <graphitemaster> that's less true these days
07:05:09 <sham1> Well if one does osdev they probably should have some experience
07:05:21 <sham1> And ability to search for themselves
07:05:51 <glauxosdever> Yes, but some things are basic. Otherwise don't claim you teach C
07:06:14 <glauxosdever> (Maybe not the interfacing with assembly part, but we didn't even do malloc/free)
07:06:35 <sham1> That's just dumb
07:06:51 <sham1> Dynamic allocation is one of those things you just do in most programming tasks most likely
07:07:00 <glauxosdever> We didn't even do structs in the lab. Go figure
07:07:23 <sham1> What kind of a C class is that
07:07:32 <sham1> How does one even C without structs
07:07:37 <sham1> How about arrays?
07:07:39 <graphitemaster> https://i.redd.it/97fovyhq1fm11.png
07:07:50 <glauxosdever> And there was a data structure that called itself for structs. No, we had to do multiple data structures and then make a two-dimensional array of pointers. Excessive casting was necessary..
07:08:16 <Shockk> I'm currently trying to cross-compile dash for my OS and I feel like it's doing something weird and wondered if someone else has had the following issue:
07:08:23 <glauxosdever> graphitemaster: lolwat
07:08:27 <sham1> I'm positive that void isn't a positive value. Or negative
07:09:05 <graphitemaster> https://processing.org/reference/mouseWheel_.html
07:09:06 <bslsk05> ​processing.org: mouseWheel() \ Language (API) \ Processing 3+
07:09:15 <Shockk> it builds mknodes.c using CC_FOR_BUILD (which I set to i386-shk-gcc), but then it attempts to execute it, it seems, and obviously this results in
07:09:16 <Shockk> ./mknodes: ./mknodes: cannot execute binary file
07:13:40 <glauxosdever> sham1: Anyway, we had just 6 or so labs, one of the instructors didn't know that hexadecimal are up to F (he thought they are up to E) and suggested to always use & with scanf(), even with %s. The other didn't know how to configure DevC++ so gcc is in -std=c99 or -std=c11 mode.
07:13:50 <glauxosdever> It's crazy.
07:14:27 <zid> glauxosdever: We were supposed to use borland with conio but there was a dev-c++ on the shared directory
07:14:29 <zid> so I did everything in C90
07:14:41 <zid> I made 3D graphics within gdi calls :p
07:14:46 <zid> they were doing scanf, badly
07:14:53 <glauxosdever> Uh, that's even worse..
07:15:07 <zid> 99% it was just for getch or whatever that conio thing is
07:20:29 <glauxosdever> Out of curiosity, what are your interesting uni stories?
07:21:46 <zid> I got arrested
07:23:06 <glauxosdever> ?
07:23:15 <glauxosdever> What did you do?
07:23:24 <glauxosdever> :o
07:23:40 <zid> Punched an emo fuck talking shit at me in the face
07:23:46 <zid> his jewellery cut his face to shit
07:24:00 <glauxosdever> lol
07:25:03 <glauxosdever> Actually, not lol, but jewellery is stupid
07:25:40 <zid> he shouldn't have been chatting shit and saying he was going to have me killed :p
07:26:32 <zid> They took a statement then released me, I guess he didn't want to have to admit he was chatting shit
07:27:37 <glauxosdever> Another story, teaching-related?
07:31:16 <Shockk> ah I fixed my issue, I had to set CC, not CC_FOR_BUILD
07:31:43 <glauxosdever> I had a java lab instructor that told us to ignore warnings
07:32:11 <zid> Might have been a "I just need to get them to infinite monkey a passing text file, that's all"
07:33:38 <glauxosdever> Not sure, probably didn't want us to think more about "irrelevant" details or having to explain them
07:33:43 <zid> exactly
07:34:01 <zid> no time to actually teach an entire language, how to program, etc
07:34:05 <zid> just need to get them through the module
07:35:56 <glauxosdever> And that's the fundamental issue of universities. Durations of modules are discrete (at semester intervals), which causes either rounding down of time (less time for details) or rounding up of time (time that could be spent on other things is spent on solving additional exercices)
07:36:26 <zid> I mean ideally they'd just be a good learning environment and you'd be learning for your own benefit
07:36:38 <zid> but too much money and 'degrees' etc is involved
07:37:13 <zid> so if it took you 4 months to get good enough at java for the final exam rather than 2, but it only took you 2 months rather than 4 for something else, that should be fine
07:37:23 <zid> and you can just show up to more than the normal amount of java labs
07:37:50 <zid> gotta railroad people through it instead so you can churn through more students and make more money etc
07:38:14 <glauxosdever> Universities here are public
07:38:25 <glauxosdever> Most of them at least
07:38:38 <glauxosdever> They are operated (indirectly) by the state
07:38:42 <zid> And they don't teach to the test otherwise they lose their grants etc?
07:38:52 <zid> that's how highschools work here, sadly
07:39:10 <zid> you'll get shut down if suddenly you have a 30% pass rate because you let people just wander between lectures learning what they feel like :P
07:39:54 <glauxosdever> Well, pass rates here are probably less than 30%. Some students need more than 10 years to finish
07:40:31 <zid> Doing that at cambridge or oxford etc would cost a fortune here, even as a non-foreign student
07:40:47 <zid> our right wing government keeps getting rid of things like free education and stuff
07:42:17 <glauxosdever> We have a "leftist" government, but universities have been free since 1970 or something
07:43:01 <glauxosdever> Nope, more like 1960. My dad, who is fairly old, had to pay for his first year of studies, but then universities became free
07:44:16 <bcos_> Literally free?
07:44:36 <glauxosdever> No, there are exams, but you pay nothing. Not even books
07:44:48 <glauxosdever> (You of course pay indirectly through taxes)
07:45:10 <bcos_> here it's mostly just "low interest government loan for students"
07:45:44 <zid> yea you can default on them here and you only pay them back as a small percentage of your income etc, they're not too bad
07:45:45 <glauxosdever> I think that's how UK does it too, am I right?
07:45:58 <zid> but it'd still be pretty expensive, it used to be capped at £3000/year
07:46:00 <zid> nwo it isn't
07:46:11 <glauxosdever> I mean, what bcos_ siad
07:46:19 <zid> I'm literally explaining how it works :P
07:46:31 <glauxosdever> Oh, right
07:47:04 <bcos_> Similar here (Australia) - you only have to pay the loan back if/when you earn more than a specific $/year amount
07:47:05 <zid> I think they expire after a couple of decades and stuff, way less of a profiteering mess than the US
07:47:41 <bcos_> ..and then it's a pecentage of income
07:47:52 <bcos_> Not sure about expiry or death
07:48:03 <zid> In the US you can't even default on them
07:48:10 <zid> if you go bankrupt, you're still in student loan debt
07:48:24 <zid> and they have interest
07:48:41 <zid> So if you fail out of college because of a health issue you should probably just kill yourself ;P
07:48:57 <bcos_> US is relatively borked for lots of things (anti citizen, pro company)
07:49:12 <zid> yea, we're trying as hard as we can to follow them atm
07:49:30 <zid> leave eu, deregulate everything, sell off services, etc
07:50:05 <glauxosdever> The first part may actually be good. The other two (and etc) not though
07:50:20 <zid> it will definitely not me
07:50:21 <zid> be*
07:50:23 <CompanionCube> glauxosdever: the problem with the first part is that it requires a competent government.
07:50:38 <zid> we're losing years of growth per year atm
07:50:40 <CompanionCube> if you don't, everything just goes to hell
07:50:40 * bcos_ thinks it should be illegal for politicians to hire/use bodyguards
07:50:53 <CompanionCube> it's a delayed form of doom, but it happens
07:51:10 <zid> We *could* leave the eu, if we had like 40 years to plan it
07:51:19 <CompanionCube> zid: or hell
07:51:25 <bcos_> zid: I lost track - who is "we"?
07:51:30 <zid> UK
07:51:33 <CompanionCube> it'd be good if there was agreement on what brexit even is
07:51:44 <zid> CompanionCube: But then nobody would want it :P
07:51:47 <bcos_> Didn't UK already leave EU?
07:51:49 <zid> so they can't do that
07:51:49 <zid> no
07:51:56 <sham1> Not yet
07:52:00 <zid> we sent a memo to the EU to start a 2 year timer on us leaving
07:52:01 <sham1> They're WIP
07:52:06 <bauen1> ^
07:52:10 <bcos_> Heh
07:52:14 <bauen1> brexit was popcorn-worthy
07:52:15 <zid> which can be retracted at any time but nobody has the balls to do it
07:52:21 <zid> because it'd cost a couple of politicians their jobs
07:52:44 <sham1> I'm starting to feel like you lot over yonder don't actually want to do this
07:52:45 <bcos_> Ah - that takes us back to "should be illegal for politicians to hire/use bodyguards"
07:53:15 <zid> sham1: Same as anywhere else, the republicans/right-wing/conservative temporarilly embrassed millionaires think they want it
07:53:17 <CompanionCube> sham1: some do
07:53:19 <zid> everybody else doesn't
07:53:31 <CompanionCube> mostly delusional people and greedy capitalist bastards.
07:53:44 <zid> the only reason it even passed is because a lot of people were using it as a protest vote against our shitty government
07:53:48 <sham1> "greedy capitalist bastards"
07:53:50 <zid> but all they'll get is a permanent version of that government
07:53:50 <sham1> Ooh
07:54:20 <zid> The only thing keeping our banks regulated, worker labor laws in-place and shit is the EU at this point
07:54:45 <CompanionCube> sham1: got a better description for mogg and ERG friends?
07:54:51 <zid> And once we've left we'll be having to compete with other glorious non-EU nations for EU trade like malaysia, and have to have similar labor costs to them to compete
07:54:53 <zid> it's going to be fun
07:55:14 <sham1> CompanionCube: only thing I can find for Mogg is a Swedish hard rock band
07:55:18 <sham1> Probably not what you meant
07:55:19 <glauxosdever> Yeah, you need a competent government
07:55:53 <bcos_> I don't think Amazon sells compentent governments - where do you get them?
07:55:57 <zid> sham1: jacob reese-mogg or something is his name
07:56:11 <glauxosdever> bcos_: lol
07:56:15 <CompanionCube> zid: yeppers
07:56:17 <zid> upper-class twat born into his job who just wants to sell the UK off and profit
07:56:33 <CompanionCube> him and his 'european research group' buddies
07:56:45 <Shockk> does anyone know what sys/param.h comes from? i.e. what standard? it doesn't appear in POSIX as far as I can see, and I've been unable to find the information on google
07:57:16 <zid> background detail: The only reason the vote even happened is because the right-wing party promised the racist party they'd absorb their agenda of having one if they gave them their votes
07:57:21 <zid> so that the left wing government couldn't get in
07:57:41 <sham1> Wait, the UKIP is racist?
07:57:43 <zid> Shockk: never heard of it
07:57:52 <zid> sham1: I hope that's sarcasm
07:58:32 <CompanionCube> UKIP very well fit the 'not-racist-buuuut' descriptor.
07:58:41 <zid> but all their members are :P
07:58:50 <zid> and several of their counceilsiors
07:58:57 <zid> and most of their twitter posts
07:59:01 <zid> other than that, totally not racists
07:59:02 <bcos_> Shockk: I think that might be a "linux specific not posix at all" header
07:59:04 <sham1> I really don't follow British politics to know anything about them except that they were very much in favour of this whole thing
07:59:26 <CompanionCube> it's their raison d'etre
07:59:39 <zid> sham1: basically lab and con were polling 40% each and ukip 10%, so con said they'd have their little racist vote that would obviously fail (right?)
07:59:41 <CompanionCube> really they no longer have a reason to exist
07:59:53 <zid> except everybody fucking hates con so the racist little vote won
08:00:09 <sham1> Is it racist to with for less immigration?
08:00:12 <sham1> wish*
08:00:19 <zid> It's racist because it's racist
08:00:32 <sham1> Although I'm starting to feel like we're going way off topic
08:00:38 <zid> they don't want those awful darkies coming over here stealing their jobs and speaking their funny languages
08:00:54 <glauxosdever> Yeah, better let's return to universities that don't teach well
08:01:01 <zid> kek
08:01:04 <booyah> if you want to not let random criminals enter your country
08:01:07 <booyah> then U R racist
08:01:07 <CompanionCube> zid: another reason for the vote was to resolve the question in cameron's own party
08:01:09 <booyah> please it's 2018
08:01:11 <zid> oh look, found one
08:01:31 <CompanionCube> that really backfired, given that the division's worse than ever.
08:01:31 <bcos_> zid: That's sounding like "I'm wet therefore it must be raining" logic
08:02:18 <zid> bcos_: boats.
08:02:31 <CompanionCube> (though may made it even worse)
08:02:48 <booyah> damn racist Natives. affraid the kind guests will put them into concentration reserve
08:03:10 <zid> booyah: we get it, you're a racist, quiet down
08:03:16 <booyah> or thoes Iran woman, affraid islamists will make them wear burkas
08:03:40 <bcos_> (there are more reasons to want to reduce immigration that just racism alone, so "less immigration" doesn't imply "definately racist")
08:03:40 <booyah> zid: yeah everyone who doesn't want to destroy own culture is
08:03:57 <zid> bcos_: Right, but the UKIP members were members for a reason
08:04:06 <zid> and not just members of the conservative party
08:04:06 <CompanionCube> reducing immigration in and of itself is not racist
08:04:12 <booyah> it is a cultural war
08:04:13 <CompanionCube> the reasons they want to do it though...
08:04:19 <zid> imagine ukip is the "Burn the boats" party and the con party is the "we don't like boats for various reasons" party
08:04:23 <sham1> Economic burden?
08:04:36 <zid> immigrants pay way more taxes
08:04:48 <bcos_> Over population?
08:04:48 <booyah> culture that denies human rights to woman (islam), or our cultures. Hmmm which to choose? Thoes daily acid attacks do sound like a cool sport
08:05:15 <booyah> wait, don't want zid to ride on his moral horse and call us racist. Proceed with burking all the woman
08:05:16 <zid> booyah: I told you, we get it, you're a racist, you don't need to go down the handbook line by line
08:05:20 <booyah> ^
08:05:31 <sham1> zid: oh if only. Over here in Finland for instance we get people who come here just for social benefits
08:05:44 <glauxosdever> booyah: Not sure if serious or trolling but, while Syrians are indeed refugees because of the war the US caused in their country, there are indeed some terrorists who mask as Syrian refugees
08:05:46 <zid> sham1: doesn't work like that here, as much as the daily mail would tell you otherwise
08:05:46 <sham1> Who don't want to put in stuff
08:05:50 * CompanionCube rolls eyes at the sterotypical responses
08:05:54 <booyah> sham1: same in europe. All current migration is about that, at BEST. (at worst its about replacing our culture)
08:06:01 <booyah> (in mainland europe)
08:06:13 <sham1> Meh, they cannot replace the culture
08:06:23 <sham1> They either isolate themselves or assimilate
08:06:24 <booyah> sham1: of course they can, and they did.
08:06:31 <zid> yea, that's why I am a black woman
08:07:32 <bauen1> how about you don't think about reasons to keep people out (usually ends with having to keep people in, eg. DDR) and start thinking about fixing the underlying problems (war in their homecountry, economic failure in the 3rd world caused by the luxury in the 1st world, and lack of education about woman rights) ?
08:07:47 <booyah> sham1: show me a country where Islamists are the majority, and woman are free to go topless, or in bikinis and christians have no problem practicing own relligion, and woman have equall rights in general
08:08:11 <zid> Let's see if sham1 is stupid enough to take the bad-faith arugment bait :P
08:08:17 <sham1> <.<
08:08:28 <booyah> bauen1: there is no problem in their country, that is a lie 95% of the time. 5% might be actuall war migrants, if even
08:08:32 <bcos_> Wait - just show me a country where woman go topless..
08:08:41 <booyah> bcos_: poland
08:08:42 <zid> for reseach purposes only of course
08:08:52 <sham1> Something about most of European countries not being majority muslim, much less islamist something
08:08:59 <bauen1> booyah: i'm not talking about war problems only, there are *economical* problems
08:09:00 <booyah> bcos_: on the beach
08:09:11 <booyah> bcos_: greece (on the beach)
08:09:21 <booyah> mostly countries in Europe allow nudist beaches right?
08:09:22 <zid> bcos_: idk like, namibia? Bastion of human rights there I think
08:09:34 <glauxosdever> THey still wear something :-)
08:09:46 <booyah> glauxosdever: on nudist beaches they do not
08:09:59 <glauxosdever> I wasn't aware we had such beaches here..
08:10:03 <booyah> wait, you really didn't knew that? well os-dev is for super-geeks... >_>
08:10:21 <glauxosdever> I don't see any correlation
08:10:31 <zid> glauxosdever: Yea but if certain immigrants didn't ever go to those beaches, your culture would be destroyed :(
08:11:11 <zid> like, all 4000 years of it
08:11:11 <glauxosdever> ?
08:11:31 <glauxosdever> I'm trouble parsing the previous line, maybe I'm tired
08:11:33 <CompanionCube> tis a joke
08:11:43 <zid> glauxosdever: it's the logical conclusion of what boo said
08:11:44 <booyah> sham1: well in Germany, and UK, isalmists will be the majority, already they are majority in the new generation afair (e.g. children in pre-school)
08:11:52 <CompanionCube> booyah: source lol?
08:12:02 <zid> letting immigrants in destroys your culture, the cultural difference is that they woudlnt' go to those beaches
08:12:12 <zid> ergo the fact they don't go to those beaches means your culture is destroyed
08:12:12 <sham1> I really don't care about Germany or the UK, assuming that that'd even be true which I very much doubt
08:12:35 <zid> I vote we just have another world war instead
08:12:55 <booyah> zid: false dichotomy
08:13:28 <booyah> go back to calling everyone who knows cultures can be incompatible a racist, that worked
08:13:46 <CompanionCube> booyah: and yet you think that letting immigrants in results in an islamist takeover?
08:13:59 <CompanionCube> not much difference between those arguments.
08:14:03 <booyah> CompanionCube: by millions? well obviously?
08:14:16 <zid> bait taken :P
08:14:41 <booyah> in democratic country obviously you lost at 51% if both cultures are as active voters
08:14:48 <zid> CompanionCube: he's arguing in bad faith, his goal isn't to 'win the argument', he can't. His job is to troll the fuck out of you and shit all over the board and stuff as many pieces as he can up his nose.
08:15:18 <booyah> zid - attack the opponent personally
08:15:18 <sham1> Back on actual topic, I curse whoever came up with AML MethodInvocation with a rain of having to parse forward references
08:15:21 <zid> The ultimate goal is that someone ELSE comes away from osdev saying "The debate on immigrantion seems unsolved", even though he had 0 arguments
08:15:50 <CompanionCube> sham1: why not curse the inventors of AML and ACPI in general :)
08:15:58 <booyah> zid: no, you had only argument "U R RACIST", while opposing arument is that islam takes away woman's right, so they should be kept away with such attitude (among many other bad things they culture does)
08:15:59 <zid> TEACH THE CONTROVERSY!
08:16:57 <bcos_> booyah: Surely opposing argument would be to let woman in so you can give them rights?
08:17:00 <glauxosdever> To be fair, I think booyah is right about a couple of things. That cultures may be incompatible, sure, western countries value human rights, while islamic countries usually do not. It's in our and their mentality respectively
08:17:10 <zid> I almost looked at pci-e, but the first step was to look at ACPI tables
08:17:14 <zid> so I am not touching pci-e
08:17:25 <zid> glauxosdever: good job no islamic countries are in europe then
08:17:33 <zid> he has 0 arguments
08:17:46 <booyah> zid: yeah keep ignoring the arguments I give
08:17:50 <zid> note how it was NEVER about immigration, it was about islam
08:18:04 <zid> which /aren't in the EU so we don't even have to let them in/
08:18:17 <zid> those are refugees, not EU nations 'abusing' right to work
08:18:20 <zid> nationals*
08:18:23 <geist> hey guys take this somewhere else
08:18:23 <booyah> yeah, small amount of migrants who respect normal human values, can be ok
08:18:29 <geist> not the right forum
08:18:32 <sham1> CompanionCube: because I like suspend and hibernation
08:19:13 <sham1> But yeah
08:19:18 <sham1> I guess I could curse at ACPI as well
08:19:33 <zid> I have no idea how ACPI even works other than you have to parse little tables
08:19:37 <geist> yah you probably dont want to start down suspend/hibernation right now. it requires a fair aount of everything else working
08:19:44 <zid> is it just sat in memory below 1M someware copied in by the bios?
08:20:00 <geist> not necesarily no. but yes there are multiple tables that you parse
08:20:12 <geist> at least the format is fairly straightforward. type/length/data repeat
08:20:17 <zid> where else might it be and/or have come from?
08:20:42 <geist> UEFI may tell you where it is if you came from there
08:20:53 <zid> the erm.. uefi bios? :P
08:21:04 <geist> otherwise iirc there's some sort of root table you can search for in <1MB that then poitns to the rest, if you dont have UEFI or dont understand it
08:21:04 <zid> maybe not below 1M there I guess though
08:21:27 <geist> the key is a full ACPI table can be pretty large, certainly much bigger than fits in any slot less than 1MB
08:21:49 <geist> my experience is the acpi table is usually thrown down somewhere near the top of memory under 4GB
08:21:51 <zid> is it marked in the e820 as used?
08:21:52 <CompanionCube> is AML needed for the table-parsing or is that just for the actually doing things?
08:22:03 <zid> I'm probably overwriting it currently if not
08:22:10 <geist> ah then there's AML. so many times are just tables. APIC/MCFG/SRAT/SLIT/etc
08:22:14 <geist> those are trivial to parse
08:22:23 <bcos_> zid: Mostly marked as "ACPI reclaimable" in E820
08:22:25 <geist> but, some of the tables have AML in them, DSDT
08:22:30 <sham1> geist: I was talking about suspend and such in generality, but true
08:22:32 <geist> these are the 'big' tables
08:22:39 <zid> bcos_: so as long as the bios isn't buggy it won't be type 1 or whatever RAM is
08:22:39 <zid> ?
08:22:56 <geist> zid: read the acpi spec, it tells you precisely how to find it. it's not hard
08:23:10 <geist> dont rely on the bios/efi telling you where it is. it might be able to, but worst case you can find it
08:23:13 <zid> geist: I've no interest in actually supporting it for now so I was avoiding it :P
08:23:24 <zid> reading the acpi spec sounds like a punishment
08:23:25 <geist> you'll need pretty quickly at least some part of it
08:23:35 <geist> well, frankly, if it does, then you probably should find another thing to hack on
08:23:39 <zid> pci-e needs it, I'm not sure what else
08:23:49 <sham1> I use the ACPI spec to fall asleep
08:23:50 <geist> because reading specs and trying to understand how stuff is done is one of the cornerstones of osdev
08:23:56 <sham1> It's useful in multiple ways!
08:24:06 <zid> geist: I also wouldn't read the C++ spec, doesn't mean I can't.
08:24:07 <geist> zid: ioapics, discovering all the cpus (it replaces the SMP bios table)
08:24:12 <zid> ah ioapics
08:24:16 <zid> good point
08:24:24 <zid> want to read*
08:25:03 <geist> look, i'm not sayign you have to read every spec on the planet, but there are some cases where you simply need to go to the source of truth instead of just grubbing stuff together by rooting around in tutorials or howtos or wikis
08:25:12 <zid> I have literally never.
08:25:14 <zid> wtf
08:25:17 <geist> frequently it's far easier to just read the spec instead of trying to cobble together stuff
08:25:22 <zid> tutorials are *always* fucking useless
08:25:26 <geist> right
08:25:27 <zid> please don't insult me like that
08:25:28 <sham1> Yeah. The ACPI and the AMD64/ia32 might be the most useful ones
08:25:38 <zid> I just said I don't *want* to read the acpi spec, not that I can't, or that I won't if I need to
08:25:40 <zid> I just don't *want* to
08:25:48 <zid> everything else is you projecting and making awful extrapolations
08:25:49 <geist> fine. point is dont insult me by completely discounting my suggestion to look at the spec
08:26:12 <zid> I didn't discount you, I gave a reason why I wasn't going to do what you said
08:26:12 <geist> and anyway, clearly you're in an argumentative mood, so i'm going to dismiss it
08:26:24 <zid> I'm allowed to do that, at least, I think so, maybe not
08:27:38 <geist> this acpi stuff is fresh on my mind because i was thinking of writing a new simple parser that just picks out the handful of tables and items we need for the kernel
08:27:50 <geist> so we can drop pulling in all of acpica to look at just a handful of tables
08:28:00 <zid> yea I imagine if I did add limited support for it I'd just add code that could pull out specifics I needed
08:28:12 <geist> yah. it shouldn't be more than a few thousand lines of code
08:28:12 <zid> rather than parsing the entire thing and making structures for it all etc
08:28:28 <geist> yah. that's whats nice about it: type/length/data is easy to skip stuff you dont caer about
08:28:30 <zid> acpi_grab_value(key, &v); or whatever
08:28:37 <geist> the hard part are the tables that have AML in them, because you can't just do that
08:28:38 <zid> and drivers just ask for various keys or whoever
08:28:47 <geist> those tables act kind of like objects with methods on them
08:28:59 <zid> I don't know what AML is (please don't tell me to go read the AML spec to find out >_<)
08:28:59 <geist> you 'call' the method to get the value, which may or may not be a substantial maount of AML
08:29:12 <geist> it's a virtual machine that's embedded in the tables
08:29:12 <zid> is it some bytecode or x86?
08:29:15 <geist> bytecode
08:29:17 <zid> sounds like byte- yea
08:29:22 <zid> sounds wonderfully gross
08:29:28 <zid> something else to put off until after more fun things
08:29:29 <geist> so a lot of the ACPI spec is the virtual machine spec
08:29:42 <bcos_> zid: AML = "ACPI Machine Language" - byte-code you build an interpreter for so that malware injected into it can do nasty stuff... ;-)
08:29:47 <geist> yah, that's why acpica (which is a decent piece of open source code) is like 300KB compiled
08:29:48 <zid> kek
08:29:50 <geist> lots of it is the AML parser
08:30:03 <zid> I don't mind writing virtual machines though
08:30:06 <zid> I enjoy writing emulators
08:30:18 <zid> It's just a lot of spec to read for very little actual OS visible payoff
08:30:33 <geist> but it has some nice advantages if you look at it. it allows the tables to describe relatively hard things to do that you dont have to write a driver for
08:30:37 <geist> ie, suspend/resume
08:30:45 <zid> compared to say, implementing pci and e1000 and a ghetto networks tack
08:30:45 <bcos_> Half the problem with AML is that it's OS specific (you tell it that you are Windows so that you don't get all the features stripped and then have to behave exactly the same as that version of Windows)
08:30:59 <zid> which might actually let me implement a webserver inside my OS and actually be able to say it does something :D
08:31:00 <geist> right
08:31:07 <geist> and of course buggy implementations
08:31:13 <graphitemaster> oh god don't remind me of AML
08:31:15 <bcos_> ..that's why you're a bit screwed if you don't use ACPICA
08:31:16 <geist> but in the grand scheme of things it could be a lot worse
08:31:28 <zid> bcos_: yea I have seen shims in linux to read the windows tables or linux tables etc for various devices
08:31:32 <sham1> Or you can be stubborn like in the case of the OpenBSD people
08:31:35 <geist> yah if you're actually running AML you basically need acpica or a substantial amount of time recreating it
08:31:43 <graphitemaster> AML wouldn't be so bad if OEMs actually filled it out correctly
08:31:44 <sham1> Although I guess they also try to act all Windows-like
08:31:52 <graphitemaster> I mean the AML spec sucks and parsing it is a chore
08:31:56 <geist> but for just the few tables the kernel parses to bootstrap the cpus and ioapics and whatnot, it's not too bad
08:32:00 <geist> oh HPET too, that's described in there
08:32:02 <zid> "Spend 2 months making a thing that doesn't even work right" sounds like paradise
08:32:03 <graphitemaster> but the fact most systems have completely invalid AML is a pain in the ass
08:32:22 <sham1> How do they mess the stuff up
08:32:32 <zid> because as long as windows does the right thing, they're done
08:32:32 <sham1> Is it like wrong or is the data malformed
08:32:36 <zid> project is complete :P
08:32:37 <geist> i've been fiddling with ACPI on ARM and apparently RISCv has a spec too
08:32:50 <glauxosdever> They also have ACPI?
08:32:55 <geist> basicaly end sup being a subset of the ACPI bits you'll get on PC and then new tagged structures for ARM specific stuff
08:33:12 <zid> windows { /* Change some default from 8 to 4, windows defaults to everything else correctly */ }; Linux { /* nop */ };
08:33:17 <zid> I am 99% sure that's how all bios projects work
08:33:22 <geist> if you grab the latest acpi spec there are subsections for some sort of reduced hardware spec version of it
08:33:36 <zid> windows boots, ship it
08:33:39 <geist> as in 'you dont need about 80% of it because it's PC shit, a non PC machine only has to implement these subsets of the tables'
08:33:45 <CompanionCube> zid: currently linux will noisly ignore any attempts to query if it's linux
08:33:49 <CompanionCube> fun fact
08:33:55 <geist> and then there are ARM and riscv specific things in it, mostly i the APIC table
08:33:56 <zid> That doesn't surprise me
08:34:39 <glauxosdever> I wonder why did they invent ACPI
08:34:55 <zid> To annoy specifically glauxosdever
08:34:57 <geist> well ifyou squint real hard it has an openfirmware like feel to it
08:35:02 <graphitemaster> OEMs and hardware companies are incapable of flashing proper meta information, look at the mess that is the EDID for displays, Windows now has a 2GiB database of "monitor EDID fixes"
08:35:10 <geist> which has a lot of the same strategy for solving things, they just use FORTH
08:35:11 <CompanionCube> isn't openfirmware actually ok though :p
08:35:14 <graphitemaster> what makes you think they'd get AML right
08:35:35 <zid> You can actually hack my monitor through edid
08:35:41 <zid> and inject malware
08:35:46 <glauxosdever> Really?
08:35:48 <zid> yup
08:35:52 <geist> CompanionCube: depends. i'm sure there are a subset of folks that have to deal with (ie os writers) that thik it's terrible
08:35:57 <zid> all dell monitors used the same chipset for the OSD for like 15 years
08:35:59 <zid> and it has a bug
08:36:01 * bcos_ would prefer standardised hardware interfaces and no "standard to deal with failure to standardise hardware interfaces"
08:36:07 <geist> the old Device Tree mess lives on from OF and has infected the linux arm world
08:36:08 <CompanionCube> geist: well it's dead now anyway so it's moot really
08:36:20 <geist> CompanionCube: right, except the DT thing
08:36:39 <zid> OSD can repaint arbitrary window portions etc, so I could trick you into thinking a chrome window has an SSL lock on it etc from within the OSD code if I wanted
08:36:45 <CompanionCube> isn't even that rather mutated from the original form?
08:37:09 <zid> I just draw over the top of your chrome with hacksite.com loaded with a "ssl encrypted bank.com"
08:37:15 <geist> CompanionCube: probably. unclear exactly how it mutated. but you get some old DTisms. the 'chosen' tree, everything being big endian, etc
08:37:16 <glauxosdever> bcos_: Time to ditch the x86 and the PC architecture? I wanted to do it, but the best I would be able to achieve would be to run an FPGA (not enough money for an actual CPU)
08:37:44 <zid> fun news, the newest iphone has an x86 core in it :P
08:37:50 <geist> really every few minutes someone somewhere thinks they can invent a new tree to store arbitrary data in it
08:37:52 <graphitemaster> wat
08:37:54 <graphitemaster> really?
08:38:01 <zid> yea, one of the baseband processors is x86
08:38:05 <graphitemaster> lol
08:38:13 <bcos_> glauxosdever: The choices are ACPI and "wild west of no standards" regardless of architecture
08:38:23 <zid> intel makes a lot of weird chipsets, my router is actually an intel chipset etc
08:38:34 <zid> apparently their latest cdma or whatever chip is x86 based
08:38:44 <geist> probably licensed something from intel
08:38:51 <geist> they bought a baseband company or two over the years
08:38:53 <zid> It's an intel branded chip, idk if they actually made it or not
08:39:01 <sham1> Makes sense for Intel to try to push their architecture
08:39:06 <geist> yah my pfsense router is a Rangeley based thing
08:39:13 <sham1> Well, their and AMD's
08:39:18 <sham1> Can't forget about AMD
08:39:23 <glauxosdever> True. Unless you pick (or create) a good standard that can be extended in a way that's backwards compatible, then tell everyone to use it. But the chances are almost zero
08:39:26 <zid> Fun fact, the hardware offloading on my router is not turned on at all
08:39:33 <geist> sham1: wow!
08:39:36 <zid> so you get bad ping spikes when an SMI hapens
08:39:38 <geist> (i just wanted to say that)
08:39:45 <sham1> ...
08:39:50 <sham1> I accept that wow
08:39:51 <zid> glauxosdever: now there are n+1 standards!
08:40:17 <glauxosdever> Yes! But for a given architecture, there is only one standard and n for the other
08:40:25 <zid> Intel Puma, is what mine is
08:40:43 <zid> https://www.ispreview.co.uk/index.php/2018/08/intel-coughs-to-puma-cpu-flaw-that-hit-virgin-media-hub-3-router.html
08:40:44 <bslsk05> ​www.ispreview.co.uk: Intel Coughs to Puma CPU Flaw that Hit Virgin Media Hub 3 Router UPDATE - ISPreview UK
08:40:58 <glauxosdever> And, after all, whoever isn't doing a POSIX/UNIX system is creating new "standards" here
08:41:05 <geist> ah interesting, yeah if you search for intel puma you get a big pile of 'dont buy it it's terrible!' links
08:41:09 <zid> I can ddos myself offline sending like 10kB/s of traffic :p
08:41:30 <zid> UPDATE 22nd August 2018
08:41:32 <zid> oh, maybe not anymore
08:41:34 <geist> the problem i have with my rangeley box is it caps out at around 800mbit/sec at the fiber link
08:41:47 <geist> turns out it's a combination of freebsd and e1000 and pppoe
08:42:08 <zid> looks like they finally enabled hw offload so that SMIs don't drop packets
08:42:14 <CompanionCube> my isp router is a broadcom MIPS chipset running linux
08:42:17 <geist> apparenlty those 3 dont work well together, and if you enable pppoe the e1000 drier in frebsd due to some limitation of the chipset/driver has to route all packets via Queue 0
08:42:27 <geist> instead of distributing it across all cpus as is usually the case
08:42:30 <geist> and that causes a bottleneck
08:42:49 <zid> https://cdn.discordapp.com/attachments/417023075348119556/490623532024332308/unknown.png
08:42:50 <zid> huzzah
08:42:56 <zid> That used to look like a modern art piece
08:43:11 <sham1> Completely blank with a single colour?
08:43:20 <zid> No, like someone vomitted their palette onto it
08:43:32 <geist> kinda interesting that a single queue can be a bottleneck like that, but the rangeley cores are not that fast, so you can load up one of the cpus to 100% just shuffling around data and deframing/framing pppoe
08:43:46 <zid> https://www.thinkbroadband.com/broadband/monitoring/quality/thumb/4b3b707d5be36379a955d4d50fc716ed8c25a4fe-01-07-2018.png
08:43:47 <bslsk05> ​www.thinkbroadband.com <no title>
08:43:56 <zid> That's july
08:44:18 <CompanionCube> are you sure that the packetloss scale is right
08:44:24 <zid> That's latency spikes
08:44:55 <zid> Packet loss is all at 0% (invisible flat line on the top)
08:46:57 <glauxosdever> bcos_: On a second though, maybe it would be better that system interfaces (RAM, buses, etc) are restandardised for a new architecture and adopt current "wild west of no standards" for peripherals. This way, a microkernel or the boot code only needs to care about standardised interfaces to turn on the other CPUs, detect system features, etc and make the drivers handle the "wild west of no standards"
08:49:24 <glauxosdever> Since ACPI AFAIK doesn't concern itself with peripherals
08:49:43 <glauxosdever> So we would be replacing ACPI with something better
08:49:54 <bcos_> glauxosdever: What I mean is that about 95% of AML could be replaced by a (standardised) "system power management, fan control and thermometer" PCI device, CPU manufacturer's making existing CPU power management architectural, and maing something like MSI mandatory for everything (to get rid of IO APIC and interrupt routing mess)
08:51:11 <glauxosdever> That too. A CPU architecture could maybe actually specify these things, just like they specify MOV and ADD
08:51:31 <zid> My system has like 5 fan controllers though
08:51:58 <glauxosdever> Only 5?
08:52:18 <zid> sadly :(
08:52:38 <bcos_> zid: So you need a PCI device that gives you five sets of "tag, length, value" describing each fan?
08:53:14 <zid> bcos_: It need to be standardized, they all use various pwm, dc offsets etc all in different formats
08:54:10 <glauxosdever> Why no one can build an architecture where all system components are standardised?
08:54:14 <bcos_> Ah, so one tag to descrive "PWM fan" and another tag to describe " digital->analogue DC fan"?
08:54:24 <geist> bcos_: troubl is you're already presupposing that PCI exists
08:54:37 <zid> They're also on different busses
08:54:48 <geist> ACPI at least only presupposes you have memory and a discovery mechanism to find it
08:54:51 <zid> Nuvoton is on ISA, gefroce is on PCI, etc
08:55:06 <zid> and the nuvo controls 3 different fans
08:55:29 <geist> furthermore, even discovering a PCI bus on a modern machine requires something to tell it wher eit is, where the root busses are, etc
08:55:38 <geist> which is also what ACPI does
08:55:49 <glauxosdever> Just have the master bus hardwired?
08:55:51 <bcos_> A table does that (MCFG ACPI table?)_
08:56:04 <geist> yes and no. sadly it's complicated
08:56:18 <geist> glauxosdever: not okay, the PCI ECAM is not at a consistent spot
08:56:35 <geist> and the number of root busses and which ones they are is not standard
08:56:48 <geist> nor are the number of ECAMS. but yes MCFG describes where the ECAM is
08:57:09 <geist> and then sadly you have to use AML equipped config tabes (DSDT I believe) to discover the root bus
08:57:16 <glauxosdever> How did they do it before ACPI then?
08:57:28 <zid> PC Jr, etc
08:57:35 <zid> you installed the right OS and firmware :P
08:57:39 <geist> you didn't have PCIe and then on PC at least you could just assume it's there via poking around in the io accessors
08:57:51 <zid> 'IBM compatible' etc
08:58:04 <geist> regular PCI was assumed to either just be there or not. i think the root bus was always 0. i have no idea what happens with multiple root bridges in that world
08:58:08 <bcos_> glauxosdever: Standard IO ports (0xCF8 and 0xCFC?), standard access mechanism, and "for each bus { for each device .."
08:58:29 <zid> isn't that actually the 1.1 method and not the 1.0 method though or something
08:58:33 <geist> note that it's not really much worse, you just need something that says where the ECAM is, how big it is, etc
08:59:09 <geist> there was also PCI bios, and i honestly dunno precisely where that fit in. if it was desired to use the bios first if present, and then fall back to the io config accessors?
08:59:11 <bcos_> Mostly, can be described by a static table provided by firmware (and doesn't need AML)
08:59:41 <zid> Just make it like an embedded system and you have to know in advance where everything is
08:59:47 <bcos_> ..it's only things that can change at run-time (e.g. fan speeds, temperature, hot-plug PCI devices, ...)
08:59:48 <zid> boot the machine without pci, tell it where pci lives by typing it in, reboot
08:59:55 <geist> it currently does, unforunately, but the question is do real implementations actually *need* AML, or are they just using a default template for that particular method
09:00:01 <zid> typing it in over ISA
09:00:20 <geist> i've disassembled a few of my more complex machines and they have a nonzero amount of AML code behind the PCI config AML methods
09:00:39 <geist> but mostly just seems to be so they can dynamically return something that they could have precomputed during bios initialization
09:00:46 <glauxosdever> Hm, I was thinking about some architecture design (not just CPU design). I think all devices could be memory mapped, RAM, "PCI-like", ROM, have the "PCI-like" device designate chunks from its address space to individual devices, etc
09:00:55 <zid> geist: here's my ACPI method, boot the machine, run a copy of dmidecode in a small environment for it, use what it says
09:01:10 <geist> trouble is like i said it's sometimes dynamic
09:01:42 <geist> here's an example: the 'get me the root bus' AML method on the threadripper i have conditionally computes the starting bus and number of busses
09:01:52 <geist> based on whether or not it detected you have a numa setup or not
09:02:16 <geist> clearly the code is designed to run on regular zens, and/or if you have some sort of EPYC vs threadripper setup, where there may be 1 2 or 4 root busses
09:02:30 <geist> so the code just does a quickie 'what kind of machine am i' and then returns something different
09:02:55 <glauxosdever> Then standardise different ways for multiple root buses, no?
09:02:57 <geist> and some sort of global variables presumably populated by some setting in the BIOS config panel
09:03:13 <geist> that's my point, that's what ACPI does. so if you toss it out you need to find a replacement
09:03:38 <geist> but i am not disagreeing with bcos_ here, it could be made 'static', in that the bios can precompute this stuff prior to loading the os
09:04:23 <geist> i highy recommend dumping and then decoding your acpi tables on your machine, it's very educational
09:04:30 <zid> dmidecode is a fun thing
09:04:51 <geist> yah as is iasl -d
09:04:59 <glauxosdever> Then we still have the issue of BIOS bugs. I think the BIOS (or UEFI even more) can be compared to a monolithic kernel in complexity
09:05:01 <geist> acpidump -b && iasl -d *.dat
09:05:02 <zid> not sure I've ran it on this machine
09:05:46 <glauxosdever> I mean, it needs to do a lot of low-level stuff, it has drivers, it has some kind of memory management for itself, etc
09:06:42 <geist> indeed. the bar is fairly low now. it's easy to slam together a little single tasking system nowadays, it's what a lot of us here will end up doing some day in their career
09:06:56 <zid> Apparently my motherboard serial is "1234567890"
09:07:00 <geist> that used to be some amount of work/time/energy, but now it's just a thing you do
09:07:51 <glauxosdever> Yes, I have done it too some three years ago. The one that didn't even have paging, do you still remember it?
09:08:45 <glauxosdever> (It had a basic EHCI driver though, lol)
09:09:39 <klys> acpidump -b; for f in *.dat; do iasl -d $f; done >> 0001.txt 2>&1
09:13:24 <glauxosdever> It's however crazy how complexity of architectures has increased since 1970s-1980s. So many components that don't even have a standardised interface, so ACPI was invented to cover the problem
09:13:55 <geist> thing is then in the last 10-20 years in PC land things have standardized somewhat. it's in much better shape than it was 15 years ago
09:14:25 <geist> since there are a lot more standard interfaces then, less chipsets floating around, less vendors making special snowflake stuff
09:14:37 <zid> Don't worry, moore's law's done we've got what we've got forever now
09:14:51 <zid> we can just clean it up a little now and never touch it again
09:15:10 <klys> I still want my framebuffer back
09:15:29 <klys> kms is not standardized enough
09:16:14 <glauxosdever> And that's another problem. The x86 is backwards compatible down to 16-bit mode and A20
09:16:32 <zid> It might be worth trying to get rid of ISA at some point I guess
09:16:46 <klys> you can just turn it off in your configs
09:17:09 <zid> unless you use regular pics, or probe pci through 3f8 etc etc
09:17:23 <zid> my fan sensors are on the isa bus
09:17:38 <glauxosdever> I think, maybe getting rid of x86 altogether would be beneficial
09:17:49 <klys> so just buy an arm board
09:17:56 <zid> It'd be neat to just treat amd64 as the only-citizen
09:18:33 <glauxosdever> ARM is worse. They AFAIK have closed specs and each ARM platform has its own standards
09:18:45 <geist> ARM does not have closed specs at all
09:18:46 <zid> ARM doesn't even have the standardization mess x86 has
09:18:55 <CompanionCube> glauxosdever: there's a standard for microsofty stuff
09:18:57 <zid> you build the board, and people have to install the right OS on it
09:18:59 <geist> ARM the architecture is extremely well documented
09:19:09 <zid> rather than the OS being able to detect what the board is and can do etc like a PC
09:19:14 <glauxosdever> geist: I thought it was, maybe it was the past?
09:19:14 <geist> zid: depends. ARM itself is pushing ACPI/UEFI/etc
09:19:20 <geist> glauxosdever: yeah like 15 years ago
09:19:22 <CompanionCube> it decided for better or for use to reuse ACPI/UEFI
09:19:42 <glauxosdever> Next time please warn me before reading old rants :p
09:19:43 <geist> in fact i just got zircon booting on UEFI ARM64 a few day ago
09:19:52 <geist> need to clean up some code and check it in
09:19:56 <zid> that's nice to hear.. I think
09:19:57 <geist> that's why i was looking at ACPI
09:20:08 <klys> if you don't want a pc, and arm is not for you, you probably want an fpga.
09:20:17 <zid> I want a 4THz z80
09:20:17 <geist> but.... nothing at allmandates it. so in general most random low end boards are just using uboot or whatnot
09:20:36 <geist> in which case a) the implicit assumption is you'e booting linux so b) you pass linux a pre-computed device tree
09:20:43 <glauxosdever> I think Linus ranted about ARM
09:20:45 <geist> and then that is the extent of standardization
09:20:55 <zid> the arm tree in the kernel is in waaaaay better shape
09:21:00 <zid> standardized device tree crap etc
09:21:13 <zid> so no more arm/chip1/ arm/chip2/ arm/chip3/ etc
09:21:26 <geist> that ARM is doing that's good is they have probably decided that low end stuff is a lost cause, so they're very strictly standardizing in the server space
09:21:29 <zid> or rather, every single model of every single arm board ever in its own file
09:21:42 <geist> hence why ARM has a SBS (server base spec or something) that mandates particular levels of uefi/acpi/etc
09:21:58 <geist> zid: right. that has happened in the last 5-10 years, ad it's a lot better
09:22:24 <geist> it's of course very linux centric, but in the ARM consumer electronics world the assumption is you're using linux
09:22:32 <geist> if you're not, then you should probably consider acting like linux
09:22:39 <zid> well, it's not like PC wasn't windows centric
09:22:43 <zid> people are capable of dealing with it
09:22:49 <zid> and at least with linux you know what linux is doing
09:22:53 <zid> and it isn't insane
09:22:59 <geist> funny how it all comes aorund. 10 years ago linux was juust starting to catch on in that space, and the assumption was you were running windows/palmos/qnx/etc
09:23:07 <glauxosdever> What about RISC-V?
09:23:09 <geist> and now 10 years later linux is a complete juggernaut in that space
09:23:28 <klys> that risc-v stuff is what you'd put on your fpga
09:23:33 <geist> to the point that vendors dont bother writing docs at all because they just consider linux to be the source of truth
09:23:41 <geist> and then they may or may not release the linux source
09:23:50 <glauxosdever> Are architectural components better specified/standardised there?
09:24:02 <geist> it's about as specified as arm is
09:24:11 <klys> no, risc-v doesn't even have a bus standard yet.
09:24:19 <geist> the mistake i think riscv is doing is they're doing what arm used to do: allow lots and lots of the architecture to be optional
09:24:33 <geist> and thus multiple vendors can pick an choose pieces, generating a binarry compatibility mess
09:24:57 <geist> armv8 largely cleaned that up in arm space, because they made a lot of the arch mandatory
09:25:04 <glauxosdever> But for some embedded systems, for example, floating point support may not be needed
09:25:22 <geist> indeed. so it's all a tradeoff
09:25:29 <zid> The question then though is do they need risc-v
09:25:54 <geist> thts why arm basically split their world into two: embedde stuff (where things like float may still be mandatory) and binary compatibility isn't a big deal
09:26:05 <geist> and non embedded, linux-class stuff, where they tightened it up a lot
09:26:13 <klys> if you're manufacturing a product designed from opencores, then you probably want something like that to avoid violating patents.
09:26:18 <geist> er float non mandatory in embedded
09:27:07 <glauxosdever> klys: So exclude parts of the specification that, when implemented, violate patents?
09:27:38 <klys> glauxosdever, have you looked into opencores yet? choose your cpu architecture?
09:28:11 <glauxosdever> I think I have looked a bit into it
09:29:00 <klys> then you see yourself designing riscv on a xilinx or intel/atmel board, and risc-v is not going to be a piece of cake
09:29:31 <geist> for busses i suspect they'll just suggest using AXI or whatnot
09:29:41 <geist> which at least as as far as i know a royalty free spec
09:29:55 <geist> seems to be something that ARM did a good job of seeding into the world
09:30:40 <glauxosdever> klys: I just asked about RISC-V for comparison
09:31:00 <klys> glauxosdever, ah, then you do wnt something that is patented.
09:31:26 <glauxosdever> Not really. Probably I'd do my own thing
09:32:05 <klys> doing one's own thing down to the transistor level is going to take you a while
09:32:52 <glauxosdever> I'd do it on an FPGA for now. I don't have enough money for a custom manufactured circuit
09:33:31 <glauxosdever> But I've already got a lot of projects: a programming language, an OS, probably more things on the way
09:33:59 <klys> that's kind of the assumption. xilinx and atmel boards have their own arch's like microblaze already on the chip
09:34:26 <glauxosdever> I've got a lattice FPGA on an OLIMEX board
09:34:48 <glauxosdever> Haven't really used it yet though
09:35:08 <klys> how many periperal connectors have you got there? usb? ethernet?
09:35:50 <glauxosdever> I don't remember, but I think none of these
09:36:23 <klys> yeah from what I recall, lattice parts are more expensive or don't have as many interfaces.
09:39:38 <glauxosdever> Anyway, I don't know if my architecture could actually pull off. Intel and AMD throw much money into making x86 efficient, ARM and RISC-V are not as efficient, but they also have financial backing from companies, and I'm alone
09:41:45 <klys> I set up a debian/linux/risc-v vm this week. was fun; though there are no graphics. you basically have to operate over a uart.
09:41:53 <glauxosdever> So, I'm left with FPGAs probably. Unless I encourage others to implement my architecture, which probably isn't going to happen
09:43:35 <klys> vhdl hello world sounds pretty fascinating. I hope you end up documenting that experience.
09:46:17 <glauxosdever> Even if it fascinating, it still isn't actually usable
09:46:42 <glauxosdever> And that's a big drawback, because I want to do fun, but usable, stuff
09:48:42 <glauxosdever> Anyway, gtg
09:48:44 <glauxosdever> Goodnight!
09:48:52 <klys> tah-tah.
09:52:24 <klys> the weather is up in the 90s here
09:52:27 <geist> klys: where did you get the riscv install from?
09:52:42 <geist> it's not on their main page, is the riscv stuff unofficial?
09:52:49 <klys> geist, I followed some instructions from the debian wiki
09:52:58 <klys> it did involve compiling the kernel
09:53:04 <geist> kay
09:53:31 <klys> If you like, I'll upload my chroot and kernel for testing purposes
09:53:46 <geist> nah that'sokay. if i were to fiddle with it, the fun is actually doing it
10:00:16 <klys> here's my qemu invocation for risc-v: https://paste.debian.net/1042366/
10:00:18 <bslsk05> ​paste.debian.net: debian Pastezone
10:02:55 <klys> the disk image presently uses most of a GiB, though it was recommended to use 10G
10:08:26 <klys> geist, if you're going to make a go of riscv, you'll probably need this file: https://raw.githubusercontent.com/torvalds/linux/master/arch/riscv/configs/defconfig
10:10:29 <oo_miguel> hmmm where would I get the precise current time from (like microseconds since start)? I want to make some book-keeping how much time each process was active. Now I have the pit ticking at 1/25 second. but what if some hardware intteruupt arrives and I want to task switch? Or is this precision enough usually?
10:12:06 <klys> oo_miguel, there's the tsc; you can read it with the rdtsc instruction
10:12:29 <oo_miguel> wow
10:12:36 <oo_miguel> thanks
10:19:32 <klys> geist, here's my dpkg -l from my riscv chroot: https://paste.debian.net/1042367/
10:19:34 <bslsk05> ​paste.debian.net: debian Pastezone
10:20:37 <klys> well it's actually an ext3 partition
10:30:43 <geist> oh cool, thanks
10:31:09 <CrystalMath> does anyone know how to make gdb unload a symbol file?
10:31:24 <CrystalMath> or list loaded symbol files
10:31:56 <CrystalMath> for some reason it just won't forget a badly loaded symbol file :(
10:32:02 <CrystalMath> i don't want to lose my debugging session
10:34:59 <geist> hmm, no idea
10:35:24 <CrystalMath> geist: it thinks it's not loaded, but it is
10:35:31 <CrystalMath> which is perhaps the worst kind of "loaded"
10:35:37 <graphitemaster> CrystalMath, "symbol-file" with no arguments unloads all symbols
10:35:49 <CrystalMath> graphitemaster: wow that worked!
10:35:57 <graphitemaster> it will ask if you want to discord symbol tables for each object even
10:36:12 <CrystalMath> it just said "No symbol file now."
10:36:15 <graphitemaster> there's also remove-symbol-file
10:36:20 <CrystalMath> and they've all been sent into oblivion
10:36:43 <graphitemaster> you can also bring up the python interpreter and assign None to the symbol table object gdb.symtab
10:38:02 <CrystalMath> python interpreter?
10:38:30 <CrystalMath> [coderain@theoracle ~]$ gdb --version
10:38:32 <CrystalMath> GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
10:38:38 <CrystalMath> just to be clear ^
10:39:11 <geist> yay good guy graphitemaster
10:39:26 <CrystalMath> graphitemaster: anyway, thank you x10000 :)
10:39:28 <graphitemaster> info gdb extending python
10:40:12 <zid> Hrmph lost my address calculator
10:40:24 <zid> I wanted to know what base address pml4[510] had
10:40:40 <zid> err 509 I guess, 510 is 0xffff000000000000 I think
10:40:55 <zid> 512GB lower than that I guess if memory serves
10:40:59 <graphitemaster> turns out when you spend a solid month learning gdb you learn a lot of useful things you didn't know it had or could do
10:40:59 <CrystalMath> graphitemaster: it seems i do have python in my gdb
10:41:12 <graphitemaster> you an also be more productive when you know all the tools at your disposal
10:41:55 <zid> FFEFF8000000000? math is hard
10:42:25 <graphitemaster> I have a script that hashes useful information about a crash in gdb and the state of the program (coredump style) and then does a git bisect to the last known working version attempting to trigger the crash the same way
10:42:31 <graphitemaster> until I find the last known version that did not crash
10:42:35 <graphitemaster> it works wonders
10:42:43 <zid> git bisect is amazing
10:42:54 <zid> no idea how anyone debugged anything in a large project before it
10:42:58 <graphitemaster> I don't even have to interact, just trigger it and it does it on its own
10:43:05 <graphitemaster> automatic debugging
11:10:36 <zid> Okay this relies on some other code I wrote being correct, that's a bad sign
11:11:02 <zid> Calling map_page() for the same virtual address that is already mapped is *suppoed* to do nothing.. we'll see
11:33:40 <tyler569> I love tracking the bug down to code I wrote a year ago
11:33:42 <tyler569> always fun
11:34:00 <zid> tyler569: I am hacking on some code I've not touched for 2 years atm :(
11:34:15 <zid> Whoever designed it was an idiot
11:34:20 <tyler569> that'll do it
11:34:22 <tyler569> ha
11:34:23 <tyler569> h
11:34:49 <zid> which, which byte offset does a page represent in virtual memory for my bitmap
11:34:50 <zid> think fast
11:35:10 <zid> each byte represents 32kB so divide by 32kB I guess
11:35:42 <tyler569> and 8 more probably, for the bits
11:35:48 <tyler569> 32k/32k is 1 byte
11:35:53 <zid> yes
11:36:01 <zid> so any address between 32k and 63.99k is byte 1
11:36:07 <zid> that's what I want
11:36:43 <zid> Now the bit offset is erm.. page/4096 I guess
11:36:56 <zid> nope
11:38:39 <zid> (page>>15)%8?
11:39:05 <zid> >>12
11:40:42 <zid> BITMAP_BASE[page/(32*1024)] |= 1<<((page>>12)%8); ..that better be right
11:49:28 <zid> oh great, massive oversight, my list split code has no virtual addresses
11:50:54 <zid> It may be finally time to bite the 'everything gets a virtual address' bullet