channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched
#osdev2 = #osdev @ Libera from 23may2021 to present
#osdev @ OPN/FreeNode from 3apr2001 to 23may2021
all other channels are on OPN/FreeNode from 2004 to present
http://bespin.org/~qz/search/?view=1&c=osdev2&y=22&m=1&d=15
04:56:00 <Jari--> mor
06:03:00 * klys played caverns of kroz for four hours this evening, though lost the saved game to bad luck.
06:40:00 <heat_> i really need pseudo fs framework code but I'm afraid to get caught up in the classic unix pseudo filesystem everything is a file extravaganza
06:41:00 <heat_> especially because I've spent time designing some saner interfaces and i'd hate to just go "haha /proc go brrrr"
06:45:00 <heat_> otoh procfs going brrrr as well is handy
07:12:00 <dh`> the problem with any kind of procfs is that you eventually end up with linux compat and that's a hellscape
07:14:00 <heat_> dh`, how so? is procfs that hard?
07:15:00 <heat_> until now the only thing that has scared me is sysfs which is practically a maze
07:15:00 <heat_> i just took a look at freebsd and they have a really minimal, trivial kind of procfs that isn't even mounted by default
07:23:00 <heat> hmm ok the netbsd procfs is way more linuxy
07:24:00 <CompanionCube> iirc solaris/illumos has a binary procfs
07:25:00 <heat> freebsd and openbsd have like 4-5 files in each pid and curproc/
07:25:00 <heat> CompanionCube, hmm I feel like that's silly
07:25:00 <heat> you're throwing away the scriptability/user-friendliness of procfs
07:25:00 <heat> at that point you can skip the procfs and use a utility/library
07:27:00 <CompanionCube> https://illumos.org/man/4/proc
07:27:00 <bslsk05> illumos.org: illumos: manual page: proc.4
07:28:00 <heat> its like a system call with extra steps
07:28:00 <heat> and everyone's favourite, sprintf
07:29:00 <heat> I've read enough linux libc code to know that only having procfs is not a good idea
07:29:00 <heat> but just having it doesn't sound horrible at all
07:37:00 <heat> ah freebsd has linprocfs
07:47:00 <heat> https://developer.apple.com/support/downloads/Apple-File-System-Reference.pdf <-- just found out apfs actually has a spec
08:32:00 <kingoffrance> "everything is a file extravaganza " eh, some files are more equal than others https://www.quora.com/Is-it-possible-to-share-the-pseudo-file-system-procfs-sysfs-over-NFS
08:32:00 <kingoffrance> its not as if that is actually true
08:33:00 <kingoffrance> point: noone actually seems to care about that
08:33:00 <kingoffrance> its a leftover slogan, that maybe once was true
08:37:00 <kingoffrance> actual extravaganza: https://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs
08:37:00 <bslsk05> en.wikipedia.org: Plan 9 from Bell Labs - Wikipedia
08:37:00 <kingoffrance> In the same manner, a distributed computing network can be composed with a union directory of /proc hierarchies from remote hosts, which allows interacting with them as if they are local.
08:38:00 <heat> well yes and no
08:38:00 <heat> look at sysfs :)
08:38:00 <heat> the everything is a file thing is definitely not dead
08:52:00 <kingoffrance> there is no single definition of "file". there is : 1) different charsets (or whatever you want to call them, "encodings") 2) see above -- some files more equal than others
08:53:00 <kingoffrance> as they say, there is no plaintext
09:05:00 <kingoffrance> ditto: https://stackoverflow.com/questions/36331101/how-can-fsetpos-be-used-to-allow-random-access-on-files-that-are-too-large
09:05:00 <bslsk05> stackoverflow.com: c - How can `fsetpos()` be used to "allow random access on files that are too large to handle with `fseek()`?" - Stack Overflow
09:06:00 <kingoffrance> not advocating that, but: File access mode flag "b" can optionally be specified to open a file in binary mode. This flag has no effect on POSIX systems
09:06:00 <kingoffrance> some files more equal than others, again
09:07:00 <kingoffrance> nor am i saying that was a "wrong" move. simply noone agrees on what a "file" is, or ever did, or ever will
09:08:00 <geist> heat: good question. probably not since the memory barrier already happened
09:08:00 <geist> but how much less of a hit is hard to quantify
09:09:00 <geist> also arch specific i assume
09:10:00 <geist> oh weird. there was a gap in time on my irc client where what heat had said like 7 hours ago was the last thing
09:11:00 <heat> I have invented time travel
09:11:00 <heat> or I broke TCP
09:11:00 <heat> either one is fine with me :P
09:11:00 <geist> but yeah my general model of atomics like that is based on some idea that the cpu is weakly ordered (ie, ARM) and it's basically yolo until a barrier comes along
09:12:00 <geist> and then there's some zero-to-nonzero hit where it has to establish a before/after barrier which reduces its ability to yolo
09:12:00 <geist> so in your case where you have a spinlock/atomic/spinlock, the first spinlock is presumably the real barrier
09:13:00 <geist> how much of a hit that is is dependent on the yolo factor the cpu was enjoying before
09:13:00 <heat> yes but for example I was imagining you'd still need to pay for things like the lock prefix on x86
09:13:00 <geist> and how much you crimped its style
09:13:00 <geist> i'm guessing x86 has fundamentally the same inner model, its just got less ability to yolo
09:16:00 <geist> but yeah that's a good question. i've always *assumed* that the first atomic after a series of non atomics is more expensive than a series of atomics
09:16:00 <geist> but i dont really know
09:18:00 <heat> otoh x86 spinlocks don't need a barrier
09:18:00 <geist> sure but it's implied
09:18:00 <geist> well it's implied in the sense that the memory model is in order
09:18:00 <heat> an atomic compare exchange with acquire semantics doesn't generate a memory fence
09:18:00 <geist> but i am guessing there's still a certain amount of yolo going on but atomics act as a fence
09:18:00 <heat> which was the scary thing in my head
09:18:00 <heat> tho lock is also scary
09:19:00 <geist> yah crazy that arm has full relaxed atomics
09:19:00 <geist> ie, no barrier in either direction
09:19:00 <geist> which it's kinda a brain teaser when yuo think about it
09:19:00 <heat> how does that work?
09:22:00 <geist> wel, i guess it tosses it in the queue and does the atomic transaction but it just doesn't order it relative to any other memotry transactions that are in flight (aside from usual rules like reads after writes)
09:22:00 <geist> it's great for things like bumping a counter
20:13:00 <heat> ok so I was trying to fit my ported packages inside CI and I just found out ncurses does really insane stuff with /usr/include inside configure
20:13:00 <heat> they _pass it as an include directory_
20:14:00 <heat> --sysroot goes out the window
20:14:00 <heat> now I don't know what to do
20:14:00 <heat> the build fails in non-trivial multilib configurations because my toolchains simply don't grok that
20:15:00 <heat> such as, erm, every debian/ubuntu system
20:15:00 <sortie> heat, damn, that kinda stuff happens :| Generally my philosophy is to patch stuff like and fix it.
20:16:00 <zid> and now we know why gentoo patches everything and builds inside a chroot
20:16:00 <sortie> The weirdest stuff I ever saw was a port that put the compiler and settings inside an installed header so dependencies could get it
20:17:00 <sortie> heat, generally patching stuff to use PKG_CONFIG / PKG_CONFIG_FOR_BUILD etc. is the cleanest solution
20:17:00 <sortie> Things like running custom foo-config programs from $PATH (except pkg-config) is verboten as it doesn't cross-compile
20:18:00 <heat> yes but I don't understand at all why they do -I/usr/include
20:18:00 <heat> it makes no sense
20:18:00 <sortie> It's some shit reason. It happens now and then. You gotta go and patch it out
20:18:00 <heat> they want to test if the C++ compiler works, they never add -I/usr/include in ANY OTHER COMMAND
20:19:00 <heat> just that one
20:19:00 <heat> for some reason
20:24:00 <heat> autoconf is cursed
20:24:00 <heat> ncursed
20:26:00 <sortie> In autoconf's defense, this is ncurses' own doing I imagine
20:32:00 <heat> sortie, what does --with-sysroot do when you already have (C/CXX/LD)FLAGS properly set?
20:32:00 <sortie> heat, to be honest that option doesn't really work
20:32:00 <heat> hmm actually didn't it do something totally different to what gcc does
20:32:00 <sortie> It's a libtool thing and it does a bunch of shit libtool stuff involving .la files and what not
20:32:00 <heat> >run
20:33:00 <sortie> Packages in general simply does not have support for injecting a --sysroot parameter to the compiler
20:33:00 <heat> yeah
20:33:00 <clever> nix solves this kind of problem, by wrapping the compiler, and injecting flags via a second set of (C/CXX/LD)FLAGS vars
20:33:00 <sortie> The solution I use is have the cross-compiler have a default sysroot set. Ports can detect that if they absolutely must but usually that's a bad thing.
20:33:00 <heat> i'm injecting --sysroot and for clang, --target, in all the flag variables
20:34:00 <clever> so even if the package explicitely sets CFLAGS, the NIX_CFLAGS still get mixed in
20:34:00 <heat> sortie, but that doesn't work if you want to try to provide prebuilt toolchains
20:34:00 <sortie> When I build ports, my tix-build(8) program essentially wraps the compiler binary and injects whatever --sysroot value onto it, and also wraps pkg-config with all the variables
20:34:00 <heat> which I do because things like LLVM are slooooooooow to build
20:35:00 <sortie> The pkg-config stuff is super important and a key part of it
20:35:00 <heat> i dont have any of that yet
20:35:00 <sortie> A whole bunch of stuff needs to run pkg-config to figure out dependencies on the build machine too, and also needs pkg-config for doing the cross-compilation
20:35:00 <heat> i have few ports
20:36:00 <heat> mostly autotools
20:36:00 <heat> zlib is cmake though which is nice
20:36:00 <sortie> https://gitlab.com/sortix/sortix/-/blob/master/tix/tix-build.c#L221 is a bit verbose shell-in-C but it makes sure all the variables are set
20:36:00 <bslsk05> gitlab.com: tix/tix-build.c · master · sortix / Sortix · GitLab
20:37:00 <sortie> pkg-config(1) has a lot of useful environment variables that means you can us the distro binary to cross-compile with
20:38:00 <sortie> I essentially emit a cross-pkg-config that cross-compiles using my sysroot (and turns on static linking, aetc.)
20:38:00 <sortie> Plus a pkg-config for the build system that's set in PKG_CONFIG_FOR_BUILD
20:39:00 <sortie> Ports are then supposed to use $PKG_CONFIG and $PKG_CONFIG_FOR_BUILD appropriately, just like they should use CC and CC_FOR_BUILD etc.
20:39:00 <sortie> (I might emit a plain pkg-config script that fails the build if it's invoked, to make sure the variables are respected)
20:45:00 <heat> i feel like that should all be python
20:45:00 <heat> and i also don't know how to feel telling people to use python
20:45:00 <heat> am i a bad person
20:45:00 <heat> probably
20:55:00 <sham1> Perl
21:00:00 <heat> we should make a build system that's entirely shell scripts with macros on top
21:01:00 <heat> then we just copy it from project to project
21:01:00 <heat> but slightly differently each time
21:02:00 <GeDaMo> I wonder if you could write it in UEFI shell
21:02:00 <clever> heat: sounds similar to the unholy mess that is gnustepbase
21:03:00 <heat> GeDaMo, I don't think it's turing complete, it's just a way to execute commands
21:03:00 <heat> clever, is that a rich man's autotools
21:04:00 <GeDaMo> Pity OpenBoot isn't more widespread, it has Forth :P
21:05:00 <heat> pity efi bytecode failed
21:05:00 <geist> hmm reminds me, i should actually try to package up that e1000 patch i did and see abot pushing it upstream
21:05:00 <heat> we could all be writing firmware and bootloaders in bytecode
21:13:00 <GeDaMo> A build system built on BPF :|
21:14:00 <j`ey> it'll be JITTed!
21:19:00 <heat> oh my god ncurses needs a patched version of autoconf 2.11 to get a working configure script
21:19:00 <heat> this is so fucking ncursed
21:20:00 <geist> been writing an ahci driver to finally have done that
21:21:00 <zid> neato
21:21:00 <kazinsal> GeDaMo: wow, I actually recoiled reading that
21:21:00 <geist> pretty straightforward. but now i'm at the part where you have to start talking real ATA. does not look pretty
21:21:00 <GeDaMo> :D
21:21:00 <geist> well not ugly per se, but ATA is just not a 'clean' design from a first glance
21:21:00 <heat> geist, how so? simple ATA is pretty simple
21:22:00 <geist> yah probably no biggie
21:22:00 <heat> 3 commands and you're done
21:22:00 <kazinsal> ATA has a whole bunch of legacy cruft just because of how old it is
21:22:00 <kazinsal> thankfully you can ignore most of it
21:22:00 <heat> IDENTIFY, READ_DMA48, WRITE_DMA48
21:22:00 <kazinsal> but the design is clearly a case of several specs being stacked together to make one grand unified spec
21:25:00 <heat> no u
21:26:00 <sham1> Sounds a lot like BIOS
21:26:00 <heat> huh?
21:27:00 <heat> the BIOS doesn't even have a proper spec
21:30:00 <geist> yah i thin it's a mistake to read the ATA or SATA spec from the looks of it
21:30:00 <geist> since it covers the nitty gritty deails, eve at the command level, of how the state machine works, etc
21:31:00 <geist> and from the looks of it AHCI handles a lot of those details for you
21:31:00 <kazinsal> yeah, the whole spec is definitely overkill
21:31:00 <kazinsal> probably just want to read the command set document more than anything else
21:31:00 <geist> like theres a whole thing about how you need to query the shadow BSY register on the device in a command slot before you issue a command
21:31:00 <geist> and then mentions 'host controllers may have additional mechanisms such that you dont have to'
21:31:00 <geist> and that's probably one of the many things AHCI seems to track for you
21:32:00 <geist> and then there's the whole NCQ tagging. which i also think AHCI handles for you
21:32:00 <geist> i get the vibe taht by issuing a command in one of the 32 slots, it handles matching up device responses with the 0-31 tags with all the outstanding buffers/FIS receive slots/etc for you
21:34:00 <geist> i *think* there's even a whole DMA buffer FIS descriptor that you have to send. perhaps before the actual command transmission? Either way i dont think the programmer has to do that with AHCI
21:35:00 <geist> so maybe it's handling all the DMA NCQ setup/teardown for you by sending these hidden commands
21:37:00 <geist> ooooh i see. neat: there's a DMA descriptor FIS that the device sends back to the host with { tag, offset, length, data } as it comes up with blocks of data. it's apparently allowed to return stuff out of order even
21:38:00 <geist> so presumably AHCI consumes those and lines them up with the DMA descriptors you set up for the transactions
21:38:00 <geist> so if you have 5 transactions outstanding, tag 0-4, it can just start feeding th ehost back arbitrary blocks out of those 5 transactions in any order.
21:51:00 <geist> hmm, so i think i screwed up when i set up my home network a while back
21:51:00 <geist> i left the default lan vlan as untagged
21:51:00 <geist> and then set up some additional vlans for private stuff
21:52:00 <geist> i probably should have set the default lan as another tag itself
22:05:00 <sham1> <heat> the BIOS doesn't even have a proper spec
22:05:00 <sham1> Exactly