channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev2&y=21&m=11&d=16

Tuesday, 16 November 2021

00:00:00 <riverdc> what I really want to do is implement a compiler and skip C entirely at this point, but I'm nowhere near being able to use that in practice
00:07:00 <riverdc> kingoffrance: I was watching a video that Sean Barrett (author of stb libraries) uploaded on this (using C as codegen output) recently, you might find it interesting
00:07:00 <riverdc> https://youtu.be/lCtALewoFjc?t=4775
00:07:00 <bslsk05> ​'Shawn and Sean on Game Programming #001' by Sean Barrett (03:58:28)
00:08:00 <kingoffrance> sure, thanks
00:10:00 <klange> okay I did something I did not thing would work this well and changed the granularity of what the disk schedular deals with from sectors to pages and... https://klange.dev/s/Screenshot%20from%202021-11-16%2009-06-42.png
00:10:00 <klange> 2+ minutes to install gcc is _not great_, but it works and seems to be stable _and_ I can actually use the system just fine while it's pumping out writes
00:16:00 <klange> Just need to fix up the little bugs on the ext2 side of things, and maybe we can bang out an installer...
00:17:00 <moon-child> riverdc: for games, unless you have very many entity types, better to transpose your representation so you can hoist your dispatches
00:17:00 <moon-child> better branch prediction that way
00:19:00 <riverdc> I'm not sure what you mean by hoisting dispatches
00:19:00 <moon-child> like, instead of saying (T|U)[] all_entities; match for each x in all_entities: |T -> render_t(x); |U -> render_u(x)
00:20:00 <moon-child> say: T all_ts; U all_us; for each x in all_ts: render_t(x); for each x in all_us: render_u(x)
00:20:00 <riverdc> so basically AoS instead of SoA?
00:21:00 <moon-child> sure, yeah
00:21:00 <moon-child> aos/soa is one way of expressing a transpose. Ditto row/column dbs
00:22:00 <moon-child> if you want to express an entity pointer heterogeneously, you do need a discriminant there. But you can skip it for core 'iterate over all entities'
00:22:00 <riverdc> yeah, I agree. one of the things I'm thinking about for the toy language I'm working on is being able to express transposition at compile time without having to restructure everything
00:23:00 <moon-child> oh, heh, video you linked talks about that. sean says 'the way the guy was proposing to me that he was doing things was he was keeping a seperate vector for every single type he had'
00:56:00 <kazinsal> klange: Didn't know your text editor had a render-file-with-highlighting-to-console option. That's pretty neat!
00:58:00 <moon-child> I think vim has that too
00:58:00 <moon-child> there's a context package to use vim for syntax highlighting
00:58:00 <junon> Vim has something that renders out the buffer with the configured colors to an HTML document.
00:59:00 <klange> junon: I have that, too
00:59:00 <klange> (very outdated) example: https://klange.dev/bim.c.htm
00:59:00 <bslsk05> ​klange.dev: bim.c
00:59:00 <junon> Does Bim use Vim keys?
01:00:00 <junon> (I assume Bim is your editor)
01:00:00 <klange> https://github.com/klange/bim/blob/master/docs/mappings.md
01:00:00 <bslsk05> ​github.com: bim/mappings.md at master · klange/bim · GitHub
01:00:00 <klange> and also https://github.com/klange/bim/blob/master/docs/vi-to-bim.md
01:00:00 <bslsk05> ​github.com: bim/vi-to-bim.md at master · klange/bim · GitHub
01:00:00 <moon-child> oh cute, the indentation is first-class
01:00:00 <moon-child> (I hate it personally, but it's a cool feature)
01:01:00 <klange> The choice of display characters is configurable.
01:01:00 <moon-child> I mean in the html. The indentation bits aren't selectable
01:01:00 <klange> They are but you aren't seeing it.
01:01:00 <klange> It's CSS wizardry so the actual text of the document is correct.
01:02:00 <moon-child> ah, even cuter! :)
01:02:00 * klange tries to update that dump
01:03:00 <klange> okay there we go now it's the latest version and doesn't randomly stop syntax highlighting a hundred lines before the end
01:05:00 * moon-child doesn't understand how people can work on such large files
01:05:00 <moon-child> s7 is even worse, 100k lines I think
01:05:00 <klange> Heh, it's an intentional stress test for itself at this stage :)
01:05:00 <moon-child> :)
01:06:00 <klange> Started as just a quirk of the build system I have set up for ToaruOS, which was "one source file per binary" - fine for the little utilities.
01:06:00 <klange> Compositor stretched the viability a bit, but it's only 3k lines. The editor ballooned as it went from a toy to the thing I actually use on a daily basis...
01:07:00 <klange> And at one point I did step back and look at breaking out, but I gave up :) at least the syntax highlighters and themes are separate now (they're Kuroko scripts)
01:11:00 <klange> Shell, file browser, terminal, and compositor are all under 3k lines... and then there's Bim at 12k.
01:11:00 <klange> Kuroko totals around 17k in its C sources, but it has its own project structure checked out as a submodule.
01:29:00 <heat> geist, don't you need libgcc/compiler-rt for constructor stuff? or does .init_array bypass that nicely as well?
01:30:00 <heat> i know _init is all libgcc
01:30:00 <geist> The latter
01:33:00 <heat> i see some archs in linux include libgcc
01:34:00 <heat> i guess I could do that in 32-bit archs
01:34:00 <heat> the biggest issue is really 64-bit divide, I really don't want to replace every divide with div_u64 or something
01:34:00 <heat> and I'd like for some possible portability to 32-bit architectures in the future, if I so desire
01:35:00 <moon-child> heat: you want to avoid linking in libgcc?
01:35:00 <heat> for x86_64 at least, yeah
01:35:00 <moon-child> why?
01:35:00 <heat> multilib for the kernel is hacky
01:35:00 <heat> and it's not really called I guess
01:36:00 <moon-child> you want ... multilib
01:36:00 <heat> ?
01:37:00 <moon-child> 'not really called' --gc-sections or some such?
01:37:00 <moon-child> anywaay re 64-bit divide you could implement it yourself and call it by whatever name gcc emits. I think I did that for one of the int128 things once
01:37:00 <heat> gc-sections is all linker
01:38:00 <moon-child> yeah. So you link in libgcc and then it gets rid of all the bits of it you don't use
01:38:00 <heat> that's not the issue
01:39:00 <heat> my issue is purely a logistical/sanity one, I'm maintaining a multilib in gcc for my kernel (and *every* gcc library needs to be built 3 times because of it, even stdc++), something that no-one does
01:39:00 <heat> also, I'm not sure it's even possible in clang
01:43:00 <geist> think is most arches that aren't x86-64 dont really have a problem using a 'regular' libgcc for the kernel (in my experience)
01:43:00 <geist> so for LK I always link in libgcc for all arches, and only on x86-64 do you need the no-red-zone version
01:44:00 <geist> but in practice it's mostly 32bit arches, and especialyl 32bit risc arches that need lots of assist from libgcc
01:44:00 <klange> I don't seem to actually be getting anything out of libgcc on my x86-64 kernel build.
01:44:00 <geist> yeah, LK has more regular code in the kernel
01:44:00 <moon-child> not so 'little', neh? :)
01:45:00 <heat> geist, do you require a no-red-zone libgcc for lk?
01:45:00 <geist> well, it's embedded, so you can also write your regular code in it
01:45:00 <geist> heat: yes. for x86-64
01:45:00 <geist> but then i have my own toolchain with my own patch for it
01:46:00 <geist> basically https://github.com/travisg/toolchains/blob/master/patches/gcc-patch.txt#L7
01:46:00 <bslsk05> ​github.com: toolchains/gcc-patch.txt at master · travisg/toolchains · GitHub
01:47:00 <heat> yeah I have one like it too
01:47:00 <klange> I was so happy to get libgcc_s.so working...
01:47:00 <heat> but I need an mcmodel variant too, else it doesn't link
01:48:00 <geist> hmm, i wonder how i get away with that in LK, I dont explicitly call out a mcmodel variant for the kernel
01:49:00 <geist> I mean i use -mcmodel=kernel but I dont explicitly build a multilib variant of libgcc for it
01:49:00 <heat> might happen to work if you don't bring in crtbegin/end
01:50:00 <heat> so you never really use libgcc lol
01:52:00 <geist> yah
01:53:00 <geist> but i thought there was codegen differences though, with how it extends pointers, etc
01:54:00 <geist> possible it is just playing with fire. honestly I dont do much work on x86. the x86 port is mostly there to run tests on a x86 host in a VM quickly
01:58:00 <clever> one case where ive run into trouble by not having a multi-lib libgcc, was the rpi pico
01:58:00 <clever> multi-lib enables things like a thumb and arm32 build of libgcc.a
01:58:00 <geist> yah
01:58:00 <clever> and without multilib, the linker will just throw an arm32 libgcc.a into your binary, and when your cpu is thumb only, that wont end well
01:59:00 <geist> https://www.irccloud.com/pastebin/iCkrWoAR/
01:59:00 <bslsk05> ​IRCCloud pastebin | Raw link: https://irccloud.com/pastebin/raw/iCkrWoAR
01:59:00 <geist> seems to be the standard it generates now
01:59:00 <clever> yeah
02:00:00 <clever> there was some mp or ma profile you could enable with ./configure
02:02:00 <heat> i'm going to try and blow up x86_64 lk now :P
02:03:00 <heat> geist, you seem to have nls enabled btw, not sure if it's on purpose
02:03:00 <geist> nls?
02:03:00 <heat> native language support
02:03:00 <geist> enabled where?
02:03:00 <heat> on the toolchains
02:03:00 <heat> http://newos.org/toolchains/
02:03:00 <bslsk05> ​newos.org: Index of /toolchains
02:04:00 <geist> ah. and what does that mean?
02:04:00 <heat> localized error messages
02:04:00 <geist> bein an 'murcan i dont know nothing that aint 7 bit ascii
02:04:00 <kazinsal> means that someone on the forums is one day going to have to debug someone's error messages in catalan
02:05:00 <geist> but isn't that desirable?
02:05:00 <geist> not for the debugginb, but for the user
02:05:00 <klange> it's a mixed bag
02:05:00 <kazinsal> yeah, it's good UX
02:05:00 <klange> it's terrible for being able to search the interwebs for relevant info
02:05:00 <kazinsal> true
02:05:00 <heat> or ask for help
02:05:00 <geist> word.
02:06:00 <geist> well, i solved that problem by not having my own forum and never checking The Forum
02:07:00 <geist> but yeah, that's a good question: can add a switch to the toolchain builder maybe
02:07:00 <geist> most likely something like --disable-nls on the ./configure i guess
02:24:00 <heat> i'm thinking about having a scripted 2-stage toolchain build without a sysroot
02:24:00 <heat> i would stop needing a prebuilt tarball
02:24:00 <heat> otoh, very slow
02:28:00 <klange> I'm still trying to get my "toolchain" down to just gcc and binutils... still have an awkward Python dependency for my ramdisks.
02:30:00 <heat> isn't kuroko similar enough'
02:30:00 <heat> ?
02:30:00 <klange> The awkward bit is that I use Python's tarfile module to produce controlled, compressed tarballs.
02:30:00 <heat> ah
02:31:00 <klange> I don't have a gzip compressor of my own yet... and I also don't have a tar far _producer_ yet, just an extractor...
02:32:00 <klange> tar file*
02:34:00 <klange> Hm, I should probably just remove -lgcc from this Makefile for the kernel so if I _do_ accidentally introduce a libgcc dependency I will know quickly instead of discover it after an agonizing debug session of why a stack has been corrupted by an interrupt...
02:36:00 <klange> I know my userspace needs it for similar things to what heat was saying earlier about constructors or whatever, even without C++. Just shared lib stuff in general. Good thing my modules are relocatable objects!
03:06:00 <klange> fsck summary for a disk that had gcc and doom installed on it: https://gist.github.com/klange/bba0ac908c3afdd51cf1f19fafe77b21
03:06:00 <bslsk05> ​gist.github.com: gist:bba0ac908c3afdd51cf1f19fafe77b21 · GitHub
03:06:00 <heat> oh god
03:06:00 <klange> I am counting blocks wrong (I think I'm forgetting to include the indirect blocks?) and something's very wrong with directories
03:06:00 <heat> no worse feeling than that
03:06:00 <zid> flawless
03:07:00 <klange> I think all the unattached inodes are the files in /usr/lib/gcc/x86_64-pc-toaru/10.3.0/plugin/include/ that got dropped
03:09:00 <geist> maybe forgot to mark them unallocated in the inode bitmap?
03:09:00 <geist> there is an inode bitmap right?
03:09:00 <heat> a few weeks ago I found out I wasn't filtering out device files and symlinks when freeing inode space
03:10:00 <heat> it just freed random blocks
03:10:00 <klange> I think it's that I made inodes for the files but failed to add them to the directory, as I recall having a problem with directories with "too many" entries
03:10:00 <heat> whatever garbage was in i_data
03:10:00 <zid> sounds like this should be relatively easy to find issues with at least
03:10:00 <zid> start with a known good fs, do a couple of ops, make sure it's still good, repeat
03:10:00 <zid> given how much garbage it's spewing it can't be hard!
03:11:00 <heat> what if it only corrupts the fs under load?
03:11:00 <zid> Then I feel sorry for klange
03:11:00 <heat> filesystems are hard :(
03:13:00 <heat> is there a way to get access to an interactive macOS shell without, erm, having a mac?
03:13:00 <heat> i'd like to have my OS buildable in macOS but debugging through github actions logs isn't easy lol
03:15:00 <klange> You could probably do a reverse shell off the CI to your local machine, but github might not like that.
03:58:00 <wleslie> pretty sure aws can rent you a macOS instance
04:01:00 <wleslie> on how I learned to get over it and just use the cloud for development: I used to always buy overspecced machines, mostly because I was doing really out there static analysis work (think 100 GB+ datalog databases)
04:02:00 <wleslie> I eventually realised that rather than spend several thousand on an overspecced machine that would mostly be going to waste, instead spend $2 to borrow a machine with +150GB of ram on the cloud for an hour or two
04:10:00 <zid> we're back to mainframe timeshares ;)
05:45:00 <kingoffrance> not just that quote: I bet some kids in 10 years are going to start designing things with bloody mailboxes ;) because of the rpi
05:45:00 <kingoffrance> https://en.wikipedia.org/wiki/Convergent_Technologies_Operating_System Each client consumes the services via its own mailbox called "exchange" and well-published message formats. The communication works on "request codes" that are owned by the service. The operating system maintains the exchanges, message queues, scheduling, control, message passing, etc., while the service manages the messages at its own exchange using "wait", "check", and "re
05:45:00 <kingoffrance> spond" macros.
05:45:00 <bslsk05> ​en.wikipedia.org: Convergent Technologies Operating System - Wikipedia
05:46:00 <kingoffrance> in 1980 again :p
05:46:00 <kingoffrance> *it is
06:19:00 <heat> why are macOS binutils+gcc builds so much smaller than linux ones?
06:19:00 <heat> zstd says 268MB vs 900MB
06:22:00 <heat> exact same target, of course
06:23:00 <zid> using their libc or yours?
06:24:00 <heat> they're both dynamically linked builds of the same toolchain for the same target, using the same sysroot (so you can't blame my OS's sysroot)
06:25:00 <zid> is the linux one including half of glibc inlined 80000 times, and the other one maybe linked to mac's crt dynamically, for example
06:26:00 <heat> fairly sure you can't inline through dynamic libraries
06:26:00 <heat> also, no lto
06:27:00 <zid> well I'm out of obvious guesses then, it's statistics time
06:27:00 <zid> which binaries are bigger and by how much
06:33:00 <kingoffrance> how about debugging info
06:34:00 <heat> cc1, cc1plus, etc are way way bigger in linux
06:34:00 <kingoffrance> copy them and strip :D
06:34:00 <heat> it's still a shared library though?
06:35:00 <kingoffrance> im just guessing, but do it
06:35:00 <heat> wait what
06:35:00 <heat> why are they not fucking stripped
06:35:00 <zid> splitdebug for life
06:35:00 <heat> WHY IS THE RELEASE BUILD NOT STRIPPED
06:35:00 <heat> ????
06:35:00 <zid> because it never gets loaded into memory?
06:35:00 <Affliction> heat | fairly sure you can't inline through dynamic libraries
06:36:00 <Affliction> hmm... store compiler IR in the file, link at load time?
06:36:00 <zid> inlined strlen absolutely destroyed valgrind for a while until all the distros and valgrind and gcc all worked on it :P
06:36:00 <heat> oh fuck off, I need to do make install-strip
06:36:00 <heat> this is
06:36:00 <heat> bullshit
06:36:00 <Affliction> absolutely wouldn't want to use that for small utilities, but it might be usable for long running programs?
06:36:00 <zid> those are builtins not true inlines though
06:36:00 <Affliction> oh right, it exists, we call it "JIT"
06:37:00 <Affliction> woohoo, reinventing the wheel
06:37:00 <heat> aot for lyfe
06:37:00 <zid> I should remake my VM probably on a bigger disk now I have some extra storage
06:37:00 <zid> my debug symbols are up to like 20GB
06:38:00 <zid> gentoo has a system wide splitdebug compressdebug -nostrip setting that's super nice, everything ends up in /usr/lib64/debug/usr/bin/blah.debug
06:38:00 <zid> and gdb will pick them up as the original symbols, even though /usr/bin/blah is 'stripped'
09:42:00 <geist> heat: re stripping my toolchain script strips afterwards for exactly this reason
09:46:00 <zid> -fwhole-program ;)
09:49:00 <geist> inded, though i'd tend to think that proper LTO is in every way better than -fwhole-program
09:49:00 <geist> i'd think at least
09:49:00 <geist> maybe it ends up with some more visibility though. i'll ask toolchain folks at work tomorrow
09:51:00 <zid> -fwhole-program is just a lot simpler than manually marking things hidden
09:52:00 <zid> if you're building a final executable you can basically just toss every symbol and every implementation of said symbol
09:52:00 <zid> fairly easy to show on godbolt or such that if you do int f(void ) { .. } int main(void) { f(); }; then it will both inline f into main, and leave a copy of the definition of f() in case someone externally calls it
09:53:00 <zid> In GCC, the whole program option (-fwhole-program) asserts that every function and variable defined in the current compilation unit is static, except for function main (note: at link time, the current unit is the union of all objects compiled with LTO).
09:53:00 <zid> So it basically just automates marking every symbol other tham main as __attribute__((visibility=hidden)) or whatever that syntax is
10:49:00 <klange> Stopped mounting a tmpfs to /var and the package system is surviving reboots correctly. There's some stuff I put in there that probably needs to stay as a tmpfs, like /var/run, but I'll look at that later...
11:38:00 <gog> nyaaaa
11:38:00 * kazinsal headpats
11:39:00 * gog purrs
11:42:00 <graphitemaster> Is there a way to trick configure into passing an ac test when you invoke it without modifying the configure script
11:48:00 <klange> graphitemaster: set the cache variable for the test
12:00:00 <graphitemaster> How do you do that
12:01:00 <klange> Same as you would set any other configure variable, `./configure ac_cv_whatever=yes`
12:02:00 <klange> If you don't know what the cv name is, perusal of either the log of a previous run, the script itself, or the ac source script should help.
12:03:00 <graphitemaster> Thanks
20:27:00 <heat> https://twitter.com/llvmorg/status/1455633858347556869
20:27:00 <bslsk05> ​twitter: <llvmorg> The 2021 LLVM Developers' Meeting will feature the keynote "Building an Operating System from Scratch with LLVM" by Petr Hosek on November 19th. Registration is free: https://llvm.swoogo.com/2021devmtg/ https://pbs.twimg.com/media/FDNzR9qXIAEcPfy.jpg
20:28:00 <heat> fuchsia's toolchain guy is giving a talk
20:29:00 <moon-child> guessing not this guy https://en.wikipedia.org/wiki/Petr_Ho%C5%A1ek
20:29:00 <bslsk05> ​en.wikipedia.org: Petr Hošek - Wikipedia
20:29:00 <geist> might be!
20:29:00 <geist> but no i think he spends too much time fixing llvm bugs for that
20:30:00 <heat> obviously its this guy: https://en.wikipedia.org/wiki/Petr_Ho%C5%A1ek_(footballer)
20:30:00 <bslsk05> ​en.wikipedia.org: Petr Hošek (footballer) - Wikipedia