channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched
#osdev2 = #osdev @ Libera from 23may2021 to present
#osdev @ OPN/FreeNode from 3apr2001 to 23may2021
all other channels are on OPN/FreeNode from 2004 to present
http://bespin.org/~qz/search/?view=1&c=osdev2&y=22&m=1&d=3
00:35:00 <Clockface> in NASM, is equ like a macro for a single value?
00:35:00 <moon-child> yes
00:36:00 <Clockface> thanks
04:26:00 <Jari--_> hi all
04:27:00 <Jari--_> pancreat values appear so good I can drink liqour
04:32:00 <kingoffrance> s/*/haiku hi all! pancreat / values appear so good that / i can drink liquor
05:13:00 <Jari--_> hi
05:13:00 <Jari--_> hmm
07:37:00 * gog mews sleepily
07:42:00 <sham1> mov :3
07:46:00 <gog> mov $gog, bed
07:47:00 <zid> movb $gog, garbage
07:47:00 <geist> yay someone using at&T
07:47:00 <zid> mov [luxury], zid
07:48:00 <sham1> AT&T is confusing
07:48:00 <gog> good morning to you too zid
07:48:00 <zid> AT&T is actually un-confusing.. until it suddenly is confusing because they tried to be TOO orthogonal
07:48:00 <zid> and now something doesn't look natural
07:48:00 <zid> indirect calls, out instruction, etc
07:49:00 <geist> yah agreed. there's a sense to it, but there are some places where it's a big head scratching
07:49:00 <geist> but then those cases it's because x86 itself is not that orthogonal
07:49:00 <sham1> It makes referencing the AMD documentation weird because, well, the arguments are shuffled
07:50:00 <zid> AT&T is a foreign language I absolutely have to translate to/from
07:50:00 <zid> idk how that could be natural
07:50:00 <moon-child> I mean, obviously it could be natural if you spent time getting familiar with it
07:50:00 <moon-child> but yeah. I don't trust myself to write at&t and not screw it up
07:50:00 <zid> if you only, maybe
07:50:00 <moon-child> it's hard enough writing assembly without screwing up, period
07:50:00 <geist> yah, like i said before it's simply just vax assembly as applied to x86
07:51:00 <zid> After seeing intel syntax any reasonable human should surely be like "oh, this is way simpler"
07:51:00 <geist> there's also a very similar variant for m68k
07:51:00 <geist> so it's really following DECs style from the 70s. same general left to right, $ %, etc
07:51:00 <gog> i actually like that tbh
07:51:00 <gog> a thing to show you what value in text is what
07:52:00 <gog> register or immediate or etc.
07:52:00 <geist> agreed. what i *dont* like is how a plain number is a dereference
07:52:00 <gog> yes
07:52:00 <gog> that has caused me many headache
07:52:00 <geist> it makes sense when you think of label, but i'd have preferred something like (0x1234) or (label)
07:52:00 <gog> yes so it's clear that it's a deref
07:52:00 <geist> especially since as soon as you have an offset it's N(register)
07:52:00 <gog> can you do 0(whatever) or is that a syntax error
07:53:00 <geist> but a lot of that is because x86 isn't regular in that regard: you can't have a plain label + a complex addressing mode
07:53:00 <geist> whereas i think you can on vax
07:53:00 <zid> the fact you guys are going "let's come up with ways to make it make sense" sorta proves my point ngl
07:53:00 <geist> no no i mean, it makes sense why it is the way it is
07:53:00 <zid> oh absolutelyu
07:53:00 <geist> i just dont like some of the mistakes they made
07:53:00 <zid> the same way it makes sense that english has words that are their own opposites
07:53:00 <gog> it was a different time and we should forgive their shortsightedness
07:53:00 <zid> it makes sense because sarcasm + time
07:53:00 <moon-child> I mean
07:53:00 <zid> it doesn't make it sensible
07:53:00 <moon-child> it makes sense
07:53:00 <moon-child> but you know what else makes a lot of sense?
07:53:00 <geist> to me it's like saying 'i can see why chinese makes sense' but i still can't read it at all
07:53:00 <moon-child> just using intel :P
07:54:00 <geist> no, what makes sense is to stick with what the tools use and just deal with it
07:54:00 <gog> you know what makes even more sense? giving me pets and snacks
07:54:00 <geist> since it's Just Another ISA and each isa has their own thing
07:54:00 <zid> all the tools use intel, so np :p
07:54:00 <sham1> Honestly I'd like a hybrid approach the most when doing x86 or AMD64. Basically use Intel notation but also have the prefixes like $, % and so on
07:54:00 <geist> not really. gcc, etc likes to default to at&t, objdump, etc
07:54:00 <zid> sham1: isn't that what NOT giving 'noprefix' to gas does?
07:54:00 <moon-child> sham1: you can do that with gas
07:54:00 <geist> you can apply switches but they frequently break, etc
07:54:00 <moon-child> .intel_syntax
07:54:00 <sham1> Yes
07:54:00 <zid> gas has att, intel noprefix and intel
07:54:00 <geist> inlien asm, etc
07:54:00 <zid> the latter should be precisely that
07:54:00 <sham1> I'm gonna have to adopt GAS again
07:55:00 <geist> clang i think has issues with it too
07:55:00 <moon-child> geist: -masm=intel
07:55:00 <sham1> NASM is slow anyhow
07:55:00 <moon-child> yeah only works w/gcc
07:55:00 <zid> -mintel too
07:55:00 <moon-child> but inline assembly needs to die anyway soo
07:55:00 <geist> right. that's my point. stick with the defaults since that's the path that's well trodden
07:55:00 <dh`> the problem is the architecture, not the assembly language
07:55:00 <zid> geist only compiles -O0 with no warnings confirmed
07:55:00 <geist> since in my book it's just another isa. there are plenty
07:55:00 <gog> moon-child: inline assembly considered harmful?
07:55:00 <moon-child> sham1: how much assembly are you writing, that the amount of time it takes to assemble matters?
07:55:00 <moon-child> gog: yes
07:55:00 <zid> ehhh inline assembly *does* have positives though
07:55:00 <sham1> Admittedly not much
07:56:00 <gog> i will continue to write inline assembly
07:56:00 <gog> idc
07:56:00 <geist> sure. use it when you gotta use it
07:56:00 <sham1> I still like my assembly swift
07:56:00 <zid> I'll avoid it where possible, but it's definitely useful to do it inline
07:56:00 <moon-child> gog: ._.
07:56:00 <zid> and 'where possible' means 'when it doesn't cause code differences', which is actually not that often
07:56:00 <moon-child> zid: ._.
07:56:00 <geist> yep. i try to use intrinsics if i can, which over the years more and more things have been instrinsics
07:56:00 <zid> moon-child: constraints are too powerful man
07:56:00 <geist> vs, say 10-15-20 years ago where you had to write a lot more inline asm
07:57:00 <dh`> any of these intrinsics documented yet?
07:57:00 <gog> i still have my ugly gdt reloading code inline :p
07:57:00 <geist> i am always going back to old projects of mine and converting things to intrinsics that have subsequently showed up
07:57:00 <geist> dh`: sure, like the atomics that gcc/clang use
07:57:00 <zid> stops you having to do mov rsi, arg1; mov rdi, arg2; mov rax, rsi; mov rdx, rdi; rtdsc.. or whatever, you can just do the rdtsc with the D:A constraint
07:57:00 <geist> used to be if yuo wanted atomics you always did it yourself. but nowadays most arches have most of them implemented
07:57:00 <sham1> Although one thing I like about using GAS is that I wouldn't need to dispatch between having a special thing for compiling for a .asm file vs. .c, just pass .S and .c into ${CC} and you're golden
07:57:00 <geist> or things like __builtin_ffs or whatnot
07:57:00 <sham1> stdatomic.h
07:57:00 <geist> or the endian swap intrinsics
07:58:00 <geist> sure but if you're *implementing* stdatomic.h
07:58:00 <geist> ie, osdev
07:58:00 <sham1> That's freestanding
07:58:00 <sham1> You don't need to implement it
07:58:00 <geist> i disagree
07:58:00 <dh`> where's it going to come from?
07:58:00 <zid> You might need to
07:58:00 <zid> you don't have to
07:58:00 <moon-child> for freestanding code I would definitely prefer the __atomic* stuff
07:59:00 <zid> (same for everything in libgcc)
07:59:00 <moon-child> you probably depend on gcc stuff anyway
07:59:00 <moon-child> so why take the buildchain complications?
07:59:00 <geist> right, and you can implement your own stdatomic, or your own atomics, etc. but in general prefer to use the intrinsics if the arch supports it
07:59:00 <geist> for LK for example i think maybe 6 or the 9 arches implement the full suite, so only a few do i have to resort to inline asm
07:59:00 <gog> just be sure your libgcc is -mno-red-zone :p
07:59:00 <moon-child> FRED go brr
08:00:00 <sham1> Hm, wait. It's not actually freestanding
08:00:00 <sham1> My mistake
08:00:00 <geist> but then also have to consider the first version of gcc they showed up in, etc
08:00:00 <sham1> Yeah, use intrinsics in that case
08:00:00 <geist> sham1: yah not a lot in freestanding
08:00:00 <moon-child> geist: the __sync* stuff is pretty old
08:00:00 <sham1> I really thought that atomics would be a part of the freestanding in the standard. After all, as said, you'd just use intrinsics for them anyhow
08:00:00 <geist> yah i think __atomic is the replacement
08:00:00 <moon-child> but, god help you if you're still using gcc 4.2
08:01:00 <moon-child> sham1: atomics are permitted to be implemented in terms of locks
08:01:00 <dh`> even when stuff is in freestanding, you still have to provide it; as an OS, you're the implementation
08:01:00 <sham1> I see
08:01:00 <moon-child> will have to be, for sufficiently large stuff
08:01:00 <dh`> maybe the compiler ships with float.h or whatnot that you'd actually want to use, more likely not
08:01:00 <zid> on alpha you have to implement them by buying an x86 machine
08:01:00 <zid> badum tish
08:01:00 <moon-child> I think on arm the rmw stuff is lockfree but still goes through a runtime library
08:02:00 <geist> yah and for example VAX doesn't impelment any of them, so you need to provide your own. or riscv doesn't implement the double word atomics, so it calls through to something you have to implement (IIRC)
08:02:00 <sham1> Oh I'd certainly use float.h or stdint.h because those are specifically stuff that is extremely tied to the processor arch
08:02:00 <sham1> Same for stdbool.h and whatever. iso646.h
08:02:00 <geist> also stdarg.h whcih is mostly just a wrapper around some builtins to implement it, but it's still tied to the compiler
08:03:00 <dh`> do you want to paste gcc's stdint.h (assuming they ship one) into your OS?
08:03:00 <sham1> That too
08:03:00 <sham1> Well I'd just use whatever stdint.h is provided by the compiler I'd port. If I was to write my own C compiler I'd use that thing's stdint.h
08:03:00 <sham1> And so on
08:03:00 <geist> yah can do a #include_next of it
08:04:00 <dh`> e.g. the stddef.h that gcc ships is full of horrifying hacks and not suitable for use anywhere sane
08:04:00 <dh`> (or at least it used to be)
08:04:00 <geist> yah part of it is how much you want your OS/freestanding binary to deal with multiple variants of gcc/clang
08:04:00 <dh`> because it was supposed to provide a semi-compliant stddef.h on top of completely broken vendor unixes of the 90s
08:04:00 <geist> it gets much harder if you expect others to bring their compiler
08:04:00 <geist> vs mandating a particular prebuilt or version or whatnot
08:04:00 <sham1> BYOC
08:06:00 <geist> moon-child: iirc gcc on arm64 has switched to a sceheme where they detect new (v8.1) vs old (v8.0) atomics dynamically
08:06:00 <moon-child> right
08:06:00 <geist> we're discussing what to do about that for fuchsia right now
08:06:00 <geist> since i think clang hasn't adopted that strategy
08:07:00 <dh`> does the difference actually matter? runtime hotpatching is the best way
08:07:00 <geist> that's the question
08:07:00 <moon-child> dh`: nah, recompilation is the best way
08:07:00 <geist> a) hjow much of a diff does the new atomics make (evidence is a lot) and b) is runtime detection worse/better than runtime hotpatching
08:07:00 <geist> or c) is recompilation suitable
08:07:00 <dh`> recompilation for different machine variants isn't practical
08:08:00 <geist> my guess is C is good for kernel, and for user space one of the b vvariants is worth it
08:08:00 <dh`> for userspace you can do it entirely with dynamic linker gimmicks
08:08:00 <moon-child> dh`: why not?
08:08:00 <geist> yah
08:08:00 <moon-child> see synthesis os
08:08:00 <geist> but that's all based on the idea that the atomic is already a function call, vs being inline
08:08:00 <geist> whereas *usually* atomic instructions can be simply inline where they're used
08:09:00 <dh`> synthesis was a research project :-)
08:09:00 <geist> but *thats* hard to patch since the new atomics are fundamentally different on arm,
08:09:00 <moon-child> yeah. Point is if you recompile you can do it all inline
08:09:00 <geist> so a) you have to be able to tolerate a function call
08:09:00 <geist> and then b) how do you dynamically switch the function call (patch or let the function figure it out or dynamic linker tweaks)
08:09:00 <moon-child> dh`: well, os research has been dead for 20 years. Soo I don't think it's that significant that nothing ever came of synthesis :P
08:09:00 <dh`> perhaps
08:09:00 <sham1> POSIX
08:09:00 <geist> so if a) is tolerable then there are a lot of options for b)
08:10:00 <geist> for a kernel it may be tolerable to just have multipl kernels for different major codegen options like this
08:10:00 <moon-child> IIRC apple makes you submit llvm bitcode with every app
08:10:00 <geist> say v8.0, v8.1 (new atomics), v9.0 (whatever may come there)
08:11:00 <moon-child> and the mill recompiles everything for each cpu model (or, at least, purports to)
08:11:00 <moon-child> hell, gpus compile shaders to a uarch-specific format
08:11:00 <moon-child> it's not like there's no precedent for runtime recompilation
08:11:00 <dh`> it's just expensive
08:12:00 <dh`> not so much from a runtime perspective as from the "stuffing a compiler into your kernel" perspective
08:12:00 <moon-child> how frequently do you install(/upgrade) new apps?
08:12:00 <dh`> anyway I know you know about tendra and TDF so I won't belabor the point
08:13:00 <dh`> I recompile my kernel periodically, I'm not the target audience
08:13:00 <moon-child> sure
08:13:00 <geist> yah i think for fuchsia we can tolerate a few different kernels, since it's a microkernel and there are literally only one or two variants of a binary
08:14:00 <geist> can even boot time select them if we didn't mind the space
08:14:00 <moon-child> also: package repos can ship packages for different feature levels. (Something like that would be nice to have for x86 too, sse/avx2/avx512 at least. But no distros have done that yet)
08:14:00 <geist> tat'st he more interesting thing yeah
08:15:00 <moon-child> but I still think the system should be capable of recompiling by itself. (What happens if you swap out your hardware? Iirc windows will choke on that.)
08:16:00 <geist> tis true though i;m not sure that's so practical on things on batteries or constrained memory
08:16:00 <geist> especially if the optimal compliation is a very complex language or LTO style codegen that is generally assumed to be the case nowadays
08:16:00 <zid> C++ being involved makes that untenable, takes hundreds of gigs of ram to use a C++ compiler to produce binaries that don't suck :p
08:17:00 <moon-child> hmm phones are kinda different though, it's an soc that's not really expected to be upgradable or swappable
08:17:00 <geist> i was going to pick on rust for it too since it's kinda similar
08:17:00 <Belxjander> moon-child: windows chokes when the "core drivers" have to deal with changed hardware... because it assumes the currently set graphics driver matches the card in use and doesn't detect for changes until AFTER core drivers are already in-use due to the loader order of launch IIRC
08:17:00 <moon-child> that said you should at least be able to do a crappy compilation, just enough to get up and running and then download an optimized version from repos
08:17:00 <moon-child> Belxjander: right
08:17:00 <zid> yea it has like.. 'platform' drivers
08:17:00 <zid> so changing your ahci controller etc is just a no-go
08:18:00 <zid> aka a mobo swap
08:18:00 <Belxjander> moon-child: if you reset a "core driver" back to a generic default driver... you can then change the hardware after that... and update the driver again to fixup acceleration features
08:18:00 <moon-child> Belxjander: makes sense. (Also kinda dumb that the core gfx driver isn't always generic but ¯\_(ツ)_/¯)
08:19:00 <Belxjander> zid: yeah... motherboard chipset, CPU feature accelerations and graphics are all in the "core driver" listing as well... Windows can't setup GDI/Video handling without a Display Driver which breaks the kernel graphical and UI handling... which is something optional on practically every other OS I am aware of
08:20:00 <dh`> realistically for most users and most situations, if you get new hardware you reinstall the OS
08:20:00 <zid> nah you phone microsoft
08:21:00 <zid> and they give you a new activation on your licence and tell you what to do
08:21:00 <dh`> once you move out of that territory it gets a lot less clear though
08:21:00 <Belxjander> moon-child: all graphics drivers in windows have to provide a basic "DisplayDriver" model API presence and then add on top of that... if you have 2 display drivers... then they have to be setup with one as default (primary) as kernel binding at launch
08:21:00 <zid> (this is the actual advice microsoft used to give, idk if it's still true)
08:21:00 <Belxjander> zid: they don't do the phone thing anymore... its re-run setup and have it go through troubleshooting...
08:22:00 <Belxjander> zid: it then "fixes" the machine setup locally for drivers and stuff... with LOTS of restarts
08:22:00 <dh`> e.g. in an ideal world you'd hotswap a new cpu board in and the system would detect it, recompile whatever it needed to, and chug along
08:23:00 <dh`> even if the new cpu board contained hw that didn't exist when the original machine was installed
08:23:00 <Belxjander> dh`: in an ideal world... the essential core would work as a generic model and everything specific would be optional addons to that core...
08:23:00 <dh`> nah because you want to be able to unplug and throw away the old cpu board eventually
08:23:00 <Belxjander> dh`: which would allow for the system to start up... and then detect the addons specific to the hardware in actual use
08:24:00 <dh`> and you shouldn't have to shut down ever, either :-)
08:24:00 <zid> Ideal world for me is I just boot the old OS, build a new kernel targetting the new cpu, then swap my grub config to the new kernel, reboot with the new hw
08:24:00 <zid> and I keep a GENERIC_AF_KERNEL in reserve that can boot anything
08:26:00 * Belxjander actually did work up a generic as hell "Launch Anything..." kernelling setup based on selecting "x86 32bit" or "x86 64bit"... basically architecture selections at boot and little else... everything after that was modularized where possible with a minimal essential to boot subset of drivers in-kernel-image
08:27:00 <Belxjander> basically enough to load drivers from disk and little else for the kernel essentials driver wise
08:27:00 <zid> It only works for somewhat generic setups though
08:27:00 <zid> glhf if you have encrypted disks in a raid or whatever
08:27:00 <zid> kernel needs to know a lot about your setup for that to work
08:27:00 <Belxjander> managed to get it capable of booting about 15-16 machines all from the same installation actually with unique hostnames and setups on all of them too...
08:28:00 <Belxjander> zid: yeah... I didn't have that for bootable requirements but did have support for post-launch scripted binding of such storage media
08:29:00 <zid> net.ifnames kernel option is required for my OS to boot right :P
15:17:00 <klys> is anyone here familiar with qemu -display qubes-gui,domid=0,log-level=0; ?
16:18:00 <Bitweasil> Playing with the gui domain?
16:19:00 <Bitweasil> (no, sorry, still learning Qubes and deciding if it's worth the hassle)
16:21:00 <zid> what even is qubes
16:22:00 <Bitweasil> Annoying? It's Invisible Things Labs "containers of isolation" OS, in which all the windows from various isolated VMs are brought together into a single combined display.
16:23:00 <Bitweasil> So you can have an untrusted VM for random browsing, a personal VM for email/etc, and other isolated domains running, all separated by Xen.
16:32:00 <Bitweasil> The concept is solid, but I don't know how much it gains you if you use multiple different machines and some are all combined system image stuff.
16:33:00 * Bitweasil waits for his office to warm.
16:34:00 <klys> so I can start my vm with `virsh -c xen:/// start vm-name' and then it appears to be running (hey, I retrieved the ivt from qemu); now, there is no display visible.
16:34:00 <Bitweasil> Wouldn't #qubes be a better place for gui domain discussion?
16:35:00 <klys> ok I get
16:35:00 <Bitweasil> Mostly because I don't think many people here actually usei t...
16:36:00 <klys> yeah well most of us know qemu and can read source
16:36:00 <Bitweasil> *shrug*
17:04:00 <gog> i can't read at all
17:04:00 <zid> None of those signs ever stop gog afterall
17:04:00 <gog> exactly
17:05:00 * kingoffrance throws gog a no-reading-required fish o<
17:06:00 * gog eats fishy
17:08:00 <gog> i feel like a fish rn
17:08:00 <blockhead> cold and wet?
17:08:00 <gog> yes
17:09:00 <zid> I was thinking poor long term memory and craving aniseed pop-up boilies
17:10:00 <gog> that too
17:10:00 <zid> is there any fishing in iceland, other than in the sea
17:11:00 <sham1> Where else would you fish on a volcanic island
17:11:00 <Bitweasil> I would assume there are lakes and freshwater sources somewhere on it.
17:12:00 <zid> I don't know enough about iceland to know
17:12:00 <zid> some places.. don't have lakes, and if they do, fish are optional
17:12:00 <Bitweasil> People live there, so I'd assume that it has some freshwater, somewhere.
17:12:00 <zid> birds spread fish around, but there have to be freshwater fish /somewhere/ for them to do that with
17:12:00 <zid> I know they have geothermal vents, and probably a brook or two
17:13:00 <gog> oh yeah fly fishing for trout is a big tourist draw here
17:13:00 <gog> there are many well-populated lakes and streams
17:13:00 <zid> fair enough
17:13:00 <zid> There's about 400 carp lakes per square mile where I live
17:14:00 <gog> and it rains 300 days a year so fresh water is plentiful. also the glaciers
17:14:00 <zid> we're below sea level so the water doesn't drain out
17:14:00 <gog> whoops
17:14:00 <zid> and the 'soil' is actually clay anyway
17:15:00 <zid> so you can just dig a hole, fill it with water, and add fish
17:15:00 <Bitweasil> Heh.
17:15:00 <gog> nice
17:15:00 <Bitweasil> Done that here... without the fish.
17:15:00 <Bitweasil> "Hrm. That hole hasn't drained in three days. No, I'm not going to put a tree there."
17:15:00 <zid> no liners or plants or need to fix the soil etc
17:15:00 <zid> literally just dig
17:15:00 <zid> you can have shear cliffs if you like
17:16:00 <zid> there's an open face clay mine in my town
17:16:00 <zid> (they make bricks)
17:17:00 <gog> nice
17:17:00 <gog> masonry is based
17:18:00 <zid> I assume icelandic houses are made of fish and moss
17:18:00 <gog> mostly
17:18:00 <gog> the tallest building in iceland is actually just an upturned boat
17:19:00 <zid> I just got done playing a fun game, "This list removal hangs if you select more than so many elements, what clever search terms will return fewer items so you can slowly delete them all?"
17:19:00 <zid> I thought the tallest building in iceland was when someone dropped an ice-cream cone
17:19:00 <Bitweasil> dd if=/dev/zero of=/dev/sda bs=1M works. ;)
17:19:00 <Bitweasil> Warning, may have other side effects.
17:20:00 <gog> i actually did that to an NTFS partition once
17:20:00 <gog> totally by mistake
17:20:00 <gog> i must've ovewrote the MFT index because i couldn't recover anything
17:24:00 <Bitweasil> Yeah, I'm pretty careful with dd, lsblk beforehand, etc.
17:25:00 <gog> a hard-learned lesson for me
17:26:00 <sham1> There's a reason why it's often known as "Disk Destroyer"
17:27:00 <Bitweasil> That's most of what I use it for.
17:27:00 <Bitweasil> Wiping drives that blkdiscard doesn't work properly on.
17:27:00 <kingoffrance> eh, its useful if one is imaging. zero first, then install OS/whatever. then make image -- if this was a priorly-used disk, much smaller images
17:27:00 <Bitweasil> You can do that after the install a lot more safely, though.
17:27:00 <Bitweasil> dd if=/dev/zero of=bigfileofzeros bs=1M
17:28:00 <Bitweasil> And then delete that file.
17:28:00 <Bitweasil> So even if the install/update process has written and deleted stuff, you still zero that space.
17:29:00 <kingoffrance> thats worse, that makes it sound like you should do both
17:29:00 <Bitweasil> The second accomplishes the first.
17:29:00 <zid> https://www.audioasylum.com/forums/pcaudio/messages/11/119979.html
17:29:00 <bslsk05> www.audioasylum.com: RE: A revolution in audio rendering - SBGK - Computer Audio Asylum
17:30:00 <kingoffrance> maybe i should ask: does uefi store stuff on disks? maybe it is more of a danger to modern people ?
17:31:00 <kingoffrance> then i would agree
17:31:00 <kingoffrance> or "recovery partitions" or whatever
17:31:00 <Bitweasil> What's the context? There's an EFI boot partition, but most of the stuff EFI stores gets stuck in some nvram somewhere.
17:31:00 <kingoffrance> i just dont see how zeroing a disk that you are totally replacing is dangerous
17:32:00 <Bitweasil> It's not. The problem is when you type the wrong partition and then zero something important.
17:32:00 <Bitweasil> I'm not saying that zeroing before an OS install is bad.
17:32:00 <kingoffrance> ok fair :)
17:32:00 <Bitweasil> I'm saying, if you're imaging it, you should zero the free space in the partition *after* the install.
17:32:00 <Bitweasil> Because if the install has downloaded and deleted temp files, those may be on the partition, as non-zeroed data, but not actively used.
17:33:00 <Bitweasil> A "big zero file" after install will overwrite those with the zeros that are properly handled in imaging.
17:35:00 <zid> just image the fs itself, make an iso
17:35:00 <zid> rather than the drive
17:35:00 <zid> note: you may want to use tar, to preserve permission bits :P
17:36:00 <Bitweasil> Depends on what you're blowing it out to.
17:36:00 <Bitweasil> An image you can slap on SD is pretty nice.
17:36:00 <gog> what about rock ridge
17:37:00 * Bitweasil mutters something about that sounds like ISO stuff and handwave mumble...
17:40:00 <kingoffrance> these are not the dvds you are looking for
17:41:00 <Bitweasil> I used a DVD to install an OS recently on something that was UEFI only, but didn't support exFAT USB sticks in EFI.
17:41:00 <Bitweasil> It... worked.
17:42:00 <gog> i don't even have an optical drive lol
17:43:00 <Bitweasil> I have a couple BluRay burners that are USB3 and I use for random stuff, but this was an old laptop with a built in and still working optical drive!
17:44:00 <zid> I don't even have an picture of an optical drive
17:44:00 <gog> wow an artifact
17:44:00 <gog> belongs in a museum
17:45:00 <Bitweasil> Hey, 100GB on an optical disc doesn't suck for backup. :p
17:45:00 <gog> true
17:46:00 <zid> it sucks because of the speed still though
17:46:00 <sham1> It's like tape in that regard
17:46:00 <sham1> And that's why it's backup
17:46:00 <zid> 100GB / 1MB/s = pain :p
17:46:00 <gog> wow LTO9 can carry 18TB
17:46:00 <Bitweasil> It's a whole lot faster than 1MB/s. And, backup. :p
17:47:00 <Bitweasil> A stack of those backs up a lot of hard to find media.
17:47:00 <gog> :|
17:48:00 <Bitweasil> *shrug* Or buy an external 5TB USB drive, either way.
17:48:00 <Bitweasil> I don't mind having optical media burners around.
17:49:00 <Bitweasil> I did discover I lack blank CDs, though. :/
17:49:00 <zid> I need a cd burner and a fat stack of CDs
17:49:00 <Bitweasil> My daughter has a CD player and I was thinking of getting her some stuff burned to audio CD.
17:49:00 <gog> i never did finish the last spool of CD's i bought
17:49:00 <zid> so I can debug code on a playstation
17:49:00 <zid> I came up with a dumb trick though so maybe I'd only need 1 disk
17:49:00 <zid> but all the assets to the disk, launch the exe by copying it into memory with the cheat cartridge
17:49:00 <gog> and some dodgy DVD+RW's where every other one failed
17:50:00 <zid> DVD+RWs are dodgey just cus barely anything could read em
17:50:00 <gog> true
17:50:00 <Oli> For measurement, I have read about floppy disk write speed to be around 100kbps and 250kbps.
17:51:00 <gog> i used to make backups with a floppy. and i had an onion tied to my belt, which was the style back in those days
17:51:00 <Bitweasil> It was the bees knees!
17:51:00 <sham1> I could probably buy a Blu-Ray drive for about 100 euros. I'd just have to find one that can also record
17:52:00 <Bitweasil> SHouldn't run you more than $60 or $70
17:52:00 <Bitweasil> I've got...
17:52:00 <Bitweasil> *looks*
17:52:00 <zid> I never made backups on floppies they were too small for anything I'd want to back up, like my goat porn
17:52:00 <Bitweasil> I have a Pioneer BDR-XD05S
17:53:00 <zid> slip of the tongue, GREATEST OF ALL TIME porn
17:53:00 <sham1> zid: Zip disks
17:53:00 <zid> phew
17:53:00 <Bitweasil> USB3, has an extra USB plug for moar power, and does everything I ask.
17:53:00 <zid> yea I've seen those weird double plugs
17:53:00 <zid> they're like.. semi fused together
17:53:00 <sham1> Siamese USB3
17:53:00 <gog> yeah makes it fun to use on a laptop with only one port on either side
17:54:00 <zid> https://img.joomcdn.net/dc802defe3f632689b21221d1f51382fe0366159_original.jpeg
17:54:00 <Bitweasil> If you have a USB3 port, usually it doesn't matter.
17:54:00 <Bitweasil> They supply enough power.
17:54:00 <gog> i'm not clicking that after what you said zid
17:54:00 <zid> it's the greatest of all time cable
17:54:00 <zid> not porn
17:54:00 <zid> ...unless you REALLY like cables
17:54:00 * gog sweats profusely
17:54:00 <Bitweasil> I hate those things. :/
17:54:00 <sham1> Does it also involve goats
17:54:00 <Bitweasil> micro-USB3.
17:54:00 <Bitweasil> A goat might eat it.
17:55:00 <Oli> Talking about PlayStation, one could debug code in, by using a exploit named as FreePSXBoot, and a computer communicating via serial port to the PlayStation port, using a software named as NOTPSXSerial: It features debugging and PS-EXE uploading options. One may build a PSX Serial cable end from a AV Multi Out cable, using the steps featured in the web page that the next hyperlink goes to:
17:55:00 <Oli> https://github.com/unirom/unirom.github.io/pull/10/commits/fe97ed7aab5a4efe3c3bdcf78e585154bcd27d4f?short_path=ddbb47d#diff-ddbb47d576bc2bc9f93d1e9c9c0ab4268f085b56d48f623aeaa63fdd7fddc3f6
17:55:00 <bslsk05> github.com: Adding documentation about making a half serial I/O port with a component AV Multi Out cable, to serial_psx_cable.md by Sharwil-Summoner · Pull Request #10 · unirom/unirom.github.io · GitHub
17:56:00 <zid> you don't need to use an exploit
17:56:00 <zid> the port on the back is on the bus
17:57:00 <zid> you can just fuck with the system in realtime, hence gamesharks
17:57:00 <zid> patch it to an infinite loop somewhere, rewrite memory with your payload, done
17:57:00 <Oli> That is using the parallel port!
17:57:00 <gog> for earlier systems with that port
17:57:00 <zid> the newer models sucked anyway (and the first model but hey)
17:57:00 <zid> imagine using a scph-75xx unironically
17:58:00 <gog> i know so little about playstation hacking
17:59:00 <zid> you're better off
17:59:00 <zid> it's a mess of a thing
18:00:00 <zid> gameboy is what you want, lovely lovely gameboy
18:00:00 <Oli> Its graphic coprocessor features a practical functions for drawing 3D graphics: It is very approachable.
18:00:00 <zid> It does not and isn't
18:01:00 <zid> It has no floating point, no perspective, etc. It's vastly inferior to the n64 in terms of accessibility, and those pale in comparison to 'literally anything else that can do 3d'
18:01:00 <zid> It's probably the hardest way to do 3D you can manage
18:02:00 <gog> i watched a vid about the guys that made crash bandicoot and all the stuff they had to do to eke out the most from the machine
18:02:00 <gog> like monkeypatching the dev libs
18:02:00 <zid> yea the sdk and kernel are both absolutely insane
18:02:00 <zid> the kernel code is all total shit, and the sony provided sdk even patches the shit out of it at boot
18:04:00 <gog> lol nice
18:04:00 <Oli> Granted, if without a custom, albeit costly approach to calculate floating point in software, it differs from serving as smooth 3D movement than the Nintendo 64 provides, however, its graphic processor unit serves an approachable set of commands to draw on the screen to me; one may read at a list and description of, in the web page that the next hyperlink goes to:
18:04:00 <Oli> http://www.problemkaputt.de/psx-spx.htm#graphicsprocessingunitgpu
18:04:00 <bslsk05> www.problemkaputt.de: Nocash PSX Specifications
18:05:00 <zid> all 2D.
18:05:00 <zid> All integer
18:05:00 <zid> You wanna link the GTE docs not the GPU
18:08:00 <zid> Then note they're about a page long and have 0 details :P
18:08:00 <Oli> Perhaps I am just content with it. The geometry transformation engine documentation in continues after it.
18:10:00 <zid> It's not horrific after you spend 40 hours writing the 1/3/12 fixed point code, and the dynamic tesselation engine etc
18:10:00 <zid> meanwhile on n64 you just submit 3D verts
18:12:00 <Oli> The Nocash psx-spx document as served me the best usage description of the platform graphic processing unit and geometry transformation engine of: I am feeling curious about if there is one describing them better.
18:14:00 <Oli> Sounds nice: I am unfamiliar yet with development on Nintendo 64 platforms: Thank you for sharing about, zid!
18:14:00 <zid> no, korth's docs are pretty much all there is
18:15:00 <zid> other docs are just spread around various emulators and stuff
18:49:00 <gorgonical> A good start to the week -- last night I hacked together an arduino with a temperature sensor to interact with my web power switch to upgrade my dumb on-off heater into a thermostat-controlled one.
18:49:00 <gorgonical> Not osdev-related, but good motivation to get some hypervisor work done today :)
18:52:00 <gog> :D
19:17:00 <Bitweasil> Nice!
19:18:00 <gog> i haven't really gotten anything done today code wise
19:19:00 <gog> i'm exhausted and would like a day off where i have no obligations or expectations to fulfill pls
19:20:00 <Oli> I have for long been debugging a feature of a sound engine; may today be the day I get it down. One step at a time!
19:29:00 <Oli> Sounds as a nice place to every now and then day off you describe, gog!
19:29:00 * geist yawns
19:29:00 <geist> good morning folks!
19:30:00 <jimbzy> Sup?
19:33:00 <Bitweasil> allo!
19:49:00 <heat> helloooo
19:49:00 <heat> belated happy new year
19:49:00 * gog passes bagels out all around
19:50:00 <heat> yo gog did you get the address allocation working?
19:50:00 <gog> almost there
19:50:00 <heat> i remember reading a question of yours about the rb tree
19:51:00 <gog> yeah i figured out the rbtree
19:51:00 <heat> the key is the key (address), the datum is a void * that's supposed to represent what's keyed
19:51:00 <heat> I didn't write that code and I honestly find it garbage
19:51:00 <gog> i actually did something slightly different
19:51:00 <gog> the key is the tuple (address, size) and i wrote a comparison function for it
19:51:00 <heat> yes
19:52:00 <gog> and i had a flash of inspiration that checking for overlaps is pointless because there are only two possible non-overlapping conditions
19:53:00 <gog> so yeah thanks :)
19:53:00 <heat> I'm way happier with https://github.com/tianocore/edk2-platforms/blob/master/Features/Ext4Pkg/Ext4Dxe/Extents.c (ctrl+f for OrderedCollection)
19:53:00 <bslsk05> github.com: edk2-platforms/Extents.c at master · tianocore/edk2-platforms · GitHub
19:53:00 <heat> it's such a well thought interface
19:53:00 <heat> the only issue with this one is that rb tree nodes aren't inline in the structure, which they should
19:54:00 <heat> for efficiency's sake
19:54:00 <heat> gog: np :)
19:56:00 <gog> so yeah now i have to actually finish writing some kind of heap so i can start really testing out vm allocation
19:56:00 <gog> and actually write the gap-finding algo
19:56:00 <gog> i have a bunch of diagrams i drew on paper of it tho
19:57:00 <heat> my heap doesn't use vm allocation
19:57:00 <heat> it's just a brk with a randomised base
19:58:00 <gog> mine doesn't directly, i need it to allocate the tree nodes :p
19:58:00 <heat> (correction: it's not a brk, but it uses one)
19:58:00 <gog> i did start writing an object pool type thing but it started looking like i just needed to actually write a heap instead of doing extra work to avoid it for the moment
19:59:00 <heat> also, warning: kernel unmaps are not fun
19:59:00 <heat> you should avoid them
19:59:00 <gog> yes
20:00:00 <heat> unmapping kernel memory means broadcasting a tlb shootdown over all the cpus
20:00:00 <heat> which isn't the case with user memory
20:00:00 <gog> yes\
20:01:00 <heat> do your vm regions have backing objects already?
20:04:00 <gog> anonymous vm regions do
20:04:00 <gog> and the only thing it contains is a pointer to the anonymous_fault_handler
20:04:00 <geist> would need them anyway to have something to stick into the tree
20:04:00 <geist> sounds like a good start
20:04:00 <gog> :)
20:05:00 <gog> yeah laying some good groundwork here
20:05:00 <gog> much further along than any of my prior experiments
20:05:00 <geist> yah the next phase after that is having the vm region point to a backing object
20:06:00 <geist> then having the backing object sharable by multiple vm regions
20:06:00 <gog> yes
20:06:00 <gog> then you have the basis for files
20:06:00 <geist> and voilá you have sharable memory
20:06:00 <gog> also that
20:06:00 <geist> or files, or basis for COW
20:06:00 <gog> shareable memory is going to be a necessary thing. in my mind it's a message-passing kernel
20:06:00 <geist> also you reduce the need to store the page mappings in the page tables
20:07:00 <geist> ie, the page tables aren't load bearing anymore and you can free memory by freeing the object (and unmapping of course)
20:07:00 <gog> yeah my end goal with that is to do what other kernels do and treat the page tables like a cache
20:07:00 <geist> but you know this of course, mostly exposition for folks following along
20:07:00 <gog> :)
20:07:00 <gog> i know all the theory
20:07:00 <gog> its putting it to code where i struggle :p
20:08:00 <geist> that's the critical part. once you understand the pieces that's a sizable chunk of modern VM design
20:08:00 <geist> or not so modern, same thing
20:08:00 <gog> what's old is new again
20:08:00 <heat> modern 1990s VM design
20:08:00 <geist> pretty much
20:08:00 <zid> disregard all that, just map things and forget about it, it'll get torn down eventually anyway. Life is fleeting, everything is meaningless.
20:08:00 <heat> didn't really change did it lol
20:09:00 <zid> Shared memory is a security issue, files are a pain
20:09:00 <geist> the only real shift i can think of is an idea i've had that i've been talking about at work with other VM folks is to completely shift everything to runs of pages, and then COW at the run level
20:09:00 <geist> ie, take something like how BTRFS handles vnodes -> file runs
20:09:00 <gog> existentialOS
20:09:00 <geist> and then treat it as vm objects -> runs of pages
20:09:00 <geist> and COW at the page level, instead of the object level like most systems do
20:10:00 <heat> isn't that kind of what the page folios patches do on Linux?
20:10:00 <heat> not sure if you've been following along
20:10:00 <geist> ie, COW a vm obect? do a straight clone of the object, no parent child relationship, and let the page level COW work
20:10:00 <geist> nope. i have not been following along on linux. mostly on purpose, to be honest
20:10:00 <heat> yeh I figure
20:10:00 <heat> GPL and all that
20:10:00 <geist> well and also you can't unsee things
20:11:00 <geist> not from a GPL point of view
20:11:00 <geist> mostly from a 'clouds your thinking' sort of view
20:11:00 <gog> yes
20:11:00 <gog> linux has many examples of how not to engineer software in my not-remotely-humble opinion
20:11:00 <heat> they're trying out a change where all the struct page * everywhere become a page_folio structure which holds N pages, and is part of their efforts into putting huge pages in the page cache and all that
20:12:00 <geist> oh that doesn't sound like what i'm thinking
20:12:00 <geist> i'm thinking far more fundamental. like all the page runs in the system are a bTRFS style btree with active copy on write
20:12:00 <heat> well, it doesn't hold, it just points to stuff like struct folio{ struct page *head, *tail; size_t nr_pages;};
20:13:00 <heat> oh
20:13:00 <geist> and if you clone a vm object it fundamentally makes a copy, but the underlying mechanism lets it lazyly COW pages individually
20:13:00 <geist> instead of doing the traditional shadow copy stuff
20:13:00 <geist> the fuchsia API design is in this direction so it sets up for it, sicne COWing a VMO is expressed simply as a 'clone this vmo'
20:15:00 <gog> rewinding slightly, that is one thing i was thinking about before wrt page tables is that relying on them for storing information about vm state feels fundamentally shaky
20:15:00 <heat> they do store information in my design
20:16:00 <GeDaMo> https://www.tesmanian.com/blogs/tesmanian-blog/starlink-cats
20:16:00 <geist> gog: agreed. for x86 you can basically get away with it if you're okay with burning lots of memory to keep page tables in ram
20:16:00 <geist> but there are arches where that's not the case
20:16:00 <heat> the most important bit is that if you know that mapping X's vm_region is marked VM_WRITE and mapping X has write disabled, mapping X is for some reason write protected
20:16:00 <geist> do they matter? probably not. linux chose the strategy of using the page tables to hold stuff. i call those 'durable' page tables
20:17:00 <geist> heat: yeah but that's precisely the kinda thing you can store in the vm region object itself
20:17:00 <geist> the permissions of the mapping
20:17:00 <heat> when I say mapping I mean mapping of a single page
20:17:00 <heat> you can't have single page granularity like that unless you allocate a separate data structure with all of that info
20:18:00 <geist> right
20:18:00 <geist> *generally* speaking the answer is you allocate a separate data structure with that granularity
20:18:00 <geist> since it's pretty rare to need to do single page permissions like that
20:18:00 <heat> Hm? It's pretty common
20:19:00 <geist> well, yeah, but not hyper aggressively
20:19:00 <heat> see COW, software dirty bit
20:19:00 <geist> ah yes but that can be stored inside the vm region/object heirarchy itself
20:19:00 <geist> hard to explain, but that's a function of how you set up the shadow objects and whatnot
20:20:00 <geist> or can be at least
20:20:00 <geist> what's the harder one is if you map say a 1MB RW anonymous object
20:20:00 <geist> and then go back and mark half of it as RO. do you split the mapping into two? (that's what zircon does currently)
20:21:00 <geist> then what if you mark every other page as RO? do you keep splitting the mapping? (yep currently)
20:21:00 <geist> most code doesnt' do that, except when it does
20:21:00 <heat> depends, if it's mprotect-like, that sure
20:21:00 <heat> if its kernel internal you may not need to
20:21:00 <Bitweasil> GeDaMo, yeah... saw that. lulz.
20:21:00 <geist> exactly. sadly there's some code that we deal with that hyper aggressively mprotects
20:21:00 <Bitweasil> My Dishy is up out of cat reach.
20:22:00 <geist> presumably because on other platforms it's no biggie
20:22:00 <geist> some security thing where its internal heap is mprotected constantly as objects are freed or whatnot
20:22:00 <heat> which platforms?
20:22:00 <geist> presumably linux/windows
20:22:00 <heat> afaik linux also splits regions
20:22:00 <heat> you can see it on pmap <pid>
20:23:00 <geist> that's what i thought too, but i guess they have an optimization there that makes it not too bad
20:23:00 <geist> as it currently stands zircon will just keep splitting regions, and coalescing them later
20:23:00 <geist> but it gets bad since you might end up with 10k regions
20:23:00 <geist> or whatnot, depending on how aggressively they mprotect their internal stuff
20:24:00 <geist> the obvious solution is to have some sort of sub-region mprotect overlay tree
20:24:00 <zid> sounds like a job for a mmap flag
20:24:00 <geist> where it doesn't have to split the region object, but the regio object itself has some way to mark runs of itself as having lesser permissions than the mapping in the first place
20:24:00 <zid> MMAP_SKIP_THE_REGION_SHIT_THIS_IS_GUNNA_GET_WILD_YO
20:24:00 <heat> so log2(10K) = ~14 lookups in the worst case
20:24:00 <geist> since we already have those permissions: if you map a RO region you can't mprotect it to RW later
20:24:00 <heat> not horrible
20:24:00 <geist> but you can take a RW region and mprotect it to RO
20:25:00 <geist> heat: no but enough to show up
20:25:00 <heat> is that scudo paranoia?
20:25:00 <gog> yeah my thinking on that is that a vm region should only be created or deleted, never split or resized. put the policy in the backing object
20:26:00 <geist> heat: i dunno. i think its in chromium
20:26:00 <geist> they have their own heap, etc
20:26:00 <heat> >they have their own heap
20:26:00 <geist> or whatever the chrome engine is
20:26:00 <heat> fuck
20:26:00 <geist> oh yeah that whole blob of code is already a huge pile of internal stuff
20:27:00 <geist> heat: but you're right, it does beg the question: *why* does it matter? we store the region list in a WAVL tree. it should be log2
20:27:00 <geist> oh oh now i remember. it's nto the tree, it's reverse lookups that are slow
20:27:00 <geist> object -> regions is now a 1:N mapping
20:27:00 <geist> which is inefficient currently
20:28:00 <heat> but they're always 1:N no?
20:28:00 <heat> at least in the design
20:28:00 <geist> since the code is not currently written to assume there is an excessive number of overlapping mappings of objects
20:28:00 <heat> ah yes
20:28:00 <geist> rght and there are a bunch of O(N) operations from object back to all its mappings
20:28:00 <geist> specificaly page mapping shootdowns when doing COW, etc
20:29:00 <heat> why RO instead of unmapping the pages? that would halve the N
20:29:00 <geist> ie, i map an object 1000 times, then if i deallocate a page in the middle of it i have to O(N) through all the mappings and make sure the page isnt mapped
20:29:00 <geist> what do you mean RO?
20:30:00 <heat> why does the chromium heap (supposedly) mprotect address space as read-only instead of unmapping it
20:30:00 <geist> i have no idea.
20:30:00 <heat> why does it even do anything like this
20:30:00 <dh`> garbage collector
20:30:00 <geist> yah possible it's in the javascript heap, indeed
20:30:00 <geist> it does heavy shenanigans with pages
20:30:00 <geist> also likes to manually deallocate pages, etc
20:31:00 * heat wonders how portable chromium and firefox are
20:31:00 <geist> both of those are bad in the zircon VM right now: creating lots of mappings of a single anonymous object + lots of page deallocates (creates a O(N) walk through all the mappings)
20:31:00 <dh`> not much you can do to avoid that
20:32:00 <geist> yah i dunno precisely how linux/windows avoid such a complex thing, but maybe they understand that there are a ton of mappings that dont overlap
20:32:00 <geist> and thus it's not a O(N) walk anymore
20:32:00 <geist> but a log2 walk from object -> mapping to find which one intersects
20:33:00 <geist> our code assumes that all mappings are equally overlapping so it has to walk through every one. it's simply a linked list of all mappings attached to an object
20:34:00 <geist> anyway, fun things to solve. just VM stuff you get into when you start running bigass code on it
20:34:00 <heat> I know linux has a bunch of reverse mapping code
20:34:00 <heat> it's not just a simple linked list of mappings
20:35:00 <geist> yah probably some sort of more clever algorithm to avoid this
20:35:00 <geist> also i guess if you had a per page -> mapping reverse lookup you could drill into the object, iterate per page and then walk back to all the mappings
20:35:00 <geist> i know the BSDs have traditionally stored that
20:38:00 <gog> yeah struct pmap
20:38:00 <gog> or something
20:38:00 <geist> yah
20:45:00 <heat> I see
20:45:00 <heat> I found out how linux does it
20:45:00 <heat> dunno if you wanna hear it, since GPL2 and whatnot
20:46:00 <geist> it's not GPL2 that i'm worried about
20:46:00 <geist> what do they do in this case?
20:48:00 <heat> for anonymous mappings, they essentially maintain a red-black tree of (start_mapping, end_mapping) for each anon backing object
20:48:00 <heat> it's all object-based reverse mapping, not per-page
20:49:00 <geist> ah and they can be sure they dont overlap because tey have a different notion of shared vs anoymous mappings
20:49:00 <heat> I was reading a lwn article from 2003 where it said they found per-page reverse mapping really slow and with large overhead
20:49:00 <geist> IIRC. a shared mapping uses different plumbing
20:49:00 <heat> I think shared mappings use internal tmpfs? not sure
20:50:00 <geist> ah so that's a fundamental difference: zircon has no concept of shared vs unshared mappings. mappings are simply mappings
20:50:00 <heat> non shared mappings can also be shared
20:50:00 <heat> with COW
20:50:00 <geist> it's very flexibe, but.... in this case it reduces the chance of optimization because you can't tell if N mappings of an object are overlapping or not since there's no distinction between mapping the same object N times
20:50:00 <geist> hmm, so then how does their red/black tree handle that?
20:50:00 <geist> does it point back to an intermediate mapping which then broadcasts out to multiple ones?
20:51:00 <sham1> Could be a special bit in the actual backing object
20:51:00 <sham1> Not sure though
20:52:00 <heat> it iterates through the tree
20:53:00 <heat> they have some code where it can look at the subtree [lower_bound, upper_bound]
20:53:00 <geist> or coudl be their red/black tree allows multiple overlapping mappings
20:53:00 <geist> yah okay
20:54:00 <heat> I don't see them calling a generic rb_tree_insert() so I imagine this is all hacky
21:12:00 <zid> wow, did you see ingo's new patchset?
21:14:00 <j`ey> 2,000 commits later...
21:14:00 <zid> ingo's new patchset is 22000
21:14:00 <zid> :D
21:15:00 <gog> jesus
21:17:00 <zid> sorry, 25000
21:17:00 <zid> 178,024 insertions(+), 74,720 deletions(-)
21:18:00 <j`ey> 25,000 changed files
21:21:00 <geist> what was it? major refactor?
21:21:00 <j`ey> header file cleanup
21:21:00 <zid> loots of header cleanup
21:21:00 <zid> which meant basically every file needed its include list fixing up too
21:22:00 <j`ey> revamping headers a ton to stop having huge headers
21:22:00 <j`ey> one example was that a preprocessed pid.c went from 90k lines to 30k lines
21:22:00 <GeDaMo> https://lwn.net/ml/linux-kernel/YdIfz+LMewetSaEB@gmail.com/
21:22:00 <bslsk05> lwn.net: [PATCH 0000/2297] [ANNOUNCE, RFC] "Fast Kernel Headers" Tree -v1: Eliminate the Linux kernel's "Dependency Hell" [LWN.net]
21:22:00 <geist> cool
21:22:00 <zid> mainly centered around the task struct stuff, by the looks of it
21:22:00 <zid> lots of things were including task.h to access some random task detail
21:22:00 <j`ey> yeah, that stuff is kinda nasty, using untyped bytes and stuff
21:22:00 <zid> which then recursively included EVERY task related header
21:23:00 <j`ey> (the new stuff is nasty, but it makes sense)
21:23:00 <zid> yea it's a shame he couldn't come up with a better workaround for that
21:23:00 <j`ey> the new way is to just attach a giant pile of bytes at the end of the task_struct, and then have accessors with unique offsets into that
21:24:00 <zid> I'd love to see a testcase for exactly why it ends up being necessary
21:24:00 <zid> I assume it's just that
21:24:00 <zid> struct task { struct bob a; struct bar b; struct foo c; .. } ends up being the task def
21:24:00 <zid> which means you need to know sizes, and thus recursively include headers
21:25:00 <geist> yah have had some recursive header problems in zircon basically concerning the same header (kernel/thread.h)
21:25:00 <geist> since ltos of stuff has anchors in the thread struct you get a recursive problem where thing can't be easily declared in header foo.h and also have foo.h include thread.h
21:25:00 <zid> I wonder if you could somehow mechanically generate 'naked' versions of all the structs and have ghost headers
21:26:00 <geist> subsystem foo has a struct it wants to put inline to the thread struct, basically
21:26:00 <zid> so turn that into struct task { struct { int, int, char, char, int; } anon; struct ... };
21:26:00 <zid> you might be able to generate it once ahead of time in the build scripts if you carefully include guarded each header, idk
21:39:00 <catern> I vaguely recall some operating system where environment variables are thread-local, anyone know what I'm thinking of?
21:40:00 <geist> not that i know of, possibly VMS
21:40:00 <geist> but it's environment variables are a bit more complex than unixy ones
21:47:00 <sham1> It technically breaks POSIX semantics, but it'd still be an interesting exercise
21:49:00 <zid> free yourself from the yoke of posix
21:49:00 <dh`> geist: hmm I figured you meant O(n) of the n reverse mappings of the same page, not all mappings
21:50:00 <geist> to do the former yo need per page mapping tracking
21:50:00 <geist> which of course may be the answer
21:51:00 <catern> (what I even more would like to do is support full-fledged function-level dynamic scope for environment variables - so I can bind an environment variable within a single function call and have it unset when I return from that call - and also have this work with setjmp/longjmp too. but I'm not sure how to do that...)
21:51:00 <catern> (so thread-locals are a first step)
21:52:00 <dh`> per-page mapping tracking seems necessary if you ever invalidate pages for eviction
21:53:00 <dh`> or even for "let's mark this inactive now so we can evict it later"
21:57:00 <geist> not necessary, just potentially more optimal
22:05:00 <heat> https://lwn.net/Articles/23732/
22:05:00 <bslsk05> lwn.net: The object-based reverse-mapping VM [LWN.net]
23:11:00 <geist> ah yeah. that makes sense. it's very old, but that's not fundamentally different from what zircon does
23:11:00 <geist> but i suspect it's out of date given that it's nearly 20 years old
23:12:00 <heat> vm, vm never changes
23:12:00 <heat> ;)
23:41:00 <gog> mew!