channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev2&y=21&m=10&d=15

Friday, 15 October 2021

00:02:00 <devcpu> kazinsal: reread it twice and did not get it =) you scp iso, then power cycle the vm? then vm loads new kernel from tftp?
00:09:00 <klange> ESXi host is presumably running the tftp server and just has a 'reload' command.
00:10:00 <klange> My only experience with ESXi is from my uni days when our ACM student chapter migrated a rack of a dozen servers to a single Dell blade running ESXi, which marked the end of our days of having our own server room (which we mostly used as a refigerated storage for soda).
00:32:00 <kingoffrance> parsed output: ( instead of [...], I just [...] ) there's 2 branches there :)
00:34:00 <kingoffrance> or is it one branch, two twigs? 2 paths
01:31:00 <kazinsal> devcpu: my old method is to power the VM off (because having the VM on locks the .iso file), scp over the new .iso file, start the VM again, and connect to the serial console over TCP from PuTTY
01:32:00 <kazinsal> the serial console that is my OS's, not ESXi's
01:32:00 <kazinsal> the new method is to just add "update-tftp" to my build command and then punch "reload" into the serial console of my OS to reboot from TFTP
01:36:00 <kazinsal> update-tftp copies the new boot images to the TFTP root, archives the old ones, and updates grub.cfg to point to the new ones
01:40:00 <clever> kazinsal: ive been using the netboot support in the rpi for similar reasons, i dont have to constantly re-flash the sd card
01:40:00 <klange> I've got all my stuff ready, I need to set up a development workflow for that...
01:41:00 <klange> Got my Pi, got the TTL-to-RS232 serial thing going (one of my adapters was DOA, but the package I bought came with two, and the other one worked)...
01:44:00 <clever> for the whole pi0-pi3 lineup, an sd card with only bootcode.bin will let it network boot with the latest version of that code
01:45:00 <clever> with the pi3, the bootrom also can do netboot if you set an OTP fuse, but that code may be older and buggy
01:45:00 <klange> This is a Pi400.
01:45:00 <clever> pi400 has all of that code in spi flash, and you just enable it with the BOOT_ORDER
01:46:00 <clever> the spi config on the pi400 also lets you set the tftp server ip
01:46:00 <clever> so the dhcp server doesnt have to agree to things
01:48:00 <clever> that tftp dir, needs all of the firmware files you would normally have had on the fat32 partition
01:52:00 <devcpu> kazinsal: i see, thanks!
02:01:00 <kazinsal> interesting. never actually looked at the e820 map on my ESXi VMs before but there's exactly one 4096-byte entry after the last usable RAM entry marked as bad memory
02:08:00 <[itchyjunk]> Someone told me a GPU "basically runs it's own OS" and this has been bugging me a bit
02:09:00 <Mondenkind> [itchyjunk]: your [intel] CPU runs its own OS, too!
02:09:00 <[itchyjunk]> right, but CPU already has an OS and in my head, CPU told GPU what to do etc
02:09:00 <[itchyjunk]> But if GPU ran it's own OS, my model of the world is shattered and i am back to square one
02:09:00 <Mondenkind> no, the CPU has two OSes
02:10:00 <Mondenkind> one of its own and one that came with the CPU
02:10:00 <[itchyjunk]> @_@
02:10:00 <[itchyjunk]> how is that possible?
02:10:00 <kazinsal> GPUs can run code independently of the host ass soon as the code is uploaded to them
02:10:00 <Mondenkind> ¯https://en.wikipedia.org/wiki/Intel_Management_Engine
02:10:00 <Mondenkind> https://en.wikipedia.org/wiki/Intel_Management_Engine
02:10:00 <kazinsal> the hard part is uploading the code to the host
02:10:00 <Mondenkind> it runs minix
02:10:00 <[itchyjunk]> oh the one that starts it up and does the basic input out put test?
02:10:00 <kazinsal> er, to the GPU
02:11:00 <Mondenkind> [itchyjunk]: also see microsoft's recent thing, directstorage. GPU can talk directly to your ssd over pcie, decode textures without getting the cpu involved
02:11:00 <[itchyjunk]> Okay, CPU has this basic input output test stuff, sure GPU doesn't get its own?
02:12:00 <[itchyjunk]> GPU can talk to my storage without taking to the CPU? that sounds .. hmm
02:12:00 <Mondenkind> bios most likely comes as part of the motherboard firmware, not the CPU. But yeah GPU has its own firmware
02:12:00 <Mutabah> BIOS/firmware is different to the SMM/IME code
02:12:00 <[itchyjunk]> the more i think i learn, the less i seem to know
02:12:00 <Mondenkind> whether you want to call the code that runs on the GPU an OS or not is...ehh it's iffy
02:13:00 <[itchyjunk]> is this "model" of GPU being it's own thing, CPU being it's own thing etc related to high performance computing model?
02:13:00 * [itchyjunk] has been curious about HPC ever since overhearing some conversation on the trian
02:14:00 <[itchyjunk]> since HPC is a lot of different CPU talking to each other right?
02:15:00 <kazinsal> I'd hesitate to call anything running a GPU an OS since it's not going to be providing I/O services etc
02:15:00 <[itchyjunk]> and here, we have GPU / CPU taking to each other in the same way
02:17:00 <Mutabah> I'd guess (with minimal evidence) that modern GPU drivers upload a small kernel/OS that handles scheduling of GPU tasks
02:17:00 <Mutabah> e.g. if you're running a 3D application along with a 3D-accelerated window manager
02:18:00 <[itchyjunk]> It's hard to believe humans have figured out this level of complicated stuff
02:18:00 <[itchyjunk]> and i was even using it blissfully unaware!
02:28:00 <kingoffrance> "It's hard to believe humans have figured out this level of complicated stuff" https://qwantz.com/index.php?comic=2406
02:30:00 <clever> [itchyjunk]: also of note, the gpu on the rpi is running threadx, an os MS created, https://github.com/azure-rtos/threadx
02:30:00 <kazinsal> we figured out how to get sand to do trillions of computations per second and then used it to burn down the planet generating tokens used for money laundering and illegal pornography
02:31:00 <Mondenkind> :D
02:32:00 <[itchyjunk]> you can't design modern CPU's by hand? even with a lot of smart people in the same room?
02:32:00 <kingoffrance> i dont know, dinosaur says no
02:33:00 <[itchyjunk]> learning about computers is a good way to have existential crisis
02:33:00 <[itchyjunk]> My idea of OS is too narrow i guess
02:33:00 <kingoffrance> it was a supposed cray joke
02:33:00 <Mutabah> You could probably design a modern-ish CPU by hand, but not a fast one
02:34:00 <kingoffrance> apple just bought a cray to design their new computer. cray just bought an apple to replace his pen and paper
02:34:00 <Mutabah> On the other hand, bootstrapping is interesting
02:34:00 <clever> pipelines, prefetch, jump prediction, those all make things faster, and much more complex
02:34:00 <kazinsal> bootstrap a new CPU by bribing Jim Keller
02:36:00 <[itchyjunk]> bootstrapping.. i came across that concept when i asked about compilers. who compiles the compiler type thing..
02:36:00 <[itchyjunk]> so for cpu's , i guess it's who starts the os that starts the computer?
02:36:00 <kazinsal> who hand assembles the assembler
02:36:00 <[itchyjunk]> is that how you make the first one?
02:37:00 <kazinsal> though these days if you're designing a CPU you can just simulate the instruction set on an existing OS
02:37:00 <clever> [itchyjunk]: https://doc.coreboot.org/soc/amd/family17h.html
02:37:00 <[itchyjunk]> i guess there is no other options
02:37:00 <clever> [itchyjunk]: the PSP (amd's version of management engine) starts first, brings ram online, and copies the bios from flash->ram, then enables the x86 cores
02:37:00 <kingoffrance> previously computers were people (look up in dictionary, or "when computers were people"), but thats getting philosophical.
02:38:00 <clever> [itchyjunk]: the bios on the x86 core then brings the rest up (pci-e and such), then loads the bootloader from your boot disk, and the bootloader loads the os
02:38:00 <kingoffrance> i mean, you can argue hand toggling is still one computer starting another :)
02:39:00 <[itchyjunk]> is the PSP considered OS?
02:39:00 <[itchyjunk]> and the GPU would have a PSP equivalent that performs something similar
02:39:00 <clever> [itchyjunk]: i think it is running its own os, and the code must be signed by amd, so you cant modify it
02:41:00 <clever> [itchyjunk]: https://doc.coreboot.org/soc/amd/family17h.html#basic-pre-x86-boot-flow
02:41:00 <clever> there is both a PSP rom, and that validates what is in flash, and then executes that
02:45:00 <[_]> dang it, computer is dying i think
02:45:00 <[_]> so nothing is needed for this PSP to get started. as soon as electricity is coursing through the veins of a computer, PSP starts doing it's job
02:46:00 <[_]> immutable firmware is to protect the user or companies secret sauce?
02:46:00 <clever> [itchyjunk]: possibly both, its also part of the trust chain for secureboot
02:59:00 <kazinsal> both, but honestly even if you had enough evidence that it was protecting the user most foss nerds would still think you're worse than the KGB for defending it
03:31:00 <[itchyjunk]> ah, are there system where such a blob doesn't exist?
03:31:00 <[itchyjunk]> riscv doesn't need such a blob?
03:34:00 <kazinsal> risc-v just specifies the ISA
03:35:00 <kazinsal> the internals of the CPU are up to the designer/manufacturer
03:36:00 <[itchyjunk]> ahhh
03:39:00 <[itchyjunk]> i guess this would make those "foss nerds" even madder
03:41:00 <kazinsal> I have interest in RISC-V but it's probably going to be another five years before off-the-shelf hardware reaches the "couple hundred bucks on amazon.ca" point
03:42:00 <[itchyjunk]> i thought arduino equivalent of riscv things already existed
03:42:00 <[itchyjunk]> riscv microprocessors is the word i guess?
03:43:00 <kazinsal> production volumes are pretty low is one of the problems
03:44:00 <kazinsal> according to mouser they've got 46 HiFive Unmatched boards in stock, with another 360 expected 2022-02-11 and another 720 expected 2022-07-06
03:45:00 <kazinsal> unfortunately they're also $700 USD or thereabouts
03:46:00 <kazinsal> PC Engines has no stock of *any* of their apu series boards, which deeply sucks... expected availability is a nebulous "2022"
03:46:00 <klange> Bit more pricey than exploring ARM with a Pi...
03:46:00 <kazinsal> though the AMD GX-412TCs in those are getting quite long in the tooth anyways
04:05:00 <clever> [itchyjunk]: in the case of the rpi, the maskrom can be dumped and decompiled, and the 1st stage it loads is on SD card for most models, so it can be replaced
04:06:00 <[itchyjunk]> ah, interesting
04:07:00 <clever> [itchyjunk]: i have been investing a lot of time into the open pi firmware, and ive re-created enough features that you can boot a pi2 to an xfce desktop, with ntsc video out
04:08:00 <[itchyjunk]> hmm sounds complicated
04:08:00 <[itchyjunk]> so you've been reverse engineering some boot up blob?
04:08:00 <clever> yeah
04:08:00 <[itchyjunk]> do you need to just look at the blob on some other computer or do you poke at the rpi hardware itself?
04:08:00 <clever> bit of both
04:09:00 <[itchyjunk]> oh very interesting
04:09:00 <clever> the 2d pipeline is entirely undocumented, but has linux drivers
04:10:00 <[itchyjunk]> i imagine its similar to what a neuroscientist would do. see some signal and try to figure out what that signal does to the hardware etc
04:10:00 <clever> [itchyjunk]: with a baremetal arm kernel, looking at the linux source, and letting the closed firmware initialize the system, i can do https://www.youtube.com/watch?v=JFmCin3EJIs
04:11:00 <[itchyjunk]> there should be one logo but you can get multiple to appear?
04:11:00 <clever> its fully hardware accelerated sprites
04:11:00 <clever> so i can move them around with zero cpu cost
04:11:00 <clever> you could then use that to make any 2d sprite based game
04:11:00 <[itchyjunk]> pi comes with a "hardware accelerator" which just means another processor?
04:12:00 <clever> this is a dedicated core, specifically designed to only do sprite rendering
04:12:00 <[itchyjunk]> ah
04:12:00 <clever> it will parse the list of sprites, fetch image data from ram, scale the image data, alpha blend the layer stogether, and generate 1 scanline of image
04:12:00 <clever> then repeat, forever
04:13:00 <clever> https://www.youtube.com/watch?v=u7DzPvkzEGA is the result of reading the linux source more, i can now bring the ntsc generator online, and repeat the same demo, without the closed source firmware being involed
04:13:00 <geist> kazinsal: at some point when i still had esxi running i figured out how to create a shared disk image between two vms
04:13:00 <geist> could then have VM A directly write to VM Bs hard drive
04:14:00 <kazinsal> yeah, that's a thing you can do. never actually found a *use* for it mind you
04:14:00 <geist> well this whole reboot the OS in test
04:14:00 <clever> [itchyjunk]: https://i.imgur.com/Yeh6wEr.jpg is then the next logical step, just doing that as fast as possible, 0.61 seconds from power-on to video output, and a working command prompt
04:14:00 <kazinsal> ah yeah I guess that would work, use one VM to write to the other
04:14:00 <geist> right
04:14:00 <geist> and you could even get the serial port of one routed to the serial of another
04:14:00 <[itchyjunk]> wait so you can completly talk to the hardware accelerator without needing their blob?
04:15:00 <clever> [itchyjunk]: correct
04:17:00 <clever> [itchyjunk]: but ive not entirely figured out hdmi init yet, and the linux drivers fail because of undocumented security features
04:18:00 <clever> [itchyjunk]: and just to see how far i could go, https://www.youtube.com/watch?v=GHDh9RYg6WI i brought the 3d core online too!
04:19:00 <[itchyjunk]> how many cores does the pi have?
04:20:00 <[itchyjunk]> is one of the core dedicated to 3d rendering?
04:20:00 <clever> [itchyjunk]: the pi2/pi3 have ~18 cores
04:20:00 <clever> 12 cores purely for the 3d rendering
04:21:00 <[itchyjunk]> wth
04:21:00 <clever> and thats probably small, compared to modern GPU's
04:21:00 * geist eyes roll
04:21:00 <geist> yeah that is
04:21:00 <clever> the 12 cores, are each 16 wide vector only cores, so you can kinda count it as 192 cores
04:24:00 <clever> geist: what are modern gpu's up to?
04:24:00 <geist> 4000, 5000? i think a rtx3090 crossed 10k
04:25:00 <clever> i can see why you rolled your eyes, lol
04:25:00 <geist> my older 1080ti seems to have 3584
04:26:00 <geist> but of course a lot of that is just artificial stuff. how many have to be ganged together in a wavefront, etc limits the amount of independence
04:26:00 <geist> much like how you pointed out that a lot of the cores there are 16 wide
04:27:00 <Affliction> geist | 4000, 5000? i think a rtx3090 crossed 10k
04:27:00 <Affliction> isn't that more, the total number of SIMD lanes?
04:27:00 <geist> i dont think so. nvidia and AMD both moved back to lots of scalar cores a while back
04:28:00 <geist> they're not VLIW or very wide individually like they were in the 2000s
04:28:00 <clever> if we are just counting how many parallel lanes can be running at once, i think you could slap a 384 on the rpi, with the exception that exactly half are only capable of addition, and exactly half are only capable of mults
04:29:00 <Affliction> hm
04:29:00 <clever> the v3d on the pi avoids a lot of stalling issues, by just interleaving different threads on the same pipeline
04:29:00 <clever> kinda like hyperthreading?
04:29:00 <Affliction> I think that's pretty common
04:30:00 <clever> even if the pipeline is 4 clock cycles long, you only execute 1 opcode every 4 cycles, for a given thread
04:30:00 <Affliction> don't recall the source, but I've heard AMD's modern scheduler can juggle 128k threads
04:30:00 <Affliction> Given the memory latency within GPUs, that makes sense
04:31:00 <geist> yah and having 10k cores or whatnot to assign them to
04:31:00 <clever> for the rpi, its less of a scheduler, and more just a fixed rotation between 4 threads
04:31:00 <[itchyjunk]> how many cores was pentium I ?
04:31:00 <Affliction> 1?
04:31:00 <geist> 1
04:31:00 <clever> each core has 4 identical sets of registers, and 1 shared pipeline
04:31:00 <[itchyjunk]> and rpi has 18? damn..
04:31:00 <clever> [itchyjunk]: not all 18 are the same
04:31:00 <clever> [itchyjunk]: 4 arm cores, 2 vpu cores, 12 qpu cores
04:31:00 <Affliction> the rpi4 has 4 ARM cores
04:32:00 <geist> again, clever is very good at hyping things up a bit here. rpi is not a particularly special arm soc, but if you start adding up gpu cores and whatnot you can find that almsot everything has something like 20 nowadays
04:32:00 <geist> but cpu cores, 4.
04:32:00 <Affliction> There could be additional cores hidden away in the hardware too
04:32:00 <geist> what's a bit different about rpi is the GPU is somewhat more general purpose than usual
04:32:00 <[itchyjunk]> hmm, are all the cores of rpi "printed" at the same time? same dye i guess?
04:33:00 <geist> and so there's a default binary blob that runs on it that gives you a fairly high level gpu interface
04:33:00 <geist> yeah
04:33:00 <Affliction> I hear the apple M1 has a cortex-m4 coprocessor in the GPU, which itself has a cortex-m0 coprocessor
04:33:00 <Affliction> there are CPUs all over the place
04:33:00 <Affliction> for every one you know about, there's probably 10 you don't
04:33:00 <geist> that's very very common nowadays. you'll find random cortex-ms as little assistant or power management processors on all sorts of things
04:33:00 <clever> the wifi and bt also have their own dedicated arm cores on the rpi
04:34:00 <[itchyjunk]> right, this is the "apple phone charger has a cpu" thing
04:34:00 <Affliction> My HDD has 5 cores!
04:34:00 <[itchyjunk]> that was shocking too
04:34:00 <geist> right plus things like SD cards, etc all have little cpus on them
04:34:00 <geist> running little rtoses
04:34:00 <clever> and the sd card firmware can even crash, leaving the card unreadable, yet the data is still perfectly intact
04:34:00 <[itchyjunk]> those tiny sd cards that you can put in phones have cpus on them?
04:34:00 <clever> [itchyjunk]: yep
04:35:00 <geist> sure
04:35:00 <[itchyjunk]> i imagined SSD's to have it.. but not an sd card
04:35:00 <[itchyjunk]> yikes
04:35:00 <geist> absolutely
04:35:00 <Affliction> That was a thing with some samsung phones, the firmware tripping itself up resulting on phones not booting
04:35:00 <clever> [itchyjunk]: that cpu and rtos manages the wear leveling and translation between sd commands and raw nand flash
04:35:00 <Affliction> Amazing the abstractions we build so we can pretend flash is spinning rust
04:36:00 <Affliction> but then, HDDs themselves are pretty much completely dependant on FEC these days too
04:36:00 <geist> yah totally
04:36:00 <geist> and the shingled stuff too
04:36:00 <geist> even does a RMW cycle and supports trim!
04:39:00 <Affliction> just reading the backlog
04:39:00 <Affliction> Mondenkind | no, the CPU has two OSes
04:39:00 <Affliction> [itchyjunk] | how is that possible?
04:39:00 <Affliction> Mondenkind | https://en.wikipedia.org/wiki/Intel_Management_Engine
04:39:00 <Affliction> I think SMM could count too
04:40:00 <kazinsal> SMM isn't an OS
04:40:00 <Affliction> since it's there in the background, possibly emulating legacy hardware interfaces
04:40:00 <Affliction> Isn't it though?
04:40:00 <kazinsal> it's no more of an OS than the 16-bit BIOS is
04:41:00 <Affliction> I figure, it runs in the background, providing services to the running main OS
04:41:00 <Affliction> To me, that's pretty much what an OS does, provide services to running software
04:41:00 <kazinsal> the point of SMM is that the actual OS doesn't even know it's doing stuff
04:41:00 <kazinsal> the software on the machine isn't making library calls or whatnot to SMM code because SMM code is just doing things. maybe. sometimes.
04:42:00 <kazinsal> that honestly makes it *less* of an operating system or kernel or whatever you want to call it than BIOS ISRs
04:43:00 <Affliction> In that case, the intel ME wouldn't count either
04:43:00 <Affliction> Since the running OS doesn't have to call into it, either
04:44:00 <clever> what do you even define as an os?
04:44:00 <clever> something that provides services to another piece of software?
04:45:00 <Affliction> as a generalisation, sure, with the extra caveat that it abstracts hardware
04:45:00 <clever> and isnt that exactly what the SMM is doing?
04:45:00 <Affliction> pretty much
04:46:00 <kazinsal> by that definition the microcontroller firmware in my mouse that lets iCUE change the LED colours is an operating system
04:47:00 <kazinsal> as it abstracts the underlying LED GPIO hardware and provides that abstraction to my PC over USB
04:47:00 <clever> kazinsal: probably!
04:48:00 <kazinsal> kinda makes the phrase "operating system" useless
04:48:00 <Affliction> I would consider the mouse another node on the network that forms your computer
04:48:00 <clever> drawing the line between firmware and os, gets kinda fuzzy
04:48:00 <Affliction> pretty much
04:49:00 <kazinsal> if everything from Boondoggle GNU/Linux v4.20(69) to the scancode translator in a model M is all just considered an "operating system"
04:49:00 <Affliction> since a lot of 'firmware' is indeed an OS
04:49:00 <kazinsal> the firmware loaded into the DSP in the mixer my SM57 plugs into isn't an OS
04:49:00 <kazinsal> it's DSP code
04:50:00 <Affliction> I mean, your mouse' firmware probably has a trivial scheduler to share its CPU resources between the LED code, button sampling, sensor reading, and communication
04:50:00 <Affliction> Even if that 'scheduler' is just a loop
04:51:00 <clever> Affliction: i think the stage-1 loader in the rpi (brings ram up, loads the next thing) has full context switching in its scheduler, but that code is hard to make sense of
04:51:00 <Affliction> The firmware in your router probably runs a full multitasking kernel
04:51:00 <kazinsal> my router is a virtual machine running OpenBSD
04:51:00 <clever> Affliction: routers are often just embedded linux
04:51:00 <Affliction> pretty much
04:51:00 <kazinsal> and a router is a much more complex appliance that runs much more complex code than a mouse.
04:52:00 <Affliction> Perhaps, I haven't designed a mouse
04:52:00 <kingoffrance> " the mouse another node on the network that forms your computer" yeah, but now you have a mainframe </disappears, like waldo>
04:53:00 <Affliction> I mean, computers haven't been 'one computer' in a long time
04:53:00 <Affliction> ANd, the microcontroller in some mice can probably outperform mainframes of decades ago
04:54:00 <kingoffrance> true
04:54:00 <Affliction> I hear some of them run multi-MHz ARM cores, not sure what that says about the state of their code :)
04:54:00 <kingoffrance> and various nasa or other spacecraft
04:55:00 <Affliction> Though I suppose, sampling effectively a camera at up to 1KHz probably does require a fair amount of compute
04:56:00 <Affliction> So yeah, I'd agree my definition of OS might be permissive, but, I don't think it's wrong
04:56:00 <Affliction> I can see why some might disagree
04:58:00 <geist> yah generally to count as an OS to me it needs to be logically separate from the task itself, and provide some functionality in a semi generic way
04:58:00 <geist> ie, a single embedded thing that just runs a loop and has no real distinction between the OS and the application doesn't count
04:58:00 <clever> geist: so LK without a userland, isnt an os? more of a firmware?
04:58:00 <geist> OS
04:58:00 <geist> because it's logically different from the app that runs on it
04:59:00 <geist> even if it's same address space, etc. like PC-DOS
04:59:00 <clever> guess that works
04:59:00 <Affliction> I can agree with that
04:59:00 <clever> though DOS had loadable apps
04:59:00 <Affliction> I would go further, compiling your app into the linux kernel would still count as an OS, they are still separate code
04:59:00 <geist> right, but LK is a project you can build stuff on top of. it provides OS like functionality, even if you mostly use it as a single binary
04:59:00 <Affliction> Sure, same binary blob but, so is most firmware
05:00:00 <geist> exactly
05:00:00 <geist> most rtoses are that way, freertos, threadx, etc
05:01:00 <clever> ive heard that the threadx based rpi firmware, still has the ability to load modules at runtime
05:01:00 <clever> but the symbols are all missing, so the runtime linker cant do very much of use
05:01:00 <geist> right
05:04:00 <clever> you kinda need more of a checklist of features
05:04:00 <clever> because id generally consider anything with threads and context switching to be an os
05:04:00 <geist> to me it's all about intent and purpose and separation
05:04:00 <Affliction> still reading up
05:04:00 <clever> but dos lacks those, and is still an os
05:04:00 <geist> technical details have changed over the years
05:04:00 <Affliction> kazinsal | both, but honestly even if you had enough evidence that it was protecting the user most foss nerds would still think you're worse than the KGB for defending it
05:04:00 <Affliction> Man, I had that debate with the TPM
05:04:00 <Affliction> It's just a chip
05:04:00 <geist> truying to make a list of concrete features for OS stuff i think is a moving target
05:04:00 <Affliction> If you have it, why not make use of it
05:05:00 <Affliction> But, apparantly the TPM is hitler-enshrined-in-silicon
05:05:00 <kazinsal> people angry about windows 11 because of the TPM need some medication
05:05:00 <Affliction> And the arguments are about what proprietary software might do with it
05:05:00 <Affliction> Proprietary software might do evil with your CPU too!
05:05:00 <geist> 2 pieces of windows 11 stuff i learned lately: a) my threadripper isn't eligable. i guess only Zen2 are eligable? WTF?
05:06:00 <geist> b) i upgraded a mostly irrelevant cheapo machine and it's not half bad
05:06:00 <geist> ui is clean at least. runs fairly smoothly
05:06:00 <Affliction> geist: at work, the shiny 2400Gs and 2500Us we rolled out a couple of years ago are nope, so we're sticking with 10 'til 2025
05:06:00 <kazinsal> I feel like they're going to drop the eligibility requirements a bit soon
05:06:00 <kazinsal> it's probably hardware speculation exploit mitigations that it's looking for
05:07:00 <geist> yagh my main desktop i'm sure is eligable but no way am i upgrading until at least they fix the zen slowdowns
05:07:00 <Affliction> That's the general rule with windows, always wait :)
05:08:00 <Affliction> I only use it for gaming, leave the serious work for a serious OS
05:08:00 <kazinsal> if I tried to do my job on Linux I'd probably be fired for not being able to use half the tools I need
05:10:00 <Affliction> Naturally, Office is still important
05:10:00 <Affliction> Because the trainwreck they call file formats are damn hard to implement all the details of, properly
05:11:00 <Affliction> "but it's open! and xml!" sure, an XML mapping of the binary format
05:11:00 <kazinsal> a lot of remote access VPNs don't have linux clients because most of the enterprise world doesn't realize that $currentYear+1 is the year of linux on the desktop
05:11:00 <Affliction> That has an ISO standard that they don't implement
05:11:00 <Affliction> no no kazinsal, I think 2022 will really be it this time! /s
05:12:00 <kazinsal> my desktop runs windows. my laptop is an m1 macbook. the only linuxes in my apartment are virtual machines on a server that lives under my desk and keeps my feet warm
05:13:00 <Affliction> My laptop is ancient and was hopeless running windows 7, it's pretty much a thin client these days.
05:13:00 <Affliction> Curious about Alder Lake, and what it could do in a laptop if it doesn't suck
05:13:00 <Affliction> Otherwise, I might get a zen3 one
05:14:00 <Affliction> Either way, holding off
05:14:00 <geist> yah i'm waiting for the next line of macbooks and will probably get one to replace my aging 2014
05:14:00 <geist> but on the flip side i dont really use laptops right now becuase that involves going outside
05:14:00 <kazinsal> work sent me a new laptop and it's a dell with an alright screen an ddecent weight but the worst keyboard and trackpad I've used in years
05:15:00 <kazinsal> I have not actually done anything on it yet
05:15:00 <Affliction> sounds about right
05:15:00 <kazinsal> I'm still remoting into my desktop at the office
05:15:00 <geist> hah yeah like the first dell i got at my current job. it was so terrible i swapped it with a macbook
05:15:00 <geist> it was only usable with external keyboard and mouse
05:15:00 <Affliction> I'm hoping to pick up something decent, that I can get another 10y out of
05:15:00 <kazinsal> they gave me the option of either a 13" or a 15" dell and I just told them to give me the 13" because I knew I wasn't going to use it unless I have to go on site somewhere
05:15:00 <kazinsal> and even then it's my second laptop, because I'll just bring my macbook
05:16:00 <kazinsal> weighs less, better screen, better battery life, better trackpad, better keyboard... probably faster than an 11th gen mobile i5 as well
05:16:00 <geist> yah usually i get a fairly high end mac laptop and then milk it for 5 or 6 or 7 years
05:16:00 <geist> the fact that my 2014 mbp is still going okay without any major malfunctions is pretty good
05:17:00 <Affliction> Well, I went low end and cheap back in 2009 and, given only a memory upgrade, it's still alive!
05:17:00 <Affliction> Single core non-HT celeron @ 2.5GHz, woo!
05:17:00 <geist> most annoying part is some sort of antiglare covering on the screen is starting to wear off in parts. where it has rubbed against the keys
05:18:00 <Affliction> as I said, thin client... Even modern websites are too much for it
05:18:00 <geist> yah that's what the 2014 is starting to feel like
05:21:00 <Affliction> that said, given the shortages, made even worse by Microsoft obsoleting a bunch of otherwise perfectly good chips...
05:21:00 <Affliction> We'll see
05:27:00 <sham1> Morning all
05:29:00 <kazinsal> hmm. hasn't started raining yet
05:29:00 <kazinsal> I should go for a quick chronic walk and then read through the PXE spec once I'm nice and relaxed
05:30:00 <kazinsal> trying to de-gnuify my setup as much as possible and apart from the obnoxious ones (gcc -> clang, gnu make -> posix make) all I've got left is grub -> not grub
05:30:00 <kazinsal> for my OS project, that is
05:34:00 <klange> I should poke at llvm again. I had a working clang build of toaru32 but never really touched the userspace aspects...
05:35:00 <klange> Meanwhile, re: gnu make → posix make... my aim is to implement my own make that's sufficiently gnu-compatible ;)
05:36:00 <klange> I'm also working on porting Python scripts to Kuroko; the last notably one I have is essentially a wrapper for making tarballs suitable for use as ramdisks...
05:37:00 <sham1> Make a thing that generates posix makefiles or even ninja
05:38:00 <klange> I have my own ISO generator as well, written in Python but definitely portable to Kuroko... and most of my build is actually Kuroko-driven magic dependency detection.
08:53:00 <geist> well, been watching the Evangelion movies. this is something
08:54:00 <geist> been years since i saw the series, but the second half was such an incomprehensible mess it didn't make an impression then
09:05:00 <zid> I know, great isn't it
09:06:00 <zid> EoE is still one of my favourite things ever
09:06:00 <zid> 3.0+1.0 existing is kind of.. sad in a way, end of an era
10:02:00 <klange> Experimenting with some startup changes that integrate kernel log messages (mostly so I can nicely say that I'm decompressing a ramdisk, which can take a few a second or two under TCG, or quite a bit longer under Bochs) https://klange.dev/s/Screenshot%20from%202021-10-15%2019-01-32.png
16:36:00 <[itchyjunk]> yesterday we said pentium one was 1 core processor and a rpi might have a 4core arm processor. but does that mean a rpi is multiple times more "powerful" that pentium one? i guess there is this weird bias in my head that makes in think of desktop as "real computers" and everything else as toy computers
16:36:00 <j`ey> you can have 4 slow CPUs slower than 1 fast CPU
16:38:00 <[itchyjunk]> ahh, so is that the case here?
16:38:00 <j`ey> I dunno about pentium 1
16:39:00 <kazinsal> the pentium 1 is *slow as hell*
16:39:00 <[itchyjunk]> hmm pentium I through V are the only ones i really remember
16:40:00 <j`ey> what makes you call a computer a desktop?
16:40:00 <[itchyjunk]> yes but i think pentium 1 was the full blown desktop where i saw GUI? i can't remember
16:40:00 <[itchyjunk]> i remember lotus,dos and some gui
16:40:00 <kazinsal> it's hard to effectively compare speeds between architectures, and still kind of hard to compare between microarchitectures because of differences in out-of-order execution and speculation and such
16:40:00 <kazinsal> and instruction latencies etc
16:40:00 <[itchyjunk]> moniter, that big white box containing motherboard with a fan on the back etc
16:41:00 <j`ey> you could put a rpi in a big white box
16:42:00 <GeDaMo> This is an 8-bit CPU at 150MHz running a 386 emulator running Windows https://www.youtube.com/watch?v=5RF45uEaFlU
16:42:00 <[itchyjunk]> take pentium II for example, i could play Aegis of Empire: Rise of rome on it smoothly. could rpi outperform that easily?
16:42:00 <kazinsal> by a longshot
16:43:00 <kazinsal> just on pure clocks (a terrible benchmark, but for comparison) alone the highest clocked Pentium II was something like 450 or 500 MHz and a Pi 4B has a 1.5 GHz quad-core 64-bit ARM CPU
16:43:00 <[itchyjunk]> oh interesting, so that board you linked to runs windows 3 smoothly
16:44:00 <[itchyjunk]> wonder how many core it has
16:44:00 <GeDaMo> 1
16:44:00 <[itchyjunk]> oh..
16:44:00 <[itchyjunk]> 1 150Mhz is all it takes?
16:44:00 <GeDaMo> Well, the 386 only went up to 25MHz I think
16:45:00 <[itchyjunk]> What happens with all this extra compute in modern computer? It ocassionally gets used to loading things and rest of the time it's idle?
16:45:00 <GeDaMo> Modern machines are doing a lot more
16:46:00 <GeDaMo> Not necessarily useful things, of course :P
16:46:00 <[itchyjunk]> lol
16:46:00 <zid> Mine's mostly always idle
16:47:00 <zid> graphics being hw accelerated accounts for most of that
16:47:00 <zid> no need to redraw the screen every frame by hand
16:47:00 <[itchyjunk]> doesn't hw accelerated mean there is some processers dedicated to redrawing the screen constantly?
16:48:00 <zid> yes, it's called an nvidia 1050 ti in my case
16:48:00 <[itchyjunk]> so someone else is busy all the time?
16:48:00 <zid> it's busy.. for a small fraction of the time, 60 times a second
16:48:00 <zid> unless I've told it to figure out which of these 14 million points lie inside this cube and then connect them with triangles and shade them according to this bytecode
16:48:00 <[itchyjunk]> what happens if you have some inbuilt hardware accelerator and also a GPU? the cpu will have to decide what to use?
16:48:00 <zid> accelerator for what
16:49:00 <zid> either it talks to the cpu in order to function or it doesn't, if it talks to the cpu.. then the cpu has to talk to it
16:49:00 <zid> which takes up some amount of cpu time
16:49:00 <zid> you're overthinking this I feel
18:30:00 <catern> hmm I hope I didn't just screw up my interview, I said spinlocks were useful in single-core situations
18:30:00 <catern> they are in userspace, right? like you can just yield every time you fail to take the lock, that seems useful right?
18:30:00 <zid> there's no point spinning
18:30:00 <zid> if nothing could possibly change the state while you spin
18:31:00 <zid> They're *possibly* useful as hw access primitives, but then you wouldn't call it a spinlock
18:31:00 <zid> just a busy wait
18:32:00 <catern> zid: sure, I'm saying in userspace, where you could get interrupted and switched out
18:34:00 <sortie> Honestly on single core a proper futex thing that wakes the other thread exactly is the only thing I can see make sense, and no cases where spin locks make sense in user-space
18:34:00 <sortie> This obviously changes on multi-core where it can in fact reduce latency if the threads actually run concurrently
18:35:00 <geist> hmm, looks like aquantia AQN-107 based 10gbe ethernet adaptors are pretty decent
18:35:00 <geist> i wonder how the chipset itself compares with equivalent intel ones
18:35:00 <geist> a bit cheaper, but seems to have a similar number of features, for basic 10gbe at least
18:36:00 <sortie> I'm afraid my inner interview would deduct a point on that answer, although not flunk you just on that since spin locks do have their very special purpose uses and perhaps you knew about one
18:36:00 <geist> agreed re: user space spinlocks
18:36:00 <sortie> If only because you explicitly stress single-core here, totally different story on multi-core
18:37:00 <sortie> I had a nice conversation a while back with geist iiirc about spin locks in a multi-core setting
18:37:00 <geist> i've seen folks try to use them in user space and a lot of the time it's kinda a bad idea
18:37:00 <geist> if anything because you end up slamming the kernel with yield() syscalls
18:37:00 <catern> well I mean, if all you have is a sys_yield, rather than a proper futex thing...
18:38:00 <zid> catern: who are you spinning for
18:38:00 <geist> it's actually a thing we found folks doing in fuchsia that we redirected. someone had dome basically something like this
18:38:00 <sortie> catern: Well yes they can be used as a basic threading primitive
18:38:00 <catern> but I agree that for sure in a single-core setting futex kind-of-thing is better
18:38:00 <geist> and suddenly we were seeing 100k syscalls in short order
18:38:00 <geist> turns out someone had rolled a spin/yield user space lock
18:38:00 <geist> when they really wanted a futex
18:39:00 <geist> it was causing some issues because it was stressing out the scheduler locks
18:39:00 <sortie> Futexes are a quite brilliant design that does very much the right thing that one needs, though their actual design and implementation is probably not usually taught
18:39:00 <raggi> Golang yields rather a lot in it's runtime
18:39:00 <geist> of course that was sort of our problem, due to less than ideal locking in scheduler stuff
18:39:00 <geist> raggi: yah go too :)
18:39:00 <sortie> I very much loved getting my OS to stop spinning altogether
18:40:00 <sortie> It's a beautiful thing to be idle.
18:40:00 <catern> so I actually feel more confident in my answer then - obviously futexes (futices) is what you would use on real Linux but in an imagined "my OS sucks and just has yield, not futexes", spinlocks are your only choice
18:40:00 <zid> Mine doesn't spin, but it also doesn't do any real work sooo
18:40:00 <zid> A spinlock is a userspace wait, where you pray some variable in your address spaces changes before you finish waiting, to avoid having to do a syscall. Who is going to change that variable? You have *one* cpu.
18:40:00 <geist> clever: but you *did* at least get the yield() part. whats worse is when folks roll spinlocks with no yield
18:40:00 <zid> You will *always* wait the full amount and *never* get the lock
18:40:00 <sortie> catern: Well what was the question and context of it?
18:40:00 <sortie> futexen
18:40:00 <zid> you will only get the lock by yielding to whoever has it, because they are /not running to release it/, you have one core.
18:41:00 <geist> oh sure if you have no futex then user space spinning + yield is about all you get
18:41:00 <zid> 'just spin until I get scheduled out' is also, not, a spinlock
18:41:00 <zid> it's a spin.
18:41:00 <geist> correct. and there's basically no reason for that
18:41:00 <catern> sortie: just some "how would you protect some memory from concurrent modification in a low-latency way" basic thing, and I answered spinlocks, but then they asked about single-core vs multi-core so I think they were trying to elicit me saying "uh well actually you wouldn't use a spinlock on a single core"
18:42:00 <sortie> Though spinning is not quiiiite correct in principle because there's a risk it doesn't terminate due to poor scheduling / starvation
18:42:00 <geist> in the Old Days, before futexes were the default thing, say the 90s, usually you just always blocked on something
18:42:00 <zid> WaitForSingleObject or select for life
18:42:00 <catern> instead I said "well um on a multi-core system you'd want a cache-coherent spinlock which creates annoying bus traffic, and so on a single-core system you can use a nice unlocked non-cache-coherent spinlock"
18:42:00 <catern> which, in retrospect, is... a weird thing to say...
18:43:00 <sortie> catern, in the context of standard programming or operating systems programming?
18:43:00 <catern> userspace
18:43:00 <catern> (er, not on a single-core system, on a single-core datastructure, you can use non-cache-coherent CAS)
18:44:00 <sortie> Eh you pass my buzzword test
18:44:00 <catern> I assume there must be some non-cache-coherent CAS in x86 right
18:44:00 <catern> in one of the many extensions
18:44:00 <sortie> Start talking about cache coherence and bus traffic and so on, the candidate knows some stuff, might get a detail wrong or so
18:44:00 <sortie> I assume this question wasn't load bearing on the entire interview
18:45:00 <catern> \( ﾟヮﾟ)/
18:45:00 <zid> that's what the nops are for
19:02:00 <gorgonical> This might be a dumb question. It was something I was thinking about in the shower today -- when you dd some data to a USB drive for booting or something and you run "&& sync" to make sure it's fully copied, where is that data being buffered? There's the VFS system, the block device interface, the USB driver. What buffer exactly are you waiting to flush when you run && sync?
19:04:00 <zid> Yes :P
19:04:00 <zid> but mainly the vfs
19:04:00 <zid> it'll absorb write() and return early for performance
19:04:00 <gorgonical> I thought it would probably be vfs
19:04:00 <zid> then block once the buffer fills up
19:04:00 <sortie> I believe sync is supposed to make sure it all bubbles down
19:04:00 <gog> it's buffers all the way down
19:04:00 <gorgonical> And you can probably tune this buffer size in vfs, right?
19:04:00 <sortie> It may vary how much of an effort sync does on your particular system, I expect there to be settings about this
19:05:00 <zid> the writes should have at least been issued, they m ight still be clogged up in the usb host or whatever though
19:05:00 <zid> so I wouldn't pull the power that quickly
19:05:00 <sortie> On Linux you can basically tune all of these parameters, I expect, there's always loads of settings
19:28:00 <sortie> Ah right, yet another case where object and A and B need to access each other asynchronously without deadlocking
20:00:00 <riverdc_> where would yall recommend looking for (cheap) x86_64 boards to test on? I'd like to try implementing a PS/2 driver for keyboard input and run it on real hw
20:03:00 <GeDaMo> Do new machines even have PS/2 interfaces nowadays?
20:03:00 <GeDaMo> Maybe second hand?
20:04:00 <riverdc_> definitely not new machines, but USB seems like a ton of work to get working
20:10:00 <jimbzy> Why is USB so funky?
20:11:00 <jimbzy> I mean, I get that anything universal is going to be complicated, but it's one of those technologies I just haven't been able to really dig into.
20:12:00 <zid> literally anything :P
20:12:00 <riverdc_> I don't know enough whether to judge the design, other folks here would be better qualified. I just know the spec is insanely complicated
20:14:00 <jimbzy> It's like the PDF of hardware or something. :p
21:00:00 <geist> not entirely true, some new gaming mobos still have ps2
21:00:00 <geist> i looked into how it's done. usually those are attached to a SuperIO chip, usually made by Nuvoton, sitting on the LPC bus
21:01:00 <geist> if you look up the data sheet for those, they basically just implement a bunch of legacy ISA stuff. com ports, parallel, ps2, etc so the mobo vendor can still stuff them if they want
21:01:00 <geist> but those things are no longer present in modern intel/amd socs or chipsets anymore
21:01:00 <zid> That's what I have, but it's 2011
21:01:00 <zid> nuvoton superio, with a ps/2
21:01:00 <geist> yah i've bought machiens as recently as 2019 that still have ps2, like my ryzen 3950x
21:02:00 <geist> reason that has hung on is because there's a subset of gamer types that still believe it is lower latency than usb
21:03:00 <j`ey> it's polling!!!
21:03:00 <geist> yes. though also modern usb stuff can poll at higher speeds such that ps2 doesn't really have an advantage
21:04:00 <geist> there's a pretty good Ben Eater vid where he compares on the wire the polling intervals of various usb keyboards against ps2
21:04:00 <geist> kidna interesting
21:04:00 <geist> TL;DR a proper USB2 keyboard with say 1khz polling is at least as good as ps2, since the clock rate of the data is much faster than ps2
21:05:00 <geist> so though there may be up to 1ms jitter of when the packet starts, the packet itself is much faster and completes at or before the best case ps2 packet
21:05:00 <geist> that being said the usb stack as it processes the hid events may still add more delay
21:07:00 <zid> yea I was messing porting an arcade machine to PC
21:07:00 <zid> and it uses 115200 baud serial for its control panel
21:07:00 <zid> that's 1ms just to clock it out
21:08:00 <geist> yah i forget the clock rate of a ps2 packet. it's at least fixed length, so the latency is more or less fixed
21:08:00 <geist> but it takes some amount of non trivial time
21:08:00 <zid> I wrote hid code to replace it
21:08:00 <zid> I am never touching the windows hid api ever
21:08:00 <zid> ever again
21:09:00 <geist> haha yeah hid is a disaster i think everywhere
21:09:00 <geist> the fuchsia hid is complicated too and i think it only pretends to handle a subset of things
21:10:00 <zid> It's not a bad api once you figure it out
21:10:00 <zid> but holy shit do they not make it easy to figure out
21:11:00 <zid> the msdn pages are literally "Returns the report collection status page of a hid device collection object"
21:11:00 <zid> x20
23:18:00 <sortie> Unix sockets file descriptor passing is complicated to implement correctly. Implementations even have to use garbage collection to handle special cases. I'm trying to see if I can avoid that if I can get away with a few restrictions:
23:18:00 <sortie> 1. A Unix socket (with file descriptors currently sent on it) cannot be sent over another Unix socket.
23:18:00 <sortie> 2. A Unix socket (currently being sent over another Unix socket) cannot send file descriptors.
23:18:00 <sortie> 3. A Unix socket cannot be sent over itself.
23:18:00 <sortie> These are the three laws of Unix sockets and they prevent reference cycles and the robot uprising.
23:30:00 <Mondenkind> what's wrong with cycles?
23:30:00 <Mondenkind> Embrace the cycles!
23:31:00 <sortie> Break the cycle!
23:31:00 <sortie> HACK THE WORLD
23:31:00 <sortie> cd /usr/src && make world!
23:33:00 <Qubasa> I just discovered that my grub memory map does not describe the memory region 0xFEE0_0000 where the APIC is located. Is this normal? Does this mean the grub memory map can be incomplete?
23:35:00 <sortie> I don't have the knowledge to definitely answer your question, but in principle, yes, I believe GRUB is asking the BIOS (or through UEFI if UEFI) about the memory map and it may be incomplete if the firmware isn't great
23:36:00 <Qubasa> shit
23:36:00 <sortie> (usually not much to do in that situation than just to use less memory)
23:36:00 <sortie> (Not really a problem in practice though, I believe)
23:36:00 <Qubasa> well and if my APIC is located in that missing memory? :(
23:36:00 <sortie> As for the APIC, I don't have the knowledge to say for sure, but I think it might be special memory
23:37:00 <Qubasa> ahhh
23:37:00 <Qubasa> i see
23:37:00 <Qubasa> thanks ^^
23:37:00 <sortie> I don't believe the memory map necessarily contains all the places where hardware memory regions are mapped (it's more designed to tell you about regular memory)
23:38:00 <sortie> In any case, you should generally not be poking at any hard-coded memory location without something telling you something is supposed to be there.
23:39:00 <sortie> I haven't done APIC so I don't know what is supposed to tell you it exists at any given location, but if the firmware/hardware registers/whatever in-memory tables you are using/standards hard-coding an address/whatever tells you it's there, then it seems fine to trust that
23:40:00 <sortie> That's generally a good design to avoid hard-coding addresses or assuming the physical memory layout of things, instead one should prefer to dynamically learn the layout of things at runtime and figure things out (sounds like you're trying to do that, which is great)
23:41:00 <Qubasa> Hmm yeah I'm checking a register that tells me where the APIC lies, however I trusted the memory map too much regarding hardware devices and generated a page table that seems unfitting now :)
23:41:00 <Qubasa> sortie: Really big thanks!
23:42:00 <sortie> Qubasa, OK so that was just my non-authoritative answer as I'm not too knowledgeable here to tell you a definitive answer, but hopefully my incomplete knowledge helps and someone can get us both a more definitive answer
23:46:00 <Qubasa> It definitely does it also aligns with what I faintly remember hearing a longer time ago~
23:49:00 <sortie> :)
23:50:00 <zid> yea just bear in mind the physical address space isn't just for ram
23:50:00 <zid> so poking it at random may.. brick your e1000 ;)
23:51:00 <Qubasa> I will keep that in mind ^^
23:51:00 <_eryjus> the IA32_APIC_BASE_MSR should have the address of the APIC
23:51:00 <Qubasa> yup I do check that
23:51:00 <zid> Trying to enumerate all the devices listening to the memory bus would be a fun challenge
23:52:00 <qookie> that MSR and also the MADT has it iirc
23:53:00 <_eryjus> qookie: i believe you are correct, but it has been a while