channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched
#osdev2 = #osdev @ Libera from 23may2021 to present
#osdev @ OPN/FreeNode from 3apr2001 to 23may2021
all other channels are on OPN/FreeNode from 2004 to present
http://bespin.org/~qz/search/?view=1&c=osdev2&y=21&m=11&d=2
02:26:00 <vin> If there is a processor chip which supports no timmer interrupt but has 1000 cores per die (hypothetical) so appliiications can have dedicated cores specified to them rather than being time shared. A spin lock would do well becuase there will be no repeated cost of context switch or provides low latency response time. But why would a spin lock not do well on this hypothetical design?
02:27:00 <vin> /s/or/and/g
02:28:00 <MelMalik> A spin lock would create heat.
02:29:00 <MelMalik> You need a cold lock to sleep the core, and an interrupt to wake it up.
02:29:00 <MelMalik> Also, consumes power
02:29:00 <vin> So it's just heat and power?
02:30:00 <clever> in such a cpu design, upon failing a spinlock in userland, i would issue a futex syscall
02:30:00 <clever> which would put the thread to sleep, and wait for an IPI signaling release of the lock
02:30:00 <clever> that also relies on an atomic increment opcode, to let the other end know you want to be woken up
02:31:00 <clever> my understanding of that api, is that userland will try to get the lock with atomic ops, and use futex to block upon failure
02:32:00 <vin> So when would a spin lock fail? It keeps waiting on the resource until availaible right
02:32:00 <clever> well, it would be less of a spinlock then
02:32:00 <clever> more of a lock_try function
02:33:00 <clever> but i have seen spinlocks that include a try function, for when you dont want to spin
02:33:00 <vin> yea more like busy wait. So spin lock just tries a few iterations and sleeps if resource is not acquired?
02:33:00 <clever> yeah
02:33:00 <clever> arm and haskell also have a different approach to things
02:34:00 <clever> with arm, you get a token from the cpu core, that says you own a given addr
02:34:00 <vin> oh then power and temprature is not an issue with spin lock if it's only a few iterations
02:34:00 <clever> then you do load/compute against the value
02:34:00 <clever> then you do a "store only if i still own it" opcode
02:35:00 <clever> arm never actually stalls or says no when you fail the lock
02:35:00 <clever> it will instead say no, when you try to write the value back
02:35:00 <clever> so if your read/modify/write contested with another core, it becomes a read/modify/abort
02:35:00 <clever> and the routines around that, are responsible for trying again
02:36:00 <clever> haskell has a similar mechanism in STM, where you read a sequence-counter for every variable you read as you compute an arbitrarily complex operation
02:37:00 <clever> and every write, just stores them into a temp buffer
02:37:00 <clever> then during commit, you grab a global mutex, check the sequence counters, and maybe commit
02:37:00 <vin> So why would a failed spin lock do a syscall?
02:38:00 <clever> so you can have the scheduler yield control to another task, or drop into a low-power state
02:38:00 <clever> until somebody else releases the lock
02:38:00 <vin> but in this chip processors aren't time shared so does it make sense to yeild control to another task?
02:39:00 <vin> each thread had a dedicated processor
02:39:00 <clever> you can still drop into a low-power state
02:39:00 <clever> where you basically just shut off the clock for the core, and halt all state changes
02:40:00 <vin> So the kernel itself is running on a bunch of dedicated cores where the user app communicate to it via some call api.
02:41:00 <clever> in such a system, i would have the kernel and userland on the same core
02:41:00 <clever> when you issue a syscall, the core just changes into kernel mode
02:41:00 <clever> and while in kernel mode, it has the power to sleep until an irq happens
02:41:00 <vin> can't clever, they can't be time shared because there is no timmer interrupt
02:42:00 <clever> you dont need a timer interrupt to share it
02:42:00 <clever> windows 3.11 never used timer interrupts for time-sharing
02:42:00 <clever> it was 100% volentary
02:42:00 <vin> I see.
02:42:00 <clever> a thread only looses cpu when it chooses to give control back to the kernel
02:43:00 <clever> and the same can be said about any thread on a real-time scheduler
02:43:00 <vin> So one issue I can think of from our discussion is that a spin lock that can't be interrupted can end up being stuck on the core forever without making any progress and no way to recover it.
02:44:00 <clever> yeah, so you would want to avoid using such locks
02:45:00 <MelMalik> personally i would have interrupts on such a cpu
02:46:00 <vin> haha yes
02:47:00 <MelMalik> because not having interrupts means you can't reliably schedule more than, eh, 700 tasks at once
02:59:00 <vin> Yes, this was just me thinking about concepts in different settings, not practical in any sense.
03:39:00 <geist> if you wanted to build such a design you'd probably do some sort of hardware event style yield
03:39:00 <geist> kinda like ARMs WFE/SEV but a bit smarter
03:39:00 <geist> then your spinlocks dont spin, they check and then go into a sleep waiting for another core to signal the same address
03:45:00 <clever> geist: yeah, if you had a way to target the SEV, you could do that far more efficiently
03:45:00 <clever> how does SEV operate, when you have multiple guests under an EL2 hypervisor for example?
03:45:00 <geist> arm64 actually does, it's implicit when you store to the address you had previously acquired an exclusive line to
03:48:00 <geist> caveats being that the size of the observation area is relatively large and defined per core
03:48:00 <geist> though i think it's always a cache line practically speaking
03:49:00 <geist> since that's really how it works: the cpu grabs the line exclusively and waits for another core to write to it, thus losing the exclusiveity
03:49:00 <geist> that triggers an explicit local SEV to the core
03:49:00 <clever> yeah, i read somewhere, that the load/store exclusive, expands the region up to a whole cache line
03:49:00 <clever> but the api only garantees for the host-size word you targeted
03:50:00 <geist> basically
03:57:00 <clever> can the EL2 api entirely hide that your even running under a hypervisor?
03:57:00 <clever> i can see how you might emulate all of the hw in a board, but masking any artifacts of the emulation would be tricky
04:07:00 <geist> there's some bits somewhere that control the scope of SEV/WFe i think
04:08:00 <geist> like, wiether or not it's restricted to a single guest or not
04:08:00 <clever> ahhh, thats what i was thinking of
05:22:00 <nur> the latest sigops newsletter I got is about graph computing
05:22:00 <nur> I wonder how that fits into OS research
05:22:00 <klange> sigops isn't actually about operating systems and I'm not sure it ever was
05:22:00 <klange> it's about "distributed systems"
05:38:00 <vin> https://www.youtube.com/watch?v=36myc8wQhLo OSDI has only 6% of paper related to oeprating systems. One of the good keynotes
05:38:00 <bslsk05> 'USENIX ATC '21/OSDI '21 Joint Keynote Address-It's Time for Operating Systems to Rediscover Hardware' by USENIX (01:06:19)
05:40:00 <vin> There are hardly any new OS designs and iimplementations being published in peer-reviewed avenues.
05:51:00 <kazinsal> yeah, new designs are pretty rare because they're universally either not useful for anything other than as a tech demonstrator for "if we had 40x the computing power and quantum coprocessors this would be disruptive!" or something so inventive it wouldn't have any papers written about it until after it received a commercial release
06:58:00 <nur> this didn't use to be the case
06:59:00 <vin> kazinsal: in the talk Timothy argues the importance of OS desgined for modern SOCs forget quantum computers.
07:11:00 <vin> what are your thoughts geist? I am interested in learning what motiviated you to start Fuchsia?
07:12:00 <geist> well, i didn't start it, i was just working at the time
07:13:00 <geist> more like being in the right place at the right time
07:13:00 <nur> Tim Roscoe's talk reminded me of Rob Pike's "systems research is irrelevant" waaay back in 2000
07:13:00 <nur> and many of the same things are still true
07:13:00 <nur> hardware has changed/software is stagnant
07:14:00 <vin> Oh I thought you started Zircon, sorry!
07:14:00 <geist> well, i was there at the beginning but i didn't personally start it
07:15:00 <geist> certainly dont have that kinda clout :)
07:17:00 <vin> geist: Nevertheless, what was the motivation for the project? Any *specific* problems google was looking to solve?
07:17:00 <geist> sadly those are precisely the things we aren't allowed to talk about
07:18:00 <geist> i mean having a modern capability based design, etc is fairly obvious
07:18:00 <geist> ie, build something from scratch with more modern security and compartmentalization from day 1
07:18:00 <vin> No worries :) I expect to see it in OSDI/SOSP 4/5 years from now.
07:18:00 <geist> hmm, whta are those?
07:19:00 <nur> OS conferences I guess
07:19:00 <vin> Yes
07:19:00 <kazinsal> OSDI is the usenix OS conference iirc
07:19:00 <geist> ah yeah
07:19:00 <nur> SOSP is the ACM one
07:19:00 <geist> probably worth doing some talks for that
07:20:00 <geist> reminds me there was a fairly good paper written about the fuchsia security model recently that i was reading that was fairly good
07:20:00 <geist> a bit inaccurate with the kernel but it was mostly talking about the component model in user space
07:20:00 <vin> Most of google's work has been published after a while in these conferences. Like GFS, Borg, Map reduce, Colossus, etc
07:20:00 <geist> https://arxiv.org/pdf/2108.04183.pdf
07:21:00 <geist> it's a pretty good paper, not written by us so there are some inaccuracies, but it has some pretty good overview
07:21:00 <vin> Oh that's nice, I will give it a read. Thanks!
07:21:00 <nur> who are the authors
07:21:00 <klange> I've thought about doing a paper.
07:22:00 <geist> looks like some guys from University of Genoa
07:22:00 <klange> I thought about it more when I was still a university student... but ToaruOS wasn't really worth talking about back then.
07:23:00 <nur> they cited some blogs and even wikipedia but not my article on fuschia *grump*
07:23:00 <geist> nur: oh that's right you wrote one too huh?
07:23:00 <nur> and all the time I spent questioning poor geist :)
07:23:00 <geist> hah
07:23:00 <nur> lessee if I can dig it up
07:24:00 <nur> https://lwn.net/Articles/718267/
07:24:00 <bslsk05> lwn.net: Fuchsia: a new operating system [LWN.net]
07:24:00 <nur> some of it already outdated :-|
07:24:00 <vin> klange: maybe you should! A talk at strangeloop would be a good start.
07:24:00 <nur> like "magenta" instead of "zircon"
07:24:00 <geist> hehyeah
07:24:00 <geist> long live magenta!
07:26:00 <klange> I did do a talk once. One of my previous employers did regular internal free-topic tech talk sessions and I did a whole deep dive "What happens when..." thing.
07:26:00 <klange> I need to redo that slide deck, since that was 2015 when I was still using newlib.
07:26:00 <Mutabah> You had a video of that, right?
07:27:00 <klange> Yeah should still be up on my YouTube but might be unlisted just because I'm terribly embrassed by it.
07:27:00 <geist> yeah i did exactly one talk in front of 1000 people and i have to say it was a bit terrifying
07:27:00 <klange> Giving that talk is actually what pushed me to do the NIH thing.
07:27:00 <geist> surprisingly less so leading up to it, i thought i had it nailed
07:27:00 <geist> but i started to lock up half way through it
07:28:00 <geist> but thankfully someone else was co-presenting and could kick me along
07:28:00 <klange> Because one of my coworkers - who had previously worked on NT before joining us - badgered me about newlib and whatnot.
07:28:00 <klange> ToaruOS is 100% built on spite.
07:28:00 <geist> so. bummer that happened, since it kinda crushed my confidence in things like that in the future
07:28:00 <kazinsal> yeah, I've done a couple internal presentations on like hypervisor architecture and stuff but only to the six or seven people in the company who find that kind of deep nerd stuff interesting
07:28:00 <geist> yah i can talk to folks i know without much trouble
07:30:00 <kazinsal> eventually I'd like to end up with an OS project I can give a talk about, but I think I also said a few years ago that I was aiming to have a release out by now, and well... ;)
07:30:00 <geist> yeah same
07:30:00 <kazinsal> I look forward to having something to give a talk about in 2028
07:30:00 <nur> me too, but then "getting a real job" got in the way
07:30:00 <nur> I now write customer service chatbots for Dell :-|
07:30:00 <zid> Finally someone who can fix my printer
07:31:00 <kazinsal> entertainingly I got my current job in part because of my earlier osdev stuff. not because it's relevant to my job, but because it was so bafflingly nerdy and clearly a tech passion project
07:31:00 <nur> zid, technically I would write you a bot that could forward you to someone who can fix your printer
07:31:00 <kazinsal> and also my director and I are both guitar nerds so that helped
07:32:00 <klange> I have a guitar now.
07:32:00 <nur> although can printers be fixed really? I feel they're cursed
07:32:00 <nur> I wonder what writing a printer driver feels like
07:32:00 <zid> printers are disposable items built around ink cartridges
07:32:00 <klange> This would not really be news except it was a preorder for a collab guitar that I placed nearly a year ago and it was delayed so the fact that it _finally_ arrived...
07:33:00 <nur> I guess I'll find out once I get to the "printer" part of the osdev exercise
07:33:00 <klange> Printer drivers are a bit of a misnomer. It's almost all document conversion, either to postscript or PDF.
07:33:00 <zid> If you only deal with networky printers that talk postscript I'm sure it's easy, it's the 'winmodem' printers that never work
07:33:00 <nur> cursed I say
07:33:00 <zid> where the drivers have to control the motors or whatever
07:33:00 <klange> And then you either blat it out over some serial interface or throw it at an HTTP endpoint.
07:34:00 <kazinsal> see, that's why I'm writing a network operating system. no one ever expects their router to have printer drivers ;)
07:34:00 <klange> I'm not aware of any of those that still exist. Everything is a PostScript or PDF printer these days, and CUPS profiles almost entirely consist of names for knobs.
07:35:00 <zid> In other words, only ever do osdev targetting precisely the easiest era of the tech you're talking about
07:39:00 <klange> This is why I target VMs. No one is going to print from their VM!
07:42:00 <nur> well, to a network printer :)
11:51:00 <froggey> kazinsal: given that a whole lot of routers ship linux I'd be entirely unsurprised if some of them included printer drivers
13:21:00 <kingoffrance> bbbbbut, but...glass ttys! ttys on fire! display postscript! its printers all the way down!
13:21:00 * kingoffrance stops injecting meta
13:22:00 <kingoffrance> some printers just happen to have, say, a keyboard and mouse to assist driving them
13:23:00 <kingoffrance> REPL!
13:59:00 <kingoffrance> https://en.wikipedia.org/wiki/IPO_model thats what i was looking for
13:59:00 <bslsk05> en.wikipedia.org: IPO model - Wikipedia
14:00:00 <kingoffrance> Many introductory programming and systems analysis texts introduce this as the most basic structure for describing a process.
21:16:00 <sham1> Good evening to all
21:17:00 * Arsen waves
21:47:00 <klange> Updated my screenfetch-alike: https://klange.dev/s/sysinfo-updated.png
23:07:00 <niedzejkob[m]> TIL, from reading the sectorlisp code, that apparently BIOSes load the MBR at 0x600 too. Is this described anywhere?
23:13:00 <kazinsal> never seen that behaviour before
23:14:00 <zid> isn't that inside the IVT
23:15:00 <zid> hmm I guess that might stop at 0x400 depending on whether I can count or not
23:15:00 <zid> It's possible that's the disk buffer location some random bios uses, and copies it to 7c00 before it runs it after checking the tag and stuff I guess
23:17:00 <clever> i have heard that some bios will jump to something like 0:7c00, while others jump to 7c0:0 ? something like that
23:17:00 <clever> and it can screw with programs that assume the segment reg when doing absolute jumps
23:17:00 <zid> they're the same linear address
23:17:00 <clever> exactly
23:17:00 <zid> 0x600 is definitely not
23:17:00 <clever> but if you jump to CS:10, you wont get the same linear addr
23:17:00 <clever> because the CS differs
23:18:00 <kazinsal> anything since like, the late 90s shouuuuuuuld use 0000:7C00
23:18:00 <zid> My 100% top guess is that it's a disk buffer used by possibly some common early bios
23:18:00 <zid> it definitely isn't standard
23:19:00 <kazinsal> CS=0 IP=7C00 is specifically defined in the BIOS Boot Specification from 1996 iirc
23:19:00 <kazinsal> but nothing about any other addresses
23:20:00 <zid> yep found it in a patent
23:20:00 <zid> https://cdn.discordapp.com/attachments/417023075348119556/905234914222407750/unknown.png
23:20:00 <kazinsal> aha, it looks like in ye olde days MS-DOS would relocate its MBR to 0000:0600
23:21:00 <zid> patent for DUAL USE MASTER BOOT RECORD
23:21:00 <zid> not quite sure who he is other than a guy with a couple of patents on bootsectors
23:22:00 <clever> i had at least one machine that was unable to boot random floppies
23:22:00 <clever> i suspect it was using the wrong CS for running the MBR
23:22:00 <moon-child> bet he made bank on that
23:22:00 <zid> I always load cs manually because I'd heard in the wild some biosen like that existed
23:23:00 <kazinsal> looks like it's assigned to Phoenix Technologies
23:23:00 <kazinsal> and expired in 2018
23:23:00 <clever> zid: ive also seen code in grub, that will do a far-jump to 0:7c10 for example, to fix that
23:23:00 <zid> yep
23:23:00 <zid> might as well, if you can spare the couple of bytes