channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev2&y=23&m=1&d=18

Wednesday, 18 January 2023

00:53:00 <gorgonical> I just need to ask one question that's also a vent: why is it so fucking hard to conclusively ascertain if a certain exception in arm64 will change exception levels?
00:55:00 <gorgonical> I am debugging non-responsiveness after turning on the MMU in secure state, and my hunch is that the page tables are mis-mapped somehow and the iftech is failing.
00:57:00 <gorgonical> But even trying to determine where the exception will end up has not been trivial. I am taking it in secure EL1 and I *think* the exception is targeting secure EL1 also. But trying to find a table, description, whatever of the exceptions has been extremely frustrating.
00:58:00 <gorgonical> For some reason the best reference for exactly how the exceptions work is the pseudocode allllll the way at the end of the manual
01:28:00 <small> heat:
01:36:00 <gog> small
01:48:00 <small> lol
01:49:00 <hmmmm> lol did you ever fix your 4-star pointer
02:46:00 <small> who
07:41:00 <dinkelhacker> gorgonical: Do you have access to a debugger an can get value in the respective ESR register? You could decode that at https://esr.arm64.dev/ which would probably give you a clue whats going on.
07:41:00 <bslsk05> ​esr.arm64.dev: AArch64 ESR decoder
07:53:00 * zid hides his dinkel
10:57:00 <epony> vdamewood, eat shit
10:58:00 <epony> retard moron ;-)
10:58:00 <sham1> That's some good shit
10:58:00 <epony> buy an arm sdk board
11:06:00 <ddevault> tfw not getting interrupts from serial
11:07:00 <epony> is the csr / rts set?
11:07:00 <ddevault> pl011
11:08:00 <ddevault> set bit 8 (TX enable) and 1 (UART enable) of UARTCR
11:08:00 <heat> does your pl011 drv work in qemu?
11:08:00 <ddevault> set UARTIFLS to 0, UARTIMSC to 0
11:08:00 <dinkelhacker> Question about heap allocation: I'm pretty early in my osdev journey and currently my kmalloc function just returns a whole 4k page. That's fine because up until now I only allocate some kernel structs and don't free anything. I've been thinking about implementing something more sophistecated and been wondering how others go about that topic. Does everybody just go down the slab allocator/buddy
11:09:00 <dinkelhacker> system route ?
11:09:00 <ddevault> enable IRQ 1 (intno 33) in GIC by writing 0b10 to GICD_ISENABLER1 and ICPENDR1
11:09:00 <ddevault> heat: everything but interrupts, yeah
11:09:00 <heat> ddevault, -trace away?
11:10:00 <heat> dinkelhacker, ok so memory allocation is pretty nuanced
11:10:00 <ddevault> anyone got a pl011 driver lying around I can study?
11:10:00 <zid> qemu source imo
11:11:00 <ddevault> ideally someone who uses named constants -_-
11:11:00 <zid> just write it as a client 'agaist' qemu is easiest
11:11:00 <ddevault> yeah I guess
11:11:00 <zid> That's how I fixed all the bugs in my e1000 when I wrote it
11:11:00 <zid> just made qemu tell me what the actual flow path is
11:11:00 <ddevault> that's how I wrote my PC serial driver too <_<
11:11:00 <ddevault> anyway I was hoping this would just werk and it just doesn't, so I'll get back to more important things and return to it later
11:11:00 <heat> dinkelhacker, I'll describe the linux system first. linux has a page allocator (buddy allocator as you may know) that gets you 2^n pages, with 2^n page alignment. the kernel keeps all the memory/a good chunk of memory consistently mapped. slab works on top of that, gets pages from the buddy allocator, divides them into chunks on top of the linear mapping of memory
11:12:00 <zid> my serial driver for PC infact, just werked
11:12:00 <zid> it is however, totally braindead on PC
11:12:00 <ddevault> "mess with my kernel while long-running process on more important task runs in the background" has become "long-running process finished and now you're blowing off the more important task to debug your kernel"
11:13:00 <heat> then if you for some reason want larger sizes that don't need to be contiguous, or in an mmap-like code path, it keeps (kept, changed a release ago) a red-black tree of virtual memory allocation areas. allocation is simply exploring the gaps between nodes and seeing if there's enough space, then doing page allocation and mapping using the MMU
11:15:00 <heat> this is more or less how things work. some systems have the kernel heap allocator also on top of MMU (like the BSDs, afaik Windows, etc)
11:15:00 <zid> buddy allocate your bytes, ez
11:16:00 <ddevault> wait, you guys aren't writing microkernels? for shame /s
11:16:00 <heat> as for your slab question, I think slab has been more or less been empirically proven as the best allocator for a kernel
11:16:00 <zid> ddevault: I may not shave or get a haircut that often, but I'm not a *total* hippie
11:17:00 <ddevault> what I'm curious to see resources on is virtual memory allocation strategies
11:17:00 <ddevault> i.e. address space management
11:17:00 <klange> my windowing system lives in a userspace process, that's micro enough for me
11:17:00 <zid> your bytey allocator is just a virtual memory allocator
11:17:00 <heat> mainly because it lowers lock contention and slab allocators tend to want to hang onto objects, which the kernel can totally deal with under memory pressure by simply force freeing them itself
11:18:00 <heat> ddevault, wdym allocation strategies?
11:18:00 <ddevault> where to place memory mappings, when to reap page tables, etc
11:18:00 <ddevault> mapping pages larger than 4K
11:19:00 <heat> last I checked, linux did not reap page tables? I think they also just do a first-fit on address space gaps
11:21:00 <zid> I'm not sure what more advanced technique you need than "big slab of shit, divide it into bytes and hand them out"
11:21:00 <sham1> Slabs of shit
11:21:00 <sham1> Brilliant
11:21:00 <zid> like, what could it add, other than complexity
11:21:00 <ddevault> well, if you never reap page tables it gets much simpler
11:22:00 <zid> userspace mallocs also have the exact same constraints to deal with and also pick that approach
11:22:00 <heat> "get slab of shit, divide it and hand them out" is a very generic idea
11:22:00 <heat> what matters is how you get slabs of shit, how large they are, how you divide them and then how you hand them out
11:22:00 <zid> The point is that you don't "manage" it at all, you specifically don't divide them, or hand them out
11:23:00 <zid> You let the person fiddling the bytes do that, you do nothing except provide the initial slab of shit
11:23:00 <zid> see: brk
11:23:00 <heat> who is "you"?
11:23:00 <sham1> As for why no microkernels, the process of actually making sure that the kernel is actually micro is certainly a weird one
11:23:00 <heat> brk is not the greatest of ideas
11:23:00 <zid> the person doing the "virtual memory allocation strategy"
11:24:00 <klange> to determine if a kernel is micro you need to get out the kernel calipers
11:24:00 <dinkelhacker> heat: Thanks for your explanation. I think for starters I'll try to keep things simple and just use contiguous (allthough I'm actually already using virtual memory). So how does the management of the slabs actually work? Does kmalloc just get one and keeps track of how many start addresses and the number of bytes it gave out for each request? To make it even simpler: how would you handle allocations
11:24:00 <dinkelhacker> of couple of bytes within a single page? Like you still have all the fragementation problems, right_
11:24:00 <heat> brk is objectively a bad idea. see: internal fragmentation
11:24:00 <klange> and they're locked in tanenbaum's basement
11:24:00 <dinkelhacker> zid: also in kernel space?
11:25:00 <heat> dinkelhacker, have you read https://people.eecs.berkeley.edu/~kubitron/courses/cs194-24-S14/hand-outs/bonwick_slab.pdf ?
11:26:00 <dinkelhacker> heat: No I haven't thanks for the pointer. I'll check that.
11:26:00 <klange> I've been using essentially the same malloc implementation since the very start; wrote it before I even got into the OS stuff
11:26:00 <sham1> klange: ah yes, the international standard microkernel. But yeah, bootstrapping one from boot seems odd, at least if one thinks about it in as "absolute" terms as I do (can't have initramfs in kernel because filesystem doesn't belong, but then have to solve problem of loading initial stuff)
11:27:00 <heat> i highly recommend you do. the slab allocator essentially works with slab caches (which are just pools of objects T), which have lists of slabs, which are essentially just a few pages of memory with N objects inside them
11:27:00 <heat> allocation is basically going to one of the partial slabs (slabs with allocations in them, but not entirely allocated) and grabbing one. if you don't have a partial slab, you allocate a new slab
11:28:00 <heat> kmalloc/malloc is traditionally implemented by keeping N slab caches for a bunch of bucket sizes and allocating from them
11:28:00 <klange> sham1: Back in the day, toaru32 was excessively modular, and had those problems - had to load dozens of modules just to get a working FS; with Misaka I just abandoned all of that and embedded the basics (tmpfs, tarfs) right in the core kernel.
11:29:00 <zid> The correct amount of modulation is approximately what linux does
11:29:00 <heat> klange, there's no such thing as excessively modular, see EFI
11:29:00 <zid> if it wasn't best, why would linux do it, QED
11:29:00 <heat> how many modules do you want? yes.
11:30:00 <heat> also reminder that EFI firmware loads EVERY driver even if you don't have hardware for it or a need for it
11:30:00 <heat> really high quality stuff
11:30:00 <sham1> Well the problem wouldn't really end with that either. Another sore point I think about is why should there be a program loader in a microkernel
11:31:00 <sham1> Of course the answer is to solve the bootstrapping problem, but that's just it, it keeps rearing its ugly head up
11:32:00 <gog> all modules matter
11:34:00 <sham1> Of course, my thinking might be closer to a nanokernel than anything else, but it's still something I've considered to be a large problem with this stuff. I like μ-kernels but I don't know how they could be made micro enough
11:35:00 <zid> the reductive version of a microkernel is objeticvely shit
11:35:00 <zid> so it's just a question of how much are you going to compromise to make it not shit
11:38:00 <sham1> I'm an academic. What is this “compromise” you speak of?
11:39:00 <zid> You should increase the iron in your diet then
11:39:00 <zid> oh, academic
11:40:00 <sham1> Yeah, not anemic, thankfully
11:42:00 <sham1> But yeah, all these problems can be solved with enough engineering effort, obviously, but the complexity is always there
11:43:00 <dinkelhacker> heat: Cool. Thx again for the useful information. Once I've digested that I'll return with more questions :D
11:43:00 <heat> np
12:00:00 <heat> ddevault, in your efi stub, why do you pad your section names with 3 \0?
12:01:00 <ddevault> padding?
12:01:00 <ddevault> cargo-culted from linux's EFI stub?
12:01:00 <heat> i don't know
12:02:00 <heat> i looked at yours in hope of easier understanding
12:02:00 <ddevault> it aligns the next field on 8 so I'm gonna guess padding
12:04:00 <ddevault> FYI if you're trying to learn from my EFI stub
12:04:00 <ddevault> it exists just to get the bootloader online
12:05:00 <ddevault> after this the kernel is loaded and this executable is abandoned in memory to be reclaimed by the allocator at some point
12:05:00 <ddevault> so the priority for gooditude here is low
13:12:00 <heat> ddevault, halp
13:13:00 <heat> error: symbol '_text_start' can not be undefined in a subtraction expression
13:13:00 <heat> .long _ro_end - _text_start
13:13:00 <heat> but you seem to do the same thing?
13:15:00 <ddevault> see linker script
13:16:00 <ddevault> I define _etext et al there
13:16:00 <heat> do you need __pecoff_data_size because you can't have 2 undefined syms in an expr?
13:16:00 <clever> heat: i forget the details, but i heard something about there also being start/end symbols that are auto-generated, seperate from what the linker script adds
13:16:00 <ddevault> maybe?
13:16:00 <ddevault> I do
13:16:00 <ddevault> most of the measurements in the linker script
13:16:00 <ddevault> I recall having similar issues before I did so
13:17:00 <heat> I guess that's what I need
13:17:00 <ddevault> to minimize suffering I recommend lifting my linker script and header wholesale and moving on with your life
13:17:00 <ddevault> I offer you an MIT license for it
13:17:00 <heat> no need
13:17:00 <heat> I can't use your linker script anyway lol
13:17:00 <ddevault> rip
13:17:00 <heat> thanks anyway
13:18:00 <ddevault> I might also be open to some effort to generalize this into a bootloader which is reusable outside of helios
13:18:00 <ddevault> to just have an EFI bootloader which loads an ELF file and moves on with our lives
13:18:00 <heat> my problem is that I'm doing like _text_end - _text_start, both of which are undefined syms that get defined somewhere else
13:18:00 <heat> where what I really need is something like CONST - und or local_sym - und
13:19:00 <ddevault> what's your ultimate goal here?
13:19:00 <heat> where you can't, you do things like .long __pecoff_data_size and do __pecoff_data_size = ...; in the linker script
13:19:00 <heat> EFI boot my kernel directly
13:19:00 <ddevault> gotcha
13:19:00 <ddevault> might have more success looking at linux than at helios, then
13:19:00 <ddevault> linux loads the whole kernel as PE/COFF, mine just uses it as an intermedite step
13:20:00 <heat> I think i'm relatively near
13:20:00 <ddevault> though this __pecoff_data_size business is in linux, too
13:20:00 <heat> (if you forget the compat issues I'll get anyway lol)
13:20:00 <ddevault> also, stupid question
13:20:00 <ddevault> are you using -fPIC
13:20:00 <ddevault> and which arch
13:21:00 <heat> i'm doing this on x86_64 for now, no fPIC
13:21:00 <ddevault> would recommend PIC
13:21:00 <heat> I don't need that
13:21:00 <ddevault> EFI binaries are loaded wherever the firmware feels like it
13:21:00 <zid> pic is for sillies
13:21:00 <heat> my boot bits are physical-address-relocatable
13:21:00 <ddevault> so you'll end up writing relocation code in PIC assembly
13:21:00 <zid> like virtual memory doesn't exist
13:22:00 <ddevault> you have some MMU usage constraints in this environment, zid
13:22:00 <ddevault> but yeah, I don't like PIC generally
13:22:00 <zid> pic is something you add later for kslr after you're super advanced
13:22:00 <ddevault> another part of why I don't want my EFI stub to be my actual kernel
13:23:00 <heat> ddevault, also fwiw there's a flag to say "I have no relocs, load me only where I want to"
13:23:00 <ddevault> oh? interesting
13:23:00 <heat> IMAGE_FILE_RELOCS_STRIPPED
13:23:00 <heat> This indicates that the file does not contain base relocations and must therefore be loaded at its preferred base address. If the base address is not available, the loader reports an error.
13:24:00 <ddevault> introduces a dependency on your platform's physical memory layout, though
13:24:00 <heat> it does
13:24:00 <heat> such is life
13:25:00 <ddevault> out of curiosity, why is EFI support in scope for your project?
13:25:00 <zid> heat loves efi
13:25:00 <heat> why is it not?
13:25:00 <zid> massive boner for it
13:25:00 <ddevault> I mean what does EFI offer you that BIOS does not
13:25:00 <ddevault> just want to better understand your goals
13:25:00 <heat> I just want to boot in EFI
13:26:00 <ddevault> for me it offers a standard boot environment with a filesystem implementation to load stuff from
13:26:00 <heat> GRUB already supports it, I already support the runtime services side of the deal, I just want the EFI stub too
13:26:00 <heat> like the cool kids
13:26:00 <ddevault> ah
13:26:00 <heat> fwiw I want to ditch EFI as quickly as possible
13:26:00 <heat> fuck that shit
13:27:00 <ddevault> ExitBootServices is called before kmain in my system :)
13:28:00 <heat> half of EFI is broken as shit and vendor code is at best very very questionable
13:28:00 <zid> I just don't see what you would need to do before your OS boots
13:28:00 <heat> particularly runtime services
13:28:00 <ddevault> load modules from the filesystem, zid
13:29:00 <ddevault> and obtain a memory map
13:29:00 <zid> either stay inside the efi shell forever, or ignore it as fast as possible
13:29:00 <heat> ddevault, btw https://github.com/ardbiesheuvel/efilite
13:29:00 <ddevault> what am I looking at?
13:30:00 <netbsduser> rust
13:30:00 <zid> wow, heat is mean
13:30:00 <heat> minimal efi implementation for qemu arm64 virt
13:30:00 <heat> if you wanna ditch ovmf as fast as possible
13:30:00 <ddevault> neat
13:30:00 <heat> (which you should)
13:31:00 <ddevault> but I use ovmf on real hardware so I want a uniform environment
13:31:00 <ddevault> also I don't care
13:32:00 <heat> fair
13:37:00 <ddevault> I fucking hate serial consoles
13:37:00 <ddevault> why does it suck so much
13:38:00 <ddevault> also thanks for making me open my helios directory and start looking at my problems again
13:38:00 <heat> i like em
13:38:00 <heat> no problem
13:38:00 <heat> happy to help
13:44:00 <dinkelhacker> jesus I haven't thought about all that BIOS/stuff at all^^.. since I started on the raspberry pi 4 it was just "put your image here we will load it to this address"
13:44:00 <ddevault> problem is when you want to target ARM, not raspberry pis
13:45:00 <dinkelhacker> what do you mean by "target ARM"?
13:46:00 <dinkelhacker> Like you want to target a generic arm based platform
13:46:00 <dinkelhacker> ?
13:46:00 <ddevault> yes
13:46:00 <ddevault> and not a specific SoC
13:46:00 <ddevault> why the hell doesn't transmit work on my serial cable
13:47:00 <dinkelhacker> True story... in retrospect targeting the pi might have been an arrow to the knee..
13:49:00 <ddevault> soon enough I'm going to end up plugging buttons into the GPIO pins to control slides
13:50:00 <zid> I hope you're allowed a backup presentation
13:50:00 <zid> for when your pi crashes out
13:50:00 <ddevault> I'm not actually allowed any presentations, I was rejected and I'm still waiting to hear back about spare slots :(
13:50:00 <kaichiuchi> hm…
13:50:00 <kaichiuchi> I might raise a gcc bug
13:50:00 <ddevault> but the backup plan is to switch to laptop, easy enough
13:51:00 <ddevault> bah!
13:51:00 <ddevault> same issue on a different serial cable
13:51:00 <kaichiuchi> with Od/O0 on msvc/clang, where appropriate parameters are passed in registers
13:52:00 <kaichiuchi> with gcc you MUST use the register specifier to do that
13:52:00 <ddevault> RX and RX enable bits (and UART enable) are set to 1 in UARTCR
13:52:00 <ddevault> FIFOs enabled in UARTLCR_H
13:52:00 <ddevault> RX errors cleared in UARTECR
13:52:00 <ddevault> and... the RX flag never pops up in UARTFR when typing into the console
13:52:00 <ddevault> also GNU screen still really really really sucks
13:53:00 <ddevault> TX and RX enable bits*
13:58:00 <kaichiuchi> interesting
13:58:00 <kaichiuchi> clang 3.0.0 generates what i’d expect
13:58:00 <kaichiuchi> 4.1.2 does not
13:59:00 <heat> who cares about clang 3.0.0
13:59:00 <zid> That's 10 worse than gcc
13:59:00 <kaichiuchi> heat: this happens on trunk too
14:00:00 * ddevault types mindlessly into a console, watching nothing happen
14:02:00 <ddevault> I've reproduced this one bizzare serial problem on two rpis, three serial cables, and minicom and screen on the host
14:05:00 <ddevault> flow control. gdi
14:17:00 <heat> ddevault, actually this was pretty painless
14:17:00 <heat> I had an issue with the PE header alignment
14:17:00 <heat> the rest Just Works
14:17:00 <ddevault> ah
14:17:00 <ddevault> nice
14:18:00 <sham1> Alignment, it's dreadful
14:18:00 <ddevault> whyyyy doesn't my serial work
14:18:00 <ddevault> it worked exactly once
14:18:00 <sham1> It's also being dreadful
14:22:00 <kaichiuchi> ok I think I’m going to file a bug and embarrass myself
14:23:00 <heat> good
14:23:00 <ddevault> ...huh
14:23:00 <heat> it's what you deserve
14:23:00 <ddevault> ah, I see
14:23:00 <ddevault> I spammed my keyboard into the console out of frustration and keys started showing up after a moment
14:23:00 <ddevault> turns out I was waiting for the FIFO to be full, not to be not-empty
14:25:00 <kaichiuchi> heat: no it isn’t
14:25:00 <kaichiuchi> :(
14:25:00 <kaichiuchi> I would probably wager this is violating the ABI
14:26:00 <sham1> Stop violating the ABI without consent
14:26:00 <sham1> In fact, just stop violating the ABI
14:26:00 <kaichiuchi> “oh baby ABI let me spill all of my parameters to your stack”
14:27:00 <sham1> <.<
14:32:00 <kaichiuchi> :(
14:39:00 <ddevault> https://mirror.drewdevault.com/helios-squared.mp4
14:39:00 <ddevault> mission accomplished
14:44:00 <sham1> Now cube it!
14:44:00 <GeDaMo> That's not squared, it's rectangulared! :P
14:46:00 <sham1> I could have also said to now raise it to a 3/2s power, but that's not quite as fun as cubing, so now it's a factor of 6
14:46:00 <sham1> And by factor I mean power
14:53:00 <ddevault> finished an ambitious demo with time to spare
14:53:00 <ddevault> and... still no slot to present it in
14:53:00 <ddevault> womp womp.
17:14:00 <heat> hey guys
17:14:00 <heat> i found a new typo in the efi spec
17:14:00 <heat> what do I get
17:18:00 <Ermine> heat you rock!
17:18:00 <heat> i am not a rock Ermine
17:18:00 <Ermine> that's why there's no 'are'
17:48:00 <zid> heat: To bask in the afterglow of knowing that EFI is bad, and then finally proving it without a shadow of a doubt
17:48:00 <zid> Your sense of taste is unrivalled and your foresight perfect
17:48:00 <heat> pog
17:50:00 <heat> zid, did you know there was sandy bridge fw that gave you a bad memory map that told you some intel gfx allocated range was free
17:50:00 <heat> so you tried to use it and it went boom
17:52:00 <zid> you mean a bad bios
17:52:00 <zid> plenty of those around
17:52:00 <zid> I don't have GMA though so I am immune
17:52:00 <heat> there are also these ranges of memory you're only supposed to use before booting the OS but it used them anyway when calling into the fw to set the virtual addressing layout
17:53:00 <heat> now there's drama in arm64 where there are devices that *need* you to set the virtual address map and devices where you must not set it
17:53:00 <heat> this is so fucking broken
17:53:00 <heat> it's depressingly hilarious
17:54:00 <heat> it has been around for 20 fucking years
17:55:00 <zid> have they considered having a BIOS
17:57:00 <heat> noooooOOOOOooooOOOoooooOOOoooooo
17:57:00 <zid> They should, it's great
17:57:00 <heat> - person booted on EFI right now
17:57:00 <zid> You can write bootloaders on top of it, and the bios handles all the grotty dtails
18:05:00 <heat> ddevault, btw your stub is kind of wrong
18:05:00 <zid> only kind of? That's good for EFI
18:05:00 <heat> ddevault, .word .Lpe_header - .L_head <-- should be .short in arm64, with a .align 4 right after it since the PE header needs to be 4-byte aligned
18:06:00 <heat> your solution should work right now but it's really non-obvious
18:07:00 <gog> hi
18:08:00 <ddevault> heat: busy atm, would you mind emailing me the details so I can follow up later?
18:08:00 <zid> so it fails to warn on 16bit truncates like it should, basically?
18:11:00 <kaichiuchi> hi
18:12:00 <heat> ddevault, just take a note, what I said is all there is to it
18:13:00 <heat> instead of .short (16 bit) + align 4 (2 bytes padding) you're doing a non-obvious .word (32-bit in arm64, 16-bit in x86_64)
18:14:00 <ddevault> aight
18:14:00 <ddevault> thanks
18:14:00 <zid> spooky size changing
18:15:00 <heat> tbf arm is the correct behavior here
18:15:00 <heat> word should be native word
18:15:00 <heat> not "native word in 1985"
18:16:00 <zid> yea but 4 people got to not update their .s files!
18:19:00 <kaichiuchi> heat: something here reminds me of you at work
18:20:00 <heat> is it a clogged toilet
18:22:00 <geist> also re: align in assmbly, be careful it’s not a power of two
18:23:00 <geist> if using gas at least, i’ve found it to be safer to use .balign, which is always bytewise
18:23:00 <zid> I fucked up an align once
18:23:00 <geist> whereas depending on arch, .align is sometimes byte, sometimes power of 2
18:23:00 <zid> I made nasm ran out of memory and crash
18:23:00 <zid> I asked for align 2^32 I think
18:23:00 <geist> yeah that’ll do it
18:24:00 <zid> gas's directive syntax is like "What if we were like C and had IDB and UB and stuff? That'd be fun"
18:25:00 <dh`> have you ever looked at the gas sources?
18:25:00 <zid> no I'm not an idiot dh` what do you take me for
18:25:00 <zid> "have you ever put your finger into a blender?"
18:26:00 <heat> i use llvm as
18:26:00 <geist> i give it a lot of slack, since gas can historically please no one. if it does whatever the prevailing arch’s syntax says, folks bitch that it’s inconsistent between arches, if it tries to invent its own syntax, they bitch that it differs from $vendors syntax
18:26:00 <heat> it's nice
18:26:00 <geist> so it just tries to do what it can
18:26:00 <zid> "You know when you stick your head into an oven?"
18:26:00 <kaichiuchi> heat: the new guy looks like BAZINGA
18:26:00 <geist> the fundamental issue is there’s no one consistent syntax between all arches, but gas tries to thread a particular needle
18:26:00 <heat> llvm binutils have made great significant progress such as, erm
18:26:00 <heat> errors with descriptions
18:26:00 <zid> heat: and that thing where they erm
18:26:00 <heat> and colors!
18:27:00 <zid> and that other thing!
18:27:00 <geist> yah a lot of that was because fuchsia. llvm as and binutils stuff was pretty bad 5 or 6 years ago, and we had a mandate to at east match gas/objdump/etc for what fuchsia needed
18:27:00 <geist> lots of linker script improivements, etc
18:27:00 <gog> hi
18:27:00 <heat> geist, they should've forced at&t on every arch
18:27:00 <heat> geist, also a fair amount of that AFAIK has also been clang-built-linux
18:27:00 <zid> I also think that
18:27:00 <heat> and the android teams
18:27:00 <geist> yep
18:28:00 <geist> but i think we were doing that somewhat before the clang-builds-linux push
18:28:00 <geist> but yeah, lots of stuff coming out of the google toolchain teams
18:29:00 <geist> also iirc the linux stuff still using binutils. freebsd did too, clang the compiler + binutils linkers/etc
18:29:00 <geist> though freebsd may have fully switched
18:29:00 <heat> you can build the linux kernel with llvm only
18:29:00 <heat> LLVM=1 make ...
18:29:00 <geist> word
18:29:00 <heat> it Just Works for the most part
18:30:00 <heat> funnily enough glibc can't be built with llvm
18:30:00 <heat> (and neither can gcc if you're cross-compiling it)
18:30:00 <geist> yah i’m going on a few year old data
18:31:00 <heat> srsly, the gcc thing is kind of annoying. it compiles well until it tries to yank some multilib data out of $CC
18:31:00 <geist> guess there’s no exact combination of an OS/distro that wants to use clang/llvm + glibc
18:31:00 <heat> sure there is, CrOS
18:32:00 <zid> I'd be cross too if I had to use clang + glibc
18:32:00 <zid> ba-dum tish
18:32:00 <geist> oh that reminds me to look into crosvm. been meaning to, just keep forgetting to
18:32:00 <geist> and now i know a bit more rust i may actually be able to read its source
18:33:00 <geist> my guess is it’ll be underwheming.
18:33:00 <geist> ie, it works, but isn’t very configurable, sinceits designed to do one thing
19:06:00 <ddevault> heat: pushed fix, thanks
20:17:00 <Bitweasil> Oh ffs. https://github.com/signup
20:17:00 <bslsk05> ​github.com: Join GitHub · GitHub
20:17:00 <Bitweasil> They've ruinsed it.
20:18:00 <Bitweasil> Like, "Literally unusably slow trying to render that starfield on my machines."
21:10:00 <ddevault> sign up for sourcehut instead :)
21:58:00 <vdamewood> Mutabah klange air sortie geist: Can I talk to one of you?
21:58:00 * Mutabah is away (Sleep)
22:19:00 <kaichiuchi> i have a question
22:20:00 <heat> geist, it's quite possible it's more versatile these days
22:20:00 <kaichiuchi> the system V ABI provides two registers used for returning values
22:21:00 <heat> there was a patch floating around edk2 for OVMF on crosvm
22:21:00 <kaichiuchi> now, of course, in C you can only return one value
22:21:00 <kaichiuchi> am I right in assuming something like `int f(int *retval) { *retval = 1; return 666; }` will store 1 and 666 in those two registers?
22:22:00 <heat> no
22:22:00 <kaichiuchi> okay, I didn't think so
22:22:00 <heat> https://godbolt.org/z/rqffGT7PY
22:22:00 <bslsk05> ​godbolt.org: Compiler Explorer
22:22:00 <kaichiuchi> jesus christ
22:22:00 <kaichiuchi> that was like... 10 fucking seconds
22:22:00 <kaichiuchi> maybe less
22:22:00 <kaichiuchi> maybe I should do that
22:23:00 <heat> if you do e.g two u32 values, they get packed onto rax
22:23:00 <kaichiuchi> right, that's what I'd expect
22:23:00 <heat> if you do 4 u32, they get packed onto rdx:rax
22:23:00 <kaichiuchi> so is it really just to return 128-bit stuff?
22:24:00 <heat> hm? the example I gave you returns a struct in rdx:rax
22:24:00 <heat> not 128-bit stuff
22:24:00 <kaichiuchi> oh okay okay
22:24:00 <kaichiuchi> I see
22:28:00 <heat> kaichiuchi, in general returning works by packing values onto rax, then rdx:rax, then on SIMD registers if SIMD is enabled, then they fall back to doing a stack allocation and passing a pointer
22:28:00 <kaichiuchi> gotcha
22:28:00 <heat> which is why returning complex structures has very little disadvantages
22:29:00 <heat> at the end of the day struct S {unsigned int a[500];}; struct S foo(); and void foo(struct S*); will have identical codegen
22:31:00 <heat> in fact in C++ you probably want to favour struct S so it directly constructs S
23:24:00 <heat> mjg, https://lore.kernel.org/linux-mm/Y8g74YJVonMHpWw%2F@casper.infradead.org/T/#md12c30055c4644bfdcf2dae1212c24596eb404ec
23:24:00 <bslsk05> ​lore.kernel.org: [PATCH 00/41] Per-VMA locks
23:24:00 <heat> mjg, cpu_relax() in the middle of cmpxchg just Feels Right(tm)
23:28:00 <mjg> >
23:28:00 <mjg> If you are right, feel free to go and remove every cpu_relax() under the
23:28:00 <mjg> kernel/locking directory.
23:28:00 <mjg> wow
23:28:00 <mjg> that's one hot take
23:29:00 <heat> hilf is the "you're an obnoxious cunt" guy from al viro
23:29:00 <mjg> :d
23:29:00 <mjg> i don't remember that comment but viro is the guy to say somethingl ike that
23:31:00 <mjg> oh someone is trying to scale the mapple syroup
23:31:00 <heat> https://lore.kernel.org/lkml/Y5fzN9ZWgXFyT+fU@ZenIV/
23:31:00 <bslsk05> ​lore.kernel.org: Re: [syzbot] WARNING in do_mkdirat - Al Viro
23:32:00 <heat> mjg, could you explain why pause is so detrimental to perf?
23:33:00 <heat> I don't get the "well, you grab the exclusive bus anyway"
23:33:00 <heat> don't you eventually lose it by just executing more instructions?
23:36:00 <Bitweasil> Isn't "pause" the "I'm in a spin loop, you can seriously go hyperthread something else now" hint?
23:37:00 <heat> Bitweasil, yes
23:37:00 <Bitweasil> *nods* I'd expect it to hurt performance in hot loops, for sure.
23:37:00 <Bitweasil> That's kind of the point. :)
23:38:00 <heat> note to self: do not click links in hexchat under the possibility of crashing the whole client
23:39:00 <heat> per our friend here: To my understanding on said architecture failed cmpxchg still grants you
23:39:00 <heat> exclusive access to the cacheline, making immediate retry preferable
23:39:00 <heat> when trying to inc/dec unless a certain value is found.
23:45:00 <mjg> heat: because the E owner fails to take advantage of it
23:45:00 <mjg> first, MESI aside, consider a case where n cpus tried a literally the same time
23:46:00 <mjg> all failed but 1
23:46:00 <mjg> now you have n - 1 cpus chilling on vacation for the duration of the pause instruction
23:46:00 <mjg> with nobody even trying
23:46:00 <heat> but for how long do you keep the excl cache line is my question?
23:47:00 <heat> https://godbolt.org/z/cjMbvxYdh for instance in this simple cmpxchg atomic inc
23:47:00 <bslsk05> ​godbolt.org: Compiler Explorer
23:47:00 <mjg> until smeone else takes it, for exampel by also executing the op
23:47:00 <mjg> or reading from the cacheline
23:47:00 <heat> really?
23:47:00 <heat> do you just not lose it on the next cycle or whatever?
23:47:00 <mjg> you lose it soon(tm) when faced with numerous cpus fucking here
23:48:00 <mjg> which is why it is crucial to retry asap
23:48:00 <heat> hmm ok I see
23:48:00 <mjg> and again
23:48:00 <mjg> everyone but 1 failing == everyone but 1 just chillin'
23:48:00 <heat> and this doesn't work for e.g spinlocks because you're not guaranteed to make progress in a spinlock loop, in a timely manner
23:48:00 <heat> right?
23:49:00 <mjg> that's weirdly stated in this context
23:49:00 <mjg> the key with spinlocks is that you are ultimatley waiting for someone else to set a speific value
23:49:00 <mjg> and you have no idea how long that's gonna take
23:49:00 <mjg> so the thing to do is to wait in least disruptive manner
23:50:00 <heat> whereas a cmpxchg inc is just a tight loop in 4 instructions where you either make progress or try again very quickly (and have a good chance of succeeding), a spinlock can block for many more cycles and shit
23:50:00 <mjg> no such consideration when flipping the counter
23:50:00 <heat> so forward progress may be unlikely, so pause is a good idea
23:50:00 <heat> yeah exactly
23:51:00 <mjg> well if you feel this way remove all pauses from onyx!
23:51:00 <heat> what the hell does pause even do?
23:51:00 <heat> besides the generic intel sdm meaning
23:51:00 <heat> hyperthread yield? is that it?
23:52:00 <Bitweasil> Far as I know, it's just a hint to the scheduler that you ought go run the other hyperthread harder for a while.
23:53:00 <mjg> it fucks off on the thread
23:53:00 <mjg> but crucially, any ht considerations aside, fucks off from the cacheline
23:53:00 <mjg> so whoever is blocking your progress has easier time cmpleting what they need to do
23:54:00 <mjg> without you constantly stealing it
23:54:00 <heat> ah so that drops the E?
23:54:00 <mjg> pause does not drop E to my knowledge
23:54:00 <mjg> i'm saying when you pause you are not doing memory accesses
23:54:00 <heat> so what does "fuck off from the cacheline" mean?
23:54:00 <heat> ah ok
23:54:00 <mjg> so someone else has easier time doing the needful
23:55:00 <mjg> if you want smoe fun patch onyx to never pause
23:55:00 <heat> and bench?
23:55:00 <mjg> just literally cmpxchg in a tight loop when trying to lock
23:55:00 <mjg> you will see perf going down the shitter
23:55:00 <heat> fwiw I think my spinlocks just use xchg
23:55:00 <heat> or some primitive I have did. no idea why
23:56:00 <moon-child> xchg is fine
23:56:00 <mjg> if (__atomic_compare_exchange_n(&lock->lock, &expected_val, what_to_insert, false,
23:56:00 <mjg> __ATOMIC_ACQUIRE, __ATOMIC_RELAXED))
23:56:00 <mjg> you pessimal motherfucker!
23:56:00 <mjg> break;
23:56:00 <heat> ... does not seem to. but I swear I had seen a xchg somewhere
23:56:00 <mjg> while (__atomic_load_n(&lock->lock, __ATOMIC_RELAXED) != 0)
23:56:00 <mjg> cpu_relax();
23:56:00 <mjg> i think intel demo spinlocks are xchg
23:57:00 <moon-child> mjg: by the by, thoughts on hle?
23:57:00 <mjg> moon-child: does not work?
23:57:00 <mjg> funny you bring it up, it was recently flamed
23:57:00 <moon-child> :\
23:57:00 <moon-child> why
23:57:00 <mjg> i mean there is hw bugs which make it unusable so convo stops there
23:57:00 <moon-child> oh sure
23:58:00 <moon-child> I meant notionally
23:58:00 <heat> mjg, I cannot actually xchg because that only works if the value you want to swap with is the same as the value that is there if the spinlock is locked
23:58:00 <heat> as in if your spinlock can only be 0 or 1
23:58:00 <mjg> whack that pause and dup1_threads -t 8
23:58:00 <mjg> you will see a massive drop
23:58:00 <mjg> assuming you got spinlocks to protect the fd table
23:59:00 <heat> yessir yes indeed
23:59:00 <mjg> moon-child: i don't have a strong opinion
23:59:00 <heat> mjg, what's pessimal in my code btw?
23:59:00 <mjg> you instantly re-read
23:59:00 <mjg> you should do { cpu_relax(); } while ....