channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched
#osdev2 = #osdev @ Libera from 23may2021 to present
#osdev @ OPN/FreeNode from 3apr2001 to 23may2021
all other channels are on OPN/FreeNode from 2004 to present
http://bespin.org/~qz/search/?view=1&c=osdev2&y=24&m=7&d=18
00:07:00 <vdamewood> FWIW, pronouncing J as English pronounces a consonental Y happens in every Germanic language except English, and every Slavic language that uses the Latin script.
00:31:00 <karenthedorf> So intel docs say there's a `JMP m16:64` encoding. Why will no assembler I've tried emit it? Seems saner than lretq when I need to long jump.
00:38:00 <Mutabah> I think that's `ljmp [mem_location]`?
00:38:00 <Mutabah> or, `jmp far [mem_location]`
00:41:00 <heat> saner how?
00:41:00 <heat> it's literally the same
00:41:00 <heat> lretq is ljmp (%rsp) with a stack adjustment along the way
00:41:00 <Mutabah> tbh, I would say it's less sane
00:42:00 <Mutabah> as you're telling the reader/cpu that you're returning - when you're actually not
00:42:00 <Mutabah> It likely doesn't matter for a once-off at bootup, but it does likely confuse the branch predictor
00:47:00 <karenthedorf> Doesn't fsck up the branch predictor or shadow stack. (But yes, probabally not relevant as a one off early in boot)
00:49:00 <heat> lol
00:50:00 <heat> seriously, immeasurable in the large scheme of things (booting and/or booting a secondary core)
00:51:00 <karenthedorf> Yeah, it's funky anyway. It wants an absolute address (and selector) in a memory location, but I have no idea how to get that value into a register in the first place tbh.
00:52:00 <karenthedorf> While lret can take a relative address I think?
00:52:00 <heat> nope
00:53:00 <Mutabah> https://github.com/thepowersgang/rust_os/blob/master/Kernel/Core/arch/amd64/start.asm#L190-L196
00:53:00 <bslsk05> github.com: rust_os/Kernel/Core/arch/amd64/start.asm at master · thepowersgang/rust_os · GitHub
00:53:00 <Mutabah> Hey look, a far indirect jump
00:54:00 <heat> :(
00:54:00 <heat> i'm lret gang, my boot.S has 8 of them
00:54:00 <Mutabah> although, interestingly, I use `jmp 0x08:start64` not too far above
00:55:00 <heat> most of them with painful absolute address calculation
00:55:00 <karenthedorf> That addressing mode doesn't exist in long mode though?
00:55:00 <Mutabah> aaah, the comment says why - yep
00:55:00 <Mutabah> The first version is jumping _and_ entering long mode
00:55:00 <Mutabah> while the second is already in long mode, just ensuring that CS is correct
01:01:00 <gog> i lretq
01:01:00 <gog> i like that
01:03:00 <gog> if the branch predictor can't read my mind then what good is it
01:04:00 <heat> why does your mind have branches
01:04:00 <gog> my mind is a whole forest
01:04:00 <gog> and it's on fire
01:05:00 <gog> also i really abuse inline assembly
01:06:00 <heat> as you should
01:07:00 <karenthedorf> https://pastebin.com/0ynNKvPX What a waste of my time. Thanks OCD.
01:07:00 <bslsk05> pastebin.com: static void flush_gdt(uint16_t cs, uint16_t ds){ uint64_t scratch0; - Pastebin.com
01:08:00 <karenthedorf> (Yes, I should not do this in inline asm)
01:08:00 <gog> nice
01:08:00 <heat> fucken
01:08:00 <heat> ahhhhh
01:08:00 <gog> i do it in inline asm too
01:09:00 <heat> i regret saying you should abuse inline assembly
01:09:00 <heat> please do not abuse inline assembly like this
01:09:00 <gog> https://github.com/adachristine/sophia/blob/main/kc/core/cpu/cpu.c#L137
01:09:00 <bslsk05> github.com: sophia/kc/core/cpu/cpu.c at main · adachristine/sophia · GitHub
01:09:00 <gog> heat look
01:10:00 <heat> this is how one should abuse inline assembly: https://github.com/heatd/Onyx/blob/master/kernel/include/onyx/x86/percpu.h
01:10:00 <bslsk05> github.com: Onyx/kernel/include/onyx/x86/percpu.h at master · heatd/Onyx · GitHub
01:10:00 <heat> features being so cutting edge not every compiler groks it? check
01:10:00 <heat> preprocessor spam? check
01:10:00 <gog> sick
01:11:00 <heat> condition code inline asm output? check
01:12:00 <gog> when you need to compile while you compile
01:12:00 <heat> https://github.com/heatd/Onyx/blob/master/kernel/include/onyx/x86/include/platform/jump_label.h#L36-L48 more dubious ideas
01:12:00 <bslsk05> github.com: Onyx/kernel/include/onyx/x86/include/platform/jump_label.h at master · heatd/Onyx · GitHub
01:12:00 <heat> it's always great when you need to google what the little letter after the % does
01:12:00 <karenthedorf> If clang and gcc agree on it, IDC, but yes, moving more of my inline assembly to actual assembler files == doubleplusgood
01:13:00 <gog> i forgot why i hated assembler files
01:13:00 <gog> there was a time peroid where i was trying to eliminate them
01:14:00 <heat> it's what the kids would say
01:14:00 <heat> woman moment
01:14:00 <heat> a man would write a whole OS in x64 assembly as a midlife crisis thing
01:15:00 <gog> my whoe life is a series of woman moments
01:16:00 <gog> also heat wtf misogyny
01:17:00 <heat> no never misogynous
01:17:00 <heat> disparage everyone
01:17:00 <heat> never focus the hate you could be spreading to everyone into a single group
01:19:00 <gog> :(
01:21:00 <heat> jk love you gog
01:27:00 <karenthedorf> https://pastebin.com/agkX0bsT There, no more inline asm.
01:27:00 <bslsk05> pastebin.com: .global flush_gdt.type flush_gdt, function.align 16flush_gdt: lea [.L - Pastebin.com
01:27:00 <heat> mov %rsi, %ds compiles?
01:27:00 <heat> well, assembles
01:28:00 <karenthedorf> appears to assemble to `mov %esi, %ds`
01:28:00 <kof673> (skims backlog) <game engine> <os> <language> and these 3 are 1. the ___ proceeds from the ___... "it is the same bird" j/k but not really
01:28:00 <heat> that makes more sense, still a little weird though
01:28:00 <karenthedorf> I changed it to esi
01:28:00 <heat> i always do mov %ax, %ds
01:28:00 <guideX> I kind of have a handle on what programs are now, but where should they come from? kind of obvious answers, an app store, the web, idk what do you guys think
01:29:00 <karenthedorf> Also I'm pushing %rdi when only %di would suffice.
01:29:00 <guideX> maybe package managers
01:29:00 <sauce> guideX: users should type them in from magazines
01:30:00 <karenthedorf> heat: mov %ax, %ds also assembles to mov %eax, %ds
01:30:00 <guideX> I was thinking, maybe this os, should expose the source code of every program alongside its visual representation
01:30:00 <guideX> so the user can jump in and edit it
01:31:00 <kof673> yes do that and work towards synthesis os :D
01:32:00 <karenthedorf> I think there's only one valid encoding of "mov %gReg, %sReg"?
01:37:00 <karenthedorf> So mov %ax, %ds and mov %rax, %ds both have encodings, there is no mov %eax, %ds in the intel manual. Lmao.
01:38:00 <karenthedorf> gcc assembler just emits `mov %ax, %ds` (opcode 0x8E) but decompiles it to mov `%eax, %ds`
01:38:00 <karenthedorf> Even if the original assembly specified %rax
01:39:00 <karenthedorf> You can only force it to emit `mov %rax, %ds` byt saying `REX.W mov %rax, %ds`
01:58:00 <geist> huh i wonder if that's a valid combination
01:59:00 <geist> i guess it just ignores the rex in that case
01:59:00 <geist> and/or what if you specify the top bit i the REX? does it let you move from r8 or whatnot?
02:00:00 <karenthedorf> It's the correct encoding for `mov %rax, %ds` if that's what you're asking.
02:00:00 <geist> hmm, actually that must be a two byte opcode, since 0x8e can't encode the source rg
02:00:00 <geist> yeah, but REX has more bits than 'i'm just 64bit'
02:01:00 <karenthedorf> Oh I haven't tried messing with the other bits. x86 instruction encoding is voodoo
02:01:00 <geist> yah i was looking at https://www.sandpile.org/x86/opc_1.htm
02:01:00 <bslsk05> www.sandpile.org: sandpile.org -- x86 architecture -- 1 byte opcodes
02:01:00 <geist> which just says it's MOV Sw,Mw
02:02:00 <geist> so i guess the rest of that must come from another byte?
02:02:00 <karenthedorf> `48 ff 2c 24`
02:02:00 <geist> unless AX is implied i guess
02:03:00 <karenthedorf> You can move from registers other than az
02:03:00 <karenthedorf> *ax
02:03:00 <geist> ah, maybe moving from ax is the single byte, but otherwise it has to go to a multibyte ff one
02:04:00 <karenthedorf> Let me find it in the manual again
02:05:00 <karenthedorf> Disregard that, wrong instruction
02:05:00 <karenthedorf> `48 8e de`
02:05:00 <geist> ah now we're talking
02:05:00 <geist> yeah the de is probably describing the two registers
02:05:00 <geist> and the 48 is the rex prefix
02:06:00 <karenthedorf> 48 is REX.W, 8e is mov gReg->sReg and de is the weird register encoding I can't parse
02:06:00 <karenthedorf> I'm guessing the 'useful' bit of adding the REX.W prefix is allowing you to mov from r11 and the like, because it just truncates the value anyway
02:06:00 <karenthedorf> With the bits other than .W
02:07:00 <geist> yah REX.R is i think the bit
02:08:00 <geist> could be a MOD R/M https://www.sandpile.org/x86/opc_rm.htm
02:09:00 <karenthedorf> The spec says it's a MOD R/M yeah
02:09:00 <karenthedorf> But I tried to handright a MOD R/M earlier with .byte and it didn't make much sense.
02:09:00 <geist> yah the de doesn't quite line up, since that would to me tell me it's DX, DS
02:09:00 <karenthedorf> *handwrite
02:09:00 <geist> 'de' that is
02:10:00 <karenthedorf> Yeah, should be rsi, not rdx.
02:10:00 <geist> ah yeah RSI
02:10:00 <geist> but you see how the REX.R bit in that case escapes it out, basically adding a top bit to get you 4 bits of register encidong
02:11:00 <karenthedorf> The opcode was `mov %rsi, %ds` sorry. was using %rax before then grabbed it out the disassmbly of my code without actualy stating I'd copied an example using %rsi.
02:11:00 <geist> ah okay that perfectly lines up then
02:11:00 <geist> a mystery solved
02:14:00 <karenthedorf> Argh, brain, get out the OCD loop of wondering if it is better to push a 16-bit (correctly reflecting what is accessed by the cpu) or just push a 64-bit value (keeps the stack aligned to 64-bits, smaller instruction encoding). It doesn't matter.
02:14:00 <karenthedorf> The cpu doesn't care, since the extra bits are all 0 anyway
02:14:00 <geist> yah
02:14:00 <geist> though the 32 may be smaller since no prefixes required
02:15:00 <karenthedorf> `push %edi` fails to assemble
02:15:00 <geist> that is really strange
02:15:00 <karenthedorf> `push %rdi` is one byte, since I'm in long mode.
02:15:00 <geist> oh yeah i guess so. that's a case where REX prefix *isn't* needed
02:16:00 <geist> and the 16 bit verion probably uses 0x66
02:16:00 <karenthedorf> `66 57` vs `57`
02:17:00 <karenthedorf> So less code, better stack alignment... in return for briefly using 3 bytes of stack in a leaf function. Why am I worrying about this.
02:18:00 <karenthedorf> Also looking at the weird and wacky encoding compilers pick for various sizes of NOP is interesting.
02:24:00 <zid> There are no 16 bit pushes on amd64 btw
02:24:00 <zid> or was it just 8 there wasn't, maybe, ignore me
02:47:00 <karenthedorf> There appears to be no 32bit pushes in long mode. Nor 8 but. At least of the form "push register", I didn't try immediates.
02:50:00 <zid> huh yea, no 32 and no 8
02:51:00 <zid> I knew it was missing stuff, I was *not* expecting there to just.. be a gap
02:54:00 <karenthedorf> Good old legacy stuff.
02:54:00 <karenthedorf> In 32bit there was the 'default' if you didn't apply a prefix to the opcode, and a '16-bit override' prefix.
02:55:00 <zid> that's all modes
02:55:00 <karenthedorf> In 64bit, the default is not 64bit, but there's no 32-bit prefix
02:55:00 <karenthedorf> *is now
02:55:00 <karenthedorf> That's my understanding anyway
02:55:00 <zid> 66s and 67s just swap to the 'other' one, 64 is weird in that it has two
02:55:00 <zid> so real mode could interact with eax by slapping on the prefix, and protected mode could interact with ax by slapping it on
02:56:00 <zid> but long mode is weird and like, defaults to 32 some of the time, and needs the prefix to make it 64bit instead
02:56:00 <karenthedorf> Yup. But 64bit is a mix of "defaults to 64bit, can slap size-override on for 16bit, there is no 32-bit" and "defaults to 32bit, can slap the size-override prefix on for 16bit, or REX.W for 64 bit"
02:57:00 <zid> yup, fun fun
02:58:00 <karenthedorf> Bring on that project intel floated of nuking all the legacy crap from x64.
02:58:00 <zid> they're eventually removing real mode with x86s
02:58:00 <zid> won't be for a LONG time though
02:59:00 <zid> it isn't going to change how anything works though
02:59:00 <zid> you'll just.. stop being able to disable bits in cr2/cr4 to get back there
02:59:00 <karenthedorf> Yeah, nuking real mode and the legacy pic and some other stuff iirc. It's a bootstrap paradox though, because no OS will boot on such hardware, and the hardware is useless without an OS to run on it.
02:59:00 <karenthedorf> no current OS will boot*
02:59:00 <zid> uefi is already a thing
03:00:00 <zid> so every os will currently boot
03:00:00 <zid> it breaks bootable cd-roms only, more or less
03:00:00 <karenthedorf> https://cdrdv2.intel.com/v1/dl/getContent/776648
03:00:00 <zid> and x86s will certainly be xeon only when it launches in 2030, and not be in consumer chips until 2040 or whatever, so it's not really going to matter if you can't boot a cd-rom on it
03:05:00 <karenthedorf> I think there's a problem booting secondary cores/cpus without OS changes?
03:05:00 <kazinsal> considering the shitfest that is the last two core generations I would expect that intel will never manage to do a widespread release of x86-S
03:05:00 <zid> depends if you're amd or intel in that regard I think
03:05:00 <zid> and all this will be sorted out in literal days, years in advance anyway
03:05:00 <karenthedorf> Intel's white paper seems to prepare for that, adding a 64-bit startup IPI as a separate feature which can presumably appear a generation or two earlier.
03:06:00 <zid> I think coreboot already has x86s whitepaper support
03:06:00 <karenthedorf> I do wonder how much smaller in terms of die area/transistor count an x86s chip would be.
03:06:00 <zid> The thing is, intel has to test all this shit, and they test it on linux, then just upstream any of the changes they make, years before the chips are even available
03:06:00 <zid> 0.01%
03:06:00 <zid> at best
03:07:00 <zid> It's mainly just to they have less test area, it's a mulitplier to everything else
03:07:00 <zid> "did we break interrupts with this change?" "Okay but what about in ring -1" "okay but what about in protected mode with an invalid selector" "okay but what about in real mode" etc
03:08:00 <zid> without real mode or ring0 protected mode, a lot of testing gets a lot quicker
03:08:00 <karenthedorf> And chance for bugs, like that one where you could remap one of the PICs over SMM memory and escalate to ring -2. Can't do something similar if support that hardware doesn't exist any more.
03:09:00 <karenthedorf> Can't remember if that was the lapic or the 8259 PIC
03:09:00 <zid> lapic
03:09:00 <zid> it had an MSR base address that was unchecked
03:09:00 <zid> 8259 doesn't have a memory interface at all
03:10:00 <karenthedorf> x86s' 'removed features' list is a nice list of things I can just *never implement support for* in my own OS.
03:10:00 <zid> tbh you shouldn't anyway, in $CURYEAR
03:11:00 <karenthedorf> That's what I mean
03:11:00 <zid> amd64 is already 20 years old
03:11:00 <karenthedorf> If intel thinks it's so obsolete to the point they want to remove it, why should I care.
03:11:00 <zid> *today* there's no reason to touch real mode, much less in 20 more years when x86s is available
03:11:00 <zid> and then 20 more until it's defacto
07:59:00 <heat> karenthedorf, fyi that was the LAPIC (or IO APIC, can't remember), not the PIC
08:11:00 <heat> also the legacy PIC isn't an architectural thing, it's a platform thing
08:48:00 <karenthedorf> Yeah, legacy PIC is a PC thing, not a true x86 thing like the LAPIC. But non-PC x86 is kind of esoteric nowadays.
08:49:00 <karenthedorf> Does the I/O APIC cound as a platform thing since it's part of the chipset and not the core? (Or am I wrong and it is part of the cpu proper?)
08:49:00 <zid> PS4 isn't *that* esoteric :P
08:52:00 <nikolar> Are there any x86 embedded boards or something
08:53:00 <zid> lots, but they tend to be PCs
08:53:00 <nikolar> Makes sense I guess
08:53:00 <nikolar> I know there were weird 486 based devices in the 90s
08:56:00 <GeDaMo> https://en.wikipedia.org/wiki/PC/104
08:57:00 <nikolar> Something like that
09:00:00 <zid> yea I should clarify I mean a PC from the software side
09:01:00 <zid> they often eschew the hardware features of PCs, like AT/ATX/PS/2 etc
09:01:00 <nikolar> Yeah I got it
09:01:00 <zid> am I allowed to be a PC if I have no ps/2 port!
09:03:00 <karenthedorf> What if your keyboard control isn't an external chip that's hooked back into one of the CPU's address bus lines?
09:03:00 <karenthedorf> But actually not that whole franken mess.
09:04:00 <kazinsal> don't have to worry about how your keyboard controller works if you only accept input via RS-232 or virtual terminals attached to berkeley sockets
09:04:00 <kazinsal> this shitpost brought to you by: network janitor gang
09:05:00 <GeDaMo> https://en.wikipedia.org/wiki/Terminal_server
09:06:00 <karenthedorf> You do if you want the A20 line to work correctly and not have your addresses wrap every MB. :D
09:07:00 <karenthedorf> I should get back to trying to get the APIC(s) to work. Urgh. I enabled the LAPIC (on the boot processor), but apparently I also need to configure the I/O APIC? Also I'm supposed to disable the legacy APIC I think?
09:07:00 <zid> there's a legacy apic?
09:07:00 <kazinsal> apic stuff is weird
09:08:00 <kazinsal> ideally you shouldn't really *need* to use the apic for interrupt routing
09:08:00 <karenthedorf> There's 2, actually (or one bit of hardware that emulates two physical chips)
09:08:00 <zid> you mean the PIC?
09:08:00 <kazinsal> almost any PCI card made after like the mid 90s shouuuuld support MSI if not MSI-X
09:08:00 <zid> It's virtualized these days, but you don't need to know that to tell it to shut up
09:08:00 <kazinsal> which is a nice low-latency interrupt routing system that bypasses the IOAPICs
09:09:00 <karenthedorf> Yeah, but I just want to get interrupts from the serial port.
09:09:00 <kazinsal> what I do is use legacy PIC routing on core 0 for ISA stuff and run everything else on MSI/MSI-X
09:09:00 <zid> APIC is the LAPIC + IOAPIC
09:09:00 <karenthedorf> Which I think means I have to set up the I/O APIC.
09:09:00 <kazinsal> so serial/legacy timers run on a legacy PIC emulation on core 0's LAPIC
09:09:00 <kazinsal> and everything else gets shunted to assigned cores via MSI/MSI-X
09:10:00 <zid> yea LAPIC is what each cpu does, IOAPIC is a per-cpu interrupt controller part, if memory serves?
09:10:00 <zid> redirection table from external sources to specific LAPICs
09:10:00 <kazinsal> meaning I don't ahve to deal with the nightmare circus that is PCI/IOAPIC routing
09:10:00 <karenthedorf> zid: That's my understanding. There's also the legacy 8259 PIC which you just need to command to commit seppuku and then pretend it doesn't exist.
09:11:00 <zid> yes, the PIC, which you tell to shut up
09:11:00 <karenthedorf> Yeah, I called it the "legacy APIC" earlier and that's wrong, it's not 'A'.
09:12:00 <zid> correct
09:12:00 <zid> there's also the x2apic and other stuff but I don't know the distinctions there
09:12:00 <karenthedorf> So anyway, kill the PIC, enable the APIC by writing 0x1FF to the spurious interrupt register, and yet no serial interrupts. Sooo... that's where I'm at.
09:12:00 <karenthedorf> I think x2apic is a mode for the I/O apic? and you should probabally use it, not using it is one of the things intel wants to kill off.
09:14:00 <kazinsal> I think if you want to use the IOAPIC in APIC mode to route external interrupts you need to do some wacky ACPI stuff
09:14:00 <karenthedorf> Oh and then there's the fun that Pentium and P6 have buggy-ass I/O APICs which has their own bus instead of being hooked to the system bus, which means The Intel Manual spends half the chapter on interrupt controllers discussing how messed up those were instead of anything relevant in the last 20 years.
09:15:00 <zid> intel manual do be like that
09:15:00 <zid> there's also the XEON chapter on that too I think?
09:15:00 <zid> cus the xeon did something else (the original xeon, not the cpu line, 'xeon')
09:15:00 <karenthedorf> kazinsal: Urggghhh... I was hoping to not have to parse APIC for a while.
09:16:00 <karenthedorf> *ACPI
09:16:00 <karenthedorf> Too many acronyms starting with A
09:16:00 <zid> acpi has the routing table in it, I printed mine
09:16:00 <zid> it was boring
09:16:00 <zid> everything was routed to core 0 as you'd expect
09:16:00 <kazinsal> your DSDT might just be wrong too, because lmao, BIOS manufacturers
09:16:00 <karenthedorf> I'ma going to assume qemu routes everything to 0 and see if it works.
09:17:00 <karenthedorf> And come back later when I have actual ACPI parsing
09:17:00 <kazinsal> ACPICA is the canonical ACPI blaster but there's also `lai` by manganarm (who was in this channel once upon a time years ago I think but doesn't seem to be now) on github that is a lot smaller and can do most of the stuff you'd need
09:17:00 <nikolar> Wasn't there a library you could use for acpi parsing
09:17:00 <zid> yea but who wants 300kB of acpi parsers in their 2kB of osdev code :P
09:18:00 <zid> DSDT, is the thing I was trying to find the name of
09:18:00 <nikolar> Lel those who can't be bothered to parse acpi
09:18:00 <zid> maybe not
09:18:00 <karenthedorf> Doesn't it have like a whole language in it you need to interpret? Sounds wild.
09:18:00 <zid> Yea, AML or something
09:18:00 <nikolar> It does, yeah
09:19:00 <zid> MADT!
09:19:00 <zid> multiple apic description table
09:19:00 <nikolar> zid see, a library doesn't sound so bad
09:20:00 <zid> You don't need AML for MADT
09:20:00 <zid> You need it for power management I think
09:20:00 <kazinsal> you need it for IOAPIC PCI routing
09:20:00 <nikolar> Yeah I forgor
09:21:00 <zid> entry type 1 in the madt is the ioapic default config or something?
09:21:00 <zid> for where it's mapped
09:21:00 <zid> I think that's all you need
09:21:00 <kazinsal> yeah
09:22:00 <zid> like when you grab the pci-e base addr out of the other table
09:23:00 <nikolar> You need apic for ipis too, no?
09:23:00 <zid> yea
09:23:00 <kazinsal> ayep
09:23:00 <zid> but the apic base addr is just an MSR, and your bootloader doesn't move it from the default anyway
09:23:00 <zid> lapic*
09:31:00 <heat> karenthedorf, APIC and IOAPIC are architectural
09:32:00 <heat> they're in the SDM manual, not chipset docs
09:32:00 <heat> (actually IOAPIC might /not/ be architectural, can't remember, but LAPIC definitely is)
09:32:00 <karenthedorf> Yeah, LAPIC is for sure.
09:32:00 <karenthedorf> I'm not sure on the I/O APIC
09:33:00 <zid> I don't have an MADT apparently? I have an APIC table though
09:34:00 <zid> and I have two i/o apics, one at fec000000 and one at cec01000
09:34:00 <zid> ISA bus delivers irq0 and irq9
09:36:00 <karenthedorf> Wiki says "So you can have, say, 2 I/O APICs, the first handling IRQs 0 - 23 and the second 24 - 47."
09:36:00 <zid> yep, that's how mine's set up
09:37:00 <kazinsal> LAPIC is "baked into" each logical core
09:37:00 <kazinsal> then there's one or more IOAPICs on top of that
09:37:00 <zid> https://gist.github.com/zid/42f18d8391b929dcc939ce8af44232c3
09:37:00 <bslsk05> gist.github.com: apic.txt · GitHub
09:38:00 <zid> line 183
09:38:00 <karenthedorf> Yup, I have no idea where the I/O APIC is. *sigh* guess I'm parsing the MADT.
09:38:00 <karenthedorf> Or the "Intel MP tables", which I'm assuming is some ye olde legacy way to do it
09:38:00 <zid> mine's in the APIC not the MADT apparently, but in qemu I am fairly sure you get a MADT
09:38:00 <heat> use the MADT
09:38:00 <kazinsal> at leas the MADT is just a TLV setup
09:38:00 <kazinsal> you don't need AML for it
09:38:00 <kazinsal> https://wiki.osdev.org/MADT
09:38:00 <bslsk05> wiki.osdev.org: Just a moment...
09:38:00 <zid> MADT has the routing info, idk where you'd get that on my machine heh
09:38:00 <kazinsal> relatively alright documentation on it here
09:38:00 <heat> you effectively need AML for it, because then you can't get PCI interrupts
09:39:00 <heat> and you need to execute \_PIC too
09:39:00 <kazinsal> well, you can tell your PCI devices to keep using legacy IRQs
09:39:00 <karenthedorf> My plan was *just* to get serial interrupts for now.
09:39:00 <kazinsal> or tell them to use MSI
09:39:00 <zid> (MCFG is the one I couldn't remember the name of, that has the pci-e base addr)
09:39:00 <kazinsal> I'm weird and my awful old kernel design used an ASMP setup, though
09:40:00 <heat> MSI would be great but requires all PCIe (welcome to 2024)
09:40:00 <heat> and the problem is that qemu doesn't emulate MSI on many devices
09:40:00 <kazinsal> MSI is in the PCI 2.1 spec iirc
09:40:00 <kazinsal> so it's valid on parallel PCI
09:40:00 <heat> yeah but was only made mandatory on PCIe
09:40:00 <zid> maybe my hypervisor hid it from me? :P
09:41:00 <heat> but really the big problem is emulators
09:41:00 <kazinsal> I think every emulation of stuff like intel 825xx handles MSI if not MSI-X
09:41:00 <kazinsal> even on parallel PCI on qemu/i440fx
09:42:00 <kazinsal> maybe the storage processors might not? idk
09:42:00 <karenthedorf> Yup, I read the wiki page on MADT, I am still nonplussed as to what I have to do.
09:42:00 <kazinsal> I haven't really done a deep dive into what eg. the ubiquitously emulated buslogic SCSI controller handles
09:42:00 <zid> nothing
09:42:00 <heat> IDE does not, e1000 does not (IIRC), some of the GPUs do not
09:42:00 <zid> as far as I know you just wake the lapic up having disabled the pic
09:42:00 <kazinsal> e1000 definitely does, I've done pure MSI-X on e1000
09:42:00 <zid> and the i/oapic in qemu is set up 'fine' already
09:42:00 <heat> might've been e1000e?
09:43:00 <kazinsal> e1000e is a higher revision thereof
09:43:00 <zid> I guess you wanna revector them so you don't get them all on the same source though?
09:43:00 <kazinsal> actually, my memory is bad -- e1000 handles MSI but e1000e handles MSI-X
09:44:00 <kazinsal> both of which are good enough to bypass ACPI PCI vectoring bullshit
09:44:00 <heat> aight
09:44:00 <zid> there's info lapic in qemu, cool
09:45:00 <heat> karenthedorf, port ACPICA, use it
09:45:00 <heat> then thank me for the time i saved you
09:45:00 <kazinsal> or port lai and write some damn documentation when you do
09:46:00 <kazinsal> because the documentation for it is basically just "here's what you need to plug into it, godspeed"
09:47:00 <heat> to me, lai is in the limine category
09:47:00 <heat> aka not using it, too amateurish
09:47:00 <zid> (and info pic, which I already knew existed, but is also useful here)
09:47:00 <kazinsal> idk, lai at least seems like it has some thought put into it
09:47:00 <kazinsal> it's a minimalist "get the grunchy parts of ACPI sorted" thing
09:47:00 <heat> with ACPICA you at least know it'll run everywhere and work correctly
09:48:00 <heat> with GRUB you know it'll... it will... uhh... boot sometimes
09:48:00 <kazinsal> quite frankly nearly every single hobby osdev dork is going to go nowhere near needing things like desktop-grade ACPI plug and play whizbang firmware bug workarounds working
09:49:00 <heat> i agree, but if i have a third party ACPI library i'd love for it to Just Work
09:49:00 <heat> instead of Just Work Sometimes, But Not Always
09:49:00 <kazinsal> so being able to get the 99% most important parts of ACPI going in a 10kb library and using post-ACPI modern stuff to do things like multicore interrupt vectoring seems like a win
09:49:00 <heat> lai is 10KB?
09:49:00 <kazinsal> like, I honestly don't care about the half meg of ACPI Reclaim space
09:50:00 <kazinsal> I just ignore it because I don't target anything with less than a gig
09:50:00 <kazinsal> ACPI can hold onto that forever
09:50:00 <karenthedorf> So this is what I currently do https://pastebin.com/m7t35e36 (and then `sti`) But... no interrupts get through except one from an `int $255` instruction I run (pretty sure I enabled them over on the serial config side)
09:50:00 <bslsk05> pastebin.com: //Disable legacy PIC outb(PIC1+1, 0xFF); outb(PIC2+1, 0xFF);//Enable - Pastebin.com
09:50:00 <zid> 255 is the spurious int vector
09:50:00 <zid> oh
09:50:00 <zid> instruction rip
09:51:00 <heat> what's pHHDM?
09:51:00 <karenthedorf> The Higher-Half Direct Map limine gave me. So all physical memory should be mapped there.
09:51:00 <heat> that sounds impossible
09:51:00 <kazinsal> I really need to figure out limine one of these days, if only to figure out how it works
09:51:00 <zid> clear disable bit, set up lint0 and lint1, set up spurious, set task prio, destination format
09:51:00 <zid> are the full steps
09:51:00 <zid> idk what the *minimum* steps are
09:52:00 <heat> it's mapping *all* of physical memory? are you sure it's not just the -2GB bit?
09:52:00 <karenthedorf> This is x64, we have way more than 2GB :D
09:52:00 <zid> we're itanium!?
09:52:00 <kazinsal> one day there'll be a multiboot 3 that supports x64 out of the box
09:52:00 <zid> fuck, I have no idea how the itanium ioapic works
09:52:00 <kazinsal> also one day the bombs will fall and we'll all be free of this shit
09:52:00 <heat> i like when my questions go above people's heads because they use limine to skip important steps
09:53:00 <heat> in any case, that's not how you configure the APIC
09:53:00 <zid> I think configuring the spurious boy might be hard required
09:53:00 <kazinsal> honestly the only reason I still support grub is because I like being able to blat a new bootimage to build:/araxes/tftp/ and then type `system reboot now` into the shell
09:53:00 <heat> https://github.com/heatd/Onyx/blob/master/kernel/arch/x86_64/apic.cpp welcome to apiccpp
09:53:00 <bslsk05> github.com: Onyx/kernel/arch/x86_64/apic.cpp at master · heatd/Onyx · GitHub
09:54:00 <heat> this is acpicpp
09:54:00 <kazinsal> and then it PXEs the new bootimage off of the TFTP server
09:54:00 <zid> with high bit set to enable?
09:54:00 <zid> 0xFF being the trivialierest
09:54:00 <zid> oh right that's your 0x1FF
09:54:00 <zid> and going to 0xF0
09:54:00 <zid> I figured your code out!
09:54:00 <karenthedorf> zid: lamo, sorry for giving you awful code
09:55:00 <zid> nah I just don't know this shit off the top of my head
09:55:00 <zid> what does info pic give you?
09:55:00 <kazinsal> christ now I'm thinking about a new bootloader design
09:55:00 <kazinsal> this is how it starts innit
09:55:00 <heat> dont
09:56:00 <heat> the based bootloader design is no bootloader at all
09:56:00 <kazinsal> just get UEFI to do it 5head
09:56:00 <karenthedorf> zid: https://pastebin.com/rRHt9nye
09:56:00 <bslsk05> pastebin.com: ioapic0: ver=0x20 id=0x00 sel=0x3f (redir[23]) pin 0 0x0000000000010000 dest - Pastebin.com
09:57:00 <zid> looks like every single pin is set to irq0 on cpu0 to me?
09:57:00 <heat> as i said, that's not how you use the APIC
09:57:00 <zid> and all are masked
09:57:00 <karenthedorf> So yeah, looks like everything is masked at the I/O apic level
09:57:00 <heat> it's all masked and uninitialized
09:57:00 <karenthedorf> heat: Yes, I am aware, I'm trying to work out what I need to do to not have it all masked and broken.
09:57:00 <zid> so you need to write a value above 0x10 to the ioapic then write some bits for where to send it and how
09:57:00 <heat> i linked you my code
09:58:00 <heat> see ioapic_init
09:58:00 <zid> afaik it's like the ppu on the nes where it has an address reg and a data reg?
09:58:00 <zid> so you write the address 0x10 to set vector table 0 entry into the address reg, then write.. some vector / level / etc info to the data reg
09:59:00 <karenthedorf> Yeah, I don't know where those regs live yet, so I should get to parsing the MADT. After I've read heat's code
09:59:00 <zid> madt or apic apparently, but mine's at FEC if you just wanna try it :P
09:59:00 <kazinsal> I'm at the point where I'm right about to just rewrite everything from scratch myself because, why not
09:59:00 <zid> idk if anybody bothers to move it, or if you even can move it, like the LAPIC
09:59:00 <kazinsal> good opportunity to make all the important parts platform agnostic
10:00:00 <heat> ioapic_init, apic_timer_smp_init, ioapic_unmask_pin are the key functions there
10:00:00 <heat> oooh kazinsal are you doing a UNIX
10:00:00 <zid> fec is in the acpi spec as the 'example' address
10:00:00 <kazinsal> the unix is my side side project
10:00:00 <heat> you're dropping pdp assembly for CCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
10:00:00 <kazinsal> the unix runs on an 8088
10:01:00 <kazinsal> the main side project is a proper modern-ish 64-bit network core OS
10:01:00 <kazinsal> routing, switching, firewalling, etc
10:01:00 <kazinsal> the unix is for when I get bored and want to hack on bad hardware
10:01:00 <heat> yah
10:02:00 <heat> i've kinda wanted to add routing to my OS but i honestly don't know where to start, and i have little hands-on experience on the matter
10:02:00 <zid> dev: ioapic, id ""
10:02:00 <zid> version = 32 (0x20)
10:02:00 <zid> mmio 00000000fec00000/0000000000001000
10:02:00 <zid> gpio-in "" 24
10:02:00 <zid> in `info qtree` in my qemu, so yea, it's there
10:02:00 <kazinsal> I figure the unix from scratch on a 45 year old platform will give me an advantage when the nukes pop and someone needs to reimplement the internet using late 1960s technology
10:03:00 <zid> tbh you could probably get away with never checking the madt or apic tables, the chances of any machine anyone ever runs your code on not having it precisely there seems unlikely :P
10:07:00 <karenthedorf> So the IOAPIC has two memory mapped registers (address and data), both 1 byte wide, but 16 bytes apart? I know that's perfectly possible just... weird. Probabally some hardware wiring choice we're all stuck with.
10:08:00 <zid> 4 bytes wide, 16 bytes apart
10:08:00 <karenthedorf> Right, I can't into maths. Urgh, I should probabally sleep
10:08:00 <zid> and each routing table entry is 8 bytes so takes two addresses
10:08:00 <zid> 10 and 11, 12 and 13, etc
10:09:00 <zid> the high bits seem to be for routing to different cpus, low bits are for edge/level/high/low/vector
10:21:00 <karenthedorf> Thanks for the help btw, even if I'm being a bit dense and a bit more sleep-deprived.
10:21:00 <zid> I'm just looking it up myself, for my own edification
10:22:00 <zid> then sperging my superior knowledge all over you
10:22:00 <zid> it strokes my napoleon complex
10:22:00 <kazinsal> yeah this is one of those things I kinda did a "set and forget" on ages ago and never really remembered all the details of myself
10:23:00 <karenthedorf> I am now returning to the font of all knowledge, the intel manual, but with more knowledge I should hopefully understand it better on the second time through.
10:23:00 <heat> i wrote my code 8 years ago
10:24:00 <heat> the IOAPIC+LAPIC thing is... funky
10:25:00 <karenthedorf> It does make sense to have an interrupt controller for the CPU, then another for each core. It just doesn't make the lives of OS devs easy :D
10:29:00 <zid> ioapic is just a memory mapped PIC with more bits for routing, doesn't seem bad at all
10:29:00 <zid> I'd never looked *super* deeply at it
10:29:00 <zid> lapic is kinda scary but the default config seems usable
10:35:00 <heat> the interrupt controller is in theory not on the CPU, but on the platform itself
10:55:00 <karenthedorf> Hmm, my LAPIC is at FEE0`0900 not FEE0`0000? Odd?
10:56:00 <karenthedorf> I guess limine or qemu could be responsible... I probabally shouldn't care though.
10:57:00 <karenthedorf> Oh, there's flags in the APIC_BASE_MSR as well as the address... ugh
11:37:00 <astroanax> ll
11:39:00 <astroanax> any good resources on low level vmem management?
11:42:00 <Ermine> heat: back to linear algebra, the really hard thing there is tensor theory
11:42:00 <mjg> :D
11:43:00 <mjg> lemme tell you how i failed to pick up a chick once
11:43:00 <mjg> at my first job we had a bunch of women around my age in customer service
11:43:00 <mjg> i was studying some algebra on my lunch break
11:43:00 <mjg> she was sitting nearby and we started chatting
11:44:00 <mjg> i thought i sound smart 'n shit
11:44:00 <mjg> turns out he was one year older and studying physics
11:44:00 <mjg> and the shit i was studying was ez stuff she already did
11:44:00 <Ermine> learning algebra isn't how you flirt with women
11:44:00 <mjg> dude
11:44:00 <Ermine> unless you're on the math faculty
11:45:00 <mjg> i picked up a chick on unix, no joke
11:45:00 <mjg> she was in the CS program though
11:45:00 <mjg> so you know, don't talk to me about women!
11:45:00 <Ermine> I was advised to prefer humanitarian faculties
11:45:00 <mjg> i don't want to sound like andrew tate, so i'm gonna leave it like that
11:46:00 <Ermine> til
11:48:00 <Ermine> Also yeah, when you get to advanced stuff, everything you've learned in the past course feels ezpz
11:49:00 <mjg> c'est la vie innit
11:49:00 <mjg> i remembered having tons of funs writing programs i could not be fucked to write today
12:10:00 <Ermine> Meanwhile, seems like I need to take a step back and learn opengl first
12:12:00 <mjg> opengl?
12:12:00 <mjg> that's still a thing?
12:13:00 <Ermine> yes, and it will be a thing
12:47:00 <vin> Is there a way to trigger memory swapping for a particular process? That is only identify cold pages for a process and swap it out. I don't think linux supports per process. I did try per cgroup by setting memory limit but it doesn't seem to work
13:50:00 <heat> mjg, mof do you ever run out of funny stories
13:50:00 <heat> real q here
13:50:00 <mjg> bro
13:50:00 <mjg> i have been alive for significantly longer than you
13:50:00 <mjg> liek talk t osomeone pushing 40
13:51:00 <heat> an oldie? i would never!
13:51:00 <heat> meanwhile the guy asked about "low level vmem management" and no one replied :(
13:52:00 <heat> vin, better solution: https://man7.org/linux/man-pages/man5/proc_pid_clear_refs.5.html
13:52:00 <bslsk05> man7.org: proc_pid_clear_refs(5) - Linux manual page
13:53:00 <mjg> this is a shitpost channel
13:53:00 <heat> plus smaps and you can get some sense of the pages used since you cleared refs
13:54:00 <heat> smaps or pagemap that is
13:58:00 <heat> mjg, why tf is that fio bench calling bdev buffer flushing in a loop
13:59:00 <heat> ofc it's causing LRU churn, it completely clears out the inode
14:01:00 <mjg> i don't think they are trying to test a real workload
14:01:00 <mjg> rather they are trying to break the kernel
14:01:00 <mjg> stress-ng-style
14:04:00 <heat> yeah but spamming a 90s bdev ioctl that clears out the inode is as silly as, idk, benchmarking reboot() speeds
14:05:00 <mjg> you do understand stress-ng, which you defend, is not doing anything better
14:05:00 <mjg> it's just pounding on something more than a real workload would
14:05:00 <heat> correct, but fio is not a break-the-system tool
14:05:00 <mjg> fio is configurable, maybe they use it this way
14:06:00 <heat> https://github.com/axboe/fio/blob/c63e8658bb776f2d50cde992ad07232a61e6d031/os/os-linux.h#L193 lord almighty
14:06:00 <bslsk05> github.com: fio/os/os-linux.h at c63e8658bb776f2d50cde992ad07232a61e6d031 · axboe/fio · GitHub
14:07:00 <mjg> here is an idea
14:07:00 <mjg> you repond to the thread
14:07:00 <mjg> and tell them "fuck you and your soft lockups"
14:07:00 <mjg> or maybe something nicer
14:08:00 <heat> yessir
14:08:00 <mjg> bottom line tho it not be possible to lockup the system
14:08:00 <mjg> so
14:08:00 <mjg> there *are* problems they do run into
14:08:00 <mjg> it should not*
14:08:00 <heat> i agree
14:08:00 <heat> just that this workload is really really stupid
14:09:00 <mjg> feel free to pester them about it
15:02:00 <Ermine> three setuptools releases in 2 days!!!
15:05:00 <Ermine> even in one fucking day!!!
15:05:00 <heat> obviously means they're great solid releases
15:05:00 <heat> no patching over bad releases nuh uh
15:05:00 <Ermine> yeah, 10/10 releng
15:06:00 <heat> releasen btw
15:06:00 <Ermine> btwen
15:22:00 <mcrod> heat hates me
16:13:00 <Matt|home> o\
16:20:00 <mcrod> how the fuck does googletest not have a mock generator
16:20:00 <mcrod> insanity
17:54:00 <Ermine> fourth release!
17:56:00 <GeDaMo> Fighting over spaces vs tabs? :P
17:57:00 <gog> tabs
17:57:00 <Ermine> stab
17:57:00 <gog> dwarf
18:29:00 <kof673> > I figure the unix from scratch on a 45 year old platform will give me an advantage when the nukes pop yes, i believe john titor warned about a similar scenario, unix would have to be restored
19:10:00 * geist yawns
19:10:00 <geist> howdy folks
19:22:00 <gog> hi
20:10:00 <Ermine> hi geist
20:15:00 <geist> hola!
20:43:00 <gog> hi
20:49:00 * Ermine gives gog a piece of cheese
20:50:00 * gog is fasincated
21:58:00 <Ermine> heat: ext4 becomes 20% more optimal! https://lore.kernel.org/lkml/20240718032730.GA2319255@mit.edu/
21:58:00 <bslsk05> lore.kernel.org: [GIT PULL] ext4 updates for 6.11-rc1 - Theodore Ts'o
22:47:00 <heat> depessimized
22:47:00 <heat> >in particular, improving IOPS and throughput on fast devices running Async Direct I/O by up to 20% by optimizing jbd2_transaction_committed
22:47:00 <heat> yeah doesn't impact normal people
22:48:00 <gog> hi
22:48:00 <heat> hi gog
22:49:00 <Ermine> why?
22:49:00 <heat> async DIO is only important for, like, databases
22:49:00 <heat> literally no one else does that
22:49:00 <Ermine> oic
22:53:00 <mcrod> hi
22:57:00 <heat> hi michael
22:57:00 <mcrod> hi heat
23:02:00 <heat> how do i find reference leaks?
23:02:00 <heat> does anyone know a nice tactic?
23:05:00 <geist> heat: well, qemu does
23:05:00 <geist> the async io stuff backing qcow files and whatnot
23:05:00 <heat> oh yeah good point, qemu also uses DIO
23:05:00 <heat> not by default though!
23:05:00 <geist> side note fiddling around the aio stuff on qemu i think it really does improve performance a lot
23:06:00 <heat> yep, i've observed that too
23:06:00 <geist> io_uring or aio and/or threading
23:06:00 <heat> particularly if you have a buffer/page cache in front, on the guest (which you usually do)
23:06:00 <geist> yah, proxmox defaults to it, with no cache so you end up with a fairly safe transaction on power fail
23:06:00 <heat> i'm running aio=io_uring,cache=none
23:06:00 <geist> right
23:08:00 <mcrod> i’m very sad
23:09:00 <mcrod> i should’ve snagged another SSD on prime day
23:09:00 <mcrod> :(
23:09:00 <geist> the question i ask myself: do i need it or did i just want it?
23:15:00 <Ermine> what do you need/want?
23:16:00 <heat> a hug
23:20:00 <Ermine> huh
23:24:00 <gog> i want keyboard mechanisms that don't break when you try to get debris from under them
23:24:00 <gog> yes i am still on about my keyboard
23:26:00 <Ermine> commodore 64 keyboard is nice in this regard
23:26:00 <Ermine> or vic 20
23:27:00 <heat> gog be thankful you don't have a mbp keyboard
23:28:00 <gog> maybe
23:29:00 <geist> youknow one of the most pleasant keyboards to type on are on the TRS-80 model 100
23:29:00 <geist> i dunno what it is about it, but it's almost perfect
23:29:00 <geist> not even mechanical, proving that even silly dome keyboards can still be lovely