channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched
#osdev2 = #osdev @ Libera from 23may2021 to present
#osdev @ OPN/FreeNode from 3apr2001 to 23may2021
all other channels are on OPN/FreeNode from 2004 to present
http://bespin.org/~qz/search/?view=1&c=osdev&y=19&m=8&d=19
08:09:13 <program> hello
08:10:24 <klange> Greetings.
08:10:42 <klys> hello, program
08:10:58 <program> so i'm trying to get the lan91c111 netcard running on qemu versatilepb and it is 'kinda' working, because i implemted the arp and icmp protocols and these works and i'm able to ping the system from the host
08:11:08 <zid`> neat
08:11:41 <program> but the problem is, that once sufficient number of packets is transmitted (so far it looks like it is around 4), the netcard stops generating receive interrupts
08:11:53 <zid`> are you updating the ring buffer crap properly?
08:12:13 <program> when this happens i can still send packets and i see them in wireshark, but no receive interrupt is generated
08:12:29 <zid`> if you're out of TX buffers you'd stop getting stuff
08:12:35 <program> i think i'm, i have been checking this datasheet:
08:12:38 <program> http://ww1.microchip.com/downloads/en/DeviceDoc/00002276A.pdf
08:12:54 <zid`> I'd add printfs to qemu, easier :P
08:13:08 <program> this is the datasheet i'm working with, also i have tried to find some examples on internet because i really don't understand what i did wrong
08:13:46 <program> i'm on windows, so i was thinking i would like to solve it without recompiling qemu :D
08:13:59 <zid`> don't be on windows
08:14:13 <zid`> how are you even compiling shit O_o
08:14:21 <klys> get an prompt
08:14:50 <program> i also observed, that if i send couple of frames before pinging it (so i can receive something) then i will receive nothing, no recv interrupt forme
08:14:53 <program> *for me
08:15:01 <zid`> hmm
08:15:11 <zid`> maybe you enabled RX interrupts or something and wedged it?
08:15:15 <program> so could it really be that just transmitting frames in a wrong way can fuckup the card this badly?
08:15:30 <program> i have enabled TX and RX interrupts at the initialization time
08:15:40 <zid`> disable TX interrupts
08:15:45 <zid`> err RX
08:15:57 <zid`> whatever, send interrupts :P
08:16:07 <program> but then i won't receive anyhing
08:16:09 <program> oh
08:16:29 <program> like i should disable transmit interrupt? why?
08:16:40 <zid`> it's useless in practice and it sounds like you may have wedged it
08:16:46 <zid`> so your code for it might just be broken anyway
08:16:51 <zid`> so try turning it off
08:17:24 <program> okay, i'll try, this should be quick
08:18:35 <program> oh fuck, now i have completely broke it :D
08:18:39 <program> now i can't receive anything
08:18:48 <zid`> Your interrupt code needs fine-toothed-combing
08:19:00 <program> oh, i'm release the packet in isr
08:19:08 <program> on tx interrupt
08:22:47 <program> this is just weird
08:22:55 <program> and frustrating
08:23:09 <zid`> I'm still not sure my e1000 code works right :P
08:23:21 <program> Pinging 10.0.2.4 with 32 bytes of data:Reply from 10.0.2.4: bytes=12 (sent 32) time=202ms TTL=64Reply from 10.0.2.4: bytes=12 (sent 32) time<1ms TTL=64Reply from 10.0.2.4: bytes=12 (sent 32) time=1078ms TTL=64Request timed out.
08:23:23 <program> :D
08:23:38 <zid`> are you doing pci or pci-e
08:24:02 <program> uhm
08:24:24 <program> i think that on versatilepb it should be some autoconfig pci
08:24:34 <zid`> none of those words mean anything to me
08:24:41 <program> or maybe its just standalone because i didn't do any pci enumeration or anything
08:24:57 <zid`> are you writing to ISA ports and doing IRQs, or writing to MMIO regions and doing MSIs
08:24:58 <program> i'm on arm target
08:25:19 <zid`> (the ISA ports may be emulated via mmio anyway though)
08:25:24 <program> the card has MMIO
08:25:39 <program> and has fixed irq
08:25:55 <zid`> Anyway, go double check the docs for how it wants you to ack its interrupts, and double check you're acking your irq on the interrupt controller properly
08:25:58 <program> its an emulation on of old arm board soc
08:26:25 <program> thats another thing - i don't ack the interrupt controller
08:26:39 <zid`> how does it know you're done interrupting then?
08:26:41 <program> it looks like it does not need an akcnoledge
08:26:57 <program> i think it just resets the irq source once it receives it
08:27:05 <zid`> but then it could infinitely stack
08:27:50 <program> well, if i don't transmit anything, then the interrupt is triggered everything i get new packet, i have tested this, and i got like 50 packets or so
08:28:13 <program> the card is on secondary interrupt controller
08:28:45 <program> so i check if the irq for secondary controller is high and then i check the source line for the netcard and if high, i just branch to proper isr
08:28:54 <program> and then return control
08:29:22 <zid`> it needs to go low again at some point else you're in an infinite loop of either always or never getting an interrupt :P
08:30:41 <program> yeah, but i thought in this case it gets low automatically - how in this case it could get stuck?
08:30:50 <program> am i missing here something :D
08:31:00 <zid`> you do know I've not seen your code right
08:31:20 <program> right, do you want to see it?
08:31:25 <zid`> not really :P
08:31:37 <program> i understand :D
08:34:45 <program> hm, it seems that when transmitting the irq is not triggered at all
08:34:46 <program> wtf
08:37:21 <program> its triggered only when frame is received first, i guess, then it gets triggered
09:25:57 <program> its working!!!!!!
09:26:17 <program> it turned out that auto release was off - i forgot to enable it
09:26:39 <program> page 51: AUTO RELEASE - When set, transmit pages are released by transmit completion if the transmission was successful(when TX_SUC is set). In that case there is no status word associated with its packet number, and successful packetnumbers are not even written into the TX COMPLETION FIFO.
09:26:58 <zid`> and yet you were relying on it?
09:26:59 <zid`> good form :P
09:26:59 <program> holy shit no i can ping the fuck out of my system :)
09:27:05 <program> yeah.. :D
09:27:11 <zid`> ping mine I'm too lazy to hook the arp code back up
09:27:23 <zid`> I had arp working in bochs, tried it in qemu and had to hack the driver to shit and disconnected the arp code
09:27:28 <program> thank you for helping me :)
09:27:34 <zid`> code looks like it works but I've yet to get around to testing it
09:27:40 <zid`> That will be 1 pilsner
09:28:01 <program> oh shit, now it died after 59 packets
09:28:12 * zid` still betting on wedged irq
09:28:17 <program> but i hope this time the reason will be something else :D
09:28:28 <program> oh
09:28:36 <program> lets keep searching then
09:29:02 <program> actually, its dead, because it hit the UNDEFINED
09:29:28 <zid`> is that thing just asking for a list of pages to use which you negotiate giving/ungiving it, rather than a ringbuffer?
09:29:30 <program> okay, thats slightly better i guess :D maybe i fucked up the stack or something
09:30:54 <program> yeah, basically yes, its not that it has a ring buffer handling like the ne2k has for example
09:31:08 <zid`> I've only done ring buffers
09:31:16 <zid`> and bochs and qemu disagree on what means full and what means empty, which is nice
09:31:19 <program> i just read the packet number register so i know what the next packet slot is and then write data into it
09:31:21 <zid`> for the e1000
09:32:05 <program> they disagree? noo i wanted to switch from bochs to qemu because qemu has better tun/tap support
09:32:07 <zid`> I had to add my own debug printfs to figure that one out
09:32:15 <program> i don't have the e1000 though
09:32:17 <zid`> I assume bochs is wrong
09:32:30 <zid`> but the doc and qemu are hard to say they do the same thing, idk
09:32:31 <program> i like the bochs for its magic instruction
09:32:34 <zid`> maybe it matters on revision
09:32:38 <zid`> yea xchg bx, bx is nice
09:32:40 <program> and very verbose hardware information it provides
09:32:43 <program> yeah
09:32:50 <zid`> qemu has some okay hardware dumps
09:33:01 <zid`> qemu has trace events though which are also nice
09:33:11 <program> yeah, i suppose, the linux version i guess, the windows version not so much
09:33:18 <zid`> it should be the same
09:33:25 <zid`> -trace e1000*
09:33:26 <program> some info commands are entirely blank
09:33:34 <zid`> might just be your soc thing
09:33:39 <zid`> is less filled out
09:33:54 <program> right, that could be it
09:34:19 <zid`> doug from here was working on adding bochs' "page" command to qemu which would be nice
09:34:21 <program> usually if something does not work and i don't understand why, i just the qemu code
09:34:28 <program> *i check the qemu code
09:34:28 <zid`> adding printfs to qemu <3
09:34:57 <program> yeah, maybe i'll recompile it :)
09:35:06 <zid`> if you're on linxu you get splitdebug and crap
09:35:11 <zid`> no idea how to generate a pdb on windows
09:35:28 <program> yeah, on windows this sucks very much
09:35:31 <zid`> I run qemu on gentoo inside vmware :P
09:35:36 <zid`> waaay nicer than using windows
09:35:45 <program> i have my virtualbox right here :D
09:35:46 <zid`> means I get to test my OS in vmware too, it works ;)
09:36:17 <program> yeah, that is how i discovered some differences between vmware/qemu/bochs/virtualbox/virtualpc...
09:36:17 <program> :D
09:36:24 <program> that was really fun debugging
09:36:38 <zid`> my bochs is super fucked though the console output is like 4s delayed
09:36:40 <zid`> no idea why
09:37:18 <program> once it run on bochs but didn't on qemu so i fixed it for qemu and it stopped working on bochs so then i fixed on both bochs and qemu only to discover that it does not work on vmware even though that real hardware would just run the damm thing :D
09:38:05 <program> qemu is bit delayed for me too, don't know why, although its not 4 seconds, thats way too much
09:38:16 <zid`> https://cdn.discordapp.com/attachments/417023075348119556/600368937296723968/unknown.png my OS is sexy
09:39:00 <program> this is excellent :) that remains me i should add some packet hex dumps
09:40:02 <zid`> how are you handing data off to the network stack, just giving it the page with the packet on from the device?
09:40:30 <zid`> or doing a copy so you can have the page free for re-use maybe
09:41:45 <program> well, the network stack is designed for low traffic, so it basically just has statically allocated list of pages for raw frames and application sockets ; the raw frames gets copied in the ISR to the free frame slot and later serviced in a network scheduler process
09:42:08 <zid`> so the network stack technically has the pages, and gives them to the device to write to?
09:42:18 <program> yeah
09:43:06 <program> system creates a generic network device, which has its slot counters and INIT, SEND, RECV, CLOSE and other functions along with driver information associated with the network device
09:43:14 <zid`> fancy
09:43:25 <zid`> I only have one of everything so I just quickly hack shit together ;)
09:43:55 <program> the network device is then assigned a driver which populates these properties, so you just call [NetworkDevice].SEND to send a frame, same goes for application sockets
09:44:11 <program> yeah, i know that :)
15:06:11 <moondeck> is the area containing the RSDP, and the area the pointer points to guaranteed to be marked as "used" by e820?
15:06:21 <zid`> you hope
15:07:20 <moondeck> so i can just assume it is?
15:07:30 <zid`> yes
15:07:46 <moondeck> ty zid
15:07:54 <zid`> might be in your best interest to parse it early though
15:07:59 <zid`> so that overwriting it no longer matters
15:08:20 <Bitweasil> moondeck, I wouldn't treat anything about the e820 map as guaranteed.
15:08:31 <Bitweasil> "More likely to be right than not, for some value of right," is about as far as I'd go.
15:08:44 <zid`> some bioses are buggy, news at 11
15:08:57 <zid`> hope it's right, collect a shim for if it isn't
15:09:03 <Bitweasil> :p Some BIOSes don't have glaringly /obvious/ bugs.
15:09:23 <moondeck> even modern ones?
15:09:35 <Bitweasil> They're just shinier piles of bugs with mouse support.
15:09:41 <moondeck> well, BIOS is hardly modern
15:09:54 <zid`> especially modern ones :P
15:10:09 <zid`> because "what windows does" and optionally "what linux does" has been crystallized
15:10:25 <Bitweasil> EFI is, depressingly, more reference code than not, which really sucks for platform security. Now, you can rely on a bug being on most platforms, instead of just one or another.
15:10:35 <zid`> like, I bet my super modern $400 m obo has a bunch of broken bios service routines at the very least
15:10:54 <zid`> cus if windows doesn't use it who gives a fuck
15:11:37 <moondeck> actually, how does windows detect memory, on a "legacy" system?
15:12:09 <zid`> windows doesn't boot on legacy systems
15:12:26 <Bitweasil> Probably either the e820 map, or asking the BIOS.
15:12:33 <Bitweasil> (I assume pre-EFI?)
15:12:33 <zid`> bios is what provides the e820, Bitweasil
15:12:45 <zid`> it's literally bios routine something, ax=e820
15:12:47 <zid`> that's why it's called that
15:12:51 <moondeck> yeah, pre-EFI
15:13:08 <Bitweasil> Right, I thought there was a separate call for max memory or something.
15:13:12 <Bitweasil> Been a while since I've been down there.
15:13:33 <Bitweasil> https://wiki.osdev.org/Detecting_Memory_(x86)
15:13:48 <zid`> 0x15 :D
16:15:21 <griddle> Will MP table CPU entries always be in order based on lapic_id?
16:30:05 <geist> griddle: i wouldn't assume that no
16:30:16 <geist> also keep in mind that apic ids are not always sequential, there are frequently gaps
16:31:03 <Bitweasil> Intel specifically says they're not guaranteed to be sequential, but one of their hypervisors sure assumed they were...
16:31:04 <Bitweasil> :/
16:31:17 <griddle> ah, okay. So I shouldnt just append to a global cpu array as I find them, I should insert them at their index in that array
16:31:23 <Bitweasil> (they're explicitly not sequential, and if you can't handle gaps, your code will break badly)
16:31:33 <Bitweasil> (also, max APIC ID != CPU count in the system)
16:31:48 <griddle> Is it common to store CPU specific information in some global array that you index by the CPU’s acpi id?
16:33:45 <griddle> Im thinking of having a list of cpu_states which you access and get a pointer to with a get_state() function. Each of them have information like the current process, state, etc..
16:34:14 <Bitweasil> You can do that, or you can store it creatively.
16:34:22 <griddle> How so?
16:34:23 <Bitweasil> Linux stores a pointer to task_struct at the bottom of the stack in the kernel.
16:34:36 <Bitweasil> I think Linux uses the GS base as a pointer to that sort of thing.
16:34:47 <Bitweasil> For early bringup, I've used an indirection array.
16:35:00 <griddle> yeah, c++ seems to use fs for threadlocal variables, so I was thinking of maybe doing something like that as well
16:35:15 <Bitweasil> So I have N CPUs, with apic IDs sparsely between 0 and M. A small table allows me to translate from APIC ID to the CPU index, so I don't have to have a sparse array.
16:35:20 <Bitweasil> (for the bigger data)
16:35:22 <griddle> not the threadlocal varables, but using segments
16:35:40 <griddle> ah, makes sense
16:35:41 <Bitweasil> If I've got 8 CPU cores, I'll have CPU structures 0-7, and then remap based on APIC ID. And then store it locally somewhere for later use.
16:35:45 <Bitweasil> I don't recall what I used, TS, maybe?
16:36:03 <Bitweasil> But if you have 8 CPUs using APIC IDs 0-64, a sparse array of CPU structures wastes a lot of memory.
16:36:23 <zid`> 230 bytes
16:36:25 <Bitweasil> You want to be able to go from APIC ID to CPU ID, but you don't want to do that every time.
16:36:26 <zid`> not even
16:36:34 <zid`> if you restrict it to short it's 120
16:36:36 <Bitweasil> Depends on how much data is in those structures.
16:36:41 <zid`> no you index it
16:37:02 <griddle> I like the indirection idea here
16:37:14 <zid`> structure[lut[index]] where lut is {0, 1, 2, 3, 4, 5, 6, 7, 0, 0, 0, 0, 0, 0} in some order
16:37:31 <zid`> you still only have 8 structures, but you have 64 element index array
16:37:34 <zid`> like 3d rendering :P
16:39:36 <griddle> Is there a big reason to use the OOP bytecode thing that acpi has?
16:39:59 <griddle> AML or whatever its called
16:40:20 <zid`> if you want to know what the things it encodes say, what choice do you have
16:41:03 <griddle> fun
16:53:03 <mrvn> It's called hashing. :)
16:53:22 <mrvn> With your has function being the 64 entry lookup table.
16:54:38 <mrvn> zid`: restrict it to short? But then you can only support 65536 cpu threads.
16:59:07 * zid` bundles gperf with his kernel source from now on
17:00:29 <geist> Bitweasil: as you say, you'll go from cpu id -> apic id much more commonly than the reverse
17:00:46 <geist> so i think you're fine. having an array of cpu number -> apic id is generally fine
17:01:06 <geist> the time you translate in the other direction is almost always just to get the current apic id, but you can store that in the current cpu structure
17:01:37 <geist> which is traditionally pointed to by the gs: pointer on x86-64 (hence the presence of the swapgs instruction)
17:01:57 <geist> or gs: points to the current thread structure, which then has a pointer to the current cpu. either way
17:02:25 <geist> point is you should build a mechanism to get to the current cpu structure quickly, and have a way to index into other cpu structures by cpu id
17:47:43 <mrvn> yeah. But do you make it the thread or cpu struct and point to the other? decisions, decisions
17:50:32 <mrvn> I wonder if it's faster to use *(gs:0x00) or mapping the thread structure at a fixed address and accessing it directly.
18:39:52 <moondeck> hi people
18:55:44 <Bitweasil> hi