channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched
#osdev2 = #osdev @ Libera from 23may2021 to present
#osdev @ OPN/FreeNode from 3apr2001 to 23may2021
all other channels are on OPN/FreeNode from 2004 to present
http://bespin.org/~qz/search/?view=1&c=osdev2&y=23&m=9&d=14
00:00:00 <heat> kof13, geist, was talking about my local linux test kernel
00:00:00 <heat> it's about 10MB compressed
00:00:00 <kof13> :/ i think that is normal nowadays, same with bsd
00:01:00 <kof13> i don't really think it matters per se, but if you are on 64M ram you will notice :D
00:01:00 <heat> it's around 34MiB uncompressed
00:01:00 <heat> this isn't a tinyconfig but it has a few bells and whistles disabled
00:03:00 <geist> gorgonical: it's coherent
00:04:00 <geist> what the inner vs outer sharable lets you do is specify how far out it needs to worry about sharing
00:04:00 <geist> so if you marked it inner sharable, and then something in the outer domain maps it it may not be coherent
00:04:00 <gog> hi
00:04:00 <geist> basically allows for mutiple levels of coherency
00:04:00 <geist> however, in practice i think nothing at all uses outer sharable, so it's kinda moot
00:05:00 <geist> i think their idea was you could have coprocessors or whatnot that are outer sharable, in which case all of your inner pages arent coherent, unless you mark them outer too
00:05:00 <geist> but then that presumably has a cost, since it causes it to need to deal with more caches farther away
00:06:00 <heat> hi gog we're talking about horrible arm64 mmu caching details
00:06:00 <geist> note that somewhere in the manual it states that for a single OS instance all of the cpus and whatnot must be inner. so the assumption is that all symmetric cpus in a symmetric system are inner by definition
00:06:00 <heat> everything has so much configurability you end up configuring it exactly the same as everyone else
00:06:00 <geist> i *thought* remember reading somewhere that outer was deprecated and may be removed in some later version?
00:07:00 <geist> the other page bit that i think is effectively unused is the sharable bit, iirc. it basically says from what i remember 'make this page coherent'
00:07:00 <geist> as if if you didn't set it the cpu is free to be incoherent with that page. nothing of course uses that so it's always set
00:08:00 <geist> also an old optimization that probably mattered in the armv6 days or something
00:10:00 <geist> you also see this inner/outer thing when it comes to stuff ike the dsb/dmb instructions, or the cache flush instructions
00:10:00 <geist> letting you specify how far out you want it to insert the barrier
00:11:00 <geist> it's commong, for example, to see something like `dmb ish`, vs `dmb sy` (or just `dmb`)
00:11:00 <geist> since the ish version only has to insert a memory barrier as much as it matters to everything in the inner domain, ie local cpus
00:11:00 <geist> but doesn't need to barrier relative to other shit like devices or whatnot
00:11:00 <heat> and the p a g e t a b l e w a l k e r
00:12:00 <geist> which is inner, iirc
00:12:00 <heat> didn't you need a dmb sy for that?
00:13:00 <heat> or am i misremembering?
00:13:00 <geist> i think page table walkers are inner, so no you only need inner
00:16:00 <heat_> oh ok i believe you then
00:16:00 <heat_> in riscv you have none of this
00:16:00 <heat_> caching bits in PTEs? what's that?
00:16:00 <geist> also i think you can configure that in the TTBR and/or TCR?
00:16:00 <heat_> haha PMP go brrrrrrrrr
00:17:00 <geist> the cache coherency of the page table walker, etc
00:17:00 <heat> maybe
00:17:00 <geist> also one of those details you never not set
00:25:00 <heat> TIL google has a funny easter egg where if you search vi it suggests emacs and vice-versa
00:25:00 <heat> it's kinda cute but both should suggest to GNU NANOOOOOOOOOOOOO
00:27:00 <bl4ckb0ne> it should suggest ed
00:28:00 <zid> nano > vi > emacs > nano, duh
00:30:00 <heat> as much as i appreciate ed as the standard unix text editor one cannot deny the superiority of GNU nano
00:30:00 <heat> other nanos? those suck
00:30:00 <heat> GNU nano? fuckin mint
00:31:00 <zid> You're just trying to save face when you have to tell me later that you have a micro
00:31:00 <heat> a microwhat
00:32:00 <zid> Yes
00:32:00 <heat> a microwave? yeah i have one
00:32:00 <heat> it's pretty standard around here
00:32:00 <heat> we're too busy having siestas to heat our own food
00:34:00 <heat> today i had an endoscopy
00:34:00 <heat> hopefully i get better from here on out
00:42:00 <bl4ckb0ne> hows the plumbing
00:43:00 <heat> it looks okay-ish but they took some samples from my stomach to take a closer look
00:43:00 <heat> i had a bit of esophagitis apparently too, probably from my recent stomach problems
00:56:00 <heat> geist, think we'll ever see a new MMU format that's not a radix tree?
00:57:00 <heat> i struggle to see it happen, the radix tree is a pretty good data structure, O(1) and everything
00:57:00 <zid> logn for write isn't it
00:57:00 <zid> in base.. 4096
00:57:00 <zid> basically 1 but not technically 1
00:58:00 <zid> base 512
00:58:00 <heat> considering the tree has a fixed depth it is O(1) isn't it?
00:58:00 <zid> I guess
01:00:00 <zid> feels like a scam though
01:01:00 <heat> MMUs are a ponzi scheme
01:01:00 <zid> MMUs are a scam by big ram to sell you more bits
01:03:00 <bl4ckb0ne> is it why downloading it is so expensive
01:03:00 <zid> they try to cover-up being able to download more ram
01:03:00 <zid> like the oil lobby burying climate change info
01:03:00 <gorgonical> geist: okay so I'm not crazy for thinking that marking my buffer as inner on both cores and cacheable in the memory attributes/MAIR would be enough?
01:03:00 <gorgonical> As in, it should be enough and maybe I'm doing something wrong?
01:04:00 <zid> so they have to hide it on the dark web and pay a lot for security etc, makes it hard to find and expensive
01:04:00 <heat> virtual memory is nonsense
01:04:00 <heat> the vm people are taking us for absolute fools
01:33:00 <geist> gorgonical: sounds correct to me
01:43:00 <gorgonical> hmm. then I'll have to check my t's and i's and see what I've done wrong. I have just now, 5 minutes ago, gotten the protocol working on hardware by setting the memory regions non-cacheable and rewriting the spinlock code and adding no-cache mmap support to my kernel
01:43:00 <gorgonical> But that will obviously be slow and I don't know why it didn't work in the first place
01:43:00 <gorgonical> But at least it works now
02:01:00 <geist> no cache against some sort of device that's doing dma?
02:02:00 <geist> like, you were dealing with a dma device and had to swtich to no cache to make it work?
02:05:00 <netbsduser> for a lark i decided to implement a PE loader and shim Windows' storport.sys port driver
02:05:00 <netbsduser> i tested it with the virtio storage driver for windows, which happily executed its first scsi request block
02:06:00 <geist> oh neat!
02:07:00 <netbsduser> there is much more to do if it's to be a useful shim, most urgently getting its interrupts and dpc infrastructure running (i just waited a moment and then checked the buffer to see if it was transfered to)
02:08:00 <zid> when iastor so you can do intel bios raid!?
03:24:00 <kazinsal> iirc iastor is a miniport driver so it gets called by storport
04:09:00 <gorgonical> geist: no, it's just regular RAM I'm using as a communication buffer
04:22:00 <geist> which device is this?
04:23:00 <gorgonical> the board? a rockpro64 that has a rk3399
04:23:00 <geist> kk
04:23:00 <geist> not some super early arm
04:23:00 <gorgonical> nah
04:23:00 <geist> i the cortex-a9 days or so you nad o actually turn that shit on
04:23:00 <geist> so no, no reason. of course you need whatever memory barriers you need
04:24:00 <geist> or something that has an implicit barrier for you, since the memory model is weak
04:24:00 <gorgonical> I guess I misread the manual then? Because to me it suggests that if you have hardware coherency then setting cacheability to wb and shareable it should "just work"
04:24:00 <gorgonical> Is that not true?
04:25:00 <geist> well. so you have a shared memory buffer, how are you signalling to each other core that something is there?
04:25:00 <gorgonical> ipi
04:25:00 <geist> and you're inserting a dsb there to make sure the data is flushed before you ipi?
04:26:00 <gorgonical> I did try some dsbs here an there, but I suppose its possible I wasn't being perfectly consistent about it
04:26:00 <gorgonical> Sometimes these late night debug sessions can get disorganized
04:26:00 <geist> ah yeah if you put in a dsb you should be okay
04:26:00 <gorgonical> in theory dsb(sy) should cover all cases right?
04:26:00 <geist> like as a general rule, write out the data, dsb, ipi, and the first core should have synchronized it's stuff before it sends the ipi
04:26:00 <geist> yep
04:26:00 <gorgonical> does the reader need to dsb sy before reading?
04:27:00 <geist> no
04:27:00 <gorgonical> I wouldn't think so if I have hw coherency
04:27:00 <gorgonical> If it were sw coherency maybe a dc would be necessary?
04:28:00 <geist> shouldnt need to between two cores no
04:29:00 <gorgonical> anyway, if I always always dsb sy before ipi on the core where the data is modified, its inner with wb caching, then the data should be coherent,m right?
04:29:00 <geist> yes
04:29:00 <geist> given what you have described to me yes
04:29:00 <gorgonical> because some instructions like linux's spin_trylock don't even work if you don't mark it as wb, ldaxr only works on very specific memory configuration
04:30:00 <gorgonical> okay I will have to really methodically test that tomorrow then to make sure I'm not losing my mind
04:30:00 <gorgonical> an extra caveat, the coherency works on the underlying pa, right?
04:30:00 <geist> yes and yes you're right, atomics only work explicitly on regular memory
04:30:00 <geist> i'd suggest validating your MAIR settings
04:30:00 <gorgonical> yep
04:30:00 <gorgonical> I'll have to do that
04:31:00 <geist> there's really only a handful of ways to set it up and 'regular memory' is pretty much what you want
04:31:00 <gorgonical> Ofc this all worked in QEMU because of memory model fakery and whatnot
04:31:00 <geist> yes it doesn't do anytig at all with that
04:31:00 <gorgonical> iirc we borrowed linux's MT_XXX settings so it should be exactly identical
04:31:00 <geist> https://github.com/littlekernel/lk/blob/master/arch/arm64/include/arch/arm64/mmu.h#L183
04:31:00 <bslsk05> github.com: lk/arch/arm64/include/arch/arm64/mmu.h at master · littlekernel/lk · GitHub
04:32:00 <gorgonical> MT_NORMAL should be exactly the same as Linux's notion of it
04:32:00 <geist> kk
04:32:00 <geist> yeah it's hust 0xff
04:32:00 <gorgonical> just got to make sure that when I'm mmaping it in I don't change the attributes or something
04:33:00 <gorgonical> The point is to create a sockets-like interface for shuttling data across the trustzone boundary between so all the data movement is originated in userspace
04:34:00 <geist> oh if it's across a trustzone boundary that might change things
04:34:00 <gorgonical> ha ha
04:34:00 <geist> i think cache coherency across that gets funny
04:34:00 <gorgonical> ofc
04:34:00 <geist> you muight have to clean the cache and whatnot
04:34:00 <gorgonical> well the memory will never be marked as secure
04:35:00 <gorgonical> the two buffers will always be non-secure
04:35:00 * geist nods
04:35:00 <gorgonical> so hopefully that won't create a "Secure dirty" situation
04:35:00 <geist> this is where i run ot of ideas.i know very practical knowledge of dealing with trustzone shit
04:35:00 <geist> yeah exactly, was thinking of a secure dirty thing
04:36:00 <gorgonical> I should check up on that to be sure
04:36:00 <geist> you can try to flush the cache and see if it makes it go away
04:36:00 <geist> ie, write some data, clean the cache lines, see if the other side starts to magically see it
04:36:00 <gorgonical> aggresively flush whole dcache before ipi?
04:36:00 <geist> no no just the cache lines
04:37:00 <geist> like write the data, then clean the cache line
04:37:00 <gorgonical> ah
04:37:00 <geist> worth a shot, see what happens
04:37:00 <gorgonical> the upshot is at least it works with noncache so if this doesn't pan out I can still continue on with work and argue future work
08:58:00 <sortie> <heat> #osdev would be more fun if we had challenges but sadly no one but me does osdev here
08:58:00 <sortie> What are you talking about
08:58:00 <sortie> I osdev circles around you mannn
08:59:00 <sortie> In the good ol' days we would race each other to implement stuff, like mention mesa and all three of us had OpenGL by dawn
09:05:00 <kof13> i interpreted it as somewhat tongue in cheek. also, os devs you </conway's law>
09:06:00 <Mutabah> I have grown tired of this simple "OS Development" thing, and have moved on to... COMPILER DEVELOPMENT!
09:07:00 <Mutabah> (... semi-jest - I do return to osdev every couple of months, but I also have a compiler that takes most of my hobby time)
09:07:00 <kof13> :D if i am here i am coding something related...just i tend to have giant sidequests, code in states of non-building. it doesn't really help to post anything
09:11:00 <kof13> here are 100 skeleton functions, compiles great. what if you try to use them? <man steps into elevator, falls into chasm>
10:50:00 <sham1> I do OS development since I keep working on my Emacs setup, making it better for work
10:53:00 <gog> i don't do development
12:43:00 <ChavGPT> lol
12:43:00 <ChavGPT> void
12:43:00 <ChavGPT> vnode_lock(vnode_t vp)
12:43:00 <ChavGPT> }
12:43:00 <ChavGPT> { lck_mtx_lock(&vp->v_lock);
12:43:00 <ChavGPT> void
12:44:00 <ChavGPT> vnode_lock_spin(vnode_t vp)
12:44:00 <ChavGPT> { lck_mtx_lock_spin(&vp->v_lock);
12:44:00 <ChavGPT> }
12:44:00 <ChavGPT> void
12:44:00 <ChavGPT> vnode_unlock(vnode_t vp)
12:44:00 <ChavGPT> { lck_mtx_unlock(&vp->v_lock);
12:44:00 <ChavGPT> }
12:44:00 <ChavGPT> darwin-xnu
12:44:00 <ChavGPT> this is what happens when webdevelopers decide to do some kernel hacking
12:44:00 <gog> :(
12:44:00 <gog> what if you're a kernel hacker that became a webdeveloper
12:44:00 <gog> even though i'm not really either
12:44:00 <gog> i'm actually a fraud
12:45:00 <ChavGPT> i can't think of a legitimate cae o that happening
12:45:00 <ChavGPT> modulo brain damage-inducing accidents etc.
12:45:00 <gog> thanks
12:45:00 <gog> maybe my brain is damaged
12:49:00 <gog> it would explain a lot about me
12:49:00 <gog> idk
13:08:00 <heat> ChavGPT, what's the problem
13:08:00 <ChavGPT> you do see it is the same loc
13:08:00 <heat> yes
13:08:00 <heat> bud
13:08:00 <heat> https://elixir.bootlin.com/linux/latest/source/include/linux/fs.h#L769
13:09:00 <bslsk05> elixir.bootlin.com: fs.h - include/linux/fs.h - Linux source code (v6.5.3) - Bootlin
13:09:00 <ChavGPT> ye and?
13:09:00 <heat> it's the same thing
13:09:00 <heat> you're just hiding implementation details
13:09:00 <ChavGPT> i'm not commenting about a routine which wraps it
13:09:00 <ChavGPT> but mixing spin vs non spin
13:10:00 <ChavGPT> lck_mtx_lock(&vp->v_lock); vs lck_mtx_lock_spin(&vp->v_lock);
13:10:00 <ChavGPT> utter wtf
13:10:00 <heat> i don't see an immediate problem with that
13:36:00 <netbsduser> ChavGPT: it seems that it works well enough
13:37:00 <netbsduser> it appears to have been added later so probably some sort of optimisation
13:47:00 <ChavGPT> i'm afraid the locking primitives are utther shite
13:47:00 <ChavGPT> their
13:48:00 <ChavGPT> adaptively spinning waiting on a blocking mutex is real-world kernel primitives 101
13:48:00 <ChavGPT> them hand-picking when to do it is a wtf
13:48:00 <ChavGPT> and so happens their locking primitives don't perform whatsoever, so...
13:49:00 <ChavGPT> i would be deeply shocked if it was an informed decision
13:56:00 <heat> netbsduser, btw are you just emulating the windows kernel primitives?
13:56:00 <heat> how hard was it to implement enough stuff to run the driver?
13:59:00 <ChavGPT> https://lore.kernel.org/linux-arch/87msxpzryd.ffs@tglx/T/#t
13:59:00 <bslsk05> lore.kernel.org: [GIT PULL] Remove Itanium support
14:00:00 <ChavGPT> what are you going to run now heat and geist
14:00:00 <heat> the pain is sharp
14:00:00 <heat> i've always been a netbsd fan
14:00:00 <heat> more than netbsduser even
14:00:00 <heat> did you know netbsd can run everywhere
14:01:00 <ChavGPT> except itanium
14:01:00 <ChavGPT> this is an impetus for me to port SOLARIS to itanium
14:01:00 <netbsduser> heat: yes and not very: you have to provide it data in the format it expects (pci information, scatter/gather lists, that sort of thing), it uses two giant variadic functions called StorPortNotification and StorPortExtendedFunction to do things like acquire/release a spinlock, enqueue a DPC, or complete a request
14:03:00 <ChavGPT> hue https://sourceware.org/git/?p=glibc.git;a=commit;h=073edbdfabaad4786e974a451efe4b6b3f7a5a61
14:03:00 <bslsk05> sourceware.org: sourceware.org Git - glibc.git/commit
14:04:00 <heat> the VENERABLE GNU OPERATING SYSTEM REFUSES TO ABANDON ITAAAAAAANIUUUUUUUUUUUUUM
14:05:00 <netbsduser> ChavGPT: it seems they do this as well: https://github.com/apple/darwin-xnu/blob/main/osfmk/arm/locks_arm.c#L2213
14:05:00 <bslsk05> github.com: darwin-xnu/osfmk/arm/locks_arm.c at main · apple/darwin-xnu · GitHub
14:06:00 <ChavGPT> they do some adaptive spinning, but it is implemented in about worst way you can
14:06:00 <ChavGPT> by walking cpu list
14:06:00 <ChavGPT> above i'm complaining about supposed selection by the caller whether to spin
14:06:00 <ChavGPT> which is one funny idea
14:07:00 <mcrod> hi
14:08:00 <heat> netbsduser, have you tried https://github.com/microsoft/Windows-driver-samples/tree/main/storage/miniports/storahci
14:08:00 <bslsk05> github.com: Windows-driver-samples/storage/miniports/storahci at main · microsoft/Windows-driver-samples · GitHub
14:09:00 <netbsduser> heat: not yet, i expect to need to do more work before this would work as at a glance it needs some more stuff implemented in my storport shim
14:09:00 <heat> i would be surprised if this isn't exactly the driver microsoft uses
14:10:00 <mcrod> heat: hi
14:12:00 <heat> hi
14:12:00 <netbsduser> i suspect it is, the "fastfat" driver in the same repo is the actual bundled fat driver of windows
14:13:00 <heat> Copyright (c) 1989-2000 Microsoft Corporation
14:13:00 <heat> haha yes
14:14:00 <heat> oh gosh they really do use exception handling in the kernel
14:14:00 <heat> oh no
14:14:00 <heat> #define SafeZeroMemory(AT,BYTE_COUNT) { \
14:14:00 <heat> try { \
14:14:00 <heat> RtlZeroMemory((AT), (BYTE_COUNT)); \
14:14:00 <heat> } except(EXCEPTION_EXECUTE_HANDLER) { \
14:14:00 <heat> FatRaiseStatus( IrpContext, STATUS_INVALID_USER_BUFFER ); \
14:14:00 <heat> } \
14:14:00 <heat> }
14:15:00 <heat> i guess it's fun that they made it a compiler-level construct while linux does it with assembly hacks and asm goto
14:52:00 <mcrod> someone at my company suggested C++ for the next product
14:53:00 <mcrod> then the examples promptly failed to compile
14:53:00 <mcrod> god i'm hungry
16:21:00 <immibis> c++ is a good choice if you don't like the environment adding 10x speed penalty to your code just for existing
16:21:00 <immibis> (but you don't mind a 10x longer compile time :P )
16:23:00 <geist> also usual rule: its good if you know what you're doing, and everyone on your team does. but as you add more folks on the team that dont fully grok the language (which hardly anyone does) you'll have a lot of acquired language debt to constantly deal with
16:24:00 <immibis> oh, social factors... just make the entire product by yourself without ever talking to anyone, and they won't bother you
16:24:00 <immibis> don't even talk to anybody outside of work
16:25:00 <geist> totes
16:25:00 <geist> ie, a hobby project
16:25:00 <immibis> don't even say hello to the cashier at the supermarket
16:48:00 <heat> rust
16:49:00 <mcrod> rust
16:50:00 <heat> have you considered writing it in
16:50:00 <heat> rust
16:52:00 <mcrod> my boss's predecessor was fascinated with rust
16:53:00 <mcrod> thanks, I'll just stick to C
17:42:00 <nikolar> C for the win
17:43:00 <mcrod> the problem is, C is playing catch up with some C++ features these days
17:43:00 <mcrod> features of which, are useful
17:43:00 <mcrod> restricted constexpr, digit separators, etc.,
17:44:00 <ChavGPT> ;)
17:46:00 <netbsduser`> gcc should really try to innovate some improvements to C, like they used to do
17:46:00 <zid> nobody gives a shit, is the problem, they're too busy trying to catch up to the C++ spec spam
17:46:00 <mcrod> case ranges would be a nice to have
17:50:00 <mcrod> i can definitely say though with absolute confidence ANSI C is still ruling the world
17:50:00 <ChavGPT> they should make C like javascript
17:53:00 <netbsduser`> i don't think anyone really writes ANSI C
17:54:00 <mcrod> sure they do
17:54:00 <mcrod> they hate it, but they do
17:54:00 <mcrod> read: embedded
17:55:00 <netbsduser`> those ones definitely don't
17:55:00 <mcrod> the mega corporate embedded compiler doesn't even really support C11
17:55:00 <mcrod> i know what you'll say: they write C99
17:55:00 <mcrod> if they're lucky enough, they will
17:55:00 <sham1> C89 KUR^W
17:56:00 <mcrod> fortunately we use C99
17:56:00 <netbsduser`> there's nothing in ANSI C that says that volatile int *someregister =0xwhatever; *someregister = 42; will actually write 42 to the address 0xwhatever
17:56:00 <mcrod> i'm not sure I believe you
17:57:00 <sham1> It's implementation-specific
17:57:00 <netbsduser`> how could it make that guarantee? the standard was designed to allow for weird "C" on a LISP machine where the compiler uses arrays to represent the storage for different types, and then there are other weird "C"'s they had to accomodate like AS/400 C
17:59:00 <zid> Good news, C doesn't have to say it.
17:59:00 <zid> Your compiler says it, it being the thing that defines the implementation.
17:59:00 <zid> Infact, int *somereg = blah; is IDB to begin with.
18:00:00 <netbsduser`> that's the wonderful thing about the C that we actually write
18:00:00 <mcrod> sham1 looks right: An object that has volatile-qualified type may be modified in ways unknown to the implementation or have other unknown side effects. Therefore any expression referring to such an object shall be evaluated strictly according to the rules of the abstract machine, as described in 2.1.2.3 Furthermore, at every sequence point the value last stored in the object shall agree with that prescribed by the abstract machine, except as modified by
18:00:00 <mcrod> the
18:00:00 <mcrod> unknown factors mentioned previously. *What constitutes an access to an object that has volatile-qualified type is implementation-defined.*
18:00:00 <zid> It's not, via just the spec, possible to write OS code in C
18:00:00 <zid> Mainly because it doesn't mandate things be.. fast? Like, a += 30; could take 800 seconds to execute and be compliant. Live with it.
18:00:00 <zid> It's not a problem, just a fact
18:01:00 <sham1> Writing malloc is technically undefined behaviour
18:01:00 <netbsduser`> it could not be specified because its strength is precisely that it doesn't specify too much
18:01:00 <netbsduser`> so true enlightenment is realising that undefined behaviour is good actually
18:01:00 <moon-child> -fno-strict-aliasing c is the only real c
18:46:00 <gog> hi
18:49:00 * kazinsal pets gog
18:49:00 * gog prr
18:56:00 <zid> hyper? agreed.
19:06:00 <ChavGPT> hyprrrr
19:06:00 <ChavGPT> you know what sucks mon
19:06:00 <ChavGPT> memory reclamation in low mem conditions
19:11:00 <netbsduser`> ChavGPT: yes, i like to page things out before it gets too severe
19:12:00 <netbsduser`> i keep a hierarchy of memory priority, so that e.g. paging i/o will not block when it tries to allocate its request blocks
19:23:00 <mcrod> hi gog
19:25:00 <gog> hi mcrod
19:25:00 * mcrod hug gog
19:26:00 <zid> gog: you made some kind of tree shenanigan at some point right? I need to add mmap(0,
19:27:00 <zid> so that I can STEAL DLMALLOC
19:27:00 <gog> i don't exactly have a mmap
19:27:00 <zid> I do, it just doesn't have mmap(0,
19:27:00 <zid> you have to pick the virtual address yourself
19:28:00 <moon-child> that's mean
19:28:00 <moon-child> why would you make me do that
19:29:00 <zid> Hostility.
19:29:00 <gog> ohh
19:29:00 <gog> i sort of have that in some branch somewhere
19:29:00 <gog> i have a redblacktree with keys (base, length) of used space
19:30:00 <gog> so i just walk that until i find a gap with the right length
19:30:00 <zid> ye that's what I remembered
19:30:00 <gog> and the tree also stores a pointer to the vm struct
19:30:00 <zid> Maybe I should revive my BITMAP
19:30:00 <gog> if i don't still have the code it'd be fairly trivial to reproduce
19:30:00 <zid> and just use that
19:31:00 <zid> bitmaps are nice and easy to use/write/understand
19:31:00 <zid> and only 'pretty' slow rather than 'painfully'
19:34:00 <gog> bitmaps are a lot of vm space but i guess trees are too when they start getting full
19:35:00 <zid> don't care about vm space, only zuul
19:44:00 <moon-child> https://media.discordapp.net/attachments/585264720957669377/1151916102385938553/image.png :<
19:45:00 <zid> That man? Ea-nasir.
20:27:00 <dzwdz> what should i check if i'm unable to receive IRQs when in ring 3?
20:28:00 <zid> interrupts being enabled
20:28:00 <dzwdz> i run sti right before sysret
20:28:00 <zid> sysret loads flags reg
20:28:00 <zid> from stack
20:28:00 <dzwdz> ...right
20:28:00 <zid> you were supposed to push 0x7202 or whatever
20:28:00 <zid> if you wanted to iret to userspace with irqs enabled
20:28:00 <zid> then every time after that you can just save/restore as normal
20:31:00 <dzwdz> oh - eflags is part of rflags, right?
20:31:00 <dzwdz> i wrongly assumed it was a separate register
20:31:00 <dzwdz> and i didn't see the docs mention that sysret writes to eflags so i assumed it preserves the current value
20:31:00 <dzwdz> thanks
20:32:00 <zid> flags -> eflags -> rflags
20:32:00 <zid> same as ax -> eax -> rax
20:32:00 <dzwdz> yeah
20:32:00 <dzwdz> obvious in retrospect
20:53:00 <gorgonical> I have managed the buffers themselves with caching
20:53:00 <gorgonical> But still the lock isn't synchronizing correctly and I get a deadlock
20:53:00 <gorgonical> But geist it did turn out to be a problem with secure dirty
21:02:00 <heat> zid, you don't need mmap for dlmalloc, just brk
21:04:00 <zid> yea but then you can't really free for shit
21:10:00 <netbsduser`> just experimented with loading storahci.sys
21:10:00 <netbsduser`> it does require much more than viostor.sys unfortunately
21:10:00 <netbsduser`> i'll eventually try and get enough shimmed for storahci.sys
21:14:00 <heat> zid, not in the kernel unless you play funny tricks with virtual memory there
21:14:00 <heat> but the TRADITIONAL! userspace hack is to discard ranges of unused pages that effectively reduces your memory consumption
21:15:00 <gog> hi
21:15:00 <heat> hi grog
21:15:00 <gog> hi hete
21:16:00 <heat> hete just woke up
21:16:00 <heat> from a noice nap
21:16:00 <gog> cool
21:16:00 <heat> poggers
21:19:00 <mcrod> i'm holding my cat
21:26:00 <gorgonical> i just adopted a cat, mcrod
21:26:00 <gorgonical> she's still in the hiding in the bathroom in her box phase
21:32:00 <zid> same
21:32:00 <gorgonical> that you got a cat or that you're in your bathroom hiding phase
21:33:00 <zid> in a box hiding
21:33:00 <gorgonical> yep
21:33:00 <gorgonical> metaphorically all of us osdevers do that
21:33:00 <gorgonical> in the greater box of our homes
21:33:00 <heat> >all of us osdevers
21:33:00 <heat> woah woah hold the phone
21:34:00 <heat> other people do osdev too?
21:34:00 <gorgonical> I just do shitty phd research and pretend i have chops
21:59:00 <zid> heat: Why would the kernel not want to free things, and how would I implement that 'userspace hack' without a way to track said allocations?
22:00:00 <heat> sure you do, you have the mmu
22:01:00 <heat> but it's not a question of "why would the kernel not want to free things", it's "give me an allocator that's trivial to implement and is Good Enough(tm)"
22:02:00 <heat> all things considered, dlmalloc isn't a great choice for a kernel allocator
22:02:00 <heat> but it works
22:02:00 <zid> you're not really answering my question
22:02:00 <zid> I'm having that question because I don't understand what you're trying to say
22:02:00 <zid> "you have the mmu" isn't really an answer to "the kernel doesn't free"
22:04:00 <heat> i think I misspoke
22:05:00 <heat> what I meant to say is "brk isn't an impediment to freeing things, unless you're in the kernel, in which case you can take the choice of still playing the vm tricks (but may potentially fault on any memory access))"
22:05:00 <zid> what vm tricks
22:05:00 <heat> <heat> but the TRADITIONAL! userspace hack is to discard ranges of unused pages that effectively reduces your memory consumption
22:05:00 <zid> that's a userspace hack
22:06:00 <zid> and doesn't answer the question either, because of what I asked, how are you tracking that?
22:06:00 <heat> it works in the kernel if you take the page faults
22:06:00 <zid> The point of brk is that you're *not* doing tracking
22:06:00 <heat> in the page tables
22:06:00 <zid> what page faults?
22:06:00 <zid> this is like 1) brk 2) ??? 3) ??? 4) ??? 5) ??? 6) handle page fault 7) ??? 8) ???
22:07:00 <heat> lets say you have a large-ish range of memory inside your brk region that is completely free
22:07:00 <zid> free conceptually, we don't actually have to way to signal that it's free
22:07:00 <heat> the linux system call for this (which you could implement kernel-side too) is madvise(MADV_DONTNEED)
22:07:00 <zid> but then we need a way to track whether each page/range is free
22:08:00 <zid> and that's.. the bitmap I suggested in the first place
22:08:00 <heat> hold on
22:09:00 <heat> so madvise(MADV_DONTNEED) basically takes the pages mapped on the range, unmaps them and frees them. the next page fault will give you brand new, zeroed ones
22:09:00 <heat> that's the tracking. it's just the page tables
22:09:00 <zid> so munmap, then
22:09:00 <heat> it's not munmap, because munmap would mean next access => fault
22:09:00 <heat> erm
22:09:00 <heat> s/fault/SIGSEGV or whatever/
22:10:00 <zid> okay but we free'd it
22:10:00 <zid> why wouldn't it fault
22:10:00 <heat> because it's not free in a virtual addressing sense and touching it will work as usual
22:10:00 <zid> what
22:10:00 <heat> that's the vm trick
22:10:00 <zid> what'a a "virtual addressing sense"
22:11:00 <zid> I know of mapped memory, and faults, those are my two options
22:11:00 <heat> you can't mmap elsewhere and get that same range
22:11:00 <zid> my cpu doesn't have a virtual addressing sense
22:11:00 <zid> 'mmap elsewhere'
22:11:00 <zid> mmap what, from where, to where
22:11:00 <zid> we were talking about freeing
22:15:00 <zid> 1) brk 2) ??? 3) ??? 4) virtual addressing sense 5) ??? 6) handle page fault 7) ??? 8) mmap 9) ???
22:20:00 <Cindy> hi
22:21:00 <Cindy> do you think virtual addressing shoudln't be on CPU?
22:35:00 <gog> mmap me
22:36:00 <ChavGPT> MAP_FAILED
22:36:00 <gog> also wasn't there a 68000 something that had an external MMU
22:36:00 <gog> or could use one
22:36:00 <gog> but didn't need one
22:36:00 <zid> yes
22:36:00 <zid> nothing needs one
22:36:00 <zid> It's just useful if you wanna run multiple programs
22:36:00 <gog> yes
22:45:00 <netbsduser`> zid: i need one
22:46:00 <netbsduser`> there was a 68851 mmu for the 68020
22:48:00 <zid> well, I haven't got one
22:48:00 <netbsduser`> unlucky
22:49:00 <netbsduser`> you can get an MMU-bearing 68030 accelerator quite cheaply, the TF536
22:55:00 <zid> I want a load bearing 6502
23:01:00 <gog> hi
23:44:00 <netbsduser`> i've made a little progress towards running the storahci driver
23:45:00 <netbsduser`> need to shim StorPortInvokeAcpiFunction (!) to get any further
23:45:00 <zid> how much towards iastor.sys?
23:48:00 <netbsduser`> i tested the XP version of that but it was out of the question
23:49:00 <netbsduser`> used dozens upon dozens of functions from the NT kernel itself rather than nicely sticking to the storport interface
23:49:00 <zid> :D
23:50:00 <zid> it's STORAGE TECHNOLOGY
23:50:00 <zid> not just a driver, smh
23:51:00 <heat> wait, you're shimming storport and not the nt kernel stuff?
23:51:00 <zid> ndiswrapper yo
23:51:00 <zid> nstorwrapper
23:52:00 <netbsduser`> i am not masochistic enough to deal with the entire hundreds of NT kernel symbols that general drivers use
23:52:00 <heat> the enviable windows storage stack
23:52:00 <heat> well i'd expect the storage drivers to still use those?
23:52:00 <netbsduser`> believe it or not, they don't
23:53:00 <netbsduser`> most of them use under 25 symbols almost all (or even entirely) from storport
23:53:00 <zid> I'd expect the odd call here and there for sure
23:53:00 <zid> given nothing is actually stopping them
23:53:00 <heat> that's bizarre
23:53:00 <heat> is the rest inlined or??
23:53:00 <netbsduser`> virtio-scsi for example uses a few things from the NT kernel proper and HAL, but the official MS drivers don't
23:54:00 <heat> like no way there are no locks in a storage driver?
23:54:00 <heat> or does storport take care of serialization for ya or?
23:54:00 <CompanionCube> is the broader goal here to support some proprietary RAID driver or what?
23:54:00 <netbsduser`> well, the storport interface is quite abstract, and it helps a lot with serialisation by e.g. allowing you to have MSI vectors associate with a spinlock and the ISR is locked by such
23:54:00 <heat> fun and fuckery i think
23:55:00 <zid> That's why I wanna see iastor CompanionCube, so I can run ICH10R raid
23:55:00 <netbsduser`> and the StorPortNotification function is a massive variadic function which includes functionality like "lock MSI no. 3's associated spinlock"
23:55:00 <netbsduser`> heat is right
23:55:00 <zid> sweet
23:55:00 <CompanionCube> ah
23:56:00 <netbsduser`> i was looking at the virtio-scsi driver for windows and struck by how it looked simple enough to shim