channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched
#osdev2 = #osdev @ Libera from 23may2021 to present
#osdev @ OPN/FreeNode from 3apr2001 to 23may2021
all other channels are on OPN/FreeNode from 2004 to present
http://bespin.org/~qz/search/?view=1&c=osdev2&y=22&m=5&d=25
00:01:00 <clever> mrvn: maybe as a kid, ive got a light pen upstairs
00:01:00 <clever> mrvn: i plan to implement it on the rpi
00:01:00 <clever> mrvn: oh, and i used the snes light canon thing a lot
00:01:00 <mrvn> clever: how? Do you still have a CRT?
00:02:00 <clever> mrvn: yeah, i have ntsc out to my crt tv
00:02:00 <clever> and i have both vsync and hsync irq
00:02:00 <clever> the light pen is just a raw photo-transistor
00:03:00 <mrvn> clever: and a little IRQ that reads the x,y from the gfx chip.
00:03:00 <zid> I just plug ntsc into my arm, you can tell the luminosity via the tingling
00:03:00 <zid> I'm not quite good enough to do colour
00:03:00 <clever> mrvn: i can only get the current scanline from the 2d core, and there is jitter between that value and the actual electron beam position
00:04:00 <clever> but if i measure timing relative to hsync, i can get column
00:04:00 <mrvn> clever: it's fun if you can write a mouse driver for it so gimp can draw with a light pen.
00:04:00 <clever> :D
00:05:00 <mrvn> for shooting in a game it doesn't have to be as accurate I guess. but for gimp you might have to filter out the jitter over multiple frames.
00:06:00 <clever> yeah
00:09:00 <mrvn> I wonder if I could build one of those gun things to work with a laser projector that I have to the Pi.
00:10:00 <mrvn> That's just a lense and a photo receptor so it gathers light from only a narrow point far away, right?
00:11:00 <clever> mrvn: https://youtu.be/2Dw7NFm1ZfY?t=962
00:11:00 <bslsk05> 'Controllers - Super Nintendo Entertainment System Features Pt. 08' by Retro Game Mechanics Explained (00:23:58)
00:12:00 <clever> this explains how the controllers, mouse, multi-tap, and light gun all work on the snes
00:12:00 <clever> and the timestamp skips to the lightgun
00:14:00 <mrvn> clever: I need the physical part, not the registers / software
00:15:00 <geist> i always assumed they looked at the timing of the electron gun
00:15:00 <clever> yeah, thats how i believe it works as well
00:15:00 <geist> and thus only worked on CRTs but from what i understand the NES one woud just blink the screen and show a white box for one frame
00:15:00 <clever> and the calibration deals with any sub-frame delay within your display
00:15:00 <geist> if the gun 'saw' the box you were pointing at it
00:16:00 <mrvn> it needs to trigger an IRQ when the CRT beam lights up the light sensor in the gun.
00:18:00 <zid> yea which is why pointing it at a lightbulb worked :P
00:18:00 <mrvn> I wonder if I can make it trigger from the reflected light of the laser projector and then somehow figure out where the point was from the delay since I send a frame to the projector to refresh.
00:18:00 <zid> 'yep, white pixel'
00:18:00 <zid> The psx one uses the retrace
00:18:00 <zid> that's why it has a weird loopback from the AV out
00:18:00 <clever> in the case of the c64/snes, the gpu will latch the current xy when you hit a special latch pin
00:18:00 <mrvn> zid: a lightbulb wouldn't got off at 50Hz.
00:18:00 <clever> and then you just apply a fixed offset for calibration
00:19:00 <mrvn> no such latch for me,.
00:19:00 <clever> then you need to do it in software, and deal with irq jitter
00:21:00 <mrvn> I'm not even sure the projector always scans the full screen. Maybe it skips dark parts or something. Timing for the position might not be linear.
00:50:00 <mrvn> I'm starting to get the hang of paramter packs in C++
00:56:00 <geist> huh: pac man is much more interesting that i thought: https://youtu.be/ataGotQ7ir8
00:56:00 <bslsk05> 'Pac-Man Ghost AI Explained' by Retro Game Mechanics Explained (00:19:33)
00:56:00 <geist> i always assumed they just all homed in on you
00:57:00 <zid> you'd have noticed if that's what they did
00:57:00 <zid> because it'd be super boring and obvious that that's what they were doing
00:58:00 <zid> It's hard to figure out they all have a different AI, but if you play it for a little bit it's obvious they're not just following
00:59:00 <geist> yah i rarely played it and/or i played the 2600 version which may have been dumber
00:59:00 <geist> also i assume ms pac man's AI is different since it's supposedly non deterministic
00:59:00 <zid> yea I have no clue bout the spin off? sequel?
00:59:00 <zid> Not sure what bucket you'd put ms. pac-man in
00:59:00 <geist> i think it's the direct sequel
01:00:00 <zid> It is mechanically but the only thematic aspect is different
01:00:00 <zid> so it'd be like if mario 64 was the direct 'sequel' to ocarina of time, using its engine
01:00:00 <geist> well, sure. but it's the direct sequel in the sense that it's officially the second game in the franchise, etc
01:01:00 <geist> but yeah the code base is probably totally different or whatnot
01:01:00 <zid> Right that's a good way of putting it
01:01:00 <zid> it's the 2nd entry in the franchise
01:31:00 <mrvn> geist: implementing PacMan true to the original sounds like a nice project. Plus an AI to play it.
01:56:00 <jjuran> geist: The 2600 "Pac-Man" was terrible.
01:57:00 <jjuran> TRON: Legacy is the sequel to TRON. Doesn't mean the technology used was at all similar :-)
02:07:00 <mrvn> When I understand the RPi 3D interface I want to re-implement Populous.
02:36:00 <geist> jjuran: yeah i think it was a crappy reimplementation to try to fit in the hardware. i suspect it just followed blinky's algorithm, which would be easy enough to implement
02:38:00 <geist> oh gosh it was much worse than i thought. i only see 2 ghosts
02:38:00 <zid> I have it somewhere
02:39:00 <zid> It had more than 2 ghosts
02:39:00 <zid> but maybe 1 of them had to be dead in order for there to be 3? :P
02:40:00 <geist> dunno was just watching smoe longplay on youtube
02:40:00 <geist> or maybe itw as a difficultly selector feature
02:40:00 <zid> ooh yea
02:40:00 <zid> Forgot about the damn hardware switch for that
02:41:00 <heat> breaking news: poltergeist wants more ghosts
02:41:00 <zid> He's finally raising his undead armies
02:41:00 <heat> >undead
02:42:00 * heat resists turning this into #darksouls again
02:42:00 <zid> I said undead not undying, to be fair
02:44:00 <zid> heat do you think maybe that the 2600 pacman might be a pretty... hollow... experience?
02:48:00 <heat> yes, but my giant dad disagrees
02:49:00 <zid> I don't have a giant dad :( He's really short like a minature german pony, you might say he's approximately a zweihander.
02:57:00 <geist> https://youtu.be/iI5vrVmJO0Q
02:57:00 <bslsk05> 'Top Secret: I Know a Little German' by James Anderson (00:00:08)
02:57:00 <zid> That's my dad yes
03:10:00 <zid> shit I was going to google something just after I did what I was doing
03:10:00 <zid> and now I've completely forgotten what it was, I blame heat
03:13:00 <heat> you were googling bing and then binging a zweihandler build
03:14:00 <zid> nah you just watch ymfah for giantdad
04:29:00 <zid> someone ring my isp for me I fucking hate ringing my isp
04:33:00 <heat> it's 5am
04:33:00 <zid> yesand
04:34:00 <heat> ringing your isp is rude at 5am
04:34:00 <heat> they might be sleeping
04:34:00 <moon-child> being an isp is rude at any time
04:34:00 <moon-child> they deserve it
04:34:00 <zid> isps are the internet borgouseisie
04:34:00 <zid> bourugourugiseousie
04:34:00 <heat> if being an isp is rude, and you're being rude, you're becoming an isp
04:35:00 <heat> do not become the thing you set out to destroy
04:36:00 <moon-child> ? no that doesn't follow
04:36:00 <moon-child> if one is an isp, then one is rude. That doesn't mean that, if one is rude, one is an isp
04:36:00 <moon-child> pls no bad logic mr aristotle
04:36:00 <zid> heat failed his iq test :(
04:36:00 <zid> came back negative
04:36:00 <psykose> does not mean you are one
04:36:00 <psykose> but it means you are closer to one
04:36:00 <heat> sorry I got brain damage from looking at x86 TBI
04:37:00 <zid> heat was that a yes btw, you'll call them?
04:37:00 <heat> WHY THE FUCK DID AMD AND INTEL GO TWO SEPARATE WAYS
04:37:00 <heat> no
04:37:00 <zid> what's TMI anyway
04:37:00 <heat> TBI
04:37:00 <zid> is it like FRED but worse
04:37:00 <heat> top byte ignore
04:37:00 <heat> address tagging
04:37:00 <zid> ah
04:38:00 <heat> somewhy they got two different solutions
04:38:00 <heat> because fuck it
04:38:00 <zid> what's wrong with pci or whatever
04:38:00 <moon-child> oh that thing
04:38:00 <moon-child> ignore top bits is convenient though
04:39:00 <heat> amd's solution sucks
04:39:00 <heat> intel's solution is better
04:39:00 <heat> zid, pci is old, use PCI-e
04:39:00 <zid> oh, sorry
04:40:00 <zid> I need a new cable modem card but in pci-e now, I take it you'll be phoning my ISP to organize that?
04:40:00 <heat> no
04:40:00 <zid> my scams don't work on heat
04:40:00 <heat> i can't im super busy
04:41:00 <zid> moon-child: How are you for being scammed?
04:43:00 <moon-child> zid: stop trying to scam me into doing your dirty work for you
04:46:00 <zid> Why don't we scam each other, I'll email your bank and you can phone my isp
04:48:00 <psykose> i think offering kisses has a higher chance of success
04:51:00 <zid> we already do that one
04:52:00 <psykose> that's very unfortunate then, it seems your isp will simply never get phoned..
04:55:00 <zid> Correct, cus I ain't doing it
04:55:00 <zid> Let's hope they notice on the remote end that my internet is fucked and magically decide to send me a new modem
05:02:00 <heat> whats your isp
05:02:00 <zid> wirgin
05:02:00 <zid> medjia
05:04:00 <heat> nah i'm definitely not calling now
05:04:00 <zid> see
05:04:00 <zid> It's a completely rational behavior
05:04:00 <heat> i'd call if it were sky
05:05:00 <zid> what about o2 or bt, do you have a league table
05:05:00 <heat> no
05:05:00 <heat> only sky because SKY IS THE BEST HOME OF PREMIER LEAGUE FOOTBALL AND THE BEST MOST INNOCENT CEO RUPERT MURDOCH
05:05:00 <heat> SKY CAN DO NO WRONG
05:05:00 <zid> natch
05:09:00 <zid> Did you remember to celebrate Supreme Leader Murdoch's birthday
05:09:00 <zid> The only man in history to hit a hole in one on every hole of a golf course
05:12:00 <zid> That's what you get for not remembering to celebrate, ping timeout
05:14:00 <psykose> oof
05:14:00 <psykose> if only the isp was called to improve the internet
05:27:00 <stephe> hey
05:27:00 <stephe> do you guys want some fresh lutefisk
06:11:00 <geist> isn't that an oxymoron?
06:12:00 <geist> (and probably the joke)
06:12:00 <geist> i had some lutefisk before and i remember it being not terrible
06:15:00 <stephe> no it's not that bad, specially with bacon and mustard sauce
16:44:00 <mrvn> Did you know that `cin>>n;
16:44:00 <mrvn> int ar[n][n]={0};` will not initialize the whole array?
16:45:00 <heat> https://godbolt.org/z/5so9EdzoP
16:45:00 <bslsk05> godbolt.org: Compiler Explorer
16:46:00 <zid> Less than 10 pages, C++ is improving!
16:47:00 <mrvn> and the real error is in line 2 and not somewhere down there
16:47:00 <mrvn> amd all the types are nicely annotated. What is your point heat?
16:48:00 <heat> doesn't work for std::array either
16:48:00 <heat> despite it being trivial to add
16:48:00 <zid> how DARE you expect two core language features to be able to interact
16:48:00 <heat> did they forget? the answer is "probably"
16:48:00 <mrvn> you are trying to read an array from stdin. What has that got to do with initialization like in my example?
16:48:00 <zid> You're supposed to virtually something something poly something friend class
16:48:00 <zid> duh
16:49:00 <heat> mrvn, ah wait sorry I read your example wrong
16:49:00 <heat> vla matrices is spooky
16:49:00 <heat> anyway, my example is still fun
16:50:00 <mrvn> https://godbolt.org/z/7fqz4sfhh
16:50:00 <bslsk05> godbolt.org: Compiler Explorer
16:51:00 <mrvn> heat: now go and write a paper to extend the STL
16:51:00 <mrvn> That could maybe even be written even more generic with ranges.
16:52:00 <jafarlihi> You know about that HN post about guy reading OpenBSD code daily? I wanted to ask why OpenBSD and not, say, Linux?
16:53:00 <jafarlihi> If you could pick one OS to read (to learn osdev), which one it would be?
16:53:00 <mrvn> Linux is too big, you would never finish
16:53:00 <heat> they're all too big
16:53:00 <heat> and that's a useless activity
16:53:00 <heat> you don't read source code front to back
16:54:00 <mrvn> top down? bottom up? back to front?
16:54:00 <heat> random access :)
16:55:00 <mrvn> I just format the source in the shape of a penguin and print a poster.
17:04:00 <jafarlihi> Is reading source not a good way of learning?
17:04:00 <zid> It's part of it
17:04:00 <mrvn> there is reading and understanding
17:05:00 <zid> You might see new techniques or styles you hadn't thought of
17:05:00 <zid> but it isn't actually learning to program
17:05:00 <zid> It's exactly like writing novels or building furniture
17:05:00 <zid> i.e it's a skill, you learn through practice, but seeing other people do it can give you ideas or techniques
17:06:00 <heat> right
17:06:00 <heat> i mean, reading source is a good way of learning
17:07:00 <mrvn> some people never leanr no matter what
17:07:00 <heat> you just don't read it *like a book*
17:07:00 <zid> "Oh I didn't think to use a slice api primative here for marshalling access to this buffer"
17:07:00 <zid> not "I memorized this guy's source"
17:14:00 <heat> haha you're screwed i memorized your source
17:14:00 <heat> i hope you're ready to get haxxed
17:14:00 <zid> I stored them on the gibson though :(
17:15:00 <zid> I put it in the garbage file so nobody would snoop
17:20:00 <zid> What's the name of that transport they use in datacenters that isn't IP
17:24:00 <GeDaMo> https://en.wikipedia.org/wiki/InfiniBand ?
17:24:00 <bslsk05> en.wikipedia.org: InfiniBand - Wikipedia
17:24:00 <zid> That might be it
17:27:00 <heat> also nvlink
17:31:00 <mrvn> IPoverIB is a thing, like most other transports datacenters might use
18:33:00 <mrvn> Oh, fun question: https://stackoverflow.com/questions/72382432/templated-constexpr-function-invocation-with-partially-defined-class-changes-sub
18:33:00 <bslsk05> stackoverflow.com: c++ - Templated constexpr function invocation with partially defined class changes subsequent results - Stack Overflow
21:14:00 <heat> linux just removed x86 a.out support
21:14:00 <heat> how will I run my 1992 programs now?
21:14:00 <j`ey> by using an older kernel!
21:15:00 <jafarlihi> How to git gud? I have dedicated 8 hours a day to osdev since a week ago but I barely can do anything outside of copying tutorial codes
21:15:00 <sbalmos> like everything else in here - qemu :D
21:15:00 <heat> j`ey are you high memory in or high memory out?
21:16:00 <heat> jafarlihi, that's because you spent those 8 hours copying tutorials
21:16:00 <jafarlihi> heat: What do you suggest?
21:16:00 <heat> learn it at your own pace and understand *everything* fully
21:16:00 <j`ey> heat: I cant tell if thats a joke im missing, im 64bit anyways, noHIGHMEM here
21:16:00 <heat> j`ey, no that's an actual debate right now
21:17:00 <j`ey> dont some things.. just need it?
21:17:00 <heat> some people including linus want to remove highmem and tell people that need more than 800MB on a 32-bit arch to run older kernels
21:17:00 <heat> some arm people are upset, so yeah
21:18:00 <heat> apparently they're still doing 32-bit ARM machines with lots of memory
21:20:00 <j`ey> yeah.. maybe in a few years
21:20:00 <j`ey> (remove it in a few years I mean)
21:21:00 <heat> oh oops hahaha I read a mailing list thread from 2020
21:21:00 <heat> am I a time traveler?
21:22:00 <j`ey> the power of archives!
21:22:00 <heat> i like highmem, it's an interesting thing
21:23:00 <psykose> is highmem the memory implementation you come up with while being really high
21:23:00 <heat> no
21:24:00 <heat> i know that's a joke, but it's accessing memory > 800MB in the kernel
21:24:00 <heat> the first 800 or so are mapped directly, so they're not highmem
21:24:00 <mrvn> j`ey: If I had drunken 64 Bit I wouldn't understand anything, too.
21:25:00 <j`ey> (it's 768MB right?)
21:25:00 <heat> just mapping it doesn't work because you can run out of mappings and it's slow
21:25:00 <j`ey> and then you have some 'temporary' mappings for other bits of memory (aka outside of the 800MB)
21:25:00 <heat> j`ey, 896MB
21:26:00 <mrvn> heat: Why is highmem even a discussion? Doesn't then kernel address space separation mitigation make that distinction obsolete?
21:26:00 <heat> at least in 32-bit x86
21:26:00 <heat> mrvn, no. how would that affect things?
21:26:00 <mrvn> Just run the kernel in lowmem with the full 4GB address space.
21:26:00 <heat> how do you touch user memory then?
21:26:00 <heat> in a *performant* way
21:26:00 <mrvn> Should be a small change to fix up copy_to_user and copy_from user to use temporary mappings.
21:27:00 <heat> get_user_pages is slow as balls
21:27:00 <mrvn> No way around it with address space separation
21:27:00 <heat> yes there is
21:27:00 <j`ey> heat: maybe its 768 for arm32 then
21:27:00 <heat> the separation is only in usermode
21:27:00 <heat> the kernel still has the lower user bits mapped
21:28:00 <heat> right now copy_(to/from)_user are literally just memcpies with some fallback "if exception" code
21:28:00 <mrvn> heat: right now, not with a low mem kernel
21:28:00 <heat> right
21:29:00 <heat> and the only way to fix those to use temporary mappings would be to do get_user_pages(again, slow as balls) and then map it (slow as balls part 2)
21:29:00 <j`ey> heat: if exception is cool code
21:29:00 <mrvn> Mapping the user pages shouldn't be a problem since you are reloading the page tables later. So no need to INVLPG or anything. Just a few writes to the page tables and memcpy()
21:30:00 <heat> you use PCID/ASID to take a good chunk of the overhead away
21:30:00 <mrvn> don't you have to get_user_pages() anyway to check the process is allowed to read/write there?
21:30:00 <heat> mov %cr3 isn't a tlb flush in those contexts
21:30:00 <heat> no
21:31:00 <heat> again, it's just a memcpy with a goto if it faults
21:31:00 <mrvn> that would allow memcpy() of kernel memory. You have to at least check for x < 0x80000000
21:31:00 <heat> yes
21:32:00 <mrvn> For me the memcpy would go wrong for pages mapped user-read | kernel-write.
21:32:00 <heat> you can't do that though?
21:32:00 <mrvn> ARM can
21:32:00 <heat> the kernel can't
21:33:00 <mrvn> what can't the kernel? map pages "user-read | kernel-write"?
21:33:00 <heat> it's not apart of its interfaces
21:34:00 <heat> definitely not mmap(), and I don't believe there's an internal API for that either
21:34:00 <mrvn> it is for me. that's why I said "For me"
21:34:00 <heat> ok
21:35:00 <mrvn> but anyway: Why 800MB? what happened to the 2:2 split?
21:36:00 <heat> first, it's not a 2:2 split but a 3:1 split
21:37:00 <heat> then that last GB is split between the lowmem (896MB) and the rest
21:37:00 <heat> the rest including ioremaps and kmaps (the highmem thing)
21:37:00 <mrvn> iirc there used to be 1:3, 2:2 and 3:1 split options.
21:42:00 <heat> *shrug*
22:27:00 <geist> I think over time 3:1 or even 3.5:.5 became more ‘standard’ at least in x86 kernels
22:27:00 <geist> Mostly because people wanted more and more user space processes
22:27:00 <mrvn> x86_64 has 3.999:0.001 for 32bit
22:27:00 <geist> And the kernel can in a new version bequeath user space more memory, being the primary advantage of putting the kernel in high at least
22:28:00 <geist> Right
22:28:00 <geist> Only 32 bit arch I know that could easily handle a pure 4:0 split is PowerPC, and that’s because when it takes an exception it drops into ‘real mode’ which implicitly disables the mmu
22:29:00 <geist> So basically you put your exception vectors in physical address space and context switch to the kernel, if you so desired to have it be a separate address space
22:30:00 <geist> You could still do a 2:2 or whatnot split, but for a brief period you’re ‘outside’ of the current mmu context
22:30:00 <moon-child> I think some 32-bit apps broke when you gave them more than 2gb, because they were stuffing pointers in signed ints and then doing things that only worked if those ints were positive
22:30:00 <mrvn> you can do the same with switching to a kernel address space (that has no userspace mapped)
22:30:00 <geist> moon-child:yah i remember that being a thing when NT based systems gave you the opportunity. For a while it was an opt in, a flag on the PE binary, iirc
22:30:00 <geist> mrvn:correct but you still need a little trampoline for most arches
22:30:00 <mrvn> moon-child: butb that's just source that is UB.
22:31:00 <mrvn> Bigger problem is that ptrdiff_t doesn't have enough bits for 4GB.
22:31:00 <moon-child> mrvn: yep
22:31:00 <mrvn> and ssize_t
22:31:00 <moon-child> so?
22:31:00 <moon-child> apps rely on ub all the time :P
22:31:00 <geist> I actually remember that being a thing when I worked at Sega. We had some build system stuff that would actually push 2GB, so we had to enable the 3GB flag on the build machines and set it in the project
22:31:00 <mrvn> moon-child: no app may rely on UB. Only on IDB
22:32:00 <moon-child> I said nothing about whether they 'may' or not
22:32:00 <moon-child> fact is they do
22:32:00 <mrvn> moon-child: apps segfault and crash a lot too. Luckily the compiler warns about most of that stuff.
22:32:00 <moon-child> you know how many things broke because they stuffed pointers into 'int', which stopped working on 64-bit because pointers were still 32-bit?
22:33:00 <mrvn> pointers in signed ints will fail spectaculary on alpha because user space is above 4GB.
22:34:00 <geist> That’s because it wasn’t actually a 32bit arch. It was very much like x32
22:34:00 <geist> Well at least the NT for alpha was
22:34:00 <mrvn> I mend linux on alpha. Not sure what NT for alpha used as address space
22:35:00 <geist> I think hey just only mapped in <4GB and then emitted code in x32 style
22:35:00 <mrvn> verry likely
22:35:00 <geist> I learned this much later when they did the first ‘true’ 64bit NT with titanium
22:35:00 <mrvn> long == 32bit under windows anyway.
22:35:00 <geist> And then x64-64
22:35:00 <mrvn> Tons of windows progs store pointers in long because surely that's the big type that fits everything, right?
22:37:00 <mrvn> geist: Does fuchsia in 64bit start userspace at 4GB?
22:37:00 <geist> Neon.
22:37:00 <geist> Nein
22:38:00 <geist> (Stupid autocorrecting client)
22:38:00 <raggi> Fuchsia is always 64bit
22:38:00 <moon-child> it's fuchsia, not neon
22:38:00 <geist> Used to set the user space base at 16MB but i think we moved it back to 2MB because Linux :(
22:38:00 <moon-child> sheesh
22:38:00 <j`ey> does anything actually rely on that?
22:38:00 <geist> Linux does.
22:39:00 <mrvn> j`ey: rely on what?
22:39:00 <j`ey> sorry I meant does anything in fuschia rely on it being 64bit
22:39:00 <geist> Since as it is well known, any feature/misfeature/etc of anything quickly becomes load bearing
22:39:00 <geist> Oh lots. Mostly in that we generally assume that we have 64bits to play with
22:39:00 <geist> And thus design algorithms around having lots of address space to play with
22:39:00 <geist> But outside of that, probably not
22:40:00 <j`ey> makes sense
22:40:00 <mrvn> Just having all physical memory mapped really limits you for 32bit.
22:40:00 <geist> And of course the kernel assumes 64bit so it can generally not worry about a tight address space
22:40:00 <geist> Right
22:40:00 <mrvn> (and we are back at linux only supporting 800MB)
22:40:00 <geist> And also informs thing like passing 64bit values around and not worrying about it being slow
22:41:00 <geist> But fuchsia had a 32bit port for the first year or so, inherited from LK
22:41:00 <geist> We just decided it wasn’t worth maintaining it
22:41:00 <mrvn> geist: is that so much of an issue? atomic_t can be a big problem.
22:41:00 <geist> What is?
22:41:00 <mrvn> 64bit atomics when the cpu has no double word atomic primitive
22:41:00 <geist> Right
22:42:00 <geist> Stuff like going hard on time is 64bit period, etc
22:42:00 <geist> Or things like 64bit object ids
22:42:00 <moon-child> cmpxch8b!
22:42:00 <moon-child> and double-width arithmetic is not that bad
22:42:00 <geist> Actually this is sort of an issue with riscv since it doesn’t have a double word atomic
22:42:00 <mrvn> Aarch64 has 128bit atomics so you can swap 2 pointers, right?
22:42:00 <geist> We have a few places in the kernel that use double word atomic
22:43:00 <moon-child> mrvn: yeah
22:43:00 <moon-child> casp
22:43:00 <geist> Yes, arm64 and x86-64 both have basically the same thing
22:43:00 <moon-child> in lse, I think also armv8.2+
22:43:00 <geist> Well, to be very specifically: x86-64-v2+. K8 didn’t have it, but later cpus all did
22:43:00 <geist> -mcx16 i think is the feature switch for x86
22:44:00 <geist> Armv8.0 implicitly has it too
22:44:00 <geist> Because arm ain’t dumb. They have to at least feature match x86
22:44:00 <j`ey> moon-child: FEAT_LSE is armv8.1
22:44:00 <moon-child> ok
22:44:00 <geist> 8.0 also has it in the now ‘legacy’ atomics
22:44:00 <moon-child> -mcx16 sort of...
22:45:00 <moon-child> clang does the right thing, but gcc will emit a library call for __atomic*
22:45:00 <geist> Youcan disable that with a switch
22:45:00 <moon-child> it will do the cas inline for __sync*, but __sync* doesn't give back both the result and the success, which is annoying
22:45:00 <moon-child> so I do it with inline asm
22:45:00 <geist> That’s a feature of gcc that lets you switch to a thing that detects it. It’s kinda a misfeature, so you can turn it off
22:45:00 <moon-child> geist: you can?
22:45:00 <moon-child> what's the flag?
22:45:00 <j`ey> -mno-outline-atomics
22:45:00 <geist> I dont know off the top of my head, but ,… yeah thats it i think
22:46:00 <moon-child> 'unrecognized command-line option ‘-fno-outline-atomics’; did you mean ‘-fno-inline-atomics’?'
22:46:00 <moon-child> ditto -mno-...
22:47:00 <geist> I dunno, but the outline-atomics is the stem
22:47:00 <geist> May only be on very recent gccs?
22:47:00 <moon-child> huh there's -moutline-atomics in the manpage, but gcc doesn't like it
22:47:00 <j`ey> aarch64 gcc?
22:47:00 <geist> I remember cause we had to set it for the gcc kernel build
22:47:00 <geist> Maybe only implemented on some arches
22:48:00 <moon-child> ah yeah 'This option is only applicable when compiling for the base ARMv8.0'
22:48:00 <moon-child> I'll keep using inline asm for x86 then...
22:49:00 <geist> Didbn’t know about it for x86 but not equipped to check that out right now
22:52:00 <mrvn> Does AArch64 have an atomic to replace 2 separate pointers?
22:53:00 <moon-child> 'multiple ll/sc', I think
22:53:00 <moon-child> but that's basically transactional memory
22:53:00 <mrvn> I want to set a.next and b.prev atomically.
22:54:00 <moon-child> oh two contiguous things
22:54:00 <geist> There’s a double wide load/store with reservation
22:54:00 <moon-child> yeah
22:54:00 <mrvn> not contiguous
22:54:00 <geist> And thus basically a 16byte cas
22:54:00 <moon-child> oh a.next and b.prev, misread
22:54:00 <mrvn> The 16byte contiguous we already covered above in the positive.
22:54:00 <moon-child> anyway, multiple cas is a thing, and I think possibly also multiple ll/sc. As is htm. But none of these are mainstream (yet?)
22:55:00 <geist> So no you cant atomically fiddle with two non contiguous things
22:55:00 <moon-child> purportedly, intel is trying to make htm work
22:55:00 <mrvn> moon-child: multiple cas allows another core to read the wrong data.
22:55:00 <moon-child> mrvn: 'multiple cas' doesn't mean 'cas one thing, then cas another thing'. It means 'cas multiple things at once'
22:56:00 <mrvn> Now I don't know if I should believe geist or moon-child
22:56:00 <moon-child> as I said, multiple cas is a thing, but it is not mainstream. So you should not expect to find it on any commodity hardware
22:57:00 <moon-child> so I agree with geist on 'can't atomically fiddle with two non contiguous things', with the impractical proviso that there are _some_ fringe contexts in which you might be able to
23:02:00 <mrvn> making a double linkes list thread safe is kind of impossible without a big list-wide lock.
23:05:00 <moon-child> I wouldn't be surprised if it were possible
23:05:00 <moon-child> people have come up with some pretty clever lock-free algorithms
23:06:00 <mrvn> moon-child: all that I've seen fix the forward link and then the backward link and just hope nobody follows the backward link in the mean time.
23:07:00 <mrvn> or they magically wait x ms to make sure no other thread uses a node before freeing it
23:08:00 <moon-child> hmmm, liblfds seems to only have singly-linked lists
23:11:00 <mrvn> The requirement for me is to implement: node.remove_from_list() while holding the nodes mutex. No other info, no lock on the list or prev/next nodes.
23:12:00 <mrvn> problem ist just going to the prev node to lock it risks a deadlock.
23:13:00 <moon-child> what if you set a 'removed' flag on the node before removing it
23:13:00 <moon-child> then if somebody reads a node, they check the removed flag
23:13:00 <moon-child> and know to retry if they see it
23:13:00 <moon-child> downside is you have to keep that flag there forever and can't reuse the node
23:13:00 <moon-child> (gc periodically, w/e)
23:14:00 <mrvn> moon-child: that's the "magically waits till nobody reads the node anymore" problem
23:14:00 <moon-child> so what's wrong with that?
23:14:00 <moon-child> synchronise periodically
23:14:00 <mrvn> it leaks memory
23:14:00 <moon-child> it doesn't
23:14:00 <moon-child> you can tie the synchronisation period to the number of nodes waiting to be reclaimed
23:15:00 <moon-child> thereby bounding the latter
23:15:00 <mrvn> yeah. and that is a stop the world operation.
23:15:00 <moon-child> it is. But it's amortised
23:16:00 <moon-child> and you don't have to actually wait until all the nodes are reclaimed before restarting the world. The actual reclamation can be done concurrently
23:16:00 <mrvn> I want to use this for IPC and syscalls so it has to have a really high throughput. And every time it locks a full page.
23:17:00 <heat_> hey guys
23:17:00 <heat_> RCU
23:17:00 <heat_> just saying
23:17:00 <mrvn> I don't want to block all syscalls and IPC 10 times a second.
23:17:00 <moon-child> stw gc tends to have high throughput
23:17:00 <moon-child> just sayin
23:17:00 <moon-child> why are you using doubly-linked lists for ipc anyway?
23:17:00 <heat> that makes sense
23:17:00 <heat> list of messages
23:17:00 <heat> struct list_head style
23:17:00 <moon-child> yeah. But that can be just singly-linked
23:17:00 <mrvn> It's all done via message passing
23:17:00 <moon-child> my queues don't need backlinks
23:18:00 <heat> but then removing sucks
23:18:00 <heat> O(n)
23:18:00 <mrvn> and I want to be able to do cancelations.
23:18:00 <moon-child> why are you removing from the middle
23:18:00 <moon-child> oh
23:18:00 <mrvn> at least it's something I would like to be able to do.
23:18:00 <moon-child> what about setting a 'cancelled' flag, then, and then knowing to ignore the message once it finally reaches the target?
23:19:00 <mrvn> My current solution is to use single linked lists and a cancelled flag to drop messages when they reach the front.
23:19:00 <moon-child> haha
23:19:00 <moon-child> well, what's wrong with that?
23:19:00 <mrvn> it locks down memory for a while.
23:19:00 <moon-child> it is technically less concurrent. But do you expect to be cancelling messages all the time?
23:20:00 <heat> spinlocks?
23:20:00 <moon-child> singly-linked list optimises the common case of a message that _does_ reach its destination
23:20:00 <heat> hm wait this is in userspace isn't it?
23:20:00 <mrvn> no, kernel.
23:21:00 <heat> ok, then spinlock
23:21:00 <mrvn> So I only have to worry about other cores, not sleeping threads.
23:21:00 <heat> if spinlock becomes a bottleneck, RCU or epoch
23:22:00 <heat> that's how I usually do queue stuff anyway
23:22:00 <heat> generally very fast stuff
23:22:00 <heat> if you indeed profile a bottleneck there, you can move to hardcore things
23:25:00 <mrvn> My tasks use a doubly linked lists too. But every core has a separate list so there is only ever congestion when moving taks between cores.
23:31:00 <heat> you don't have a global list of tasks?
23:32:00 <mrvn> heat: I have that too but that rarely changes. The running/waiting/sleeping/wait_queue list changes all the time.
23:44:00 <mrvn> If I have a class List { List *next; ...}; class Derived : List { ... }; can I say that the List must ba all Derived objects?
23:45:00 <mrvn> List::get_next() should return Derived for whatever derived class it is
23:47:00 <mrvn> In ocaml an object has the classes type and a self type, which is the derived class when you derive.
23:49:00 <moon-child> mrvn: you have covariance issues there, I think
23:49:00 <moon-child> suppose I say List x; Derived *y = new Derived; List *z = new List; x.next = y; x.next->next = z; Derived *yn = y->next
23:50:00 <mrvn> That's ok, then all objects are Lists.
23:51:00 <mrvn> Derived *y = new Derived; List *z = new List; y.insert(z); should give a type error.
23:51:00 <moon-child> I don't know the details of C++ oop. But I assume that subclassing is meant to create a subtyping relationship?
23:52:00 <moon-child> if List::insert can take a List*, then Derived::insert must be able to do so too. (Or, you should be able to convert a Derived* to a List* and call the corresponding insert method)
23:52:00 <moon-child> otherwise a Derived is not a List
23:52:00 <mrvn> moon-child: x.insert(std::some_caset<List>(y)); would work.
23:52:00 <mrvn> -e
23:53:00 <mrvn> In c++ that happens implicit a lot of the time.
23:53:00 <moon-child> fine. But if I then say x.next->insert(z), then y->next is now not a Derived*
23:54:00 <mrvn> In ocaml you can specify it both ways, wether Derviced:foo() should take Lists or Deriveds.
23:54:00 <moon-child> ocaml has mutation, though, right? So how does it avoid this problem?
23:54:00 <moon-child> does it punt to runtime like java?
23:55:00 <mrvn> It has mutation but that has nothing to do with it.
23:55:00 <moon-child> how do you avoid the situation where you have a Derived *y st y->next is a List* and not a Derived*?
23:56:00 <mrvn> moon-child: because "List.insert : 'self -> unit" means that y.insert only accepts objects of the same type as y.
23:57:00 <mrvn> You can't insert a List onto a Derived list.
23:57:00 <moon-child> can I insert a List into an std::some_cast<List>(Derived*)?
23:57:00 <mrvn> And you can only insert a Derived into a List if you cast it to List first.
23:58:00 <mrvn> It kind of looses it's Derived-ness.
23:59:00 <moon-child> oh--wait--you want something like this? class List { List *next_list; ... }; class Derived: List { Derived *next_derived; }, but without separate names for 'next_list' and 'next_derived'?