channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev2&y=22&m=6&d=5

Sunday, 5 June 2022

00:00:00 <mrvn> implementation wise c can be anything but anything outside 0/1 is UB.
00:01:00 <mrvn> ===> thise example should not warn but others should even when you cover all members of the enum.
00:10:00 <zid> I want a secure hash, gotta make sure nobody else knows what my files contain just because they know the hash!
00:13:00 <mrvn> zid: nonon, you want them to see that the file contains one thing while it contains another.
00:14:00 <mrvn> If they can't see what's in the file that's suspicious already.
00:16:00 <mrvn> A hash is usually so small there is really no way to deduce the content of a file from it. Encrypting a file is another matter, but indexing? How are you going to say what a file contains from 256 bit unless you already know?
00:32:00 <geist> but yeah after thinking about it and reading a bit, sha256 is pretty difficult to collide
00:33:00 <geist> as in, no one has ever been able to do it, and given the probability of it there *may* have been a collision, especially with all the mining, in the sense that enough sha hashes have been rolled over the years it's possibile that someone has identically made one
00:33:00 <geist> but of course no one detected it
00:37:00 <mrvn> Practically there is also a big difference between an accidental and purposeful collision.
00:38:00 <zid> yea the chances of detecting it are also miniscule
00:38:00 <zid> on top of the chances of there being one
00:38:00 <mrvn> For the mining a single signature might have a collision but a) you couldn't make money of it and b) the other packet would throw errors if you substituted it in the chain there.
00:38:00 <zid> 2^256 is like.. the number of total transistors ever made, squared squared
00:39:00 <mrvn> What does the birthday paradox say the probability of a collision is with 2^32 hashes? or 2^64?
00:43:00 <mrvn> ; (1-1/(2^256-2^32)) ^ (2^32)
00:43:00 <mrvn> Raising to very large power
00:43:00 <mrvn> *damn*
00:43:00 <mrvn> ; ((1-1/(2^256-2^32)) ^ 2) ^ (2^16) ~1.00000000000000000000
00:43:00 <mrvn> It's just an estimate but still pretty collsion free.
00:44:00 <mrvn> ; 1 - (((1-1/(2^256-2^32)) ^ 2) ^ (2^16)) ~0.00000000000000000000
00:44:00 <mrvn> I think apcalc doesn't use enough digits.
00:46:00 <mrvn> Lets ask Wolfram Alpha: 3.709 * 10^-68
00:46:00 <mrvn> 2^64 hashes: 1.593 * 10^-58
00:47:00 <mrvn> I think I messed up the estimate.
00:57:00 <mrvn> I broke Wolfram alpha: ((2^256-2^64)/(2^256))^(2^64) == 10^(10^5.417461594016476), should be less than 1.
04:33:00 <_73> I just thought this was interesting - A processes kernel stack must always be kept in memory because it is the processes responsibility to handle a page fault and the kernel stack is needed to run the handler, so if the process page faulted without the kernel stack there is nothing the kernel or process can do, and the kernel just panics because this scenario can only occur if there is a major bug in the kernel.
04:34:00 <_73> FreeBSD btw
04:34:00 <Mutabah> A fault without a kernel stack will double-fault (on x86)
04:35:00 <Mutabah> which (if the kernel is defensively designed) will use a different (statically allocated) stack
04:35:00 <Mutabah> and if that's also bad, triple fault!
04:36:00 <mrvn> my processes have no kernel stack.
04:36:00 <mrvn> CPU cores have kernel stacks, not processes, for me.
04:37:00 <_73> what is the OS called?
04:38:00 <mrvn> moose
04:39:00 <_73> Cool, ill read about it.
04:43:00 <Mutabah> ^ per-cpu stacks is another approach. More complex, but forces good design and lower memory usage
04:44:00 <mrvn> I disagree, it's much simpler.
04:45:00 <_73> How do per cpu stacks work?
04:45:00 <mrvn> It's geared towards microkernel though as you can't sleep in kernel. Can't do any thing complex really.
04:46:00 <mrvn> _73: you never sleep or block in the kernel and task switching happens when you leave kernel mode.
04:46:00 <mrvn> SO you only ever switch the processes stack and never the kernel stack.
04:47:00 <_73> ok I see how that is a fundamentally different design.
04:47:00 <mrvn> Also means you save the processes registers when entering kernel mode in the process structure, not on the stack.
04:54:00 <mrvn> It's raining cats and dogs. Time to implement a suffix tree.
05:09:00 <Andrew> Hashes, when hashing things bigger than the hash itself, is never collision free
05:13:00 <Andrew> mrvn: I don't know of one, but it's not intuitive why there *can't* be - unless we're magically compressing data ... using 256 bits to represent files of large size, I mean, how?
05:16:00 <mrvn> Andrew: the question isn't if you are hashing things bigger than the hash but if you are hashing more things than the hash result can represent
05:17:00 <Andrew> - yes
05:17:00 <mrvn> Potentially you can has 2^256 files to unique hashes regardless of their size.
05:17:00 <mrvn> +h
05:17:00 <Andrew> 115792089237316195423570985008687907853269984665640564039457584007913129639936
05:17:00 <Andrew> note: 'potentially'
06:30:00 <Jari--> Hi People, my kernel is crashing unless I use same CPU frequency (synched) as Bridge Speed?
06:30:00 <Jari--> So should I do some power management in this case?
06:30:00 <Jari--> I am basically hitting CPU idle all the time.
06:30:00 <Jari--> Is this normal in NT and Linux kernels? It idles all the time?
06:31:00 <Jari--> CPU bug
06:31:00 <Jari--> I notice this in I/O sensitive tasks like Hard Disk Driver.
06:31:00 <Jari--> Basically I/O should not be poked faster than the bridge speed is.
06:32:00 <Jari--> And each device has their throughoutput limitations.
06:32:00 <Jari--> Impossible to do without power management or related regulation needed.
06:32:00 <Jari--> On Virtual Machines I notice it throws up virtualization exception.
06:34:00 <mrvn> That makes no sense. you are doing something wrong.
06:34:00 <Jari--> mrvn: well in that case my kernel embedded procmalloc (property-allocation) is broken still :P
06:35:00 <Jari--> wrote down a note, I noticed later, re-read
06:35:00 <Jari--> making kernel memory dynamic allocation system stable is a todo
06:36:00 <Jari--> but its just running a single thread so it should not be so unstable
07:50:00 <Jari--> https://forum.osdev.org/viewtopic.php?f=1&t=36406
07:50:00 <bslsk05> ​forum.osdev.org: OSDev.org • View topic - That pesky FPU again
07:50:00 <Jari--> I was just designing the task-switching for my OS, and as part of that, there is the question of FPU handling. That is, the handling of any register set besides the general purpose ones (for simplicity referred to as FPU from here on). First, there is the question of laziness or eagerness. In an age of increasing logical CPU counts, lazy FPU save seems like a bad idea, since a sleeping task with an unsaved
07:50:00 <Jari--> FPU cannot be transferred to another CPU. So lazy FPU save would make the scheduler more complicated, and it already is plenty complicated.
07:51:00 <Jari--> -This is interesting factual talk about the same issue I have, OS/2 3.0 has this etcs.
07:51:00 <Jari--> If I recall, on *some custom hardware, like cloned minilaptops I have, ..just bigger monitor here.
07:52:00 <Jari--> Could be related with the core handling, I have two types of cores e.g.
07:52:00 <Jari--> 2 type A and 2 type B
07:53:00 <Jari--> cheapest minilaptops = lots of fun for low level I/O programming (this is common according to my experience)
07:54:00 <Jari--> https://lore.kernel.org/lkml/20211104225226.5031-1-chang.seok.bae@intel.com/T/
07:54:00 <bslsk05> ​lore.kernel.org: [PATCH 0/4] x86/fpu: Make AMX state ready for CPU idle
07:55:00 <Jari--> VMware has better hardware support and handling than VirtualBox
07:55:00 <Jari--> less crashes, tried more than 20 different OS platforms
07:58:00 <Jari--> and unhandled exceptions might be reason I am not having crashes on non-functioning code
07:58:00 <Jari--> handling all the common 386 processor faults
08:05:00 <mrvn> have you tested how many of your tasks use the fpu regulary?
08:07:00 <mrvn> If you don't tell the compiler to not use SSE that number probably approaches 0
08:07:00 <mrvn> (just a guess)
08:12:00 <Jari--> Apparently I can't write cache code, or I wrote this in a confused state of mind.
08:12:00 <Jari--> Tempting to remove the cache entirely.
08:12:00 <Jari--> E.g. rewrite the cache problem solved.
08:13:00 <Jari--> This is actually buffers, but I call it a cache.
08:13:00 <Jari--> FAT buffer e.g.
08:17:00 <moon-child> here is a question, not really related but it reminded me:
08:18:00 <moon-child> suppose you have big.little with actual architectural differences between the cores. Cf new intels--had avx512 on the big cores, not on the littleones--before they fused off avx512 on the latter
08:18:00 <moon-child> as an application developer, I would actually not mind writing code tuned both for the big cores and for the little cores
08:18:00 <moon-child> but, what interfaces should the os provide to support this?
08:19:00 <GeDaMo> Feature flags?
08:19:00 <moon-child> obviously you can't just move a thread from a big core to a little core. But can you provide some way of migrating control state, perhaps at a safepoint?
08:20:00 <GeDaMo> I seem to remember a problem on some ARM where an instruction was only supported on some of the cores
08:21:00 <moon-child> (and given such an interface, would it make sense to keep running the thread on a big core until it reaches a safe point, or move it to a little core and use sw emulation for the unsupported features?)
08:23:00 <mrvn> if it's using the big features once it's probably going to do it a lot. emulating seems pointless.
08:23:00 <GeDaMo> https://community.arm.com/cfs-file/__key/communityserver-blogs-components-weblogfiles/00-00-00-26-71/3162.Software_5F00_Techniques_5F00_for_5F00_ARM_5F00_big.LITTLE_5F00_Systems.pdf
08:23:00 <mrvn> You can migrate when it throws an illegal instruction
08:23:00 <moon-child> mrvn: the idea is that the safepoints should come at fairly close intervals, so it only has to do a small amount of work before it can switch to the little-specific code
08:24:00 <mrvn> If you design software to switch codes then migrate it only on safepoints
08:24:00 <mrvn> it's not like a core disappears randomly.
08:25:00 <moon-child> but suppose all the big cores are tied up, and a little core is free. Isn't it better to let it make progress, even with sw emulation?
08:26:00 <mrvn> If all your cores are tied up you need a bigger mobile
08:27:00 <moon-child> I have m big cores and n little cores. I should be able to run m+n threads at once. But your way, the latency of safepoints becomes a bottleneck
08:27:00 <mrvn> not really. you just have to run m big threads and n little threads
08:28:00 <moon-child> suppose I used to have m+1 big threads and n little threads. And then one of the little threads died. So I need to turn one of the big threads into a little thread
08:28:00 <moon-child> with sw emulation, I can start on that right away
08:28:00 <moon-child> without it, I have to wait until one of the existing big threads hits a safepoint
08:29:00 <mrvn> As I see it the usual case will be running a single game that want to get all out of your mobile. It shouldn't have m+1 big threads. And other apps that occasionally run should run little code.
08:29:00 <mrvn> And you don't have to wait for a safepoint to schedule. Just to migrate.
08:30:00 <moon-child> indeed. Would be bad if you did :P
08:32:00 <mrvn> I think you are making too much of an issue out of a made up problem. Worst case you run m+1 threads on m cores till a safepoint is reached. That's not the end of the world.
08:34:00 <mrvn> I wouldn't even think of designing such safe points. Switching between big and little code seems too much work already. At most I would mark threads that use big opcodes to run only on the big cores from now on.
08:59:00 <bauen1> would there really be that many tasks that actually benefit from the better opcodes, or have higher performance requirements ? in your average linux system there's quite a few (system) tasks that probably wouldn't benefit from better simd instructions (e.g. network services)
08:59:00 <bauen1> obviously if you're running games you probably have a way to make use of those faster instructions
08:59:00 <jafarlihi> What are some eaiser system software projects you can take on before being ready for osdev?
09:00:00 <mrvn> bauen1: can't think of anything but games or hpc that really would benefit.
09:00:00 <bauen1> jafarlihi: the average "introduction to operating systems" course seems to involve writing your own toy shell in C, and e.g. parsing a FAT32 filesystem
09:47:00 <moon-child> bauen1: well, sure, yes; if you don't need to go fast, no need to do any tuning
09:48:00 <moon-child> but in cases where performance does matter, you frequently profit by taking advantage of available isa extensions, as well as uarch minutiae
09:53:00 <j`ey> you could also use taskset to just force it to only execute on the big cores
09:58:00 <moon-child> I want to use all the cores though
10:11:00 <mrvn> then don't run big code on all threads
10:11:00 <mrvn> And no, generally you want to run as few cores as possible. that's the idea of big/little.
10:14:00 <moon-child> mrvn: idea of big.little, I thought, is that, if you don't have compute intensive work, you run it on the little cores and save power
10:14:00 <mrvn> exactly
10:14:00 <moon-child> if you _do_ have compute intensive work, use the big cores
10:14:00 <moon-child> use all the cores
10:14:00 <Andrew> little endian sucks!
10:15:00 <moon-child> eh, why?
10:15:00 <mrvn> Andrew: it's not about endianness
10:15:00 <Andrew> i know, just made me think of it
10:15:00 <j`ey> moon-child: the best way is to, obviously, just have symmetrical cores :P
10:16:00 <moon-child> Andrew: oh, i thought it was a non sequitur
10:16:00 <mrvn> moon-child: do you have the memory bandwidth to run all cores?
10:16:00 <Griwes> But symmetrical cores make for less problems to solve in your schedulers!
10:17:00 <moon-child> mrvn: good question. Only big.little hardware I have is a mac, so the answer to 'do you have the memory bandwidth' is 'yes' :P
10:18:00 <moon-child> but more generally maybe not
10:22:00 <mykernel> i am currently learning i386 assembly, how calling convention works and how stack works, i think i understand stack alignment (before call you need to have stack word aligned), but i have seen some people talking about using nop to do alignment (like aligning instruction address), can someone give me a push in a right direction? Preferably with practical examples using nop
10:22:00 <mrvn> big mistake. learn at least x86_64
10:22:00 <j`ey> moon-child: high five for m1
10:23:00 <mykernel> mrvn: i will learn it afterwards
10:23:00 <moon-child> mykernel: usually, you will not manually insert nops, but will instead direct the assembler to insert them for you, since you won't know ahead of time what offset your code will be at
10:24:00 <moon-child> e.g. in gas use '.align k' for a k-byte alignment
10:24:00 <moon-child> that said, you should not be worrying about aligning code at this stage
10:25:00 <mykernel> what needs to be aligned?
10:25:00 <j`ey> a non-x86 example is the exception/vector table on arm needs to be aligned
10:25:00 <moon-child> whatever the abi says needs to be aligned--it depends on the abi
10:26:00 <mykernel> thanks
10:26:00 <moon-child> the stack, as you know; also, frequently integers must be self-aligned
10:26:00 <mrvn> it's not that code needs to be aligned, it's that some alignments are faster to jump to
10:26:00 <moon-child> there are usually no requirements regarding code alignment
10:26:00 <moon-child> yeah
10:27:00 <mrvn> Also it's sometimes faster to issue an opcode a cycle later, or half or a quarter cycle so you insert a NOP to delay it
10:27:00 <moon-child> mrvn: afaik it's not even usually that aligned is faster to jump to; it's that if an entire loop (or w/e) fits in some self-aligned small chunk of memory, it's faster
10:28:00 <mrvn> lots of architecture and model specific stuff anyway
10:28:00 <moon-child> yeah
10:28:00 <mrvn> Does x86 icache have a 64byte cache line too?
10:29:00 <moon-child> I think so
10:29:00 * moon-child looks at agner
10:30:00 <moon-child> https://www.agner.org/optimize/microarchitecture.pdf p 144
10:31:00 <mrvn> On Alpha the cpu would read opcodes in pairs so if you jump to an odd slot you waste half the opcodes for that cycle.
10:54:00 <zid> just don't look at agner in anger
11:02:00 <clever> mrvn: i believe cortex-m0+ does the same, its thumb based, so 16bit opcodes, but the opcode fetch is always 32bit wide and 32bit aligned
11:12:00 <mrvn> except thumb can be misaligned I believe
11:13:00 <kazinsal> conveniently we have the ARM extension "magic" for that
11:30:00 <clever> mrvn: ive not heard of that being allowed
11:34:00 <mrvn> geist mentioned that you could have one 16bit thumb and then 32bit opcodes.
12:01:00 <clever> mrvn: but you can only switch between thumb and arm when you branch, and one of the address bits (bit0 i think) signals which mode the dest is
12:01:00 <clever> the cortex-m0+ also doesnt support 32bit opcodes
12:01:00 <clever> so its forever thumb
12:46:00 <mxshift> Thumb2 32-bit opcodes are kinda like a pair of 16-bit opcodes under the hood so no need for 32b alignment
13:11:00 <clever> ahh, thats something different from arm 32bit
18:46:00 * geist yawns
18:47:00 <geist> good morning everyone
18:49:00 <zid> we just had one of those though
18:52:00 <jjuran> What about second morning
18:54:00 <bauen1> jjuran: time to move to a planet in a solar system with multiple stars i guess
18:55:00 <bauen1> i mean how many stars would you need so you could say "good morning" any time and be right ?
18:57:00 <zid> 1
18:57:00 <zid> just move yourself
18:57:00 <zid> sunrise travels at 1600kph on earth
18:57:00 <zid> so you can either go backwards at that speed and get them twice as often, or go forwards at that speed and have it be perpetual morning
18:58:00 <zid> or whatever combination you fancy
19:00:00 <geist> tis true, could have a tatooine morning
19:01:00 <nur> watch out for the krayt dragons
19:14:00 <bauen1> couldn't you move to the north or south pole, that should work for roughly half a year right ?
19:15:00 <zid> you can only go so far north before you get to twilight instead of night
19:15:00 <zid> like say, the UK in summer
19:15:00 <bauen1> but moving around the earth at 1600kph sounds tiring, I would opt to just slow down earths rotation enough to be locked to the sun
19:15:00 <zid> you'd still get a 'sunrise' but the more north you go to cut the speed you'd need down, the less 'morning' makes sense as a concept
19:16:00 <zid> Yea you could just go chill out on mercury, that one is tidally locked
19:18:00 <geist> or on the north (or south pole) of uranus
19:18:00 <geist> oh wait i dont think that's tidally locked either, just sitting sideways
19:18:00 <geist> but probably pointing the sun for a few hundred years at a time
19:19:00 <bauen1> if your only requirement is "always morning for a human life", then I guess even the moon would fulfill that, you just wouldn't have a very long morninng
19:20:00 <\Test_User> "nearly anywhere with a non-breathable atmosphere ought to fulfill that"
19:21:00 <\Test_User> ...though it fulfills it by reducing the timespan required rather than extending the morning
19:21:00 <zid> You can have 'morning' forever if you just stop moving, watch out for passing planets. Or falling into the sun. Or the sun orbitting the galactic core away from you.
19:21:00 <zid> (bring your own fun gasses, not supplied)
19:24:00 <jimbzy> Hallo urrybody.
19:25:00 <geist> howdy
19:25:00 <jimbzy> How's things?
19:26:00 <sortie> I finally have a day off from my crazy life and just am sitting here looking out over the city doing some osdev :)
19:26:00 <jimbzy> Nice
19:26:00 <geist> yay
19:27:00 <sortie> Working on my xkcd 1579 compliance :)
19:27:00 <sortie> https://xkcd.com/1579/ ← Even got the "irc for some reason" part nailed down
19:27:00 <bslsk05> ​xkcd - Tech Loops
19:28:00 <sortie> Once I get audio and can play youtube videos, I might actually have connected it to "things I actually want to use my computer for"
19:31:00 <geist> kazinsal: got the new mobo yesterday, seems stable so far
19:37:00 <bauen1> sortie: oh wow, didn't know that existed, checking in with what is technically my own debian derivative lol
19:37:00 <sortie> :D
19:38:00 <sortie> geist, neato :) What motivated getting a new one?
19:38:00 <geist> oh was having some stability problems with my ryzen server. kinda a long ordeal
19:38:00 <sortie> Ah :)
19:38:00 <geist> eventually settled on the old motherboard being unstable, so got a new one
19:39:00 <geist> will see. the MTBF was like 3 days, so i need to run it for a week or so to really see if it fixed everything
19:40:00 <zid> does this one have more volts
19:42:00 <bauen1> speaking of old hardware, i still have a macbook air 2013 that i want to setup with a proper linux installation again, which is always a joy. and at the same time also see if i can improve the performance a bit (e.g. new thermal paste)
19:44:00 <bauen1> now of course i can't reproduce it right now, but i think the last time i tested the _read speed_ of the internal SSD it ended up dropping to just 40MB/s weird ...
19:45:00 <geist> zid: i think so, yeah
19:45:00 <geist> only annoyance is this mobo turns the socket 90 degrees so my cpu cooler now blows air against the top of the case
19:45:00 <geist> so it's not quite as effective
22:09:00 <scripted> hello
22:09:00 <scripted> been a long time
22:09:00 <scripted> oh, it's in the middle of the night!
22:09:00 <scripted> anyways, for those who care, I moved to blockchain programming
22:09:00 <scripted> godo night
22:17:00 <geist> uh okay.
22:17:00 <geist> good to know
22:37:00 <gog> what should i compile
22:37:00 <gog> no blockchains pls
22:43:00 <j`ey> vim, llvm, linux
22:46:00 <klange> bim, kuroko, toaruos
22:47:00 <j`ey> nice
22:54:00 <gog> i wanna see how fast this baby can compile something
22:54:00 <j`ey> what're the specs?
22:54:00 <gog> Ryzen 5 4600H, 8GB RAM
22:55:00 <gog> gtx 1650 on the side
22:56:00 <j`ey> nice, good that you finally got a new computah
22:56:00 <gog> yes i'm vhappy with it
22:56:00 <gog> prime offload works much better in linux than i anticipated
22:59:00 <geist> i sometimes use compiling qemu as a benchmark, though it has variable options so it's a bit hard to do a proper 1:1 comparison with another machine
23:00:00 <geist> also yay join the ryzen club!
23:00:00 <gog> yes
23:01:00 <gog> this machine came with windows 11 and i immediately wiped it all out and installed manjaro
23:01:00 <gog> works great, no complaints
23:02:00 <Ermine> Oh, you can even use nvidia open source drivers
23:02:00 <gog> i'm using proprietary rn
23:06:00 <geist> yah experience is nvidia linux drivers are pretty solid
23:17:00 <gog> they've improved a lot since last i used them
23:17:00 <gog> which was years ago
23:18:00 <j`ey> talking about open source gpu drivers.. https://twitter.com/AsahiLinux/status/1532035506539995136
23:18:00 <bslsk05> ​twitter: <AsahiLinux> First triangle ever rendered on an M1 Mac with a fully open source driver! 🎉🎉🎉🎉 [https://twitter.com/LinaAsahi/status/1532028228189458432 <LinaAsahi> 🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺 ␤ 🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺 ␤ 🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺 ␤ 🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺🔺 ␤
23:18:00 <gog> aaay
23:19:00 <gog> next step render thousands of them
23:20:00 <j`ey> :)
23:22:00 <Ermine> gog: rendering thousands of them was a problem: https://rosenzweig.io/blog/asahi-gpu-part-5.html
23:22:00 <bslsk05> ​rosenzweig.io: Rosenzweig – The Apple GPU and the Impossible Bug