Search logs:

channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present


http://bespin.org/~qz/search/?view=1&c=osdev&y=19&m=5&d=15

Wednesday, 15 May 2019

12:00:16 <lava`> https://www.youtube.com/watch?v=wQvgyChrk_g this is real time
12:00:41 <lava`> it's pretty fast
12:02:02 <ZombieChicken> fast enough I'm a wondering if this was on purpose or not
12:04:18 <ZombieChicken> but then I'm the paranoid sort, soo...
12:10:55 <aalm> .theo
12:10:55 <glenda> I could not give a rats ass what you say.
12:30:42 <johnjay> hey how good is zfs?
12:30:45 <johnjay> can i use it in windows?
12:31:09 * johnjay wants something redundant without buying another extra hd
12:32:56 <ybyourmom> https://github.com/openzfsonwindows/ZFSin
12:33:05 <ybyourmom> https://www.reddit.com/r/programming/comments/9w7qhe/zfs_on_windows/
12:44:26 <aalm> redudant on a single disk ?
12:47:57 <johnjay> aalm: basically
12:48:07 <johnjay> i'm paranoid about losing data after a hard drive was half corrupt one time
12:52:21 <_mjg> that's why you backup
12:52:23 <aalm> paranoid+running on a single disk with something important.. backing up somewhere off of it is your only option really.
12:52:31 <_mjg> zfs wont auto solve daata loss
12:53:26 <ZombieChicken> 3-2-1; If you want to keep any data, you back it up to 3 places on 2 different media, at least 1 of which is off-site
12:54:49 <johnjay> _mjg: then why did old usb hard drive lose half its files since i didn't power it on for so long?
12:55:14 <ZombieChicken> banged around too much?
12:57:26 <_mjg> oh it's you
12:57:30 <_mjg> right
12:57:53 <_mjg> pendrives are basically supposed to be operational indefinitely
12:58:01 <_mjg> so you got a point
12:59:12 <aalm> they do die too
01:00:34 <_mjg> only if defective or set on fire
01:01:00 <_mjg> which was the case with all the pendrives i had problems with
01:01:32 <regreg> i backup on google drive, i have a 1Tb account
01:01:39 <regreg> it's very cheap
01:01:52 <aalm> now that's redudant, i guess
01:45:47 <hh1848__> hello, gentlemen
01:46:49 <aalm> o hy
01:51:11 <hh1848> a
03:26:10 <jjuran> So pen drives last forever — unless they're defective, and you know one's defective when it fails?
03:26:33 <aalm> :]
03:26:42 <ZombieChicken> by pen drive you mean like a USB thumb drive?
03:28:38 <aalm> i haven't yet seen a pen drive not start spitting errors in <1year when used with daily writes ie. system /-root etc. on it.
03:31:00 <aalm> branded or not, with abusive use expecting +6months of use is non-realistic
03:31:28 <ZombieChicken> aalm: What about sdcards?
03:31:32 <ZombieChicken> same issue?
03:32:12 <aalm> about same
03:33:18 <aalm> i do have sdcards that have been in read-only service for +year tho.
03:38:06 <Mutabah> IME, they last quite a long time - Unless you're constantly writing to them
03:38:15 <aalm> i usually buy like 10cards or 10usb thumb drives, and replace them all after third one fails, so i got plenty of 'might work for a while' cards&drives..
03:40:25 <aalm> wish the hw i use them with had enough ram to participate builds w/distcc w/o writing to disk:(
03:46:03 <doug16k> I did a little experiment with a drive with bad sectors. I wrote a script that just dd filled it with urandom all day every day for a week or so, reading the whole disk to null in between. it actually cleared up most of the bad sectors
03:46:17 <doug16k> it didn't fix it fully though, but it was neat to see it relocate a lot of them
03:48:00 <doug16k> most likely I was invoking the drive firmware's block relocation to a spare
03:49:02 <doug16k> presumably reading let it log somewhere that sectors were hard to read, then later I gave it an opportunity to relocate because I was explicitly writing it
03:57:01 <jjuran> ZombieChicken: I call them flash drives, myself
05:20:22 <AerodynamicBrick> breif offtopic: any audiophiles here?
05:20:43 <AerodynamicBrick> if so, thank me later: https://open.spotify.com/album/5zYrdM5vkTrLL2BSL2WIMx?si=aWWxb2dYTgyLY6dLV7BWCQ
02:10:51 <regreg> hello
02:11:17 <regreg> i've had a little bit of talk on #asm about virtual bytecode but people have been quite hostile
02:11:28 * zid plays a tiny violin
02:11:38 <regreg> zid :)
02:12:13 <regreg> do you ppl have a good alternative to llvm intermediary code? something more similar to MASM/NASM? what should I focus on when developing such a bytecode?
02:12:32 <zid> I'd entirely ignore assembly-like language
02:12:40 <zid> and make it more like xbox xcodes or whatever
02:12:53 <zid> something you could write a driver in would be perfect
02:13:27 <zid> so like a mmap 'opcode', a 'pci config whatever' opcode, etc would be super cool for an OS imo
02:13:44 <zid> drivers don't tend to be performance critical, and a neat meta-langauge for them would be interesting to see
02:14:14 <Mutabah> regreg: Maybe look at cranelift?
02:14:25 <Mutabah> Depends on what you want to do with it
02:16:17 <regreg> Mutabah, i want to write my kernel in portable assembler
02:16:35 <regreg> Mutabah, because i want to write my own toolchain but C is too complicated
02:16:51 <zid> Like I said, so do that then, just don't pretend it's because it'll be fast or useful or whatever
02:16:55 <zid> and you should start 10 years ago already
02:17:12 <zid> instead of talking about it, because you won't know your requirements until you've written half of it then deleted it and started over
02:17:18 <regreg> zid don't make fun of my battlestation! you silly! :)
02:17:36 <bcos> C isn't that complicated
02:17:44 <Mutabah> zid is right - you won't know what you want until you've tried and failed a few times
02:17:58 <zid> I was right over in #asm too he just doesn't want to hear it :P
02:18:22 <regreg> i'm seeking fast incremental compilation
02:18:31 <zid> C
02:18:32 <regreg> for run-time code generation
02:18:38 <Mutabah> If you don't want to do C, then make your own simple language instead - don't go for "portable assembly", do something structured
02:18:41 <zid> hmm, maybe not C then
02:19:11 <regreg> zid basically i want to specialize syscalls with common values
02:19:23 * Mutabah wanders off for bed
02:19:30 <zid> Then do what I said 5 minutes ago
02:20:04 <regreg> https://academiccommons.columbia.edu/doi/10.7916/D80R9XHX
02:20:39 <regreg> An Overview of the Synthesis Operating System
02:20:39 <regreg> Pu, Calton; Massalin, Henry
02:20:57 <regreg> Using hardware and software emulating a SUN-3/160 running SUN OS, the current version of Synthesis kernel achieves several times to several dozen times speed up for UNIX kernel calls.
02:21:52 <aalm> sounds good, now implement and profit! :]
02:21:54 <aalm> .theo
02:21:54 <glenda> What you are trying to do is insane, wrong, and entirely unsupported.
02:22:12 <zid> hah nice theo
02:22:18 <bcos> regreg: When, and how?
02:23:49 <regreg> bcos i'm looking for a fast assembly-like bytecode, somewhat similar to llvm IR
02:23:55 <zid> nooo kissedagirl is down
02:24:28 <bcos> regreg: Have you considered LLVM IR?
02:24:28 <zid> https://web.archive.org/web/20160311055446/https://cryptnet.net/mirrors/texts/kissedagirl.html
02:26:14 <regreg> bcos, yes but i generally dislike it, also it looks like it's slow to do recompilation at runtime
02:26:44 <bcos> Oh my... that Synthesis paper is so obsolete I'm surprised Noah had room on the arc for it
02:27:11 <zid> bcos: Read kissedagirl after it's funny
02:27:51 <bcos> regreg: You're forced to make a compromise between "compile fast (very little optimisation), poor execution speed" and "compile slow (extreme optimisation), fast execution speed"
02:28:13 <bcos> LLVM is mostly intended for the latter (fast execution speed) not the former (fast compile/optimise)
02:28:59 <bcos> ..but that doesn't mean you can't write a different back-end (or more likely, skip a lot of the optimisers that come with it)
02:32:12 <regreg> what the bytecode should look like?
02:32:41 <regreg> i propoe using a real assembly language as bytecode, this way at least one arch runs at full speed with no penalty
02:32:46 <regreg> *propose
02:33:18 <zid> exacting semantic meaning from assembly and translating it is an unsolved CS problem
02:33:22 <bcos> Ideally, it should probably look like random numbers (any recognisable patterns probably indicate lost opportunities for code density)
02:33:30 <zid> extracting*
02:34:05 <regreg> zid i'm not trying to extract semantic meaning from the bytecode, just a basic peephole optimizer
02:34:31 <zid> regreg: Okay then, I'll write you some amd64 code using 16 registers, you can recompile it to 6502?
02:35:00 <aalm> zid xD
02:35:13 * bcos would avoid "stack machine" and "register machine" for the byte-code - they both kind of suck for conversion to machine code
02:35:16 <zid> without the semantic meaning all y ou can do is emulate it
02:35:42 <regreg> zid yes emulate it
02:35:55 <regreg> except for that one architecture where it would run at native speed
02:36:15 <zid> meanwhile over in the real world, I could write it in C ten times faster, support every architecture, and almost certainly beat your 'one architecture's speed
02:36:25 <zid> and be able to debug it
02:36:41 <regreg> how do you debug it if you have no debugger?
02:36:43 <zid> If you just want to write some assembly or whatever you should just go do it, instead of making excuses
02:37:07 * bcos isn't sure that (e.g.) "64-bit 80x86 machine code" has been excluded as a potential "portable byte-code emulated on other types of CPUs"
02:37:19 <regreg> assembly toolchain is much simpler to implement but it's not generating portable code (except the emulation part)
02:38:14 <regreg> bcos yes i plan to use x86 (or a sane subset of it) as portable bytecode
02:42:32 <Bitweasil> zid, "drivers don't tend to be performance critical"? They're... usually exactly that.
02:42:56 <zid> Bitweasil: Got a counterexample of where a bytecode wouldn't work?
02:43:42 <Bitweasil> I'm still struggling to figure out exactly what you want to do.
02:44:00 <Bitweasil> I mean, you /can/ do just about anything, but things are typically done the way they are for tolerably good reasons.
02:44:35 <zid> sounds like a no? :P
02:45:21 <zid> If you can measure the cputime being spent inside your drivers I think you've done something wrong, making them 3x slower shouldn't be measurable
02:46:06 <Bitweasil> Are you referring to the... oh, can never remember which half of the interrupt handling is called which. The interrupt service routine for hardware, or the scheduled workqueue for it?
02:46:21 <Bitweasil> I've certainly been driver-limited in performance in the past on hardware.
02:46:24 <zid> neither
02:46:39 <Bitweasil> When your loopback device encryption is CPU bound, that's sure measurable and a limit on performance.
02:47:07 <regreg> hmm
02:47:09 <Bitweasil> Drivers are so insanely hardware specific (at least at the level I play) that cross-platform drivers are kind of a concept that doesn't meaningfully exist.
02:47:30 <Bitweasil> And if they're not terribly platform specific, then C works fine.
02:47:55 <regreg> Bitweasil, the problem with C is that the toolchain is quite large to write
02:48:08 <regreg> Bitweasil, i want to write a portable macro assembler
02:48:21 <regreg> that is a smaller project than a C compiler
02:48:24 <zid> Bitweasil: Almost all drivers are cross-platform
02:48:39 <zid> Unless you've been writing them in assembly or something
02:49:39 <bcos> regreg: I get the impression that you want the performance that can only be gained through extremely large and complex code while also wanting to avoid anything that isn't trivial to implement; which suggests maybe you're guaranteed to be unhappy with anything you attempt
02:50:17 <Bitweasil> If you're in kernel space, the C toolchain isn't that large.
02:50:50 <Bitweasil> At least not at a basic level.
02:51:01 <zid> bcos: I get the impression he wants what he wants, and he's just going to argue against reality
02:51:21 <Bitweasil> https://en.wikipedia.org/wiki/Tiny_C_Compiler
02:51:35 <jjuran> Less yakking, more hacking :-)
02:51:37 <Bitweasil> Yes, the modern LLVM chains and such are large for optimization, but they're not required.
02:51:46 <regreg> i want a portable assembler, something like intermediate llvm code or cranelift
02:52:02 <bauen1> the smallest (most likely) selfhosting c toolchain in lines of code would be binutils + tinycc (tinycc can compile binutils, you might have to modify make, bash or some of the gnuutils and port newlib, musl or your own c library)
02:52:08 <bauen1> your kernel should also compile using tcc
02:52:13 <bauen1> and then you're self hosting
02:52:39 <regreg> bauen1, i don't want just self hosting, i also want to write my own toolchain
02:52:43 <bauen1> well
02:53:22 <bauen1> i think you're better off contributing to tcc, but if you really want to assembly would be the point to start at
02:53:51 <regreg> i'm thinking about booting right into the kernel debugger
02:54:07 <regreg> you write bytecode and it gets executed instantly
02:54:15 <regreg> debugger-as-a-shell
02:57:26 <Bitweasil> Have you actually done much ring 0 work?
02:57:40 <Bitweasil> Nothing you're talking about is /impossible/, it's just... a bit loony.
02:57:55 <regreg> i have memory management and a basic sheduler
02:58:16 <regreg> i'm basically at the place where i must make decisions about the future architecture
03:05:21 <bauen1> i would suggest that you try to get somewhere near selfhosting using your current toolchain, since even that will be extremely challenging to get right (filesystem, syscalls), you can surely write your own toolchain, but it's really really challenging and requires a lot of dedication (i would guess it's somewhere between 20-100 times the amount it takes to get multi tasking usable)
03:05:37 <bauen1> i don't want to discurage you, but you it's a long rabbit hole
03:05:58 <bauen1> if you want to implement your own toolchain you should probably pick up some books / papers / pdfs about compiler design
03:06:17 <bauen1> and compilers are a really interesting subject
03:07:21 <bauen1> writing a one-pass simple assembler is quite easy, but it quickly gets complex the more features you need (second pass, e.g. resolve labels)
03:08:31 <bauen1> and if you want your toolchain to be self hosting you would have to write your assembler in another language once and then reimplement it in it's own language
03:09:18 <bauen1> a lot of people also suggest writing a formal proof for your compiler, but if you don't know what that means (or don't know functional programming) you can skip it
03:09:28 <bauen1> formal proof just makes mathematics happy
03:10:01 <regreg> i've written formally verified code before
03:10:11 <regreg> in C using the frama-c framework
03:10:19 <regreg> but only tiny samples
03:10:22 <regreg> nothing large scale
03:10:53 <regreg> i plan on having formally verified assembly code
03:11:10 <regreg> at least for specific bug classes
03:11:31 <regreg> i don't trust the verifier anyway :) (see spectre and meltdown)
03:11:51 <bauen1> i don't think you can reasonably apply formal verification to assembly or atleast i haven't seen it tried
03:12:10 <regreg> well if you can apply it to C you can reasonably expect to apply it to assembly too
03:12:16 <bauen1> formal verification just helps you eliminate a lot of bugs and also allows you to ensure that the compiler will always generate the same output for the same input
03:14:06 <regreg> formal verification is kind of useless if your formal spec is not solid
03:23:40 <Bitweasil> Formal verification is cute when the hardware doesn't behave. :p
03:30:54 <regreg> Bitweasil, :)
03:34:37 <Bitweasil> I can formally verify OS isolation all I want, but that's typically based purely on the architectural model of the processor, and... well, see the last year.
03:43:11 <jjuran> You may as well assume a spherical core of uniform density.
03:45:56 <zid> jjuran: with no gravity
03:46:03 <zid> or air resistance
05:09:24 <johnjay> jjuran: people assume that a lot in phys10
05:09:26 <johnjay> 1
05:10:17 <johnjay> isn't Z the formal verification language?
05:12:05 <zid> johnjay: that was the joke he was making, yes
05:12:35 <zid> Doing formal verification and ignoring the realities of manufacture etc, is like doing physics where you assume everything is a sphere
05:12:51 <johnjay> i guess that's why i'm not a physicist then
05:13:17 * johnjay wonders if you could make a formal assembly language and then formally verify that
05:14:10 <regreg> johnjay, it's definitively possible, frama-c does it for C
05:14:19 <regreg> spark does it for ada afaik
05:14:56 <johnjay> ah ok. having a different one for each language eh
05:15:45 <Bitweasil> johnjay, you could, and what good would it do you, when you cannot know or reason about the microarchitecture?
05:16:00 <Bitweasil> I can formally verify a kernel that says that any process cannot read another process's memory without permissions.
05:16:30 <Bitweasil> Yet, on a modern x86 CPU, running that formally verified kernel, I'm pretty sure I can read another process's memory, depending on how things are set up with the mappings.
05:16:58 <Bitweasil> If it's using a single page table (which provides isolation, per the architecture) and the kernel has a physical memory map (as most 64-bit OSes do), then that verification is worthless, because I can just Meltdown the memory out.
05:17:18 <Bitweasil> (or potentially L1TF, or... whatever, pick the one that works best).
05:18:20 <bauen1> you can also formally verify hardware ...
05:20:41 <Bitweasil> bauen1, and Intel apparently doesn't bother to do it beyond architectural behavior.
05:21:09 <Bitweasil> If they even do that. See... oh, what was it, Skylake's ?H registers, tight loops, hyperthreading, uop cache, and catastrophic corruption of architectural state...
05:27:51 <regreg> Bitweasil, what about catastrophic corruption? i wasn't aware of that
05:29:20 <johnjay> Bitweasil: i didn't know about the meltdown attack
05:29:24 <johnjay> it sounds like a timing based thing
05:30:50 <Bitweasil> Go read up on it. It's one of the early speculative execution gadgets that lets you read whatever you want from a process.
05:30:58 <Bitweasil> regreg, just a sec looking.
05:31:14 <Bitweasil> https://arstechnica.com/information-technology/2017/06/skylake-kaby-lake-chips-have-a-crash-bug-with-hyperthreading-enabled/
05:31:19 <Bitweasil> http://gallium.inria.fr/blog/intel-skylake-bug/
05:39:33 <regreg> thanks for the links
05:46:15 <Bitweasil> np, it was pretty bad. :/
05:46:35 <Bitweasil> The uarch stuff, OK, I get it. But the Skylake bug was a failure in the architectural state, which is new ground for Intel.
06:39:34 <mquy90_> I read this instruction https://wiki.osdev.org/GRUB#HDD_Image_Instructions_for_OS_X_users, but I am not sure about mbr.img, fs.img
06:40:35 <mquy90_> I need to write those, how it relates to my kernel which is built from grub https://github.com/MQuy/mos/blob/master/src/build.sh#L15
06:56:40 <doug16k> mquy90_, fs is probably a partition image. mbr goes at the beginning of the disk image, fs goes wherever partition table says it is in disk image
06:57:56 <doug16k> if you want to automate that fdisk stuff into your build, you can use variations made for scripting: sfdisk and sgdisk
06:58:24 <mquy90_> can I reuse what I already had, for example kernel, grub
06:58:40 <doug16k> so you don't have to answer 20 questions in that fdisk when you make a clean disk image
06:58:42 <mquy90_> or I neeed to write my bootloader
07:12:26 <doug16k> you want to make a disk image to debug right? in your build?
07:14:00 <doug16k> you can put grub in the disk image and put your kernel in the partition file, is that what you mean?
07:14:07 <mquy90_> I want to include other files, for example, text, image files to read
07:14:21 <doug16k> what filesystem?
07:14:35 <mquy90_> anything, fat32 or ext2
07:15:05 <doug16k> fat32 you can use mcopy -i partiotion.img source dest
07:15:28 <doug16k> dest would look like ::/ for the root IIRC
07:16:23 <mquy90_> does it affect the way grub load it? sorry I am a bit confused :)
07:17:04 <mquy90_> currently, I use something like this i386-elf-grub-mkrescue -o mos.iso isodir
07:17:10 <doug16k> in my build I truncate/mkfs.vfat/mcopy to make the partition, then I truncate/sgdisk/dd to create the disk image, partition it, copy the mbr/gpt and partition into it
07:19:28 <mquy90_> from what I understand, in this case, we write our own bootloader, right?
07:21:00 <mquy90_> so we can make our bootloader as the bootstrap or 1 sector to initiate the process (including file system)
07:21:03 <mquy90_> I might be wrong
07:21:37 <adu> are you talking about the MBR or VBR?
07:22:28 <mquy90_> mbr, I think
07:24:38 <adu> "TODO: Explain this."
07:24:57 <adu> lol sounds like something I would write
07:43:02 <bauen1> i'm pretty sure we have an article about CHS <-> LBA conversion already somewhere that could be linked there
08:10:34 <Vercas> This is incredible. The bug that made me add ubsan shit has nothing to do with undefined behaviour.
08:12:03 <Vercas> Guess I'm just that unlucky.
09:54:29 <amazoniantoad> I'm trying to create a bootloader using this tutorial: http://www.brokenthorn.com/Resources/OSDev3.html and I'm at the point where I need to create a virtual floppy drive but I can't seem to use the program provided or even use powershell (I'm running windows)
09:54:45 <amazoniantoad> Does anyone have any suggestions on how I could create a virtual floppy image
10:43:59 <Bitweasil> I'd probably use Linux loopback devices, but I haven't messed with floppies in a LONG time. There's a way to set up an ISO to boot in a legacy floppy mode, might look at that.
11:19:41 <t58> I think VFD is abandoned, you are better off using something like cygwin to get a hold of dd to make your floppy image and just use VirtualBox to run it IMO
11:25:48 <bcos> amazoniantoad: Eventually you're going to want to write a nice utility does things like modify values in the BPB (so you can generate 1440 KiB, 2880 KiB, 1200 KiB, ... images easily) and maybe automatically decides if there's space for "redundant duplicates" of files and maybe formats some kind of filesystem (although for floppy I don't see the point - not enough space for boot code+kernel+initrd means no space for normal files anyway). This kind of thing
11:25:50 <bcos> usually takes less than 1 day to write.
11:28:22 <bcos> Of course once you've done that you use your utility to generate a floppy image and configure a virtual machine to use the image/file as a floppy disk, and..
11:33:07 <Kazinsal> At which point you sit down and realize you've spent N hours on ways to format floppy disk images, and then comes the liquor and introspection...
11:33:56 <amazoniantoad> So here is my goal. I want to write a kernel and then eventually learn enough about kernels to modify an existing, more complicated, kernel like Ubuntu uses. My question to you all is where should I start?
11:35:01 <amazoniantoad> I want to try to integrate WINE into the kernel. it's been done before, i know. But I want to see how hard doing something like that really is
11:52:08 <dormito> amazoniantoad: its probably easier to examin the portions of the linux kernel that you want to change than it is to learn comprehensive kernel dev. After all that sounds like a relatively minor modification to the kernel
11:53:30 <dormito> also: the way wine works now is much more similar to how windows apps natively work than it would be if it were initrated into linux (unless you are simply saying you would like to write a PE/COFF loader in the kernel instead of userspace)
11:53:46 <dormito> *intigrated
11:55:55 <amazoniantoad> thanks for the info