channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev&y=18&m=7&d=18

Wednesday, 18 July 2018

00:00:00 --- log: started osdev/18.07.18
00:02:03 --- quit: sysfault (Remote host closed the connection)
00:02:33 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
00:02:41 --- join: spare (~user@unaffiliated/spareproject) joined #osdev
00:06:28 --- join: hmmmmmm (~sdfgsf@pool-72-79-160-70.sctnpa.east.verizon.net) joined #osdev
00:08:53 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
00:09:35 --- quit: hmmmmm (Ping timeout: 256 seconds)
00:12:07 <klys> doug16k, ever run that nm -s command?
00:18:16 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
00:19:30 --- quit: tadni (Remote host closed the connection)
00:25:30 --- quit: [Chorus] (Quit: Du coeur à l'outrage.)
00:27:59 <klange> oh hey you *can* give qemu 32 megs of video ram
00:28:32 --- join: asymptotically (~asymptoti@gateway/tor-sasl/asymptotically) joined #osdev
00:31:27 * klange resurrects qemu display harness
00:31:36 <klange> BEHOLD MY 4K GLORY https://i.imgur.com/DKUACRh.jpg
00:32:55 <klange> This is a lovely little hack since QEMU's interfaces don't support communicating window size back and forth. Supposeldy one of the more advanced backends has support for this, but the hardware interface is really annoying.
00:33:50 <mischief> 2big4me
00:33:58 <klange> A Python script starts QEMU with serial over TCP, finds QEMU's X window, and then reports changes in its size over the serial interface to an app in the guest that then sets display resolution to match.
00:34:18 <klange> But then qemu gets blurry, so it also sends a ping back to the Python script which then sends ctrl-alt-u to tell qemu to unblurrify
00:34:21 --- join: [Obsidian] (~while@unaffiliated/awaxx/x-0928682) joined #osdev
00:34:41 <klange> By default QEMU has 16mb of vram, not enough for 4k
00:34:58 <klange> And my handling for that was... well, to crash (I think just the compositor).
00:40:18 <klys> doug16k, turns out what I needed was a gcc downgrade. gcc-4.9 works. I now have the Csm16.bin
00:42:55 <klys> and of course we need to recompile ovmf in order to use it...
00:52:26 <bcos_> klange: Text in top bar ("applications", "local@base:~", ...) looks too small, like resolution independence is broken somewhere.. ;-)
00:52:53 <klange> bcos_: or, you know, non-existent
00:53:35 <klange> I need to regenerate my assets at higher resolutions first...
00:53:45 <klange> Some of them are easier to do than others.
00:54:16 <klange> I think I could do the decorations, though were SVGs to begin with. The buttons in them I drew pixel-by-pixel, though, so... might need to work around that.
00:54:25 <klange> Top panel I think was also done in GIMP, so...
00:56:08 <klange> Hm, but where are my SVG sources for the decorations...
00:56:24 <klange> cursors and icons are here: https://github.com/klange/toaru-art
00:56:25 <bslsk05> ​klange/toaru-art - SVG sources for some ToaruOS assets (0 forks/1 watchers)
01:01:26 --- join: nortega (~nortega@gateway/tor-sasl/deathsbreed) joined #osdev
01:04:47 --- quit: zeroSignal (Changing host)
01:04:48 --- join: zeroSignal (~ovi@fsf/member/zeroSignal) joined #osdev
01:13:55 --- quit: nwm (Ping timeout: 264 seconds)
01:14:11 --- quit: sixand (Ping timeout: 256 seconds)
01:17:15 --- join: Asu (~sdelang@7.43.136.77.rev.sfr.net) joined #osdev
01:18:36 <klange> ah, found them
01:19:44 --- quit: Asu (Remote host closed the connection)
01:20:08 --- join: Asu (~sdelang@7.43.136.77.rev.sfr.net) joined #osdev
01:22:19 --- quit: MarchHare (Ping timeout: 264 seconds)
01:27:00 --- quit: zeitue (Quit: Leaving)
01:27:20 <klange> I'll build some @2 assets and try to hack something together.
01:27:23 <klange> I imagine it will be slow.
01:28:04 <izabera> you're so hardcore
01:30:41 --- quit: flacks (Ping timeout: 260 seconds)
01:31:41 --- join: flacks (flacks@184.91.69.131) joined #osdev
01:33:09 --- join: Jari-- (~vai@mobile-access-6df015-6.dhcp.inet.fi) joined #osdev
01:33:11 <bcos_> Sadly, Qemu doesn't support EDID or multi-monitor; so (without real hardware) it's hard to test the initial steps (e.g. figuring out where the physical screens are within a larger virtual desktop) needed
01:33:56 <klange> VirtualBox supports multiple monitors.
01:34:59 <klange> You have to signal through its guest-host interface that you support it, and I'm not sure how the second framebuffer is set up.
01:35:31 <doug16k> qemu supports notification and resizing with the window with virtio-gpu
01:35:46 <doug16k> virtio-gpu supports multiple monitors too
01:35:48 <klange> Only with certain clients.
01:36:23 <doug16k> up to 16
01:36:52 --- quit: celadon (Quit: ZNC 1.7.0+deb1+b1 - https://znc.in)
01:37:13 --- join: celadon (~celadon@66.157-14-84.ripe.coltfrance.com) joined #osdev
01:40:21 * bcos_ looks forward to "16 * 4K" screenshots! :-)
01:42:11 <klange> that's like 500MiB just in framebuffer space
01:44:22 <bcos_> Depends on video mode - maybe only 128 MiB if you use "256-colour"
01:44:34 <klange> > 4K
01:44:36 <klange> > 256 color
01:44:40 <klange> lol
01:50:26 --- quit: hmmmmmm (Remote host closed the connection)
01:55:55 <Jari--> hi all
02:01:10 <klys> wb.
02:02:44 <klys> halftoning 256-bot color on 4k would produce a new image...
02:02:54 <klys> bit*
02:03:23 --- quit: spare (Quit: leaving)
02:09:39 --- join: sortie (~sortie@static-5-186-55-44.ip.fibianet.dk) joined #osdev
02:13:43 --- quit: hunterlabs (Ping timeout: 240 seconds)
02:16:59 --- join: hunterlabs (Elite20801@gateway/shell/elitebnc/x-vlbfuajuslacmjwa) joined #osdev
02:19:54 --- quit: bemeurer (Ping timeout: 276 seconds)
02:20:22 --- quit: Jari-- (Ping timeout: 256 seconds)
02:23:00 --- quit: mavhq (Read error: Connection reset by peer)
02:24:17 --- join: mavhq (~quassel@cpc77319-basf12-2-0-cust433.12-3.cable.virginm.net) joined #osdev
02:31:12 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
02:33:07 --- join: zeus1 (~zeus@154.225.146.190) joined #osdev
02:36:09 --- quit: bemeurer (Ping timeout: 276 seconds)
02:43:22 --- quit: nortega (Quit: Vivu lante, vivu feliĉe!)
02:45:27 --- quit: zeus1 (Ping timeout: 248 seconds)
02:47:40 <S_Gautam> I've no problems with 256 bit color
02:47:54 <S_Gautam> I wish more games had that
02:48:21 <S_Gautam> I don't need 4 billion colours to differentiate between a tree and some enemy
02:50:30 --- quit: xerpi (Quit: Leaving)
02:53:19 <bcos_> S_Gautam: What if the enemy is properly camouflaged (and looks like a tree)?
02:53:53 <S_Gautam> Just use a rocket launcher and nuke the tree
02:54:00 <S_Gautam> That's a good question though
02:54:27 <S_Gautam> I haven't played any competitive games in that resolution yet. Time to get a 800x600x256 monitor?
02:54:46 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
02:55:12 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
02:56:54 --- join: zeus1 (~zeus@81.199.19.241) joined #osdev
02:57:23 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
02:59:13 --- quit: froggey (Ping timeout: 240 seconds)
02:59:40 <FreeFull_> 256 colour = 8 bit
02:59:42 <FreeFull_> Not 256 bit =P
03:01:03 --- join: froggey (~froggey@unaffiliated/froggey) joined #osdev
03:02:03 --- quit: sysfault (Remote host closed the connection)
03:02:32 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
03:02:36 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
03:02:51 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
03:06:03 --- quit: sysfault (Remote host closed the connection)
03:06:14 --- quit: Belxjander (Client Quit)
03:06:33 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
03:06:35 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
03:07:03 --- quit: sysfault (Max SendQ exceeded)
03:07:59 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
03:08:08 --- quit: flacks (Quit: Leaving)
03:08:57 --- nick: FreeFull_ -> FreeFull
03:12:24 --- join: flacks (flacks@184.91.69.131) joined #osdev
03:12:26 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
03:12:42 <S_Gautam> Yeah
03:12:46 <S_Gautam> That's what I mean
03:12:58 <S_Gautam> Good old VGA mode 13h style
03:13:20 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
03:15:22 <klange> "256 bit color" would be pretty insane - we're still working on getting 16 bits-per-channel (48-bit color, or 64 if you really like those high-resolution alpha channels)
03:16:34 <klys> freefull, ah, you have corrected me.
03:17:20 <klange> 256 bit color, with alpha channel, would mean 64 bits per channel, and that's a LOT of color.
03:17:30 <FreeFull> Yeah
03:17:39 <klys> smellovision
03:17:44 <FreeFull> I don't know any device that would be actually capable of displaying that many
03:18:12 <klange> There are 16bpc "half-precision" floating point color specs.
03:18:22 <klange> So 256 would just be double precision :D
03:18:59 <klange> great for that USMHDR (Ultra-Super-Mega-High Dynamic Range)
03:19:15 <FreeFull> Thinking about it
03:19:54 <FreeFull> cairo_set_source_rgba () takes four doubles
03:19:57 <FreeFull> So it is 256-bit
03:20:28 <FreeFull> I don't even know why it takes double precision rather than single precision
03:20:35 <klange> I'm not sure it has any texture modes that use it.
03:21:41 <FreeFull> The most it does is 10 bits per channel
03:21:55 <FreeFull> Without alpha
03:21:58 <FreeFull> Or 8bpp with alpha
03:22:31 <FreeFull> It doesn't seem to have a floating point surface format
03:23:43 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
03:24:13 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
03:24:47 <FreeFull> Sorry, I meant 8 bits per channel, not per pixel
03:25:27 --- quit: Ranhir (Ping timeout: 248 seconds)
03:30:10 <m712> what do you guys think about managed languages for systems programming?
03:30:13 --- quit: bemeurer (Ping timeout: 240 seconds)
03:30:22 <m712> my idea was to add a VM in the scheduler that runs the bytecode for the operating system's programming language
03:30:46 --- quit: Belxjander (Quit: AmigaOSv4.1.6+//PowerPC native)
03:31:29 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
03:32:01 <bcos_> m712: What kind of VM - insanely slow, or insanely insecure?
03:32:32 <m712> bcos_: hah
03:32:40 --- join: ljc (~ljc@unaffiliated/ljc) joined #osdev
03:33:02 <m712> i won't don a defeatist attitude, though. i think that it is doable
03:33:10 <klys> m712, all attempts at inducing programming are ultimately reduced to scripts, and so instead of inventing an intelligible syntax, you should focus on debuggable bytecode.
03:33:40 <m712> klys: yeah, that would be one of my man focuses
03:33:52 <m712> unix and all of its descendants really lack proper debugging
03:34:15 <mischief> define proper
03:34:46 <m712> mischief: for one, i think the operating system should pop you into a debugger (if you want it to) when a program fails due to a memory referencing error
03:34:52 <m712> or some other fatal error
03:35:06 <klys> proper debugging should work like a calculator from layer 7 to layer 1.
03:35:16 <m712> klys: ?
03:35:21 <klys> :)
03:35:50 <klys> I think of writing a debugger occasionally
03:35:54 <m712> mischief: symbol stripping is a dumb thing in my opinion
03:36:04 <bcos_> m712: It's not doable, "doable" is a myth. Either it's extremely simple ("while(running) { fetch instruction; execute instruction }") and insanely slow; or it's incredibly complex (a high performance JIT) and impossible to guess how many thousands of security problems there are
03:36:58 <m712> bcos_: i don't think the current option (just running the machine code) is any better, though
03:37:02 <bcos_> Of course there's plenty of middle ground - e.g. "slow and insecure"
03:37:14 <m712> bcos_: fast and relatively secure
03:37:31 <m712> being pessimistic won't get us anywhere
03:37:35 <bcos_> Just running machine code (with hardware isolation/protection) is faster and relatively more secure
03:38:27 <m712> bcos_: but it comes at the disadvantage of no memory and type safety, hard debugging and the need to compile your code
03:38:38 --- join: navidr (uid112413@gateway/web/irccloud.com/x-weriogsxjegoxynd) joined #osdev
03:38:55 <bcos_> IMHO, "ideal" is compiling byte-code to machine code ahead of time (and doing as much sanity checking and security checking then); and then running machine code (with hardware protection)
03:39:13 <m712> not to mention compiled binaries are completely unportable to another architecture and *maybe* portable to another system
03:39:33 <m712> bcos_: could be a better idea than running bytecode raw or JIT
03:39:51 --- quit: zeus1 (Ping timeout: 276 seconds)
03:40:21 * bcos_ thinks this was one of the ideas behind .NET - pre-compile (and cache the resulting machine code) to get performance + portability
03:41:56 <bcos_> (ironically, it's even faster and more portable than the traditional "download and install machine code" approach, because compiler can optimise for a specific CPU and not just "generic lowest common denominator")
03:42:05 <m712> a good idea killed by microsoft's hostile anti-open source and anti-anything-not-windows back then -- why am i surprised
03:42:17 <m712> not
03:42:27 <bcos_> I think LLVM can be used like that too..
03:43:20 <m712> bcos_: it would be nice to be able to embed LLVM into a scheduler so you can run your code on it
03:43:51 <m712> being able to run source code is a big plus imo
03:43:52 <bcos_> WIth scheduler in user-space?
03:44:06 <m712> bcos_: can you even put a scheduler in user-space?
03:44:28 <bcos_> Yes, but with severe restrictions (e.g. cooperative scheduling only)
03:45:04 --- quit: sysfault (Remote host closed the connection)
03:45:38 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
03:45:57 <m712> that wouldn't be very viable then
03:46:08 --- quit: sysfault (Max SendQ exceeded)
03:46:43 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
03:47:09 --- join: m_t (~m_t@p5DDA257A.dip0.t-ipconnect.de) joined #osdev
03:47:09 <klys> putting a scheduler in userspace should be a requirement
03:49:00 <m712> klys: back to DOS?
03:49:18 <klys> m712, I've been coming from that angle for a while now
03:50:04 <Lowl3v3l> i am not certain purely cooperative scheduling is the best way though, but maybe thats one of those things that could be useful to abstract
03:50:56 <m712> i think preemptive is the way to go
03:51:02 <m712> but to each their own
03:51:23 <klys> hmm wonder what happened to mrvn
03:54:18 --- join: Shikadi (~Shikadi@cpe-98-10-34-205.rochester.res.rr.com) joined #osdev
03:57:52 --- join: zeus1 (~zeus@160.119.149.218) joined #osdev
03:59:09 --- quit: sysfault (Remote host closed the connection)
03:59:39 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
04:02:19 --- join: Kimundi__ (~Kimundi@i577A952E.versanet.de) joined #osdev
04:15:49 --- quit: immibis (Ping timeout: 240 seconds)
04:19:01 --- join: Kimundi_ (~Kimundi@i577A93F7.versanet.de) joined #osdev
04:19:44 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
04:22:14 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
04:22:31 --- quit: Kimundi__ (Ping timeout: 248 seconds)
04:24:27 --- join: CrystalMath (~coderain@reactos/developer/theflash) joined #osdev
04:25:38 --- quit: zeus1 (Ping timeout: 265 seconds)
04:37:12 --- quit: kasumi-owari (Ping timeout: 268 seconds)
04:37:53 --- join: kasumi-owari (~kasumi-ow@ftth-213-233-237-007.solcon.nl) joined #osdev
04:46:40 --- quit: ybyourmom (Quit: Lost terminal)
04:48:04 --- quit: sysfault (Remote host closed the connection)
04:48:37 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
04:49:14 --- quit: sysfault (Max SendQ exceeded)
04:50:06 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
04:50:36 --- quit: sysfault (Max SendQ exceeded)
04:51:09 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
04:52:55 --- quit: Kimundi_ (Ping timeout: 256 seconds)
04:53:41 --- join: nortega (~nortega@gateway/tor-sasl/deathsbreed) joined #osdev
04:53:43 --- quit: bemeurer (Ping timeout: 240 seconds)
04:57:18 --- join: zeus1 (~zeus@160.119.149.218) joined #osdev
05:06:06 --- quit: sysfault (Remote host closed the connection)
05:06:32 <yrlf> Does anyone know if it's possible to create an ELF containing only symbols with objcopy?
05:06:36 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
05:07:40 <yrlf> It's possible with an asm file (empty sections with just symbol assignments and .space), but I haven't found a way to do it with objcopy yet
05:08:05 <Lowl3v3l> yrlf: why do you want to use objcopy specifically?
05:08:30 <yrlf> I already need objcopy to prefix the symbol and section names
05:08:40 <yrlf> If objcopy can do that I could do it in one command
05:09:46 <yrlf> In essence I want to create prefixed aliases for all symbols in an elf that refer to the LMA of the symbol, so the early kernel code can reference it instead of erroneously looking up stuff at the VMA
05:09:56 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
05:11:04 --- join: bauen1 (~bauen1@x59cc8223.dyn.telefonica.de) joined #osdev
05:12:19 <yrlf> My current Idea is to compile 32bit startup code with -m32, copy that into a 64bit elf with objcopy so it can link with the rest of the kernel, and then do some symbol, section and linkscript magic so I can say something like __PHYS_entry64 in the 32bit code to refer to the load address of entry64, which would otherwise be VMA'd to 0xffffffff800000
05:12:19 <yrlf> 00+some_offset
05:12:58 <yrlf> The load address would be just a little over the 1M mark, which 32bit code can very easily access
05:14:56 --- quit: bemeurer (Ping timeout: 265 seconds)
05:15:07 <Lowl3v3l> hum your way sounds complicated. i just do it the other way around : i build a 64 bit elf with some 32 bit startup code in assembly, then i objcopy to a 32 bit elf and its done
05:18:03 <yrlf> The thing is: I want to write the startup code in C
05:18:59 <Lowl3v3l> why?
05:19:26 <Lowl3v3l> at best you end up with a naked function with 5 dozen additional attributes containing inline assembly
05:19:49 <Lowl3v3l> and thats a function you never call ( it just gets called by the bootloader)
05:20:51 --- join: Kimundi_ (~Kimundi@dynip-129-217-122-042.wifi.tu-dortmund.de) joined #osdev
05:21:46 <yrlf> For me it's just one function with naked and noreturn, which just sets up a stack and then calls a regular c function, which just uses inline asm for stuff that doesn't exist in C (lgdt and the like)
05:26:36 <Lowl3v3l> so you go through all this just for a single mov and some gdt stuff? ;)
05:26:58 <Lowl3v3l> okay, some paging structures are needed as well, but you need to do those in inline asm as well
05:27:28 <yrlf> And the linker then thinks "ohh, entry()?, that function surely is at 0xffffffff80100000", which just doesn't fit in a 32bit relocation
05:28:20 <yrlf> It's to fix up a uni osdev project where the base thing has relied on an ugly hack to get this to work, the hack broke on GCC8
05:28:37 <yrlf> They really dislike asm fes
05:28:40 <yrlf> *files
05:28:52 --- quit: adam4813 (Quit: Connection closed for inactivity)
05:29:22 <Lowl3v3l> if they dislike assembly entry points they are idiots.
05:29:43 <yrlf> The linker then bails because the relocation is wrong, or truncates it, so the address is flat-out wrong (0x80100000)
05:30:02 <yrlf> They even just assume esp points to a useful stack after multiboot entry
05:30:45 <Lowl3v3l> you really should scrap this entry stuff and start anew
05:30:46 <yrlf> This thing has been used for teaching osdev for years. It's a base os we students have to extend with threading, swapping, and so on
05:31:03 <yrlf> I know
05:31:24 <Lowl3v3l> tell your professor some smart guy from the internet said it's all shit
05:31:28 <yrlf> For my private project I take whatever looks cleanest
05:34:29 <yrlf> The whole way the image is set up is strange. The 64bit part is understandibly linked to the upper 64bit address space. The problem is: the 32bit code is too. This logically blows up in all sorts of ways
05:35:01 <yrlf> The old havk involved prefixing the 32bit files with asm(".code32") and compiling them with -m32 -Wa,--64
05:35:49 <yrlf> This even blows up with the old compilers once you enable optimizations as it just reorders the file so code is interpreted as 64bit again
05:38:06 <yrlf> And then they had a macro that adds 0x80000000 to the address, but with lots of casts and splitting it into adding 0x7fffffff and 1, so somehow ld doesn't notice the address overflows and happily links it
05:42:23 <Lowl3v3l> wow, that sounds fucked even for "university teaching project"-standards xD
05:42:47 --- quit: SopaXorzTaker (Remote host closed the connection)
05:42:56 --- quit: light2yellow (Quit: light2yellow)
05:43:12 --- quit: ljc (Quit: ayy)
05:44:03 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
05:45:07 <yrlf> The whole code is on github, publicly. github.com/iaik/sweb
05:45:08 <bslsk05> ​IAIK/sweb - SWEB Educational OS (60 forks/42 watchers)
05:46:04 <yrlf> the code in question is at arch/x86/64/source/boot.32.C and arch/x86/64/CMakeLists.include
05:46:14 <Lowl3v3l> oh graz. finally someone from the right side of the pool :D
05:47:02 <yrlf> Except for the weird codebase, the osdev course is actually really good
05:47:43 <Lowl3v3l> i usually look at "types.h" first. usually you see everything wrong with the project in there^
05:48:36 <yrlf> one of the meltdown/spectre guys is doing the lecture for osdev
05:48:40 <Lowl3v3l> they only use their doxygen comments partially. And when they totally overdo it xD
05:48:42 <Lowl3v3l> thats funny
05:49:23 --- quit: bauen1 (Read error: Connection reset by peer)
05:51:49 <lava> doxygen doc is outdated...
05:52:01 <lava> i removed that over the last years
05:52:39 <Lowl3v3l> lava: i am not even sure wether i like those systems at all
05:52:58 <lava> comments are pure evil
05:53:01 <lava> they diverge from the code
05:53:02 <Lowl3v3l> i always just see 2 possibilities : total overkill with trivialities. Or undocumented.
05:53:06 <lava> good code does not have comments
05:53:29 <lava> i invested a lot of time in rewriting code to make it self explaining to get rid of comments
05:53:52 <bcos_> "zero comments" is purely retarded.
05:54:07 <Lowl3v3l> some ocmments are good or even necessary. but take a look at this example :
05:54:10 <Lowl3v3l> https://github.com/IAIK/sweb/blob/master/arch/x86/32/include/ArchMemory.h
05:54:12 <bslsk05> ​github.com: sweb/ArchMemory.h at master · IAIK/sweb · GitHub
05:54:25 <lava> yeah, haven't gotten to remove the comments there ;)
05:54:29 <lava> lots of other things to do
05:54:41 <Lowl3v3l> literally the only fucking thing that should have been documented there is not
05:54:42 <lava> i'd merge pull requests that clean up comments though
05:54:47 <Lowl3v3l> :D
05:54:55 <Lowl3v3l> lava: oh you wrote this stuff?
05:55:06 <yrlf> He's Daniel Gruss :)
05:55:09 <lava> not all of it by far not
05:55:14 <Lowl3v3l> yrlf: brought it up, thats why i picked this :P
05:55:26 <Lowl3v3l> in that case : shame on you :P
05:55:35 <lava> i joined the project in 2009 and started reshaping things in 2010
05:55:49 <lava> the comments there are from 2005
05:55:53 <lava> i'd like to get rid of them
05:56:24 --- join: vmlinuz (~vmlinuz@2804:431:f725:a4f:294:da3:f40d:c058) joined #osdev
05:56:24 --- quit: vmlinuz (Changing host)
05:56:24 --- join: vmlinuz (~vmlinuz@unaffiliated/vmlinuz) joined #osdev
05:56:29 <Lowl3v3l> like i suggested above, throw out the whole thing and start anew :D
05:56:42 <lava> i'm actually quite happy with most of it
05:56:48 --- quit: yrlf (Quit: The Lounge - https://thelounge.chat)
05:57:21 <lava> we use this in a second year undergraduate course and people extend this significantly
05:57:33 <lava> in one year a group ported doom3d to it
05:57:39 <Lowl3v3l> lava: yeah i just see so many bad practices in there
05:57:48 <lava> like?
05:58:06 <lava> I've had a lot of discussions here and also elsewhere about this
05:59:26 <lava> the goals of an educational OS differ from goals of non-educational OSes
06:01:08 <Lowl3v3l> parts of the comments, mixing c++ and c and then calling pure c++ headers ".h" without a c-filter, mixing #pragma once and include guards, i strongly advocate for assembly entry points, i dont understand the way you guys mix c and c++ ( e.g. in modern c++ you should just not use constant-macros) and your cmake stuff looks way to complicated on the first glance^^
06:01:20 <Lowl3v3l> as you see, just little things that irk me :D
06:01:55 <lava> yeah I didn't write the cmake stuff, I don't like that part either
06:02:02 <lava> luckily students don't have to touch it
06:02:07 <lava> so I'm fine with it
06:02:27 <lava> c++ headers without a c filter are potentially bugs that should be fixed
06:03:11 <lava> i strongly advocate against assembly, because I lose ~30% of the students there
06:03:36 <lava> it works quite well with C and C++, the students actually learn a lot
06:03:57 <lava> i mean we get excellent students out of the lecture, so we can't be completely wrong
06:04:01 <Lowl3v3l> you misunderstand me
06:04:10 <Lowl3v3l> i am not saying "do all in assembly".
06:04:31 <Lowl3v3l> but if you're gonna have 100 line functions with inline assembly, just taking assembly is clearer^^
06:05:34 <lava> i think in virtually all cases the C functions we have are clearer
06:06:13 --- join: yrlf (~yrlf@core.recodex.at) joined #osdev
06:06:26 <lava> and i've seen that reflected in the course evaluations over the past 10 years (the course is twice per year)
06:07:02 <lava> as we replaced assembly files with C equivalents, the students found the course easier and found that they had to invest less time
06:07:44 <Lowl3v3l> https://github.com/IAIK/sweb/blob/master/arch/x86/32/common/source/boot.cpp take a look at the actual function. This IS assembly. its just inside of quotes, lots of escaped newlines and asm(). those parts are what i mean^^
06:07:46 <bslsk05> ​github.com: sweb/boot.cpp at master · IAIK/sweb · GitHub
06:08:39 <lava> and still it's easier to read for someone who knows no assembly
06:08:48 <Lowl3v3l> and if i thought your project was completely crap i wouldn't offer criticism. I'd joke about you guys starting to work in redmonde ;)
06:08:51 <lava> because many things there are functon calls, print macros
06:08:57 <bcos_> lava: If you switch to QuickBASIC (and reduce the functionality - single tasking, etc) they might be able to invest almost no time at all!
06:08:59 <doug16k> those fragments of assembly should at least be separated into inline functions
06:09:37 <lava> doug16k: maybe, yeah
06:10:30 <lava> i don't like functions in header files though
06:10:43 <lava> i had bad experiences with students having troubles with them quite often
06:10:44 <Lowl3v3l> and i am not sure wether you dont need __attribute__((naked)) for the function, i am not sure wether the function prelude might make it creash without a proper stack
06:11:06 <lava> attribute naked did not exist on x86 until gcc 8 or 7?
06:11:19 <doug16k> I wrap each thing like that in a function -> https://github.com/doug65536/dgos/blob/master/kernel/arch/x86_64/cpu/control_regs.h
06:11:20 <yrlf> GCC8 instroduced it
06:11:21 <bslsk05> ​github.com: dgos/control_regs.h at master · doug65536/dgos · GitHub
06:11:28 <yrlf> *introduced
06:11:31 <lava> yrlf: yeah, that's quite recent
06:11:50 <lava> we didn't have time in the last 12 months due to rowhammer stuff and our discovery of meltdown+spectre...
06:11:52 <lava> was quite busy
06:12:13 <lava> Lowl3v3l: the function has a proper stack, control flow is transferred there from grub
06:12:15 <yrlf> Oh, and maybe sweb should switch to the boottime stack earlier, multiboot doesn't specify a usable stack at entry time
06:12:32 <yrlf> Grub does create one, but it's nonstandard
06:12:33 <Lowl3v3l> lava: does multiboot give you a proper stack though? afaik it didn't.
06:12:44 <lava> yrlf: I'm very fine with being non-standard
06:13:46 <lava> if I can educate students with that to be able to build awesome things (like the kaiser patch which evolved into kpti - that was written by a student who did the course in one year, was tutor for it the next year, and then he wrote the kaiser patch)
06:14:05 <yrlf> Oh, and the "please give me VESA graphics" part in the multiboot header seems to be placed wrong. I thing there should be 4 int32s of zeroes in between (see multiboot_header_t in multiboot.h)
06:14:05 --- quit: sysfault (Remote host closed the connection)
06:15:17 <doug16k> lava, did you get a rowhammer implementation to work on ryzen yet? I'd love to try it :)
06:15:27 <lava> now you're all using something that was initially written by a student who learned in that OS ;D#
06:15:53 <lava> doug16k: why wouldn't it work on ryzen?
06:15:55 <yrlf> I totally think that sweb is a great thing, it does what it should: educate students about OsDev
06:16:25 <yrlf> Isn't rowhammer mostly dependent on the DRAM used? What has ryzen got to do with that?
06:16:31 <lava> yrlf: exactly
06:16:50 <doug16k> lava, doesn't the implementation have to know exactly how addresses translate into rows, banks, and columns?
06:16:55 * bcos_ thinks that it's impossible to make an OS that is simultaneously simple enough to be useful for educational purposes and complex enough to be useful for educational purposes
06:17:16 <lava> bcos_: i think we're pretty much there ;)
06:17:33 <lava> doug16k: sure, but why would that be a problem?
06:17:42 <lava> doug16k: just run our drama tool and get the functions
06:18:11 <doug16k> well I've tried rowhammer implementation and it didn't work and (I think you?) told me it was set up for intel and I had to figure out what constants to use to make it work
06:18:32 --- join: Asu` (~sdelang@48.17.90.92.rev.sfr.net) joined #osdev
06:19:44 <bcos_> Hrm - Google/Andriod fined $5 billion for anti-trust violations..
06:20:08 <bcos_> Geist will be eating ramen noodles for a while now
06:22:07 <doug16k> lava, I tried. it failed. could that be because I happen to have memory that isn't vulnerable?
06:22:27 --- quit: Asu (Ping timeout: 256 seconds)
06:23:34 <doug16k> anyway, no pressure. if you feel like it, try it on a ryzen. feel free to tell me you told me so if you get it to work :)
06:23:59 --- join: palk_ (~palk@unaffiliated/palk) joined #osdev
06:24:14 <bcos_> doug16k: Didn't you say yours has ECC?
06:24:19 <doug16k> yes
06:24:24 <bcos_> ...
06:25:10 <bcos_> Get yourself some of the cheapest/crappiest RAM that you can\
06:27:13 --- quit: palk (Ping timeout: 240 seconds)
06:31:03 --- quit: iknosis (Quit: Ping timeout (120 seconds))
06:31:30 --- join: iknosis (~iknosis@185.183.104.140) joined #osdev
06:34:23 <lava> cheap doesn't help
06:35:26 <lava> it's often different modules that were bought together where one is vulnerable and one is not, although all identifiers on the dram board and the dram chips are identical
06:36:08 <lava> we use this tool for dram function reverse engineering: https://github.com/IAIK/drama/tree/master/re
06:36:09 <bslsk05> ​github.com: drama/re at master · IAIK/drama · GitHub
06:36:14 <lava> has also been used by many others by now
06:36:49 <lava> it works very well on ivy bridge and haswell laptops... anything else needs tweaks and playing a lot around with the parameters
06:37:07 <lava> this tool reverse engineers the dram addressing functions by performing timing attacks
06:37:30 <lava> (and timing attacks easily do not work identically on every system, that's why tweaking things is necessary)
06:39:00 <klange> i really need an alt+f2 prompt again...
06:41:25 --- join: heat (55f7cf84@sortix/contributor/heat) joined #osdev
06:42:24 <Prf_Jakob> lava: A stupid question, doesn't ECC mostly stop RowHammer attacks?
06:42:42 --- quit: Lowl3v3l (Remote host closed the connection)
06:42:54 <lava> mostly
06:43:08 <lava> i have seen reports about bit flips on ECC memory
06:43:26 <lava> some uncorrected, some halt the server (also not good)
06:43:47 <lava> but I've never seen any bit flips on ECC memory on a machine with my own eyes ;)
06:44:12 --- quit: heat (Client Quit)
06:44:30 <Prf_Jakob> lava: Ah okay, thanks.
06:44:42 <Prf_Jakob> So ECC memory in laptops/phones when?
06:45:16 <lava> iphones already have ECC memory
06:45:16 <lava> but
06:45:25 <klange> good ol' gsudo back in action means I think I just need to port the toast notification daemon and that'll be everything not-python from the core ToaruOS... https://i.imgur.com/wZ02pN3.png
06:45:29 <lava> apple at the same time reduced the refresh rate
06:45:46 --- join: heat (~heat@sortix/contributor/heat) joined #osdev
06:45:50 <lava> rowhammer is the result of pushing an optimization problem too far in one direction
06:46:00 <klange> And I'd rather port the Python toast daemon than dig up the old C toast daemon...
06:46:47 <lava> you have refresh interval --> increases power consumption + reduces performance; better chips --> much more expensive because you throw away more chips that do not pass the tests; ECC everywhere --> increases power consumption + reduces performance
06:48:02 <lava> so you want to find the optimum for those three which minimizes power consumption, maximizes performance, minimizes chip costs, while not showing any rowhammer bit flips
06:48:25 --- join: nj0rd (~nj0rd@mue-88-130-48-026.dsl.tropolys.de) joined #osdev
06:48:35 * bcos_ isn't sure about that - if you have ECC then you can reduce power consumption/increase performance and let ECC fix any errors caused by reducing power consumption/increasing performance
06:48:55 <Prf_Jakob> Oh didn't know that iPhone had ECC memory, cool. And thanks for the info/explination.
06:49:37 <lava> bcos_: that's what i said
06:49:42 <lava> apple does that
06:49:47 <lava> but still it's an optimization problem
06:50:03 <lava> if you introduce ECC and then reduce the refresh rate too far, you again have bit flips
06:50:19 <bcos_> Ah - OK
06:50:38 --- join: MarchHare (~Starwulf@mo-184-5-202-5.dhcp.embarqhsd.net) joined #osdev
06:50:49 --- quit: bemeurer (Ping timeout: 240 seconds)
06:51:48 --- quit: nj0rd_ (Ping timeout: 256 seconds)
06:52:17 --- quit: navidr (Quit: Connection closed for inactivity)
07:01:10 --- join: promach_ (~promach@bb219-74-174-136.singnet.com.sg) joined #osdev
07:02:43 --- quit: MarchHare (Ping timeout: 240 seconds)
07:08:31 --- quit: Kimundi_ (Ping timeout: 265 seconds)
07:12:14 --- join: Kimundi_ (~Kimundi@dhl1116.cs.uni-dortmund.de) joined #osdev
07:17:41 --- quit: heat (Remote host closed the connection)
07:17:47 --- join: baschdel (~baschdel@2a01:5c0:1c:6201:bb0:858:a281:6ed8) joined #osdev
07:20:00 --- join: shakesoda (uid36406@celestiaradio/shark/soda) joined #osdev
07:32:59 --- join: hmmmm (~sdfgsf@pool-72-79-160-70.sctnpa.east.verizon.net) joined #osdev
07:34:47 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
07:44:54 --- join: JusticeEX (~justiceex@pool-98-113-143-43.nycmny.fios.verizon.net) joined #osdev
07:55:55 --- quit: jack_rabbit (Ping timeout: 264 seconds)
07:58:49 --- quit: m_t (Quit: Leaving)
08:00:19 --- quit: JusticeEX (Ping timeout: 240 seconds)
08:02:11 --- quit: zeus1 (Ping timeout: 256 seconds)
08:03:56 --- join: zeus1 (~zeus@160.119.149.218) joined #osdev
08:07:42 --- quit: bemeurer (Ping timeout: 245 seconds)
08:13:20 --- join: drakonis (~drakonis@unaffiliated/drakonis) joined #osdev
08:19:49 --- quit: grouse (Quit: Leaving)
08:23:44 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
08:28:22 --- quit: manzerbredes (Quit: WeeChat 2.2)
08:28:56 --- quit: bemeurer (Ping timeout: 260 seconds)
08:31:17 <geist> bcos_: hmm, only 5? not too bad
08:31:22 <geist> could have been a lot worse
08:35:52 <geist> bcos_: it's actually a bit hard to visualize how much 5 billion is not a significant impact to the bottom line
08:36:02 <geist> it's the secndary effects that will leave folks scrambling, i bet
08:40:29 --- quit: zaquest (Quit: Leaving)
08:48:37 --- join: freakazoid0223 (~frkazoid@pool-108-52-244-197.phlapa.fios.verizon.net) joined #osdev
08:51:43 * bcos_ wonders if other countries (e.g. US) will follow
08:52:05 <bcos_> (not sure why, but this doesn't seem to happen)
08:58:23 --- join: d45h0 (~d45h0@x5ce486ce.dyn.telefonica.de) joined #osdev
08:58:40 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
08:59:00 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
09:00:23 --- join: isaac_ (~isaac@host81-129-159-113.range81-129.btcentralplus.com) joined #osdev
09:06:34 --- join: m3nt4L (~asvos@2a02:587:a01d:4100:3285:a9ff:fe8f:665d) joined #osdev
09:11:40 --- quit: magnificrab (Remote host closed the connection)
09:12:03 --- quit: nortega (Remote host closed the connection)
09:12:17 --- nick: isaac_ -> isaacwoods
09:12:32 --- join: nortega (~nortega@gateway/tor-sasl/deathsbreed) joined #osdev
09:12:48 --- join: magnificrab (~pi@189.171.148.122.sta.dodo.net.au) joined #osdev
09:14:33 --- join: bauen1 (~bauen1@ipbcc18c77.dynamic.kabel-deutschland.de) joined #osdev
09:26:27 --- quit: vmlinuz (Ping timeout: 245 seconds)
09:32:31 --- quit: bemeurer (Ping timeout: 260 seconds)
09:32:31 --- quit: flacks (Ping timeout: 260 seconds)
09:33:14 --- nick: puck -> pucc
09:34:30 --- join: flacks (flacks@184.91.69.131) joined #osdev
09:40:50 --- quit: brynet (Quit: leaving)
09:42:17 --- quit: Kimundi_ (Ping timeout: 245 seconds)
09:43:43 <hgoel> been studying the intel wifi driver from freebsd lately, it's not exactly complicated, but so many steps
09:43:54 --- join: vmlinuz (~vmlinuz@201-0-120-198.dsl.telesp.net.br) joined #osdev
09:43:54 --- quit: vmlinuz (Changing host)
09:43:54 --- join: vmlinuz (~vmlinuz@unaffiliated/vmlinuz) joined #osdev
09:44:52 <hgoel> I just figured out how to configure the rx/tx rings so I can upload the firmware
09:45:54 <hgoel> and after that I still have to initialize the radios and antennas, as well as bluetooth
09:47:27 --- join: lldd_ (~atrapado@unaffiliated/atrapado) joined #osdev
09:47:43 <hgoel> Sort of reminds me of Intel HD Audio though
09:56:15 --- nick: pucc -> puck
10:03:07 --- quit: baschdel (Ping timeout: 245 seconds)
10:06:46 --- quit: glauxosdever (Quit: leaving)
10:09:37 --- quit: Amaan (Quit: Connection closed for inactivity)
10:09:49 --- quit: drakonis (Remote host closed the connection)
10:12:00 --- join: Kimundi_ (~Kimundi@i577A93F7.versanet.de) joined #osdev
10:12:07 --- nick: ornitorr- -> ornitorrincos
10:12:09 --- quit: ornitorrincos (Changing host)
10:12:09 --- join: ornitorrincos (~ornitorri@unaffiliated/ornitorrincos) joined #osdev
10:17:26 --- quit: pie_ (Ping timeout: 244 seconds)
10:19:36 --- join: Tazmain (~Tazmain@unaffiliated/tazmain) joined #osdev
10:21:47 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
10:24:59 --- join: baschdel (~baschdel@2a01:5c0:1c:6201:bb0:858:a281:6ed8) joined #osdev
10:30:07 --- quit: bemeurer (Ping timeout: 264 seconds)
10:30:26 --- join: brynet (~brynet@brynet6.biz.tm) joined #osdev
10:30:43 --- quit: zeus1 (Ping timeout: 264 seconds)
10:30:48 --- join: dennis95 (~dennis@i577BCEA9.versanet.de) joined #osdev
10:32:46 --- quit: Asu` (Quit: Konversation terminated!)
10:33:05 --- join: Asu (~sdelang@48.17.90.92.rev.sfr.net) joined #osdev
10:33:07 --- quit: promach_ (Ping timeout: 245 seconds)
10:39:01 --- quit: sysfault (Remote host closed the connection)
10:39:49 <hgoel> actually, once you get through the initialization process, it's pretty simple, you just submit commands for scanning/transmitting
10:42:01 --- join: tavish (~tavish@unaffiliated/tavish) joined #osdev
10:47:01 <geist> which wifi chipset is this in particular?
10:51:38 <hgoel> the Intel WiFi 7000 and 8000 series
10:52:14 <hgoel> there aren't a lot of differences between the two, although I'm focusing primarily on the 7000 series, as both my computers have chips from it
10:53:02 --- join: MrOnlineCoder (~MrOnlineC@195.225.231.218) joined #osdev
10:53:08 <hgoel> 7000 series would be the 7260, 3160, 3165, 3168 and 7265
10:54:01 <hgoel> considering that my 5 year old laptop, and 6 month old desktop have chips from the same series, I'm assuming they're fairly common
10:55:03 --- quit: brynet (Quit: leaving)
10:56:57 <hgoel> https://github.com/himanshugoel2797/Cardinal-semicolon/blob/master/notes/drivers/iwifi/device_doc.md here's what I have so far, I stopped documenting each individual register read/write about 1/3rd of the way through, instead just referring to the function in question.
10:56:58 <bslsk05> ​github.com: Cardinal-semicolon/device_doc.md at master · himanshugoel2797/Cardinal-semicolon · GitHub
11:02:28 --- join: bemeurer (~bemeurer@stdcognition.static.monkeybrains.net) joined #osdev
11:03:30 --- join: xobroll[m] (xobrollmat@gateway/shell/matrix.org/x-xdbniunfxhuctyxo) joined #osdev
11:03:45 --- join: brynet (~brynet@brynet6.biz.tm) joined #osdev
11:05:46 --- join: tacco| (~tacco@i59F4D53E.versanet.de) joined #osdev
11:11:36 --- quit: m3nt4L (Remote host closed the connection)
11:27:17 --- quit: flacks (Ping timeout: 245 seconds)
11:30:00 --- join: angel0xff (~zzz@158-58-227-127.sf.ddns.bulsat.com) joined #osdev
11:30:49 --- join: flacks (flacks@184.91.69.131) joined #osdev
11:40:38 --- join: leinne (~leinne@gateway/tor-sasl/leinne) joined #osdev
11:42:16 <geist> neat
11:44:57 --- join: m_t (~m_t@p5DDA257A.dip0.t-ipconnect.de) joined #osdev
11:46:22 --- quit: Lucretia (Quit: Konversation terminated!)
11:47:27 --- quit: nortega (Quit: Vivu lante, vivu feliĉe!)
11:48:36 --- join: SopaXorzTaker (~SopaXorzT@unaffiliated/sopaxorztaker) joined #osdev
11:49:00 --- join: Lucretia (~Luke@pdpc/supporter/active/lucretia) joined #osdev
11:54:17 --- quit: dennis95 (Quit: Leaving)
12:02:08 <klys> perhaps they are very common, mine are 4965 and 6205, though.
12:07:21 --- quit: asymptotically (Quit: Leaving)
12:11:41 <qeos> hi
12:13:15 <qeos> do someone made print callstack for kernel? do u use any libs or write own code for this?
12:13:53 <geist> there are a few ways to do it
12:14:04 <geist> if you're willing to build with frame pointers you can easily just walk up the stack manually
12:14:06 --- join: zeus1 (~zeus@154.225.146.190) joined #osdev
12:14:07 <geist> very little code required
12:16:20 <qeos> can show it? can I take you code?
12:17:38 <doug16k> qeos, this is a function that traces the stack of the calling thread -> https://github.com/doug65536/dgos/blob/c834ba31a9c94f09e285e5a7ed13fe4ec8a994be/kernel/arch/x86_64/stacktrace.cc#L12
12:17:40 <bslsk05> ​github.com: dgos/stacktrace.cc at c834ba31a9c94f09e285e5a7ed13fe4ec8a994be · doug65536/dgos · GitHub
12:18:48 <qeos> oh.. __builtin_frame_address
12:18:50 <doug16k> you can change it to trace an arbitrary thread by changing it to take a parameter telling it the thread's rbp instead of using __builtin_frame_address(0)
12:19:10 <qeos> looks good
12:19:15 <qeos> thank u
12:19:31 <geist> yep. that's the trick. you have to be able to start with the current frame, and then you can easily walk up the tree
12:19:52 <geist> you have to build with frame pointers, but on modern architectures (ie, not x86-32) the frame pointer overhead is fairly negligable
12:20:07 <geist> or at least for early development i wouldn't worry about it. it shouldn't impact the performance of your kernel on the whole
12:20:22 <doug16k> the way that works is, at function entry, the stack is pointing at the return address. the function prologue code pushes the parent's rbp and points rbp to where it pushed the parent's rbp
12:20:57 <doug16k> therefore, on the stack there will be pairs of values adjacent on the stack, a pointer to the parent frame, followed by the return address
12:21:23 <doug16k> it follows the parent pointers and peeks at each return address that is adjacent to the parent frame pointer
12:23:09 <doug16k> the compiler only produces code that does this with -fno-frame-pointer. you'll need to compile with that to reserve rbp to have the frame pointers work as described above
12:23:55 <geist> you mean the opposite?
12:24:09 <geist> as in with frame poitners
12:24:25 <doug16k> oops, right, DON'T pass -fno-frame-pointer
12:24:30 <doug16k> thanks :)
12:24:41 --- join: glauxosdever (~alex@ppp-94-66-200-106.home.otenet.gr) joined #osdev
12:25:01 <doug16k> you need -fno-omit-framer-pointer
12:25:19 <doug16k> -fomit-frame-pointer makes the stack walk not work
12:28:03 <doug16k> dammit, -fno-omit-frame-pointer
12:29:25 --- quit: MDude (Quit: Going offline, see ya! (www.adiirc.com))
12:34:00 <klys> heat around
12:35:16 <geist> i always think about that epic shootout in the movie every time i see it
12:35:28 --- quit: lldd_ (Quit: Leaving)
12:35:51 <qeos> o_/
12:43:25 <doug16k> I got virtio-gpu putting commands on the available ring and getting success responses. still have to figure out why I don't get any completion IRQ(s)
12:44:14 <klys> doug16k, what condition is dgos in this week?
12:46:33 <doug16k> working perfectly as far as I can tell
12:46:37 --- join: pie_ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
12:46:51 <klys> cool I'll check it out~
12:49:49 <doug16k> I have msix enabled, I set each virtqueue's vector index, I unmasked the function in pcix config, all of the msix interrupt entries are unmasked, eflags.IF = 1, configuration interrupts are using msix table entry 0
12:50:48 <doug16k> I must be missing poking a value into the used ring to request an IRQ when it reaches that point
12:51:08 <qeos> doug16k: do u have screenshots of u own os?
12:52:18 <klys> /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.a when searching for -lgcc
12:52:55 <doug16k> I had a desktop started for a bit but I was doing way too much in the kernel so I backed out from that. I'm now getting started again on a GUI, with 3d acceleration support this time
12:54:18 <klys> doug16k, does this build 32 bits by default? according to debian, binutils:i386 conflictS: binutils:amd64
12:54:53 <klys> I noticed it was linking mbr-elf
12:54:58 <doug16k> 64 bits. I haven't bothered with a 32 bit version
12:55:13 <doug16k> the amd64 binutils can handle 32 bits too
12:55:37 <doug16k> you don't use the system compiler anyway
12:55:52 <klys> it is using gcc-7 and got that error on mbr-elf
12:56:01 <doug16k> yeah, don't do that
12:56:32 <doug16k> there is a cross compiler build script in there, and there's a script that runs configure for you with the appropriate options
12:57:51 <doug16k> give me a chance to add build/run/debug instructions to the readme
12:58:31 <doug16k> I'll try to get to it later today
12:59:41 <doug16k> eventually I need to beef up the build so it magically conjures up a cross compiler when necessary
13:01:13 --- quit: xenos1984 (Ping timeout: 240 seconds)
13:01:18 <klys> I got it going from my last attempt's tree, which had built gcc. this clone was missing build-crossgcc.sh.
13:01:50 <doug16k> I moved it
13:02:11 <doug16k> I was dumping too much crap at the top of the source tree. moved some stuff into directories
13:02:11 <klys> so ./dobuild will need to change then.
13:02:21 <doug16k> ah missed that I guess. thanks
13:02:38 <klys> yw.
13:02:39 <doug16k> haven't touched that for a while, forgot it existed
13:05:57 --- join: JusticeEX (~justiceex@pool-98-113-143-43.nycmny.fios.verizon.net) joined #osdev
13:06:30 --- join: spare (~user@unaffiliated/spareproject) joined #osdev
13:06:52 <DusXMT> Hmmm... I wonder, when it comes to memory allocation, how do I ensure that memory I allocate isn't used as a memory-mapped IO by some device? (pc / 32bit x86)
13:07:39 <klys> you keep a structure containing free memory
13:08:05 <DusXMT> Of course, but how do I know that the memory is free, and doesn't belong to some device which I don't know of yet
13:08:21 <klys> the memory map you get a tboot
13:08:27 <klys> at* boot
13:09:18 <klys> is either a multiboot(1,2)(grub), e820 (bios), or efi(uefi) memory map.
13:09:49 * DusXMT nods
13:09:59 <klys> also you can tell which memory is mapped, I gather, by reading acpi tables (aml).
13:10:14 <klys> not too clear on that last tidbit.
13:11:41 <DusXMT> Okay, then that answers my question; I'm just asking because I have been using the multiboot memory map thus far, but didn't know if there's any regions of memory that I need to avoid using as general-purpose memory
13:12:25 <klys> that should all be general purpose memory until you tell your device to map.
13:12:27 <doug16k> DusXMT, the type field in that array of items tells you whether memory is usable
13:12:59 <DusXMT> doug16k: And I have indeed been checking that field
13:13:27 <DusXMT> klys: thanks a lot :)
13:13:43 --- quit: X-Scale (Ping timeout: 248 seconds)
13:17:03 --- quit: Halofreak1990 (Ping timeout: 268 seconds)
13:17:26 --- join: drakonis (~drakonis@unaffiliated/drakonis) joined #osdev
13:23:58 <klys> doug16k, how many cores does the build use by default? my machine is pegged. It was last time, too.
13:25:07 --- join: stee3 (~junk@66.252.139.92) joined #osdev
13:25:26 --- join: nicht (~nicht@2804:7f1:2080:1ba5:b65:467a:a348:a252) joined #osdev
13:27:53 --- quit: ptx (Quit: ZNC - http://znc.in)
13:28:05 --- quit: stee (Ping timeout: 244 seconds)
13:29:03 --- join: MindlessDrone (~MindlessD@unaffiliated/mindlessdrone) joined #osdev
13:29:46 --- join: ptx (~ptx@563400ad.rev.stofanet.dk) joined #osdev
13:31:17 <klys> http://show.ing.me/paste/dobuild1.nng
13:31:21 <bslsk05> ​show.ing.me <no title>
13:32:26 --- join: X-Scale (~ARM@83.223.227.106) joined #osdev
13:34:47 --- quit: SopaXorzTaker (Ping timeout: 245 seconds)
13:34:55 --- quit: klysm (Ping timeout: 264 seconds)
13:38:57 --- join: CheckDavid (uid14990@gateway/web/irccloud.com/x-sgdjnahofgictcnp) joined #osdev
13:48:12 --- join: Halofreak1990 (~FooBar247@5ED0A537.cm-7-1c.dynamic.ziggo.nl) joined #osdev
13:50:07 --- quit: pie_ (Ping timeout: 256 seconds)
13:56:36 --- quit: elevated (Remote host closed the connection)
13:57:17 --- quit: zeus1 (Ping timeout: 245 seconds)
13:57:18 --- quit: tavish (Quit: Leaving)
13:57:19 --- join: elevated (~elevated@gateway/tor-sasl/elevated) joined #osdev
14:03:09 --- quit: FManTropyx (Ping timeout: 256 seconds)
14:03:35 <MrOnlineCoder> hi, before enabling paging, I should setup an identity paging directory for my kernel or it does not matter?
14:04:37 --- quit: vmlinuz (Quit: Leaving)
14:07:27 <doug16k> klys, you mean when you run make? it's whatever you say to use, default is 1
14:07:58 <doug16k> ah, I have it using 16 in dobuild :) I have 16 CPUs
14:08:10 <doug16k> maybe a bit much for a more modest processor
14:09:53 <doug16k> MrOnlineCoder, yes, you must set up paging before you enable it
14:10:12 <MrOnlineCoder> ikr, i mean it must be only identity paging?
14:10:17 <MrOnlineCoder> 1 to 1?
14:10:21 --- quit: baschdel (Ping timeout: 276 seconds)
14:10:36 <doug16k> the page containing the code that enables paging must be identity, the rest don't matter
14:10:53 <doug16k> you probably want the stack it is using at that moment identity mapped too
14:11:58 <doug16k> so, at least that ^ or as much as everything, of course
14:12:20 <MrOnlineCoder> what a typical mapping for a kernel in paging?
14:12:30 <doug16k> which architecture?
14:12:49 <MrOnlineCoder> x86
14:13:19 <doug16k> typically the kernel base is either 0x80000000 or 0xC0000000
14:13:36 <MrOnlineCoder> you mean the higher half?
14:13:38 <doug16k> which gives user mode 2GB or 3GB of address space, respectively
14:13:46 <doug16k> yes
14:14:17 <MrOnlineCoder> but if i map my kernel in this way, wouldn't my heap allocator break down?
14:14:24 <MrOnlineCoder> it is used before paging
14:14:33 <MrOnlineCoder> and it has a "heap start address"
14:15:09 <doug16k> do you use multiboot?
14:16:06 --- join: FManTropyx (~fooman@82-203-190-31.bb.dnainternet.fi) joined #osdev
14:17:28 <MrOnlineCoder> grub
14:17:34 <geist> generally speaking yes you will have bootstrapping issues. usually theidea is you immediately get the kernel bumped to the high address
14:17:44 <geist> before really running any C code at all. thus the heap is off limits
14:18:13 <doug16k> if so, you'll need to have a trampoline that is loaded to a known physical address, and also linked at that same address (virtual address == physical address). that code would initialize the page tables and jump to the actual entry point
14:18:16 <sortie> I kinda have thought about having the initial code page after enabling paging be different from the page before, since the next instruction will be executed
14:18:37 <sortie> You just have to make sure the right code shows up in the current place
14:18:50 <geist> there are a handful of ways to go about it, and it's annoying. same on other architectures too, generally speaking
14:19:27 --- quit: Halofreak1990 (Ping timeout: 276 seconds)
14:19:32 <sortie> geist: When pid 1 exits, what do you think the kernel should do if there's other processes / threads around?
14:19:47 <chrisf> do it all in assembly before jumping to your c entrypoint
14:21:03 --- quit: glauxosdever (Quit: leaving)
14:21:17 --- join: Halofreak1990 (~FooBar247@5ED0A537.cm-7-1c.dynamic.ziggo.nl) joined #osdev
14:21:19 --- quit: jakogut (Quit: jakogut)
14:21:33 <MrOnlineCoder> is it ok to store my kernel paging directory as a static member in my code? or it should be dynamically allocated somewhere?
14:21:35 <geist> sortie: probably should either reboot or panic
14:21:44 --- quit: nicht (Ping timeout: 256 seconds)
14:21:52 <geist> unless there's a way of restarting pid 1, but then if there's any state left around as a result of that
14:21:57 --- join: jakogut (~jakogut_@162.251.69.147) joined #osdev
14:22:04 <geist> ie, mounted fs, devices in different states, etc.
14:22:05 <MrOnlineCoder> sortie, how can pid 1 exit?
14:22:14 <sortie> MrOnlineCoder: Normally. Like any other process.
14:22:23 <geist> MrOnlineCoder: re: static kernel dir, fine with me
14:22:26 <doug16k> MrOnlineCoder, you'd do something like having a .init section whose virtual and physical address are the same in the linker script, and it would pull in the trampoline code which is in assembly and specifies section .init, your kernel's entry point would be in that section. code in there sets up paging to make the actual kernel be mapped to the high half, then it would jump into the normal .text section whose virtual address is in the
14:22:26 <doug16k> high half
14:22:51 <MrOnlineCoder> oh
14:22:51 <MrOnlineCoder> ty
14:22:52 <sortie> geist: Well, I power off / reboot when pid 1 exits by design. But the way I do stuff, those orphan processes keep running in a weird system state. The kernel is preparing to shut everything down, and these processes hang around?
14:22:53 <geist> that's a fairly easy way to get the kernel high mapped without needing an allocator, plus it'll never be freed (or that would be a panic)
14:23:21 <geist> sortie: ah i read 'pid 1' exiting as 'there are no running processes'
14:23:22 <sortie> geist: I don't see any real harm actually, at least for now since I reboot immediately. But I try to get rid of these weird system states.
14:23:43 <geist> if it's pid 1 exiting but there are other things, then i guess it's whatever you define pid 1 being special or not
14:24:03 <geist> in posix, dunno what it's supposed to entail, but honestly i'd assume that pid 1 exiting would automatically clean up everything underneat it
14:24:04 <sortie> pid 1 is not so much special, as much as the kernel shuts down when it's done
14:24:13 <geist> since presumably pid 1 is the process group leader for all other processes
14:24:15 <sortie> POSIX says nothing on this (out of scope)
14:24:32 <sortie> Process groups are not involved here
14:24:35 <MrOnlineCoder> hm, if I map my kernel to 0xC0000000 for example, can't my heap start be placed 1-2 mb after that point (leaving 1-2 mb for kernel itself)?
14:24:43 <sortie> (process groups are not nested)
14:25:00 <geist> guess i've been thinking too much in zircon land, where the jobs are *only* nestable
14:25:08 --- quit: flacks (Ping timeout: 256 seconds)
14:25:17 <geist> and thus if pid 1 (or the root process/job) died, it would automatically kill every other process in the system
14:25:19 <doug16k> MrOnlineCoder, you startup your heap awfully early. why does it need to be up so early?
14:25:45 <geist> yeah what doug16k said. you really should wait until your VMM is up and running, and there's just going to be some early code you have to run pre-heap
14:25:49 <geist> and it's messy but that's life
14:25:50 --- quit: jakogut (Client Quit)
14:26:02 <sortie> geist: The solution I've implemented yesterday is that the kernel SIGKILL's every process just before pid 1 is deallocated. Then when the kernel has awaited pid 1's exit, there's no processes left, and the kernel can start shutting down drivers that need it without interference.
14:26:14 --- join: jakogut (~jakogut_@162.251.69.147) joined #osdev
14:26:37 <sortie> I also disallow new processes / threads when pid 1 has exited, so that part doesn't run forever
14:26:41 <geist> sounds legit, though in that case wouldn't you have to wait until all the children have exitted to? or is the SIGKILL delivery synchronous
14:26:58 <MrOnlineCoder> well ty doug16k / geist / sortie / chrisf. i've read paging article on osdev about 15 times and still do not know what I should code :D
14:27:02 <sortie> geist: It also waits for them to exit
14:27:12 <MrOnlineCoder> maybe on the time 16 i ll figure it out
14:27:16 <geist> so it's more of a kernel_process_kill() which is syncrhonous
14:27:17 <geist> got it
14:27:40 --- join: xenos1984 (~xenos1984@22-164-191-90.dyn.estpak.ee) joined #osdev
14:27:44 <geist> MrOnlineCoder: well at least you're hitting the usual roadblocks
14:27:51 <geist> which means you're probably on the right path
14:28:11 <geist> these are problems that most folks bump into. bootstrapping the paging unit and the VMM is actually a bit difficult
14:28:16 <sortie> geist: My original implementation deadlocked because the process cleanup was done in a worker thread, and now that blocked the worker thread waiting for the worker thread to clean up another process. So instead I detect the last thread to exit in a process and run a hook there before the thread shuts down.
14:28:30 <geist> simpler when the kernel is a hello world thing, but as you add more complexity to it the bootstrap phase gets more complicated
14:28:43 <sortie> MrOnlineCoder: This is fairly normal, paging is tricky
14:29:35 <MrOnlineCoder> ah, and to figure out the terms, when someone is talking about physical memory allocator, do they mean page frame allocator?
14:29:46 <sortie> geist: The delivery of the signal is synchronous (subject to just a mutex). However the dispatch of the signal is async. It delivers SIGKILL and then just awaits the child process using waitpid's backend.
14:29:57 --- join: flacks (flacks@184.91.69.131) joined #osdev
14:30:05 <geist> yah
14:30:23 --- quit: leinne (Disconnected by services)
14:30:43 --- join: leinne_ (~leinne@gateway/tor-sasl/leinne) joined #osdev
14:32:21 --- quit: isaacwoods (Quit: isaacwoods)
14:34:54 --- join: pie_ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
14:35:08 <doug16k> MrOnlineCoder, you need a piece of code that takes a virtual address and a physical address as parameters. it takes the virtual address, extracts the value in bits 31:22 (pd_index) and extracts the value in bits 21:12 (pt_index). then it looks up the pd_index in the page directory and ensures a page table is mapped there, mapping one if necessary. then it writes an entry to slot pt_index of that page table with the physical address
14:35:52 <graphitemaster> yeah, you'll get roadblocked by paging again once you move to porting your kernel to x86_64
14:36:08 <graphitemaster> I wrote a super nice microkernel for x86 a long time ago
14:36:19 --- quit: pie_ (Read error: Connection reset by peer)
14:36:26 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
14:36:27 <graphitemaster> and was like :man, I can't wait to port this to x86_64: and all my assumptions about paging went out the window
14:36:31 <graphitemaster> because it would no longer work
14:36:33 <doug16k> hopefully by the time you do x86_64 paging, x86 paging will feel simple and you'll be able to handle the increased difficulty
14:37:34 <graphitemaster> paging can be really easy if you just identity map everything and ignore the rest of the permission bits and gating
14:38:06 <MrOnlineCoder> identity mapping would be ok for kernel, but i want simple multitasking to be implemented
14:38:24 <MrOnlineCoder> so permission bits and more or less clever mapping would be good
14:38:41 <MrOnlineCoder> doug16k, the code right ^, how it should be called? like when that 'piece' of code is called? just name the function :D
14:38:59 <doug16k> map_page
14:39:32 --- join: robert_ (~hellspawn@72.186.97.130) joined #osdev
14:39:32 --- quit: robert_ (Changing host)
14:39:32 --- join: robert_ (~hellspawn@objectx/robert) joined #osdev
14:39:37 --- quit: robert_ (Client Quit)
14:39:46 --- join: robert_ (~hellspawn@72.186.97.130) joined #osdev
14:39:46 --- quit: robert_ (Client Quit)
14:39:47 <MrOnlineCoder> what should cause this function to be executed? page fault?
14:40:04 --- join: robert_ (~hellspawn@72.186.97.130) joined #osdev
14:40:04 --- quit: robert_ (Changing host)
14:40:05 --- join: robert_ (~hellspawn@objectx/robert) joined #osdev
14:40:18 <MrOnlineCoder> ah
14:40:23 <MrOnlineCoder> sorry
14:40:25 <MrOnlineCoder> figured it out
14:40:26 <MrOnlineCoder> thanks
14:40:49 <doug16k> your linker script should have symbols you can use to know the start and end of .text, .data, .bss, .whatever
14:41:46 <doug16k> so, say .text begins at 0xc0000000, you'd know/compute where your linker script put that in physical memory, and you'd map_page 0xC0000000, 0xC0001000, ... in a loop, for each range
14:42:32 --- quit: d45h0 (Read error: Connection reset by peer)
14:42:41 <doug16k> and you'd make map_page calls that identity map your .init section, then you'd initialize cr3 and enable paging, and jump into your .text in high half
14:42:47 * UNIVAC ​reports changes to wiki page "Descriptor" by Mronlinecoder: fix typo <​https://wiki.osdev.org/index.php?title=Descriptor&diff=22527&oldid=17438>
14:43:03 <doug16k> jmp to some symbol in your normal kernel code's .text
14:44:36 <MrOnlineCoder> i must do that jump because my eip at the moment when i enable paging would point to an unmapped page?
14:45:18 <doug16k> the jump is what you do at the end of the paging-init trampoline (which is identity mapped in .init section), to jump into the actual kernel which is in high half in .text section
14:45:45 <MrOnlineCoder> thx
14:46:27 <doug16k> so big picture: grub loads your kernel at addresses you said in your linker script, and entry point is in the identity mapped .init. code there sets up the paging to map .text/etc in high half, and enables paging, then you jump into high half actual kernel
14:47:57 <doug16k> also, your linker script tells the linker to link the high half stuff to assume it is at a high address. by the time paging is enabled, things really are at those high virtual addresses, because the page tables made them be at those addresses
14:49:09 <doug16k> the actual kernel pages are probably going to be put somewhere close to 1MB line, just after your .init trampoline, in physical memory. you only make them look like they are way up at 0xC0000000 by mapping those physical pages to those places in the page tables
14:49:51 <MrOnlineCoder> and then I map my kernel in the same way to every process's virtual address space?
14:50:22 <doug16k> yes, by making the last 1/2 (2GB user space) or 1/4 (3GB user space) of the page directory be the same in every process
14:50:40 --- join: NightBlade (~user@ip-173-247-149-50.user.start.ca) joined #osdev
14:50:54 <klys> /root/src/sys/dgos/boot/elf64.cc:240: undefined reference to `__udivdi3'
14:50:59 <doug16k> which translates into the last 512 or 256 slots of the page directory being the same everywhere
14:51:28 <doug16k> klys, are you using the cross compiler?
14:51:51 <klys> x86_64-elf-g++
14:52:28 <doug16k> old cross compiler or recent? I improved libgcc build
14:53:08 <klys> It's probably the old one since I copied that script earlier
14:53:24 <MrOnlineCoder> doug16k, if have a normal pointer to 0xB8000 for printing some nice messages to the screen, wouldn't it break after I enable paging? if so, then I should somehow dynamically figure at what address my video memory will be depending on the paging state?
14:54:09 <doug16k> MrOnlineCoder, then map_page 16 pages with 0xb8000 virtual address, 0xb8000 physical address
14:54:28 <doug16k> you have god-like control over address space, you can do whatever
14:55:01 <MrOnlineCoder> should I care about code below the 1 mb mark (below my kernel) like grub or boot code?
14:55:29 <Icefoz> Usually you have your physical address space mapped 1-to-1 into part of your virtual address space (identity mapped)
14:55:37 <Icefoz> At least to start with
14:55:58 <Icefoz> And then you can add more mappings from there as necessary.
14:56:39 <doug16k> easy mode is to at first just identity map the first 1MB. I strongly recommend making the first 4KB or 64KB or so not present to catch null pointers, and map those physical addresses to some other virtual address if you find you need to access them for whatever reason
14:56:40 <MrOnlineCoder> well ok, ty, i'll try something
14:57:28 --- quit: Halofreak1990 (Ping timeout: 244 seconds)
14:57:41 <MrOnlineCoder> ah, and the last one: in c, what type should my virtual and physical adresses have? pointers or unsigned ints or whatever?
14:58:21 <MrOnlineCoder> the addresses that I pass to map_page or other_function()
14:58:28 <doug16k> uintptr_t is going to be the right width to represent an address on most architectures
14:58:43 <doug16k> it will be 32 bits in 32 bit build, or 64 bits on 64 bit build, etc
14:58:53 <doug16k> typically
14:59:32 --- quit: pie__ (Remote host closed the connection)
14:59:57 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
15:00:14 <doug16k> don't use pointers to represent physical addresses that aren't identity mapped - you can't actually use them to access memory once paging is up. try to use uintptr_t (or typedef something like `typedef uintptr_t physaddr_t`) to reprent physical addresses. virtual addresses can be validly represented as void * or whatever
15:00:58 <doug16k> because you can dereference a virtual address to actually access memory, it makes sense for it to be a pointer
15:01:29 --- join: zeus1 (~zeus@154.226.249.25) joined #osdev
15:02:30 --- join: Halofreak1990 (~FooBar247@5ED0A537.cm-7-1c.dynamic.ziggo.nl) joined #osdev
15:02:32 <doug16k> you can also represent virtual addresses as uintptr_t or `typedef uintptr_t linaddr_t` or whatever, your choice
15:02:45 <doug16k> it's nice to have different type names to make intent clearer in the code
15:02:58 <doug16k> you won't get mixed up as easily
15:03:38 <MrOnlineCoder> yea i try to create typedef for different things all the time
15:03:40 <MrOnlineCoder> thanks
15:04:50 <doug16k> representing virtual addresses with uintptr_t is actually a better idea than a pointer, because you'll be doing bit shifts and masks, and it's very weird to do that to a pointer with casts
15:05:50 <doug16k> the high level apis might like to take/return pointers, because from the caller's perspective, they are pointers
15:06:12 <doug16k> ...for the virtual addresses I mean
15:08:29 <MrOnlineCoder> haha it is 1:07 night here and i am trying to write paging implementation
15:08:37 <doug16k> best time, lol
15:09:22 <doug16k> it'll probably be a mixture of "oh cool!" and "what was I thinking?" tomorrow though
15:10:53 --- quit: Kimundi_ (Remote host closed the connection)
15:11:02 --- join: Kimundi_ (~Kimundi@i577A93F7.versanet.de) joined #osdev
15:11:56 --- quit: Halofreak1990 (Ping timeout: 244 seconds)
15:14:10 --- join: lucus16 (~quassel@relto.rasusan.nl) joined #osdev
15:14:59 --- join: Halofreak1990 (~FooBar247@5ED0A537.cm-7-1c.dynamic.ziggo.nl) joined #osdev
15:21:17 <MrOnlineCoder> doug16k, in terms of convenience, what is the better way to store a page table entry (a single page) - as a c struct or as plain uint32?
15:21:28 <klys> doug16k, http://paste.debian.net/1034318/
15:21:30 <bslsk05> ​paste.debian.net: debian Pastezone
15:23:05 <klys> I would use a uint32, though I don't use paging.
15:23:06 <doug16k> MrOnlineCoder, I'd represent it as simply a pointer. I typedef `uintptr_t pte_t` and use pte_t * to represent pointers to page tables, page directories, or page table entries
15:23:48 <doug16k> then have some constants for PTE_PRESENT, PTE_WRITABLE, etc for setting/masking the flag bits
15:23:50 <MrOnlineCoder> and set the needed flags using bitwise logic, yea?
15:23:52 <MrOnlineCoder> ah
15:23:54 <MrOnlineCoder> xd
15:24:03 <MrOnlineCoder> ok
15:25:01 <doug16k> then, if you have a pointer to a page directory, for example, you can lookup the appropriate page directory entry with pte_t * pd_ptr = ...; pd_ptr[linear_addr >> 22]
15:25:27 <doug16k> and once you have a pointer to a page table, lookup the appropriate entry with pt_ptr[(addr >> 12) & 1023]
15:27:36 --- quit: Shikadi (Quit: Leaving)
15:27:36 <doug16k> you'd probably have a mask PTE_ADDR which is 0xFFFFF000. then you can get a pointer to a page table (when it's still identity mapped) with pte_t pt_ptr = (pte_t*)(pd_ptr[linear_addr >> 22] & PTE_ADDR)
15:27:54 <doug16k> oops, pte_t *pt_ptr...
15:27:55 --- join: Shikadi (~Shikadi@cpe-98-10-34-205.rochester.res.rr.com) joined #osdev
15:28:15 <MrOnlineCoder> yea already found it ty
15:29:10 <doug16k> I usually don't name things by their type (i.e. "ptr") just trying to maximize clarity
15:33:55 <doug16k> klys, yeah, or even better, try to auto-detect how many cpus are present, instead of hard coding 16
15:34:20 <doug16k> you'd think someday, make would take an option to automatically use j == number of cpus D:
15:34:30 <doug16k> ridiculous
15:35:02 <klys> please accept this patch
15:35:10 <klys> :)
15:35:58 <doug16k> want to send me a pull request so you get credit or shall I just put it in?
15:36:13 <klys> I don't care just put it in iguess
15:36:30 <klys> not that familiar with git rly
15:38:06 --- quit: Halofreak1990 (Ping timeout: 276 seconds)
15:38:46 <doug16k> crash course doing it on github: you clone the repo into your account and add a remote to your working directory which points to your account's clone url, name it fork or something, then you commit the change in your working directory and push it to fork master or whatever, then in the github site it will offer to make a pull request, from there it's obvious
15:39:41 <klys> my account has only one small project, and forking yours would be a nightmare
15:39:54 <doug16k> one click is a nightmare?
15:40:05 <doug16k> you click the fork button at the top right. done
15:40:15 <klys> making a fork there would be no way I could keep up with you
15:40:39 <doug16k> once you have that set up you can go to your working directory and git pull origin master and poof, all my changes merge in
15:41:21 <doug16k> you are underestimating the level of awesome that is git
15:42:22 <doug16k> then you can update your fork after doing that by doing git push fork master
15:43:22 --- quit: Asu (Remote host closed the connection)
15:43:29 <doug16k> learning a bit of git is very worthwhile. you get 90% of the benefit by learning 10% of it
15:43:39 <klys> let's focus on cloning it into my account. I logged into the website. Do I clone it into my account from git on the commandline or the website?
15:43:59 <doug16k> be logged into git, go to my repo, and click fork at top right
15:44:05 <doug16k> logged into github*
15:44:19 <klys> did thx
15:44:28 <doug16k> poof, now you suddenly have dgos in your account
15:44:30 --- join: Halofreak1990 (~FooBar247@5ED0A537.cm-7-1c.dynamic.ziggo.nl) joined #osdev
15:44:37 <doug16k> now, go to your page and click dgos...
15:44:38 --- quit: pie__ (Remote host closed the connection)
15:44:58 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
15:45:12 <doug16k> then grab that clone link and in your working directory: git remote add fork <paste the clone link>
15:45:55 <doug16k> now you can commit changes and push them to fork master. after doing that you can go to github page and it will offer to send me a pull request to merge your change into my repo
15:46:47 <doug16k> you'll be the author of those changes in my git log and your name will come up on those lines in git blame
15:46:59 <doug16k> if I merge it that is
15:48:01 <doug16k> if I/the-repo-owner doesn't merge it, screw them, you have the desired changes in your repo :)
15:48:17 <klys> so I need to do something like "git commit" dobuild?
15:48:34 <doug16k> that would work
15:48:45 <klys> it gives me an edit session
15:48:49 <doug16k> you can also prepare to commit files by doing a series of `git add` commands
15:49:01 <doug16k> yeah, give it a commit message and save and exit. briefly describe what you did for the log
15:49:08 --- quit: palk_ ()
15:50:12 <doug16k> you can abort the commit by exiting the editor without saving any changes
15:52:12 <klys> and do you have my pull request?
15:52:37 <doug16k> did you `git push fork master` yet?
15:52:54 <klys> something like that
15:52:58 <doug16k> commit just does it on your hard drive. push puts the change up there
15:53:06 <klys> git push https://github.com/mdasoh/dgos.git
15:53:09 <doug16k> did you go to your fork on github and tell it to send me a pull request?
15:53:15 <klys> and then I did that
15:54:29 <klys> ok
15:54:40 <klys> you'll for sure have it now.
15:55:38 <klys> I "closed the pull request"
15:55:49 <doug16k> merged
15:55:51 <klys> thanks doug16k
15:57:54 <klys> the only way I can figure how to do that is with a utility that parses /proc/cpuingo
15:58:07 <doug16k> yeah, that's a typical approach
15:58:16 <klys> cross-tools build complete
15:59:22 --- quit: pixelherodev (Ping timeout: 245 seconds)
15:59:39 <klys> dgos build complete.
15:59:56 <klys> bbl. suppertine
16:00:11 --- quit: earlz (Ping timeout: 265 seconds)
16:00:13 --- quit: pecan (Ping timeout: 240 seconds)
16:00:18 --- quit: nshp (Ping timeout: 260 seconds)
16:00:27 --- quit: lokust (Ping timeout: 256 seconds)
16:00:28 --- quit: Icefoz (Ping timeout: 268 seconds)
16:00:43 --- quit: edr (Ping timeout: 264 seconds)
16:00:51 --- quit: sircmpwn (Ping timeout: 276 seconds)
16:00:51 --- join: ryonaloli (ta@unaffiliated/ryonaloli) joined #osdev
16:03:18 --- quit: Tazmain (Quit: Leaving)
16:04:43 --- quit: angel0xff (Ping timeout: 240 seconds)
16:08:42 --- quit: Brnocrist (Ping timeout: 256 seconds)
16:12:19 --- quit: sortie (Quit: Leaving)
16:15:05 --- join: immibis (~chatzilla@222-155-163-212-fibre.sparkbb.co.nz) joined #osdev
16:15:59 --- join: Brnocrist (~spartak@unaffiliated/brnocrist) joined #osdev
16:16:16 --- quit: leinne_ (Remote host closed the connection)
16:16:49 --- quit: pie__ (Remote host closed the connection)
16:16:52 --- quit: bemeurer (Ping timeout: 245 seconds)
16:16:57 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
16:19:21 --- join: leinne (~leinne@gateway/tor-sasl/leinne) joined #osdev
16:22:27 <klys> oh dear, bootefi-amd64 is not recognized by ovmf.
16:22:43 --- join: MarchHare (~Starwulf@mo-184-5-202-5.dhcp.embarqhsd.net) joined #osdev
16:26:19 --- quit: jakogut (Quit: jakogut)
16:26:44 --- join: jakogut (~jakogut_@162.251.69.147) joined #osdev
16:28:18 <doug16k> klys, the one it really uses is bootx64.efi
16:28:33 <doug16k> that one you cited is the one I point my debugger at
16:28:50 <doug16k> bootx64.efi has the debug info stripped
16:29:31 <doug16k> it should be at /EFI/boot/bootx64.efi if you are manually doing it from the EFI shell
16:30:18 --- quit: MrOnlineCoder (Read error: Connection reset by peer)
16:30:50 --- quit: jakogut (Client Quit)
16:31:14 --- join: jakogut (~jakogut_@162.251.69.147) joined #osdev
16:31:48 <doug16k> if you attach the disk image, it should just go straight into the EFI bootloader, assuming you don't spam Esc
16:32:06 <klys> hmm how do I make this file with your tree?
16:32:28 <klys> I now I did
16:33:41 <klys> yayy
16:34:37 --- quit: ryonaloli (Quit: She was Lo, plain Lo, in the morning, standing four feet ten in one sock. She was Lola in slacks. She was Dolly at school. She was Dolores on the dotted line.)
16:35:38 <klys> doug16k, Taking 1000 at 63100
16:35:42 <klys> 0
16:35:45 <klys> doug16k, Taking 1000 at 631000
16:36:16 <klys> assert failed boot/p...something
16:36:38 <klys> probably because I need a root
16:38:24 --- join: pixelherodev (znc@irc.sircmpwn.com) joined #osdev
16:39:11 --- join: sircmpwn (znc@irc.sircmpwn.com) joined #osdev
16:40:34 <klys> http://show.ing.me/paste/dgos0001.png
16:40:47 <bslsk05> ​show.ing.me <no title>
16:41:04 <klange> ./boot/include/inttypes.h:2:5: warning: "__LONG_WIDTH__" is not defined [-Wundef]
16:41:14 <klange> doug16k: Your build system doesn't like my compiler.
16:41:54 --- quit: sircmpwn (Excess Flood)
16:42:00 <klange> I also notice your build system modifies checked-in files (generates Makefile.in despite that being checked in) and creates a lot of files not included in your gitignore?
16:42:03 <klange> RIP sircmpwn
16:42:46 <klys> cool beans that I have a bootx64.efi built with gcc now
16:42:51 <klange> but maybe those will be deleted later if the build wasn't failing?
16:43:59 --- join: nwm (~nwm@d162-156-46-158.bchsia.telus.net) joined #osdev
16:44:45 --- quit: flacks (Ping timeout: 255 seconds)
16:45:45 --- quit: xenos1984 (Quit: Leaving.)
16:46:05 --- join: sircmpwn (znc@irc.sircmpwn.com) joined #osdev
16:47:49 --- join: flacks (flacks@184.91.69.131) joined #osdev
16:50:05 <doug16k> yeah I check that in so you don't have to run autoconf before configuring
16:50:39 <doug16k> should probably .gitignore it though, yeah
16:52:22 <doug16k> klys, screen output appears to stop because EFI has it in graphics mode, so my text output has no effect from there on
16:52:42 <doug16k> I'm in the middle of fixing that right now, implementing virtio-gpu
16:53:21 <doug16k> that's what the screen looks like when it has successfully booted in efi :)
16:53:51 <doug16k> if you do make monitor-debug-output in the build directory in another terminal, and relaunch it, you'll see all the things happening
16:54:26 <doug16k> I'd like to know what assertion that is that failed, too
16:54:49 --- quit: Kimundi_ (Ping timeout: 240 seconds)
16:55:25 <doug16k> klange, it can't be built 32 bit. that compiler is 64 bit?
16:55:44 <doug16k> I didn't bother with a 32 bit port, though I have come close to doing it a few times :)
16:56:33 <doug16k> I should do a PAE version sometime, since it is almost the same as x86_64 paging
16:57:12 <doug16k> and the vast majority of the compiled code is written so it should work. would have to make a 32 bit interrupt dispatcher
16:57:35 <klange> doug16k: should be my system gcc, which is 6.3 targetting x86-64
16:58:22 <doug16k> ah. it almost works with system compilers. libgcc will not work since I compile the bootloader using register parameters :(
16:59:04 <klange> Your build system neither requested nor built a cross compiler, and your INSTALL file was generic and did not tell me I should provide one.
16:59:21 <doug16k> at first I was using my system compiler, and I had a workaround hack to do 64 bit divide
16:59:40 <doug16k> yeah I need to update those readmes. those are thrown in by default setup of autotools
17:00:23 <doug16k> if you use doconfigure it will run configure appropriately. I don't setup my compiler totally right yet, has to have -nostdlib
17:00:41 <doug16k> that's why I asked about sysroot the other day. to make it configure more properly
17:01:19 --- quit: zeus1 (Ping timeout: 240 seconds)
17:01:21 <doug16k> that part sucks I know. gotta get the cross compiler set up more properly with sysroot and start files before it builds really properly without hacks
17:02:10 <doug16k> glad you guys are bringing up this painful crap. I gotta get it working more elegantly
17:02:25 <doug16k> I hate it when stuff is hard to build too
17:02:33 <klys> doug16k, my qemu is still invoked thus: qemu-system-x86_64 -L /usr/share/qemu -bios OVMF.fd -drive file=/c/sys/fat32osdata.img,media=disk,index=0,format=raw,cache=none,if=ide -serial stdio -boot c
17:03:03 <klys> perhaps I should change this to accommodate
17:03:52 <doug16k> klys, if you run `make monitor-debug-output` in another terminal, then you can do something like: `make run-smp-efi-hd-fat-nvme-kvm` to launch it with everything set up
17:04:04 <klys> o cool
17:04:21 <doug16k> then you can see all the debug spam as it boots up too
17:04:38 <doug16k> and tell me what that assertion failure says :)
17:05:10 --- quit: elevated (Quit: bye)
17:06:23 <doug16k> if your ovmf isn't in the standard place, you can do: make QEMU_BOOT_efi='-bios OVMF.fd' run-smp-efi-hd-fat-nvme-kvm
17:07:14 --- join: palk (~palk@unaffiliated/palk) joined #osdev
17:07:17 <klys> mang I don't have 5 GiB free on this disk
17:07:30 <doug16k> really? :O
17:07:35 <doug16k> k 1 sec
17:08:26 <klys> the /c disk has 104 GB free
17:08:47 <doug16k> do this: touch random-mem.img && make QEMU_BOOT_efi='-bios OVMF.fd' QEMU_MEMFILL="" run-smp-efi-hd-fat-nvme-kvm
17:09:02 <klys> oh I was reading that wrong
17:09:09 <klys> no problem so far
17:09:38 <doug16k> that 5GB thing is to fill all VM memory initially with garbage to catch bugs
17:10:40 <doug16k> makes a 5GB memory image on disk and configures qemu to use it as the initial RAM contents
17:10:52 <doug16k> runs `shred` over it to fill it with random crap
17:11:20 <doug16k> 1st time, then uses it from then on. to make new random garbage, you'd delete random-mem.img and it would shred a new file
17:11:58 <doug16k> that caught several bugs for me
17:14:34 <doug16k> klange, I suppose once I get the compiler working more properly I can add something to configure to detect whether I need to use the 64 bit divide hack and get it to build with a system compiler or bad libgcc
17:14:59 <doug16k> it's close to working on a system compiler
17:16:06 <klange> idk, I'd just get it building a cross compiler
17:16:12 <doug16k> it does
17:16:13 --- quit: Benjojo (Ping timeout: 240 seconds)
17:16:27 <klange> does it? it didn't build one here, it just tried to use my system gcc to build the bootloader
17:17:09 <doug16k> just not automatically
17:17:20 <doug16k> ../dgos/toolchain/build-crossgcc.sh -a x86_64-elf -o toolchain-build -p toolchain-install -j 16
17:17:36 <doug16k> then add $PWD/toolchain-install/bin to your PATH
17:17:47 <doug16k> then ../dgos/doconfigure
17:17:51 <doug16k> then it should build fine
17:17:54 --- join: Benjojo (~sid4563@srv-69.nsa.me.uk) joined #osdev
17:18:25 <doug16k> s/16/however-many-cpus-you-have/
17:18:29 <klys> remember to nice the build script
17:18:55 <doug16k> -j 16 is muscle memory for me now :)
17:19:16 <geist> i wrote a bash function 'mj' that just figures it out for you
17:19:22 <klys> isn't there a tool called lscpu that reads cpuinfo and parses it?
17:19:36 <klys> yay geist
17:19:53 <geist> well, making that run on linux/mac/freebsd/etc is a little more work
17:19:59 <CompanionCube> lscpu is indeed a thing
17:20:03 --- quit: pie__ (Remote host closed the connection)
17:20:09 <geist> it's easy enough on linux to just count the number of instances of cpu: or something in /proc/cpuinfo
17:20:19 <geist> something like grep cpu /proc/cpuinfo | wc -l
17:20:21 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
17:20:58 <doug16k> that gives 80 for me but yeah, just figure out what to put there instead of 'cpu'
17:21:24 <geist> oh hang on, i have it here
17:21:31 <doug16k> grep bogomips /proc/cpuinfo | wc -l gave correct number for me
17:21:48 * klys reclaimed 4.8 GiB by moving the old dgos stuff off the partition
17:21:57 <geist> https://pastebin.com/HhcSxaLG
17:21:58 <bslsk05> ​pastebin.com: function makej() { case `uname` in Linux) N=`cat /proc/cpuin - Pastebin.com
17:22:52 <chrisf> is there some reason not to just use `nproc` ?
17:23:08 <geist> probably not. depends on if that's in all distros
17:23:17 <geist> i certainly was unaware of nproc until just now
17:23:23 <chrisf> it's in coreutils
17:23:33 <doug16k> chrisf, thanks
17:23:37 <geist> even better than
17:23:45 <geist> probably will just switch that
17:24:05 <doug16k> I figured there'd be a sane way, just had no clue what it was :)
17:24:09 --- join: promach_ (~promach@bb219-74-174-136.singnet.com.sg) joined #osdev
17:24:59 <doug16k> grepping and that darwin thing is a half decent fallback though
17:25:18 <geist> yah whether or not you do the *2 thing is up t you
17:25:30 <geist> that's just an old habit of mine. most of the time there's little difference, if you have the ram
17:25:47 <doug16k> *2 helps at all? I figured it would be worse
17:26:19 --- quit: jakogut (Quit: jakogut)
17:26:22 <geist> so the theory goes in the Bad Old Days of slow spinny disks, you want to at least have some extra jobs so as some of them block up on a disk there's a chance that another will fill the gap if it happens to have cached data
17:26:28 <geist> and i think there's a bit of validity to that
17:26:44 <doug16k> ah, yeah. I'm talking about fast nvme. it would be better with real I/O delays
17:26:45 --- join: jakogut (~jakogut_@162.251.69.147) joined #osdev
17:26:47 <geist> if you have a fast disk and/or stuff is in cache, then -j <nproc> is almost always totally fine
17:27:20 <geist> i fiddle with it sometimes. usually on a machine with a lot of ram there's no real difference from -j <nproc> all the way to -j <infinity>
17:27:22 <chrisf> depends where you're limited, i think.
17:27:29 <klys> doug16k, is changing QEMU_RAM supported? I got a netboot screen back from qemu.
17:27:45 <geist> but if you have a slow io subsystem then something highter than nproc can help a bit
17:28:05 <chrisf> -j this_build_doesnt_expose_enough_parallelism
17:28:18 <doug16k> klys, yeah, this should work -> make QEMU_BOOT_efi='-bios OVMF.fd' QEMU_MEMFILL="" QEMU_RAM="1G" run-smp-efi-hd-fat-nvme-kvm
17:28:22 <geist> yah. if you have a lto of ram you can do some fun stuff like make -j on say a qemu build
17:28:38 <geist> which will absolutely start thousands of compiles and chew up 20 GB ram, etc
17:28:41 <geist> it's fun to watch on top
17:29:06 <geist> for a smallish os project or whatnot, make -j is fun just for lulz (on a beefy enough machine)
17:29:11 <doug16k> geist, try that watching `sudo perf top`... it's amazing how much time gcc spends in heap and paging
17:29:16 <chrisf> 20G? are you compiling on your toaster? :P
17:29:47 <geist> i haven't really observed any total time slowdown if you just keep adding more jobs, as long as there's ram to hold all of them and a decent disk cache
17:30:05 <geist> ie, the time slicing that linux does is coarse enough that any cache eviction effects seem to be lost in the noise
17:30:17 --- quit: spare (Quit: leaving)
17:30:18 <doug16k> big compiles are basically lots of fork'ing and malloc'ing, with a pinch of actual compiles mixed in there
17:30:32 <geist> chrisf: what do you mean?
17:30:49 <geist> also holy shit the new workstations are starting to show up at work. drool: 72 threads, 192GB ram
17:30:50 --- quit: jakogut (Client Quit)
17:31:09 <klange> true story, had a PSU issue where $EMPLOYER's build was causing my dev machine to just shut off half-way through a build when too much parallelism was used.
17:31:14 --- join: jakogut (~jakogut_@162.251.69.147) joined #osdev
17:31:27 <chrisf> geist: 2x18 xeons?
17:31:41 <geist> yah
17:32:22 <chrisf> geist: what are you trying to build, chrome? :P
17:32:49 <geist> it's the standard 'high performance workstation' that chrome, android, and our team gets
17:32:57 <geist> since we do lots of local big compiles
17:33:18 <doug16k> with 72 threads, 192GB gives you 2.6GB per process on average
17:33:20 <klange> "yes, actually" also fuck building chrom(e|ium)
17:33:35 <doug16k> so 192 sounds about right
17:33:37 <klange> have to build it as part of Qt's webengine and it's fucking terrible
17:33:38 <geist> yah building chrome is gnarly
17:33:46 <geist> doug16k: yeah
17:34:05 <geist> for personal stuff i like at least 1GB per hw thread, but 2+GB per is a win
17:34:08 <chrisf> building android is fun too.
17:34:16 <geist> especially if you're doing builds that like to fire up javac or dart or whatnot
17:35:03 <klys> assert failed: boot/physmap.cc(379): result.size == size
17:36:05 <klys> last line says: mmu_init: Creating new PML4
17:37:55 --- quit: lansiir (Ping timeout: 264 seconds)
17:40:18 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
17:45:49 --- join: zeus1 (~zeus@154.226.249.25) joined #osdev
17:53:14 --- join: pie_ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
17:54:09 --- quit: pie__ (Read error: Connection reset by peer)
18:00:50 --- quit: sysfault (Quit: Leaving)
18:01:12 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
18:01:19 --- quit: m_t (Quit: Leaving)
18:04:41 --- join: nicht (~nicht@2804:7f1:2080:1ba5:b65:467a:a348:a252) joined #osdev
18:04:42 --- quit: nicht (Max SendQ exceeded)
18:05:09 --- join: nicht (~nicht@2804:7f1:2080:1ba5:b65:467a:a348:a252) joined #osdev
18:07:20 <klys> size = ( PAGE_SIZE + 4095 ) & -4096; // 0x1000
18:08:25 <klys> I cite PAGE_SIZE because no other values are passed in
18:09:53 <doug16k> klys, is that with 1G?
18:10:16 <klys> yes and 2G
18:10:23 <doug16k> EFI?
18:10:26 <klys> yes
18:11:31 <doug16k> klys, I reproduced it, thanks
18:13:52 --- join: ybyourmom (~ybintell@vampire.ertos.nicta.com.au) joined #osdev
18:15:38 <doug16k> klys, it's harmless. I removed the assert
18:15:47 <doug16k> I put that there to see if that ever happens
18:17:40 <doug16k> it didn't happen until I added ASAN, that made bss quite a bit larger and caused it to actually happen. I just updated it to omit the asan code/data when asan is off
18:18:03 <klys> did it abort anything important, perhaps
18:18:19 <doug16k> no, asserts don't abort, they just break into the debugger
18:18:22 --- join: bemeurer (~bemeurer@stdcognition.static.monkeybrains.net) joined #osdev
18:18:25 <doug16k> if there's a debugger
18:19:13 <doug16k> oh then it froze later after "mmu_init: Creating new PML4" ?
18:19:17 <doug16k> looking...
18:19:18 <klys> yes
18:21:15 <doug16k> didn't repro, it booted all the way up
18:21:34 <doug16k> wait, that's with KVM right?
18:22:17 <klys> ok yeah it has a few options to qemu that I saved
18:23:07 --- quit: bemeurer (Ping timeout: 245 seconds)
18:23:27 --- quit: pikhq (Ping timeout: 240 seconds)
18:23:32 <doug16k> worked for me without whatever you added
18:24:19 <klys> I just used what make spat out, sec, will try without -enable-kvm.
18:24:44 <doug16k> you can change -kvm at the end to -tcg and it won't use kvm
18:24:57 --- quit: pie_ (Remote host closed the connection)
18:25:07 <doug16k> i.e. run-smp-efi-hd-fat-nvme-tcg
18:25:28 --- join: pie_ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
18:26:05 <klys> It's giving me PIT Time messages this way
18:26:14 <doug16k> good, that's booted up
18:26:22 <doug16k> says shell started right?
18:26:44 <klys> shell running
18:26:49 <doug16k> ya
18:27:11 <klys> should I be able to type "ls" or something?
18:29:13 <doug16k> soon. I'm reworking the screen output significantly.
18:29:22 * klange bumps a kernel version after implementing / fixing a bunch of fs permission and syscall erno stuff
18:29:31 <doug16k> I'm mostly focusing on adding lots of hardware support
18:29:51 <klange> toaru base 1.4.0-8fefbe9 nih Jul 19 2018 10:28:11 i686
18:30:06 <doug16k> so close to virtio-gpu working. can't seem to get an IRQ out of it when it puts a descriptor index on the used ring
18:30:38 <doug16k> once that's up I'll add a display driver and get screen output unbroken
18:30:39 <klys> klange, I have toaruos building make toolchain
18:32:32 <klys> so I did git clone git://github.com/klange/toaruos
18:32:33 <bslsk05> ​klange/toaruos - Hobby kernel + userspace, built mostly from scratch. Composited GUI, dynamically linked ELF binaries, networking, Python applications, and more. (270 forks/2323 watchers)
18:32:46 <klys> .git
18:32:57 <klange> why are you building that old thing
18:33:03 <klange> did you even read the README?
18:33:13 <klys> oh? now I remember you have a new one... where?
18:33:18 <klys> ok
18:33:34 <klange> https://gitlab.com/toaruos/toaru-nih
18:33:36 <bslsk05> ​gitlab.com: ToaruOS / ToaruOS NIH · GitLab
18:34:02 <klange> the build for mainline toaruos is really messed up and convoluted
18:34:11 <klys> can it use some of this toolchain nonsense or should I clean house
18:34:22 <klys> building gcc-6.4.0 now
18:34:25 <klange> nih uses its own toolchain
18:34:33 <klys> k
18:34:35 <klange> which is just gcc and binutils, and only one of them
18:34:48 <klange> mainline toaruos uses a separate bare elf target for the kernel
18:35:12 --- quit: zeus1 (Ping timeout: 245 seconds)
18:36:03 --- join: adam4813 (uid52180@gateway/web/irccloud.com/x-zmvuwytdufvttelp) joined #osdev
18:37:15 <klys> klange, is toaru-nih gouing to build for 32 or 64-bits?
18:37:20 <klange> 32
18:37:24 <klys> coo
18:38:02 <klange> I still have a bunch of things I want to finish with this before I resurrect the 64-bit kernel project.
18:39:00 <klys> now that I have x86_64-elf-gcc... perhaps others use this
18:39:07 --- quit: palk (Read error: Connection reset by peer)
18:39:15 --- quit: promach_ (Quit: WeeChat 2.2)
18:39:31 --- join: palk (~palk@unaffiliated/palk) joined #osdev
18:41:04 <klys> gitlab.com[0: 52.167.219.168]: errno=Connection timed out
18:41:14 <klange> RIP gitlab
18:41:45 <klange> you can also clone from github http://github.com/klange/toaru-nih
18:41:45 <bslsk05> ​klange/toaru-nih - A completely-from-scratch hobby operating system: bootloader, kernel, drivers, C library, and userspace including a composited graphical UI, dynamic linker, network stack, etc. (3 forks/33 watchers)
18:42:08 <klange> or from my own server running Gogs https://git.toaruos.org/klange/toaru-nih
18:42:11 <bslsk05> ​git.toaruos.org: klange/toaru-nih: A completely-from-scratch hobby operating system: bootloader, kernel, drivers, C library, and userspace. - ToaruOS Git
18:43:02 <klys> is this all up to your recent annoucement?
18:43:50 <klange> All of these are the same thing, current HEAD of master should be 8fefbe9e73ddd0002188c682c4a646836c55e87c "With all these errno and perm fixes, let's bump kernel to 1.4.0"
18:44:13 <klys> very good
18:45:00 <klange> Just run `make`. It'll let you know if you need to install things for the toolchain to build, but you probably won't need to if you started the mainline build process already. It'll prompt you to build gcc, respond y and let it go. Since it's just a single binutils and gcc it shouldn't take too long.
18:45:15 <klange> It'll automatically complete the rest of the build after the toolchain is built.
18:46:12 <klys> the thing I wonder about "git clone," is can I change it to "git pull" later?
18:46:18 <klange> yes?
18:46:24 <klys> cool
18:47:51 <klange> I don't make changes the toolchain since my gcc patches are fairly minimal (and the libc is part of the main repo) so in theory toaru-nih won't need another toolchain build until I decide to update to a newer gcc.
18:49:01 <klys> dgos used objcopy to change a binary so it would work as BOOTX64.EFI
18:49:27 <klys> perhaps that's what's needed in some of those other cases
18:49:35 <klange> I do the same thing. objcopy ${EFI_SECTIONS} --target=efi-app-x86_64 $< $@
18:50:21 <klys> would yours work with gnu-efi 3.0.8 installed too
18:50:45 <klange> gnu-efi/stable,now 3.0.4-2 amd64
18:50:48 <klange> I would hope so.
18:51:02 <klys> :) there appear to have been some changes...
18:51:12 <klange> I don't believe they broke anything between the two.
18:51:17 <klys> k
18:53:51 <klange> I should probably check if the system compiler is sufficient for building my bootx64.efi and maybe make it optional if it's not...
18:55:03 --- quit: flacks (Ping timeout: 276 seconds)
18:56:57 --- quit: pie_ (Remote host closed the connection)
18:57:24 --- join: pie_ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
18:57:40 --- quit: Benjojo (Read error: Connection reset by peer)
18:58:06 --- join: Benjojo (~sid4563@srv-69.nsa.me.uk) joined #osdev
18:58:44 --- join: flacks (flacks@184.91.69.131) joined #osdev
19:02:50 --- join: angel0xff (~zzz@158-58-227-127.sf.ddns.bulsat.com) joined #osdev
19:07:13 --- quit: angel0xff (Ping timeout: 240 seconds)
19:19:08 --- join: zeus1 (~zeus@154.226.249.25) joined #osdev
19:19:45 <doug16k> klys, yeah I use objcopy but tell it what to remove, klange tells objcopy what to keep. roughly the same idea
19:20:10 <doug16k> mine basically says "remove everything that makes OVMF's head explode"
19:21:31 <doug16k> I suspect that OVMF rejects the whole executable when it encounters a NOLOAD section that is at an address it doesn't like. debug sections are like that
19:21:51 <doug16k> it shouldn't care where NOLOAD sections are
19:22:32 --- join: zaquest (~notzaques@5.128.210.30) joined #osdev
19:23:24 --- quit: CheckDavid (Quit: Connection closed for inactivity)
19:24:05 <doug16k> perhaps there is something about PE/COFF that prevents the section from being obviously not loaded, I forget a lot about COFF now
19:28:22 <geist> hmm, what requirements does UEFI really have from the PE binary?
19:28:33 <geist> I assume it uses the entry point, but doesn't reference the symbol table
19:28:40 <geist> i guess it has to relocate it, though i dunno how PE relocation works
19:28:52 <geist> if it's simple you can probably just manually construct a PE header and skip all the toolchain nonsense
19:29:10 <geist> but the relocation stuff is probably a deal breaker
19:30:19 --- quit: JusticeEX (Ping timeout: 240 seconds)
19:32:11 * bcos_ cheated and wrapped "flat binary" with a PE header, and used RIP-relative in the flat binary (mostly because NASM doesn't support PE)
19:32:36 <geist> yah, that's doable up to a point. inevitably you'll end up needing a relocation
19:32:56 <bcos_> ..which isn't incredibly horrible until you try to make it work for 32-bit and end up with "call getEIP -> mov ebp,[esp];ret"
19:32:57 <geist> but if you can keep it simple and/or not relocate then that's fairly easy to do. linux has a PE header inside the kernel for ARM
19:34:42 --- join: pikhq (~pikhq@c-73-181-126-9.hsd1.co.comcast.net) joined #osdev
19:38:49 --- quit: tacco| ()
19:45:15 <doug16k> it tries to load it at its natural load address but if it can't it requires the relocations
19:45:37 --- quit: nicht (Ping timeout: 245 seconds)
19:45:39 <doug16k> I load mine at 0x400000 and it ends up actually being there
19:45:53 <doug16k> link mine*
19:46:37 <doug16k> 0x400000 is about as windowsy as it gets for a load address
19:49:01 <doug16k> bcos_, you can also call .next_line ; .next_line: pop ebx to get the code address. ryzen actually optimizes that case and doesn't screw up the return address stack
19:49:09 <doug16k> no idea if intel special cases it though
19:50:29 <bcos_> Unless you're writing/compiling for one specific CPU, better to just assume you'll screw the return stack
19:51:47 <bcos_> (I cached the value in EBP most of the time though, so it probably wouldn't matter much either way)
19:52:25 <doug16k> I had to do some fun stuff to make my EFI SMP entry trampoline be position independent from real mode to protected to long mode. position independent real mode was fun :)
19:52:59 <bcos_> Position independet real mode is trivial if you use "CS relative"..
19:53:25 <doug16k> of course. can't do that to populate the GDTR and stuff
19:53:41 <bcos_> Erm?
19:54:28 --- join: spare (~user@unaffiliated/spareproject) joined #osdev
19:54:39 <bcos_> Can "lgdt [cs:0x1234]"
19:54:51 <bcos_> (if you don't want to copy CS into DS)
19:54:58 <doug16k> and what, have the value at that address miraculously be correct?
19:55:11 <bcos_> Yes
19:55:24 <doug16k> what do you think lgdt does? I have to put the base address in there
19:55:50 <bcos_> (where "miraculously" = the OS/BSP sets up values the trampoline before starting the AP)
19:56:29 <doug16k> I made it position independent instead of hardcoding crap into somewhere else in the code
19:56:49 <doug16k> it can just land on the 1st instruction with only cs loaded and just work (tm)
19:57:55 <doug16k> you can just slap it anywhere on a 4KB boundary without touching it
19:59:35 <doug16k> https://github.com/doug65536/dgos/blob/master/boot/mpentry_efi.S
19:59:37 <bslsk05> ​github.com: dgos/mpentry_efi.S at master · doug65536/dgos · GitHub
20:00:57 --- quit: pie_ (Remote host closed the connection)
20:00:58 <bcos_> Heh - almost none of that is needed..
20:01:11 <doug16k> yes it is
20:01:29 <doug16k> none of that is linked at 0
20:01:33 --- join: pie_ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
20:01:43 <doug16k> it's linked anywhere
20:03:43 <doug16k> ap_entry: could be at 0x1234 and it just works
20:04:02 --- quit: pie_ (Read error: Connection reset by peer)
20:04:04 <doug16k> when you copy it to a 4KB boundary and SIPI it
20:04:21 --- join: pie_ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
20:04:37 <bcos_> you don't need a real mode stack (you aren't using it at all), you can set CR0 to a constant (without reading the old value), the "linear address of trampoline" can be set by BSP and doesn't need to be patched in trampoline, you don't need a stack in protected mode (also not used), ...
20:05:13 <bcos_> Can bring it down to about 12 instructions (still without caring about load address)
20:05:47 <doug16k> no you can't
20:05:49 --- quit: drakonis (Remote host closed the connection)
20:06:07 <bcos_> Well, I can, but maybe you can't
20:06:18 <doug16k> what's up with you today?
20:06:26 <doug16k> condescending much?
20:06:32 <bcos_> :-)
20:08:02 --- join: lansiir (~oldtopman@unaffiliated/oldtopman) joined #osdev
20:11:37 <doug16k> I don't assume I can just stuff hardcoded values into registers. I compensate for ap_entry being nonzero for each time I access something in the trampoline, because all of the symbols in there are off by whatever address ap_entry was at. It is set up so I can put some values into the trampoline by linking them in the .smp.data section and they end up somewhere in the relocatable chunk and still be naturally accessible from other places
20:11:46 --- quit: lansiir (Read error: Connection reset by peer)
20:11:52 --- quit: sysfault (Remote host closed the connection)
20:12:21 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
20:12:55 <doug16k> I don't need to write code that explicitly writes stuff into it, it's linked there to begin with and has the values by the time the code deploys it to a page in low memory
20:13:27 <doug16k> so it's not all just stupidity
20:18:00 --- join: jack_rabbit (~jack_rabb@c-73-211-181-224.hsd1.il.comcast.net) joined #osdev
20:21:12 --- quit: jack_rabbit (Client Quit)
20:23:00 * bcos_ mostly just has: write to "I started" flag, LGDT, set CR0, load CR3, set EFER, far jump out of trampoline to 64-bit; where it all uses CS prefixes (and things like loading data seg regs, IDT, TR, RSP, ... aren't done in trampoline)
20:24:00 --- quit: zeus1 (Ping timeout: 244 seconds)
20:24:21 --- quit: sysfault (Remote host closed the connection)
20:24:22 --- quit: flacks (Ping timeout: 245 seconds)
20:24:49 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
20:25:20 --- join: Goplat (~Goplat@reactos/developer/Goplat) joined #osdev
20:27:16 --- quit: pie_ (Read error: Connection reset by peer)
20:27:21 --- join: flacks (flacks@184.91.69.131) joined #osdev
20:27:31 --- join: pie_ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
20:47:26 <buhman> I feel like I'm missing something fairly fundamental; MII defines several vendor-agnostic control registers, but nothing related to sending/recieving ethernet frames. How does this work, mechanically, other than "dma magic"?
20:49:24 <klange> MII?
20:50:54 <buhman> the 802.3u MII
20:52:28 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
20:52:35 --- quit: pie_ (Read error: Connection reset by peer)
20:52:43 <klange> Fast Ethernet?
20:53:14 <buhman> I wouldn't be asking if I was doing 10gig :)
20:53:18 <klange> Isn't that just a hardware spec? You still need to communicate with a NIC through whatever device-specific interface it has.
20:56:16 <buhman> It doesn't seem like it? The tx driver in the PHY is responsible for all signaling, and the only way that connects to the MAC is MII
20:56:16 --- quit: CrystalMath (Quit: Support Free Software - https://www.fsf.org/)
20:56:36 --- quit: nj0rd (Ping timeout: 276 seconds)
20:56:41 <klange> That's at least one layer beyond what I know network-wise, though...
20:56:58 <buhman> it seems pretty clear how PHY configuration works, but the mystical thing to me is how you actually do something with it, like make an actual ethernet frame
21:00:40 <geist> it's not hard. you just shove bytes over the MII bus and it'll slam out a packet
21:00:58 <geist> and vice versa. it tends to put the exact bytewise representation of the packet
21:01:17 <geist> also there's GMII and RGMII for gigabit eth. different signalling, but basically the same thing
21:02:57 <geist> the ethernet MAC then deals with decoding the eth frame, doing the checksum, and dealing with the cpu/memory interface
21:03:09 <geist> that's the register level stuff that you commonly see as an 'ethernet controller' or whatnot
21:04:59 --- quit: pie__ (Read error: Connection reset by peer)
21:05:37 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
21:08:13 --- quit: flacks (Ping timeout: 240 seconds)
21:08:28 --- join: jack_rabbit (~jack_rabb@c-73-211-181-224.hsd1.il.comcast.net) joined #osdev
21:10:17 --- join: xenos1984 (~xenos1984@22-164-191-90.dyn.estpak.ee) joined #osdev
21:10:18 --- join: nj0rd (~nj0rd@200116b84534b5004b34fa64f20861d2.dip.versatel-1u1.de) joined #osdev
21:15:38 <buhman> hmm, it seems like I can't directly write on the MII bus with this mac; it has these non-memory-mapped "transmit/recieve descriptors" that I'm supposed to access with DMA exclusively, I guess
21:16:05 --- join: flacks (flacks@184.91.69.131) joined #osdev
21:17:30 <buhman> (where the fully-formed ethernet frame is dereferenced from the descriptor)
21:17:57 <buhman> geist: does that sound like how a MAC might actually work, or am I not understanding this correctly?
21:21:18 --- quit: freakazoid0223 (Read error: Connection reset by peer)
21:23:49 <buhman> I think this is what I'd ultimately want, I just wanted to avoid learning how the DMA block and MAC block worked at the same time.
21:24:06 --- join: zeus1 (~zeus@154.226.249.25) joined #osdev
21:32:47 <klange> I feel like I'm sitting here listen to someone contemplate fuel injection and how do get fuel into a car, while I think "you put the gas in the tank and the car does the rest."
21:34:19 --- quit: pie__ (Read error: Connection reset by peer)
21:34:26 --- join: pie___ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
21:41:36 --- join: zeus2 (~zeus@154.226.249.25) joined #osdev
21:42:16 --- quit: zeus1 (Read error: Connection reset by peer)
21:47:46 <klys> klange: genext2fs -B 4096 -d base -D util/devtable -U -b `util/calc-size.sh` -N 2048 fatbase/ramdisk.img; ... objcopy -j .text -j .sdata -j .data -j .dynamic -j .dynsym -j .rel -j .rela -j .reloc --target=efi-app-x86_64 boot/efi64.so fatbase/efi/boot/bootx64.efi util/mkdisk.sh cdrom/fat.img fatbase; fallocate: fallocate failed: Operation not supported; mkfs.fat 4.1 (2017-01-24); mkfs.fat: unable to discover size of cdrom/fat.img; init :: could not read boot secto
21:48:52 --- quit: sysfault (Remote host closed the connection)
21:49:18 <klange> klys: uh, what filesystem are you using?
21:49:24 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
21:49:41 <klys> that looks like a FAT failure from make
21:49:44 <klange> no
21:49:46 <klange> fallocate failed
21:50:00 <klys> I'm upgrading util-linux now
21:50:13 <klange> then mkfs.fat was like "uh, there's no file here? dunno how big you want this filesystem to be"
21:50:30 --- join: pie__ (~pie_@unaffiliated/pie-/x-0787662) joined #osdev
21:50:38 --- quit: pie___ (Read error: Connection reset by peer)
21:50:45 <klange> like what filesystem did you check out the tree into? because apparently it doesn't support fallocate
21:50:53 <klange> I can dd from /dev/zero instead...
21:50:56 <klys> oh I'm building on ntfs
21:51:01 <klange> ewwwww
21:51:10 <klys> ran out of room on ext2, so...
21:51:12 <klange> wsl?
21:51:15 <klange> or...
21:51:22 <klange> wait did you check out my repo into an NTFS partition?
21:51:27 <klys> yes
21:51:30 <klange> please do not do that
21:52:03 <klys> the obvious argment ensues...
21:52:19 <klange> there is no obvious argument here, you checked out a git repo into a filesystem that doesn't support Unix permission bits
21:53:09 <klys> I'll lug over that usb thing and have a nice day.
21:55:51 <klange> anyway the failure here is because fallocate doesn't work on NTFS
21:56:15 <klange> it was being used to creaste an appropriately sized blank image to build the FAT into
21:57:05 <klange> I'm not sure if I want to fix that for the NTFS case.
21:57:41 <klange> If some actually-viable filesystem has that issue... which I doubt... I would fall back to the old method of dd'ing from /dev/null to make a blank file.
22:02:34 --- join: Lowl3v3l (~Lowl3v3l@ulbp2362.ulb.uni-jena.de) joined #osdev
22:08:57 --- quit: pie__ (Ping timeout: 245 seconds)
22:13:55 <buhman> oh
22:14:28 <buhman> I did not realize SMI is distinct from MII
22:18:07 --- quit: flacks (Ping timeout: 245 seconds)
22:19:22 --- join: MDude (~MDude@c-73-187-225-46.hsd1.pa.comcast.net) joined #osdev
22:23:02 --- join: flacks (flacks@184.91.69.131) joined #osdev
22:23:56 <Sjors> <klys> oh I'm building on ntfs
22:24:00 <Sjors> I too like to live dangerously
22:24:11 <Sjors> sometimes you just have to beat the odds
22:25:18 <klange> klys: try replacing fallocate with the appropriate dd in util/mkdisk.sh and see what happens, but I suspect you may have problems with the ramdisk genext2fs built
22:25:24 <klys> this move...what's gcc/gcc/lto1 and gcc/gcc/go1 ?
22:26:08 <klys> ...removing old files...
22:27:13 <klange> Are you now trying to move the repository, including the gcc you built, to another path? gcc's going to be really unhappy with that
22:27:56 <klange> One does not simply move gcc. Too many embedded paths, especially the sysroot in this case, but it also gets ornery about its own system files.
22:28:15 <klys> no, fallocate simply again fails to work on ext3fs.
22:28:19 --- join: allight__ (allight@nat/google/x-mnrtijjvrawhvrez) joined #osdev
22:28:22 <klange> fallocate failing on ext3?
22:29:04 <klange> wait, why are you using ext3?
22:29:05 <geist> buhman: yeah, you usually dont get direct access to the MII bus, except the registers you can access on the side
22:29:24 <klys> util/mkdisk.sh cdrom/fat.img fatbase
22:29:25 <klys> fallocate: fallocate failed: Operation not supported
22:29:30 --- join: adam4813_ (uid52180@gateway/web/irccloud.com/x-fomjqvxzvgvvoaoe) joined #osdev
22:29:33 --- quit: adam4813 (Ping timeout: 276 seconds)
22:29:33 --- quit: sircmpwn (Ping timeout: 276 seconds)
22:29:33 --- quit: pixelherodev (Ping timeout: 276 seconds)
22:29:35 --- nick: adam4813_ -> adam4813
22:29:35 <klys> oh ext4 is too fancy
22:29:37 <geist> iirc there's some little protocol for reading registers there for configuration. some ethernet controllers expose it directly, hence why you get PHY drivers in linux or whatnot
22:29:38 --- join: Affliction` (affliction@2402:1f00:8101:135::) joined #osdev
22:29:44 <klange> you moved from an unsuitable FS to an outdated one
22:29:45 --- join: shakesoda_ (uid36406@gateway/web/irccloud.com/x-yfefmmkfxtrhytbm) joined #osdev
22:29:46 --- join: stux- (stux@cosmo.lunarshells.com) joined #osdev
22:29:46 <geist> or some handle it entirely within the controller
22:29:53 <klange> fucking fine I'll make it use dd
22:30:00 <klys> don't work with my old kernels, etc.
22:30:11 <klange> HOW FUCKING OLD IS YOUR KERNEL?
22:30:26 <klange> ext4 has been supported as stable for A DECADE
22:30:51 <klys> you'll be proud to know I gave my nephew all debian releases since potato on his 12th b-day
22:30:51 --- quit: nj0rd (Ping timeout: 276 seconds)
22:30:52 --- quit: allight_ (Ping timeout: 276 seconds)
22:30:54 --- quit: ybden (Ping timeout: 276 seconds)
22:30:54 --- quit: stux (Ping timeout: 276 seconds)
22:30:55 <geist> i thought he was on ntfs
22:31:05 <klange> He was, but now he's on ext3!
22:31:16 --- quit: shakesoda_ (Changing host)
22:31:17 --- join: shakesoda_ (uid36406@celestiaradio/shark/soda) joined #osdev
22:31:19 <geist> huh. and that driver doesn't have fallocate huh
22:31:27 --- quit: shakesoda (Disconnected by services)
22:31:29 --- join: sferrini_ (sid115350@gateway/web/irccloud.com/x-uqerhncypeqwkwvm) joined #osdev
22:31:31 --- quit: hunterlabs (Ping timeout: 276 seconds)
22:31:32 --- quit: Affliction (Ping timeout: 276 seconds)
22:31:32 --- quit: mpetch (Ping timeout: 276 seconds)
22:31:33 --- quit: sferrini (Ping timeout: 276 seconds)
22:31:35 --- quit: glenda (Ping timeout: 276 seconds)
22:31:35 --- quit: m712 (Ping timeout: 276 seconds)
22:31:35 --- quit: PyroLagus (Ping timeout: 276 seconds)
22:31:39 --- nick: shakesoda_ -> shakesoda
22:31:41 <klys> "extents." I have no clue what they are.
22:31:42 <geist> note that you can use ext4 driver against an ext3 fs, i believe
22:32:02 <geist> extents are one of the primary physical differences between ext3 and 4
22:32:12 --- join: mpetch (~mpetch@vps2.gnubg.com) joined #osdev
22:32:14 <klys> yeah I know that much ofc.
22:32:38 <geist> it switches the way inodes store file location to a list of runs of blocks
22:32:40 <geist> instead of a table
22:32:41 <klys> time for mount -t ext4, we'll see
22:32:47 <klange> extents are continuous blocks so that large files can be read in one go
22:32:50 <geist> also iirc ext4 driver can mount ext3
22:33:07 <geist> one of the bigger changes with the driver i remember was it switched to delayed allocation. which is generally a big win
22:33:31 <geist> since that has a lot to do with the path the allocator works in, it may be that fallocate() only works internally against delayed allocation fs drivers or something
22:33:43 <klys> /dev/sdb4 on /e type ext4 (rw,relatime,data=ordered)
22:33:47 <klange> The ext4 driver was kinda special because it was a complete rewrite, while the ext3 one was an extension of the ext2 one.
22:33:49 --- quit: Xal (Ping timeout: 240 seconds)
22:33:49 <klys> fallocate fails again.
22:34:06 <klange> klys: because it's still an ext3 filesystem
22:34:10 --- join: ybden (ybden@coleridge.vehk.de) joined #osdev
22:34:11 <geist> yeah but is it actually 4 or 3?
22:34:12 <klys> true
22:34:19 --- quit: Goplat (Ping timeout: 240 seconds)
22:34:27 <klange> klys: just pull the repo and it'll use dd instead
22:34:34 <klys> perhaps I should make a loopback ext4 for ye
22:34:37 --- nick: ybden -> Guest30994
22:34:58 <klys> or what I'd rather do, view the patch.
22:35:02 <geist> klange: is it really important that it be allocated up front? just setting the file length and letting it be sparse isn't okay?
22:35:05 <klange> I try to use the current best practice for generating a big empty file and of course *someone* comes along with a filesystem that doesn't support it...
22:35:10 <bcos_> Surely (for portable code) you'd use the feature test macro for "posix_fallocate()" and then fall back to boring old "seek to end-1 and write a byte"
22:35:25 <klange> bcos_: the fuck do I do that in a shell script
22:35:42 <bcos_> Oh, shell script?
22:35:53 --- join: m712 (~annoying@unaffiliated/thefam) joined #osdev
22:36:07 --- join: PyroLagus (PyroLagus@i.have.ipv6.on.coding4coffee.org) joined #osdev
22:36:10 --- join: glenda (~glenda@tunnel272116-pt.tunnel.tserv29.fmt1.ipv6.he.net) joined #osdev
22:36:12 <bcos_> ..maybe write a "myfallocate" utility.. :-)
22:36:12 <klys> hmm, this will require some deliberation it seems.
22:36:13 --- join: hunterlabs (Elite20801@gateway/shell/elitebnc/x-xwyapyybrtarlzzn) joined #osdev
22:36:19 <klange> klys: just update the damn repo!
22:36:25 <geist> i'd also point out that none of this is particularly portable to non linux, but i guess since it's futzing around with fs images it's likely to be highly linux specific, right?
22:36:46 <geist> vs say OSX or freebsd or whatnot
22:36:50 --- join: Xal (~Xal@S010664777dabacc3.vw.shawcable.net) joined #osdev
22:37:28 <klys> ohmies, klange is focussing on me instead of his job. well, I would like apartial update then. I did a clone earlier.
22:37:46 <klange> Do you not know how to use git?
22:38:03 <klys> I should not anger you
22:38:04 --- join: pixelherodev (znc@irc.sircmpwn.com) joined #osdev
22:38:13 --- quit: flacks (Quit: Leaving)
22:38:36 <geist> you nave enraged teh klange
22:38:39 <geist> everyone run!
22:38:51 <geist> throw the weak ones into the volcano
22:39:06 <klys> I've just been laughing for mebby five minutes. no pain no gain, it seems
22:39:16 <klange> I don't accept volcano sacrifices.
22:39:56 <klange> klys: But seriously, do you not know how to use git? Running `git pull` in the repository will pull the newest changes to your current branch.
22:41:00 --- quit: leinne (Remote host closed the connection)
22:41:26 <klys> you were right about something else. bbl.
22:42:01 --- join: flacks (flacks@184.91.69.131) joined #osdev
22:42:07 --- join: leinne (~leinne@gateway/tor-sasl/leinne) joined #osdev
22:43:57 --- join: awordnot (~awordnot@c-73-210-61-59.hsd1.il.comcast.net) joined #osdev
22:44:18 --- join: sircmpwn (znc@irc.sircmpwn.com) joined #osdev
22:44:23 <klange> I'm actually surprised that fallocate(1) doesn't work on FSes that don't support fallocate(2) as its manual doesn't suggest that - it just says that FSes that *do* support it will be "fast".
22:44:24 --- join: nj0rd (~nj0rd@200116b84534b5004b34fa64f20861d2.dip.versatel-1u1.de) joined #osdev
22:47:57 <bcos_> klange: Did you give it an "-x" or "--posix" parameter?
22:48:23 <clever> klange: ive found that the truncate function on darwin, will allocate the entire file
22:48:43 <bcos_> ("Enable POSIX operation mode. In that mode allocation operation always completes, but it may take longer time when fast allocation is not supported by the underlying filesystem.")
22:49:25 <klange> There are no such arguments listed in the manual page of my system's fallocate...
22:49:40 <klange> which claims to come from util-linux... manpage last modified april 2014
22:49:40 * bcos_ is looking at: http://man7.org/linux/man-pages/man1/fallocate.1.html
22:49:40 <bslsk05> ​man7.org: fallocate(1) - Linux manual page
22:50:10 <klange> fallocate: invalid option -- 'x'
22:50:11 <clever> i see it in my system, which also claims april 2014
22:50:28 <klange> So did Debian patch it out, is my util-linux outdated and did they never bother updating the manpage timestamp...
22:50:28 <bcos_> Seems like an incredibly silly default (a "non-standard let's be broken by default and have an option to do the right thing")
22:50:32 <clever> klange: what year does your fallocat manpage have at the bottom?
22:50:40 <clever> $ fallocate --version
22:50:40 <clever> fallocate from util-linux 2.32
22:50:50 <klange> fallocate from util-linux 2.29.2
22:51:47 <klange> https://github.com/karelzak/util-linux/blame/master/sys-utils/fallocate.c#L98
22:51:48 <klange> Interesting.
22:51:49 <bslsk05> ​github.com: util-linux/sys-utils/fallocate.c at master · karelzak/util-linux · GitHub
22:51:54 <klange> Maybe I should just stick to dd'ing from /dev/zero
22:52:51 <bcos_> Hrm - "man fallocate" (from 2011!) doesn't have it on my system
22:53:23 <klange> interestingly, I have man pages for posix_fallocate
22:53:45 <klange> my Ubuntu has the -x option
22:53:48 * bcos_ wonders how old fallocate is - whether Linux cowboys slapped together their own without bothering with any formal standardisation process, and then POSIX standardised it in an incompatible way after that
22:54:12 <klange> probably, given that the posix version had to slap posix_ on the front.
22:56:08 <klange> Interestingly I don't see it being patched *out* in Debian stretch, so are they just building it in an environment that doesn't have posix_fallocate?
23:00:57 --- quit: zeus2 (Ping timeout: 255 seconds)
23:02:11 <klys> local/local
23:03:00 <klys> hmm bcos_
23:03:26 <klys> I liked how you said you used nasm for efi
23:04:21 <klys> well I suupose I need the efi instructions for toaru-nih first
23:04:33 <bcos_> Not sure I'd recommend it for most people..
23:04:36 <klange> what do you mean by instructions?
23:05:00 <klys> well this all needs to be documented
23:05:07 <klange> what needs to be documented?
23:05:21 <klys> what can be done, I suppose
23:09:21 --- join: quc (~quc@host-89-230-164-98.dynamic.mm.pl) joined #osdev
23:11:53 --- quit: sysfault (Remote host closed the connection)
23:12:29 --- join: sysfault (~exalted@pool-108-53-157-115.nwrknj.fios.verizon.net) joined #osdev
23:17:13 --- join: bemeurer (~bemeurer@c-24-6-228-111.hsd1.ca.comcast.net) joined #osdev
23:17:57 --- join: manzerbredes (~loic@myriads-lg.irisa.fr) joined #osdev
23:21:37 <klys> hmm efi works like a charm. thanks, klange!
23:21:55 * geist cheers for klange
23:22:45 <geist> clever: re: truncate on darwin. HFS+ traditionally didn't support sparse files I believe
23:22:52 <geist> so it always allocated the entire file on resize
23:22:59 <geist> APFS though i think can do sparse
23:23:08 <clever> geist: and it will zero out every single block on-disk
23:23:23 <clever> geist: i take advantage of this "feature" to zero unused space, so zfs can compress the disk image better
23:23:25 <klange> Linux's ftruncate indicates it's not producing sparse files by default, it's allocate blocks but just not clearing them
23:23:32 <geist> yah, that's the only real way to do it, except for storing the run + 'unitialized' flag
23:24:03 <geist> well, responding to clever there. ftruncate usually makes a sparse file
23:24:21 <geist> fallocate however does force an allocation. based on the feature it may or may not need to blat out a big pile of zeros though
23:24:38 <geist> it's all verfiable with filefrag -v
23:24:43 <geist> very informative
23:24:56 <clever> though zfs doesnt support filefrag
23:25:13 <geist> many fses, like ext4, can either store a sparse run or an allocated run with the 'uninitialized' bit set
23:25:27 <geist> which allows it to allocate a run but not need to write zeros
23:25:58 <clever> my gut says zfs would just reserve the free space, but not actually select blocks for the allocation
23:26:07 <geist> NTFS has had this forever in a smpler form: the file has a length and an unitialized length which is always <= length
23:26:24 <geist> that way ou can create a large file and allocate it instantly without the zeroing effect
23:26:35 <geist> s/uninitialized/initialized
23:27:06 <klys> bcos_ is sneaky enough that I can't find your efi code
23:27:30 <geist> filefrag -v and btrfs is fun too: you can see if a run is shared between files
23:27:42 <geist> and observe the COW behavior as you snapshot stuff and fiddle with it
23:27:56 --- quit: flacks (Ping timeout: 244 seconds)
23:28:15 <geist> and since there's a dedup ioctl() you can go in and dedup stuff and verify it with filefrag
23:29:57 --- quit: iknosis (Ping timeout: 240 seconds)
23:30:34 --- join: flacks (flacks@184.91.69.131) joined #osdev
23:32:27 --- join: iknosis (~iknosis@185.183.104.140) joined #osdev
23:33:54 --- quit: bemeurer (Quit: WeeChat 2.1)
23:35:29 <klys> wauw http://web.archive.org/web/20161013052431/http://bcos.hopto.org/log.html
23:35:35 <bslsk05> ​bcos.hopto.org: Project Log
23:41:12 --- quit: leinne (Disconnected by services)
23:41:33 --- join: leinne_ (~leinne@gateway/tor-sasl/leinne) joined #osdev
23:42:21 <geist> yeah bcos_ has this whole thing about protecting his source
23:42:45 <klys> I should ask mischief about some compilation or something
23:43:22 <geist> if you're looking for another opinion we have an EFI loader in zircon too
23:43:57 <geist> source is mostly under https://fuchsia.googlesource.com/zircon/+/master/bootloader/
23:43:58 <bslsk05> ​fuchsia.googlesource.com: bootloader - zircon - Git at Google
23:44:02 --- quit: quc (Remote host closed the connection)
23:44:15 <klys> mischief, /root/src/sys/efivim/edk2/AppPkg/Applications/Lua/src/ldo.c:122:1: error: ‘noreturn’ function does return [-Werror]
23:45:04 --- join: zeus2 (~zeus@160.119.149.222) joined #osdev
23:45:11 <klange> "GigaBoot 20X6"
23:45:25 <geist> yah
23:46:20 <klys> sjors, what'ya think of that
23:46:22 <bcos_> klys: https://pastebin.com/q5MjJ4a5
23:46:23 <bslsk05> ​pastebin.com: [C] //"Fix EFI" EFI Application File Format Fixer //fixefi //gcc -g -Wall -Wextra - Pastebin.com
23:46:29 <klys> wauw
23:46:35 <klys> ``````````````thanks bcos_
23:46:35 <bcos_> ^ source for ugly "slap PE headers on flat binary" tool
23:49:37 --- join: onecoder779981 (~onecoder7@103.29.141.158) joined #osdev
23:50:02 <mischief> klys: what is the problem?
23:50:15 <mischief> not sure i use Werror
23:51:40 <klys> mischief, I just have to be sure I'm editing the right Makefile
23:52:16 --- join: fornew (~wherenew@120.78.87.130) joined #osdev
23:54:04 --- join: grouse (~grouse@83-233-9-2.cust.bredband2.com) joined #osdev
23:54:32 --- part: onecoder779981 left #osdev
23:54:37 --- quit: bauen1 (Remote host closed the connection)
23:54:43 <mischief> klys: what is it you're building exactly?
23:55:08 <klys> efivim/edk2, so edk2 I suppose
23:55:24 <mischief> doesn't it build already?
23:55:29 <klys> naw
23:56:06 <mischief> i don't understand, efivim works here. what's the problem?
23:56:20 <klange> Sounds like a build issue with Lua.
23:57:04 <mischief> in edk2 you don't really want to edit makefiles
23:57:15 <mischief> edk2 has this incorrigible build system that generates makefiles itself
23:57:18 <klys> there aren't any, it looks like
23:57:20 <mischief> so you have to deal with that
23:59:03 <mischief> ah, i see now
23:59:08 <mischief> my gcc flags use Werror
23:59:16 <mischief> that triggers in edk2 lua for you?
23:59:23 <klys> pulling
23:59:59 --- log: ended osdev/18.07.18