channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev&y=18&m=5&d=13

Sunday, 13 May 2018

00:00:00 --- log: started osdev/18.05.13
00:06:16 --- quit: glenda (Ping timeout: 250 seconds)
00:14:05 --- join: SGautam (3bb6f9ab@gateway/web/cgi-irc/kiwiirc.com/ip.59.182.249.171) joined #osdev
00:15:14 --- join: glenda (~glenda@216.252.162.212) joined #osdev
00:17:49 --- quit: NaNkeen (Ping timeout: 240 seconds)
00:27:34 --- nick: r3n_ -> r3n
00:28:30 --- join: CheckDavid (uid14990@gateway/web/irccloud.com/x-tzulaxrzplsraonl) joined #osdev
00:29:19 --- quit: masoudd (Ping timeout: 240 seconds)
00:31:10 --- quit: Halofreak1990 (Ping timeout: 255 seconds)
00:31:25 --- join: Halofreak1990 (~FooBar247@5ED0A537.cm-7-1c.dynamic.ziggo.nl) joined #osdev
00:40:09 --- join: quc (~quc@host-89-230-168-176.dynamic.mm.pl) joined #osdev
00:43:39 --- join: masoudd (~masoudd@5.115.237.66) joined #osdev
00:52:14 --- quit: dbittman_ (Ping timeout: 265 seconds)
01:18:15 --- join: vdamewood (~vdamewood@unaffiliated/vdamewood) joined #osdev
01:19:35 --- quit: SGautam (Remote host closed the connection)
01:20:44 --- join: fddsafsd (~sdfgsdfg@176.33.236.80) joined #osdev
01:20:59 --- join: w41 (~w41@unaffiliated/w41) joined #osdev
01:21:17 --- join: Kimundi__ (~Kimundi@i577A9E33.versanet.de) joined #osdev
01:22:37 --- quit: vdamewood (Client Quit)
01:23:43 --- join: vdamewood (~vdamewood@unaffiliated/vdamewood) joined #osdev
01:30:31 --- quit: AverageJ0e (Ping timeout: 260 seconds)
01:34:19 --- quit: masoudd (Ping timeout: 240 seconds)
01:34:47 --- part: dograt left #osdev
01:36:39 --- quit: zeus1 (Ping timeout: 248 seconds)
01:39:57 --- join: msk (~mskreczko@sub243-90.elpos.net) joined #osdev
01:45:24 --- join: Asu (~sdelang@AMarseille-658-1-167-153.w86-198.abo.wanadoo.fr) joined #osdev
01:51:11 --- quit: hmmmm (Remote host closed the connection)
01:53:56 --- join: light2yellow (~l2y@185.220.70.174) joined #osdev
01:56:55 --- join: booyah (~bb@193.25.1.157) joined #osdev
01:57:54 --- quit: Guest65529 (Quit: Leaving)
01:59:54 --- quit: msk (Remote host closed the connection)
02:06:54 --- join: glauxosdever (~alex@ppp-94-66-39-211.home.otenet.gr) joined #osdev
02:07:10 --- join: zeus1 (~zeus@197.239.5.87) joined #osdev
02:10:22 --- join: deyzie (65b0e863@gateway/web/freenode/ip.101.176.232.99) joined #osdev
02:12:43 --- join: NaNkeen (~nankeen@161.142.119.190) joined #osdev
02:16:19 --- quit: aalm (Ping timeout: 240 seconds)
02:16:49 --- quit: oaken-source (Ping timeout: 240 seconds)
02:18:12 --- quit: deyzie (Quit: Page closed)
02:18:24 --- quit: w41 (Ping timeout: 268 seconds)
02:21:20 --- quit: cantbetouched (Ping timeout: 255 seconds)
02:27:43 --- quit: MarchHare (Ping timeout: 255 seconds)
02:31:21 --- quit: ACE_Recliner (Remote host closed the connection)
02:35:14 --- join: nortega (~nortega@gateway/tor-sasl/deathsbreed) joined #osdev
02:38:33 --- join: Asd_ (b036a381@gateway/web/freenode/ip.176.54.163.129) joined #osdev
02:41:05 --- quit: Asd_ (Client Quit)
02:47:31 --- quit: tomaw (Ping timeout: 615 seconds)
02:51:59 --- join: daniele_athome (~daniele_a@5.170.120.81) joined #osdev
02:58:50 --- join: KernelBloomer (~SASLExter@gateway/tor-sasl/kernelbloomer) joined #osdev
02:58:51 --- join: aalm (~aalm@37-219-119-194.nat.bb.dnainternet.fi) joined #osdev
03:01:39 --- quit: Asu (Ping timeout: 246 seconds)
03:06:11 --- join: Asu (~sdelang@122.43.136.77.rev.sfr.net) joined #osdev
03:07:51 --- quit: nortega (Quit: Vivu lante, vivu feliĉe!)
03:09:10 --- join: sortie (~sortie@static-5-186-55-44.ip.fibianet.dk) joined #osdev
03:09:23 --- join: nortega (~nortega@gateway/tor-sasl/deathsbreed) joined #osdev
03:10:25 --- join: cantbetouched (~SASLExter@gateway/tor-sasl/kernelbloomer) joined #osdev
03:13:32 --- quit: KernelBloomer (Ping timeout: 255 seconds)
03:18:12 --- quit: FreeFull (Quit: rebooting)
03:20:40 --- quit: NaNkeen (Ping timeout: 256 seconds)
03:21:07 --- join: FreeFull (~freefull@defocus/sausage-lover) joined #osdev
03:24:53 --- join: naortega (~nortega@gateway/tor-sasl/deathsbreed) joined #osdev
03:27:02 --- quit: nortega (Ping timeout: 255 seconds)
03:32:18 --- quit: Asu (Remote host closed the connection)
03:33:11 --- nick: naortega -> nortega
03:41:23 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
03:41:43 --- quit: ALowther (Remote host closed the connection)
03:51:27 --- quit: daniele_athome (Ping timeout: 240 seconds)
03:56:13 --- join: daniele_athome (~daniele_a@5.170.124.122) joined #osdev
03:56:55 --- quit: immibis (Ping timeout: 248 seconds)
04:03:54 --- quit: fddsafsd (Read error: Connection reset by peer)
04:14:36 --- join: NaNkeen (~nankeen@123.136.118.234) joined #osdev
04:17:20 --- join: dennis95 (~dennis@mue-88-130-61-093.dsl.tropolys.de) joined #osdev
04:20:08 --- quit: glauxosdever (Quit: leaving)
04:21:39 --- join: glauxosdever (~alex@ppp-94-66-39-211.home.otenet.gr) joined #osdev
04:26:18 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
04:29:54 --- quit: nortega (Quit: Vivu lante, vivu feliĉe!)
04:40:51 --- quit: daniele_athome (Read error: Connection reset by peer)
04:46:31 --- join: geordi (~geordi@ns510081.ip-192-99-4.net) joined #osdev
04:47:38 --- quit: vdamewood (Quit: Textual IRC Client: www.textualapp.com)
04:47:42 --- join: daniele_athome (~daniele_a@5.170.124.122) joined #osdev
04:50:38 --- quit: geordi (Remote host closed the connection)
04:51:06 --- join: geordi (~geordi@ns510081.ip-192-99-4.net) joined #osdev
04:54:41 --- quit: daniele_athome (Ping timeout: 260 seconds)
04:56:08 --- quit: ALowther (Remote host closed the connection)
04:56:41 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
05:00:49 --- quit: ALowther (Ping timeout: 240 seconds)
05:04:16 --- join: ljc (~ljc@unaffiliated/ljc) joined #osdev
05:12:11 --- join: daniele_athome (~daniele_a@5.170.125.165) joined #osdev
05:14:14 --- join: spare (~user@unaffiliated/spareproject) joined #osdev
05:28:07 --- quit: samirhali (Ping timeout: 248 seconds)
05:30:48 --- join: tomaw (tom@freenode/staff/tomaw) joined #osdev
05:33:30 --- join: samirhali (~samirhali@80.211.230.116) joined #osdev
05:35:52 --- quit: ljc (Quit: ayy)
05:46:34 --- join: CrystalMath (~coderain@reactos/developer/theflash) joined #osdev
05:51:47 --- quit: daniele_athome (Read error: Connection reset by peer)
05:52:27 --- join: tacco| (~tacco@i59F5F85A.versanet.de) joined #osdev
05:59:01 --- join: t3hn3rd (~Logic_Bom@cpc91224-cmbg18-2-0-cust219.5-4.cable.virginm.net) joined #osdev
05:59:09 <t3hn3rd> Hey all \o_
06:00:08 --- quit: zeus1 (Ping timeout: 264 seconds)
06:00:09 <t3hn3rd> Anyone any idea why no ICMP packet would be generated from a test_machine->My_OS_Machine when the IP->MAC for My_OS_Machine exists in the ARP Cache? Have wireshark running and no ICMP packets are being sent at all.
06:01:07 <lkurusa> What ICMP packet are you expecting?
06:02:32 <t3hn3rd> Well from my test machine I am attempting to ping my OS Machine to check if my implementation of ICMP is working, but the test machine doesn't generate an icmp 0x8 packet at all, just says host unreachable, despite the target machine existing in the ARP Cache
06:03:50 <lkurusa> is there a path in the routing table on your host machine to the OS machine
06:04:42 <t3hn3rd> doesn't seem like it
06:04:55 --- quit: NaNkeen (Ping timeout: 248 seconds)
06:05:28 <lkurusa> add a route in your routing table with the gateway being the MAC of your OS machine
06:05:31 <lkurusa> assuming it’s the same LAN that should work
06:05:42 --- quit: Philpax_ (Ping timeout: 256 seconds)
06:05:42 <t3hn3rd> Any idea why this isn't happening automatically?
06:06:35 <lkurusa> is the IP you gave to your OS machine via DHCP?
06:06:36 <lkurusa> or via static IP/
06:06:39 --- join: daniele_athome (~daniele_a@5.170.124.175) joined #osdev
06:06:43 <t3hn3rd> static
06:07:16 <lkurusa> is it within the same mask as the host machine’s netmask?
06:07:23 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
06:07:31 <t3hn3rd> Yeah, both machines are on a /24 network 255.255.255.0
06:07:49 --- join: adam4813 (uid52180@gateway/web/irccloud.com/x-bslmzzjptfnezjws) joined #osdev
06:07:57 <t3hn3rd> Gateway is 192.168.1.1, My test machine is 192.168.1.128, my OS Machine is 192.168.1.243
06:08:08 <lkurusa> hm, then I think it should be working - assuming your ARP replies are correct
06:08:59 <t3hn3rd> https://i.gyazo.com/101f4e69579976ae0b6065fba15ceb0a.png
06:10:02 <lkurusa> ah windows :-)
06:10:04 <lkurusa> yeah that looks correct
06:10:16 <t3hn3rd> Aha, have tested in linux too :P
06:10:29 <t3hn3rd> Even used arping to check, and the replies came back OK
06:10:38 <lkurusa> if you think your host machine doesn’t even try an ICMP request then that probably implies that it doesn’t know a path to the IP
06:11:06 <lkurusa> did adding to the routing table do nothing useful?
06:11:18 <t3hn3rd> Could it be due to the fact that the OS Machine is a Virtualbox Guest using a Bridged Adapter?
06:12:07 --- quit: ALowther (Ping timeout: 264 seconds)
06:12:59 <lkurusa> I think that’s just like a “real” machine, so that shouldn’t make a difference
06:13:10 <t3hn3rd> But saying that, I'm able to ping my Ubuntu environment, and that's also a Guest using a bridged adapter
06:13:15 <t3hn3rd> That's what I thought :/
06:14:58 <lkurusa> just to be sure - are you bridging the correct adapter?
06:15:33 <t3hn3rd> I am, yeah.
06:16:40 <lkurusa> can you paste your routing table here?
06:18:43 --- join: Shikadi (~Shikadi@cpe-98-10-34-205.rochester.res.rr.com) joined #osdev
06:19:15 <t3hn3rd> Sure thing
06:19:22 --- join: lachlan_s (uid265665@gateway/web/irccloud.com/x-jxmjjvlfmzwwnksp) joined #osdev
06:19:35 <lachlan_s> Morning everyone
06:19:59 <t3hn3rd> https://i.gyazo.com/5f510f3c5ef7b74a1c7f76e6bffd390a.png
06:20:26 --- join: deyzie (65b0e863@gateway/web/freenode/ip.101.176.232.99) joined #osdev
06:20:27 <aalm> 16:20, not so morning here anymore
06:20:31 <t3hn3rd> just added it to the table for that screenshot
06:21:49 <t3hn3rd> It's just annoying me that it's not automatically finding the route...
06:27:46 --- join: JonRob (jon@gateway/vpn/privateinternetaccess/jonrob) joined #osdev
06:29:55 <lkurusa> that routing table is odd
06:30:06 <lkurusa> why .243 sent to .1 ?
06:30:20 <lkurusa> that’s your router
06:30:28 <lkurusa> you want to send .243 to the link
06:32:54 <t3hn3rd> have changed that, realised it was wrong
06:33:06 <t3hn3rd> but still no packet generated
06:33:40 <t3hn3rd> https://i.gyazo.com/291dc88c356614632731334486d50472.png
06:34:03 <t3hn3rd> Haven't manually messed with WIN routing tables in a longgg time
06:37:15 <aalm> i prefer just not to.
06:38:07 --- quit: deyzie (Quit: Page closed)
06:38:22 <lkurusa> that looks correct now
06:38:36 <lkurusa> your OS should be generating the ICMP packet and putting it on the LAN
06:39:06 <t3hn3rd> doesn't seem to generate anything as far as I can see in wireshark
06:39:33 <lkurusa> your OS == host machine, oops
06:39:43 <lkurusa> any ARP packets?
06:39:52 <t3hn3rd> ARP Packets appear perfectly
06:40:44 <aalm> uhmm
06:40:46 <t3hn3rd> https://i.gyazo.com/c10f748257158cf065ee3ef2b9bd82e2.png
06:41:00 <t3hn3rd> That's an arping from my ubuntu environment
06:41:06 --- quit: JonRob (Quit: Lost terminal)
06:41:18 <t3hn3rd> a normal ping to 192.168.1.243 from the same environment fails and generates no ICMP packets
06:42:53 <aalm> from where are you trying to reach and what? sry, lazy to read scrollback + your and lkurusa's nick are both in the same color
06:43:48 <t3hn3rd> trying to ping my OS from any machine at all, ARP works perfectly, every machine I try to ping from fails and doesn't generate any ICMP packets, but when I check their ARP Cache my OS Machine is there.
06:44:04 --- join: zeus1 (~zeus@197.239.5.87) joined #osdev
06:44:51 <lkurusa> can you check with traceroute ?
06:44:52 <aalm> so, are you sure your VM host is passing the packets?
06:45:02 <lkurusa> check if that generates any ICMP packets
06:46:24 <t3hn3rd> aalm: No ICMP packets are being generated AFAICT. Period. So nothing for the OS to process. The OS processes ARP packets absolutely fine.
06:46:36 <aalm> i'd have begun from having two working guests pinging each other, then replace one of them with "my OS" so everything not related to "my OS" can be ruled out easy.
06:47:34 <t3hn3rd> lkurusa: Tracert didn't seem to generate any ICMP requests either
06:47:37 <t3hn3rd> Very very odd.
06:47:49 <t3hn3rd> Starting to wander if wireshark is missing something.
06:48:06 --- join: JonRob (jon@gateway/vpn/privateinternetaccess/jonrob) joined #osdev
06:48:11 --- join: freakazoid0223 (~IceChat9@pool-108-52-244-197.phlapa.fios.verizon.net) joined #osdev
06:48:20 --- join: SGautam5 (3bb6f9ab@gateway/web/cgi-irc/kiwiirc.com/ip.59.182.249.171) joined #osdev
06:48:56 --- nick: SGautam5 -> SGautam
06:49:30 <lkurusa> are you sniffing the right adapter? :)
06:49:38 <t3hn3rd> aalm: well, it has to be something wrong with my OS, as every other machine can ping another absolutely fine. The question is, what is my OS doing wrong? My OS IP->MAC exists in the ARP caches, am I missing something else I'm supposed to do to announce myself?
06:49:46 <t3hn3rd> lkurusa: I definitely am :P
06:50:09 <aalm> does those "every other machine"s run under the same host?
06:50:20 <t3hn3rd> I have them re-named for ease of use, and eth0 is my main adapter, the same adapter the guest is bridged too
06:50:32 <t3hn3rd> aalm: yes, my Ubuntu environment is under the host.
06:51:42 <t3hn3rd> Ubuntu environment is .112
06:51:46 <t3hn3rd> https://i.gyazo.com/690bea4cad48fb0752a13ea92351fb8e.png
06:52:00 <t3hn3rd> https://i.gyazo.com/d09fba54c1f45ba5b5ccacb58ff10de6.png
06:52:22 <aalm> so windows is the host?
06:52:28 <t3hn3rd> It is
06:55:44 <t3hn3rd> Crude network topology: https://i.gyazo.com/6121206fb9f3faf03764248a49d92391.png
06:59:45 <aalm> ok, that explains it a bit better, if you want a good tcpdump of whatever goes on eth0, OpenBSD is fast&easy to install :)
07:02:25 <lkurusa> this is odd, as far as I can tell everything is the same LAN
07:02:49 <lkurusa> which should imply that ICMP packets are routed directly to the link (eth0)
07:03:14 <t3hn3rd> Right?
07:03:51 <lkurusa> you see the ARP packets in wireshark btw?
07:04:08 <t3hn3rd> I do, yeah
07:04:17 <bauen1> can you try to send a udp packet from windows to your os ? (iirc there might be some problems with ICMP and bridged networks (?))
07:04:50 <lkurusa> i’ve used a bridged network (albeit with qemu) and ICMP worked seamless
07:06:14 --- join: t3h_n3rd (~Logic_Bom@cpc91224-cmbg18-2-0-cust219.5-4.cable.virginm.net) joined #osdev
07:06:25 <t3h_n3rd> Hello, I'm back, did a router reset to rule that out
07:10:04 --- quit: t3hn3rd (Ping timeout: 250 seconds)
07:10:15 <lachlan_s> How long does it normally take to write a simple network stack?
07:10:32 <t3h_n3rd> Define simple?
07:10:43 <t3h_n3rd> You've got to be kidding me.
07:10:56 <lachlan_s> Simple is relative in this context
07:10:56 <t3h_n3rd> A router reset seems to have solved it, OS is now seeing the ICMP packets -.-
07:11:22 <t3h_n3rd> Not responding correctly just yet, but... It's a step closer
07:13:44 <t3h_n3rd> https://i.gyazo.com/ce27445c8e848cd8b09dc2a073619205.png Woo
07:14:06 --- join: Shamar (~giacomote@unaffiliated/giacomotesio) joined #osdev
07:14:22 <lkurusa> lachlan_s (IRC): it’s not as much as you think it is, and not as little as you’d hope :)
07:14:50 <aalm> t3h_n3rd, cool:)
07:14:51 <t3h_n3rd> lachlan_s: Well, It's taken me a week thus far to get ETH/ARP/IPv4/ICMP(ish)
07:14:58 <lkurusa> really it depends, I got UDP working on LAN in about a week
07:15:01 <lkurusa> the quality was... questionable, but it worked well as a PoC
07:15:02 <lkurusa> adding TCP was two weeks because of all the quirks and bugs I had to fix in my previous implementation
07:15:58 <t3h_n3rd> I used to work in network forensics, so I thought this should be my bread and butter, but I couldn't have predicted the number of oddities I've come across.
07:16:45 <lkurusa> t3h_n3rd (IRC): great job getting it to “work” :-)
07:17:46 <t3h_n3rd> Aha, it may not be fully working, but at-least the ICMP request is in the realm of my OS, so I can actually debug it and stop rocking back and forth crying whilst asking "WHY?!"
07:18:05 <lkurusa> aye, that’s progress
07:20:07 <t3h_n3rd> ARP was much more odd than I was expecting, I ended up passively monitoring a lot of the ARP traffic and then iterating over the hosts in my Cache to find new hosts when needed, as my router seems to refuse to respond to ARP requests.
07:22:40 --- quit: Belxjander (Max SendQ exceeded)
07:25:56 --- nick: SGautam -> S_Gautam
07:26:24 --- join: climjark_ (~climjark@c-24-0-220-123.hsd1.nj.comcast.net) joined #osdev
07:27:13 <lachlan_s> I'm planning on using smoltcp, which should make it significantly easier to get it working
07:31:32 --- join: EvanCarroll (~ecarroll@172.58.100.104) joined #osdev
07:32:12 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
07:39:37 <lkurusa> it’s less fun though :)
07:40:59 <lachlan_s> Yeah, but limited time :/
07:42:28 --- join: Goplat (~goplat@reactos/developer/Goplat) joined #osdev
07:44:26 --- quit: cantbetouched (Ping timeout: 255 seconds)
07:53:33 --- quit: awang_ (Ping timeout: 246 seconds)
07:59:36 --- join: m3nt4L (~asvos@2a02:587:a023:f000:3285:a9ff:fe8f:665d) joined #osdev
08:02:12 --- quit: spare (Quit: leaving)
08:05:20 --- join: promach__ (~promach@bb219-74-174-136.singnet.com.sg) joined #osdev
08:08:12 --- quit: quc (Ping timeout: 265 seconds)
08:11:00 <doug16k> t3h_n3rd, the obvious approach would be to coerce an existing OS to do that same ARP probe, capture that in wireshark, and compare it to what your OS sends
08:11:56 <doug16k> your code may be doing exactly what you told it to do, and you might be telling it to do something subtly wrong
08:12:33 --- join: banisterfiend (~banister@ruby/staff/banisterfiend) joined #osdev
08:12:44 <Ameisen> well
08:12:50 <Ameisen> I feel dumb today and can't solve this
08:12:54 <Ameisen> trying to convert one floating point type to another.
08:12:58 --- quit: Shamar (Quit: Lost terminal)
08:13:05 <Ameisen> So far I'm able to convert it to the point that I lose the decimal value
08:13:10 <Ameisen> 100.123123123f becomes 100.0
08:13:33 <doug16k> converting float to double?
08:13:49 <Ameisen> shifting the significand right by the difference in input and output significand bits
08:14:09 <Ameisen> and subtracting the difference between the input and output exponent bias from the input bias.
08:14:11 <Ameisen> err, input exponent
08:14:20 <Ameisen> that gives me an integral value
08:14:25 <doug16k> just copy the mantissa filling LSBs with zeros and adjust the exponent bias
08:14:27 <Ameisen> float to double or any other particular floating point type.
08:15:02 --- join: quc (~quc@host-89-230-160-119.dynamic.mm.pl) joined #osdev
08:15:10 <Ameisen> pretty sure that's what I'm doing unless I'm missing something
08:16:03 <Ameisen> single: mantissa 0x00189680 exponent 0x96 -> double: mantissa: 0x000312d003f09fe7 exponent: 0x416
08:16:08 <Ameisen> err
08:16:17 <Ameisen> mantissa 0x312d000000000 for the double
08:16:21 <Ameisen> what I printed is what it _should_ be
08:16:24 <Ameisen> roughly
08:16:39 --- join: fddsafsd (~sdfgsdfg@176.33.236.80) joined #osdev
08:16:47 <doug16k> the exponent must be correct, it would be way off if not. the mantissa must be truncated
08:17:12 <Ameisen> well, I'm shifting the mantissa based on the difference in significand bits
08:17:19 <Ameisen> I'm unsure how to bring in the fractional bit elements though
08:17:35 --- quit: adam4813 (Quit: Connection closed for inactivity)
08:17:55 --- quit: daniele_athome (Ping timeout: 268 seconds)
08:19:35 <doug16k> there is no distinction between fractional bits or otherwise, just left align the mantissa in the destination. you must be handling the exponent correctly already
08:19:40 <Ameisen> I presume that's what you mean when you say 'adjust the exponent bias'.
08:19:47 <Ameisen> I'm just not sure what that means exactly.
08:20:07 <Ameisen> as said, right now I shift the input mantissa right by the difference in input and output significand bits
08:20:33 <Ameisen> since a single has 23, and a double has 52, I effectively shift it left by 29 bits
08:20:46 <Ameisen> but then there are basically _no_ lower bits in the mantissa, and no decimal figures are expressed
08:21:04 <Ameisen> basically the difference between 0x312d000000000 and 0x312d003f09fe7
08:22:28 --- join: daniele_athome (~daniele_a@5.170.129.1) joined #osdev
08:27:38 <Ameisen> Thus... not sure how to rectify that exactly
08:27:40 <fddsafsd> is that 64 bit
08:27:55 <fddsafsd> hey, btw... is it possible to run my 64bit cpu 32 ?
08:27:59 <Ameisen> I'm converting a single (float) to a double (...double)
08:28:10 <fddsafsd> I heard of the compatibility mode or smth similar
08:28:11 --- join: spare (~user@unaffiliated/spareproject) joined #osdev
08:28:26 <fddsafsd> so it should in theory be possible to run an OS 32 bit with a 64bit cpu right ?
08:28:30 <Ameisen> when I encode the same value directly in a double to see what the compiler emits, it emits the mantissa as: 0x312d003f09fe7
08:28:43 <Ameisen> note that it has a bunch of lower bits.
08:28:45 <fddsafsd> whats a mantissa
08:29:01 <Ameisen> So, I'm very close, but lacking any decimal precision.
08:29:07 <t3h_n3rd> Never in my life has a ping felt so damn good: https://i.gyazo.com/5dd879f48a73d790cc5326462b11df62.png
08:29:15 <Ameisen> The difference, in binary, is 0b11111100001001111111100111
08:29:23 <Ameisen> no clue how it's getting that though
08:29:38 <doug16k> geordi: { double d = 100.123123123; char b[8]; memcpy(b, &d, 8); for (int i = 7; i >= 0; --i) cout << hex << setw(2) << setfill('0') << (unsigned)(unsigned char)b[i]; }
08:29:39 <geordi> 405907e13fceaaa6
08:30:11 <doug16k> geordi: { float d = 100.123123123f; char b[4]; memcpy(b, &d, 4); for (int i = 3; i >= 0; --i) cout << hex << setw(2) << setfill('0') << (unsigned)(unsigned char)b[i]; }
08:30:11 <geordi> 42c83f0a
08:31:34 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
08:31:57 --- quit: janemba (Read error: Connection reset by peer)
08:32:07 <Ameisen> https://pastebin.com/e9sK7vYP
08:32:08 <bslsk05> ​pastebin.com: Input Single: 10000000.123123123 Mantissa: 0x00189680 0b110001001011010000000 - Pastebin.com
08:32:10 <Ameisen> those are my results
08:32:34 --- join: janemba (~janemba@unaffiliated/janemba) joined #osdev
08:33:13 <doug16k> Ameisen, float can't represent that many digits
08:33:26 <doug16k> geordi: << 10000000.123123123
08:33:26 <geordi> 1e+07
08:33:58 <Ameisen> hrmm
08:34:00 <Ameisen> you're right
08:34:05 <Ameisen> I'm just dumb
08:34:11 <Ameisen> :|
08:34:18 <Ameisen> 100.1f -> double works fine
08:34:22 <Ameisen> I blame my cold
08:34:34 --- join: gareppa (~gareppa@unaffiliated/gareppa) joined #osdev
08:34:44 <Ameisen> double->float also works
08:35:35 <t3h_n3rd> lkurusa: Is all working, getting ping replies successfully!
08:36:21 --- quit: ALowther (Ping timeout: 260 seconds)
08:37:04 <S_Gautam> Hold on a sec, shouldn't your compiler automatically handle that conversion? Typecasting float -> double?
08:37:26 <doug16k> I think Ameisen is doing embedded system soft float stuff, correct me if I'm wrong
08:37:30 --- quit: gareppa (Remote host closed the connection)
08:37:55 --- quit: Goplat ()
08:39:22 <doug16k> fddsafsd, yes, x86_64 cpu can run 32 bit operating systems just fine, they are completely backward compatible
08:41:03 <fddsafsd> that is absolute ducks nuts
08:41:13 <fddsafsd> ill get myself an x86 system now then
08:41:32 <fddsafsd> the numbers are shorter and easier to read
08:41:55 <doug16k> also, a 64 bit operating system can run 32 bit processes, with a bit of work
08:41:56 <S_Gautam> ??
08:41:57 --- quit: zeus1 (Ping timeout: 240 seconds)
08:42:02 <doug16k> on x86_64
08:42:24 <S_Gautam> Oh well, yeah. I used to take this for granted until I researched carefully about it.
08:42:36 <S_Gautam> I guess the major thing is syscall compatibility?
08:43:02 <doug16k> yes and dealing with different alignment/sizes in structures
08:44:28 --- quit: elderK (Quit: WeeChat 1.9)
08:44:55 <doug16k> if you know up front it has to work on both, then it's not bad. no matter what, pointers are different sizes, so any struct members in APIs that are pointers need special handling. if you have a ton of old 32 bit code that assumes long is the same as int, you're in trouble
08:45:27 <doug16k> old 32 bit OS code*
08:46:21 <t3h_n3rd> I'm working on the basis that 64-bit is just a fad and we don't need any more than 4GB of memory space, that's why Asuro is 32-bit, at-least, that's what I keep telling myself ;) /s.
08:47:02 <t3h_n3rd> Definitely not laziness, honest gov'.
08:49:32 <lachlan_s> Yeah, who needs more than 4 gigs of memory?
08:49:40 <doug16k> o/
08:49:57 <t3h_n3rd> ^ Nah, he's just using it wrong ;)
08:50:09 <doug16k> I have 64GB
08:50:24 <t3h_n3rd> Not with Asuro you don't. ;P
08:51:36 <S_Gautam> lachlan_s: The gaming industry
08:52:07 <t3h_n3rd> Nahh, that's all marketing.
08:52:10 <doug16k> the performance is insane with okular set to "greedy" memory usage. I can find text in the combined intel manual in a few seconds. that manual takes 30GB to load on greedy
08:52:33 --- join: epelesis (~epelesis@2600:380:667a:5b66:d9f0:6228:b1f3:3f3c) joined #osdev
08:52:50 <t3h_n3rd> I'm being 100% non-serious, just to clarify - Asuro is 32-bit due to my laziness and nothing more. xD
08:54:02 <doug16k> t3h_n3rd, it's hardly more difficult to implement an x86_64 kernel. the only thing that is more difficult is the paging code. everything else is actually easier than 32 bit code
08:54:57 <t3h_n3rd> Oh no, I don't disagree, I just started with 32-bit and I really CBA to re-do all of my x86 specific code.
08:55:16 <t3h_n3rd> If I'd started with x86_64 from the start it'd be fine.
08:56:10 <doug16k> if you implement PAE, then your paging code will be roughly equivalent to x86_64 paging code and i386 is actually then completely more difficult than an x86_64 kernel
08:57:22 <t3h_n3rd> It'll be something to look at in the future, but as it stands now, all of my OS Dev time is taken up with the netstack.
08:57:23 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
08:58:02 <t3h_n3rd> I'd also need to work out how my compiler handles native x86_64
08:58:25 <t3h_n3rd> It's FPC, so will guarantee there will be quirks.
08:59:20 --- quit: epelesis (Quit: Mutter: www.mutterirc.com)
09:02:01 --- quit: ALowther (Ping timeout: 260 seconds)
09:02:41 <t3h_n3rd> Which timer would be best to time milliseconds since boot?
09:05:41 --- join: MarchHare (~Starwulf@mo-184-5-204-232.dhcp.embarqhsd.net) joined #osdev
09:06:28 --- join: asdafdsasd (~sdfgsdfg@176.33.236.80) joined #osdev
09:06:45 --- quit: fddsafsd (Read error: Connection reset by peer)
09:07:01 <doug16k> t3h_n3rd, rdtsc would be the simplest, but then you'd have to calibrate
09:07:20 <doug16k> assuming x86 architecture
09:10:52 <doug16k> depends on what timers you are using and what you've implemented. available timekeeping mechanism on PC are PIT, CMOS, APIC timer, RDTSC, ACPI timer
09:13:04 <Ameisen> doug16k - it would handle the conversion for me
09:13:10 <Ameisen> if I weren't implementing half and float24
09:13:12 <Ameisen> :|
09:13:16 <Ameisen> I have to implement those myself.
09:13:23 <doug16k> Ameisen, ah
09:14:16 --- quit: daniele_athome (Read error: Connection reset by peer)
09:14:32 <Ameisen> I considered float8
09:14:37 <Ameisen> I sincerely cannot find a use case though
09:15:09 <Ameisen> So far it appears to be allowing me to do additoin/subtraction/etc of float24/float16 without any additional work. Not sure why.
09:15:23 <doug16k> Ameisen, what, 1 bit sign, 0 bits exponent, 7 bits mantissa, range -1 to 1?
09:15:29 <Ameisen> it is only complaining about missing the expand/truncate intrinsics
09:15:40 <Ameisen> maybe
09:15:50 <Ameisen> what woudl be neat is a builtin that let you define float types on the fly
09:16:11 <Ameisen> __builtin_getfloat_t(signed, mantissa, exponent)
09:16:41 <doug16k> I assume you are emulating that with templates already
09:16:52 <Ameisen> Not yet.
09:16:58 <Ameisen> I could. I would rather they be built-in to the compiler.
09:17:04 <Ameisen> Makes treating them as POD/primitives a little easier.
09:17:30 <Ameisen> builtins, however, cannot return types. They _can_ return values of an arbitrary-type, though...
09:17:38 <Ameisen> to which decltype() could be used on it
09:18:25 --- quit: noldor (Quit: Connection closed for inactivity)
09:18:38 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
09:18:54 <doug16k> Ameisen, are you intending to handle denormal and NaN?
09:19:08 <doug16k> ...and infinity
09:20:50 <Ameisen> at the moment, no
09:20:52 <Ameisen> I should
09:20:57 <Ameisen> denormal? No reason to.
09:21:08 <doug16k> just flushing them to zero then?
09:21:08 <Ameisen> software don't care
09:21:19 <Ameisen> but yeah, I do need to make this more 'correct'
09:21:23 <Ameisen> this is just a quick implementation for testing
09:21:53 <t3h_n3rd> doug16k: Have RTC implemented along with RDTSC, trying to work out how Unix calculates milliseconds since epoch.
09:22:14 <t3h_n3rd> I'd like something similar for timestamp comparison & file timestamps.
09:22:35 <doug16k> t3h_n3rd, do you have on-update IRQ enabled on CMOS? you can get a once-per-second IRQ that way
09:23:05 <t3h_n3rd> I'd like millisecond resolution ideally.
09:23:05 --- join: KernelBloomer (~SASLExter@gateway/tor-sasl/kernelbloomer) joined #osdev
09:23:13 --- join: daniele_athome (~daniele_a@5.170.129.1) joined #osdev
09:23:20 <doug16k> in the CMOS ISR, you can snapshot RDTSC value and store it... then in the function to get time, you can do RDTSC again, subtract the RDTSC at last CMOS IRQ, and get fractional seconds from last CMOS IRQ
09:23:30 <doug16k> then obviously increment a seconds-counter in the CMOS IRQ
09:24:33 <doug16k> so it would be roughly (seconds_since_boot*1000)+(rdtsc_now - rdtsc_at_last_cmos_irq)/(rtdsc_freq / 1000)
09:24:35 <t3h_n3rd> Hmmm... that's not a bad call, I already know the Hz of the CPU, so could probably use that with RDTSC to get Milliseconds
09:25:52 <t3h_n3rd> Cheers!
09:28:06 <doug16k> or (seconds_since_boot*1000)+((rdtsc_now - rdtsc_at_last_cmos_irq)*1000)/rtdsc_freq
09:28:55 --- quit: KernelBloomer (Remote host closed the connection)
09:30:39 <doug16k> to get microsecond precision, just change the 1000's to 1000000's
09:31:38 --- join: hmmmm (~sdfgsf@pool-72-79-167-28.sctnpa.east.verizon.net) joined #osdev
09:36:32 <Ameisen> an unsigned float type mgiht also be useful
09:39:34 --- quit: promach__ (Quit: WeeChat 2.1)
09:39:56 --- quit: daniele_athome (Ping timeout: 260 seconds)
09:42:32 --- join: zeus1 (~zeus@197.239.35.229) joined #osdev
09:46:42 --- quit: tomaw (Read error: Connection reset by peer)
09:46:58 --- quit: ALowther (Remote host closed the connection)
09:47:27 --- join: tomaw (tom@freenode/staff/tomaw) joined #osdev
09:48:04 --- quit: tomaw (Read error: Connection reset by peer)
09:54:44 --- join: gareppa (~gareppa@unaffiliated/gareppa) joined #osdev
09:56:33 --- join: tomaw (tom@freenode/staff/tomaw) joined #osdev
09:58:26 --- quit: gareppa (Remote host closed the connection)
10:00:16 --- join: KernelBloomer (~SASLExter@gateway/tor-sasl/kernelbloomer) joined #osdev
10:17:28 --- quit: sdk93 (Quit: Connection closed for inactivity)
10:17:42 --- quit: S_Gautam (Ping timeout: 250 seconds)
10:18:28 <lachlan_s> How does a kernel figure put cpu frequency?
10:19:30 <doug16k> lachlan_s, measure from a known time source like PIT or ACPI timer
10:19:51 --- join: jrheaton (~jrheaton@162.235.12.210) joined #osdev
10:20:06 <lachlan_s> Ah, and then use rdtsc
10:20:11 <doug16k> ya
10:20:20 <lachlan_s> Does the frequency stay constant?
10:20:39 <doug16k> depends on the processor. newer ones yeah, it's the base frequency
10:20:48 <doug16k> there's a cpuid bit that tells you whether it is stable freq
10:21:08 <doug16k> older ones fluctuated with turbo/throttling
10:22:04 <doug16k> it has to be quite old to be unstable frequency
10:22:36 <doug16k> if you use the APIC timer (you should), then you also have to detect that frequency
10:23:10 <doug16k> either by measuring, or on really new cpus, there's a cpuid leaf for it
10:24:54 <lachlan_s> So, how do you measure the apic frequency if you use the apic to measure cpu frequency?
10:25:25 <doug16k> you don't, you use the PIT or ACPI timer, both of which have fixed/known frequency counters
10:25:36 <doug16k> APIC != ACPI :D
10:25:48 <doug16k> annoyingly similar looking
10:26:15 <lachlan_s> Whoops, misread it sorry
10:27:33 <lachlan_s> Thanks
10:27:59 <doug16k> note that qemu+kvm will report that the frequency of rdtsc is unstable (even if it is stable) unless you have -cpu migratable=false
10:29:29 <doug16k> they lie and say it is unstable in cpuid, just in case you migrate it to a host with unstable rdtsc frequency
10:30:02 <doug16k> migratable=false disables the lie and lets you test your code to handle stable frequency bit being set
10:30:44 --- quit: zeus1 (Ping timeout: 264 seconds)
10:32:02 <doug16k> you can also say -cpu invtsc to just enable "invariant timestamp counter" without going fully unmigratable
10:32:47 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
10:33:16 --- quit: zaquest_ (Ping timeout: 255 seconds)
10:39:26 --- join: zaquest (~notzaques@5.128.210.30) joined #osdev
10:41:48 --- join: variable (~variable@freebsd/developer/variable) joined #osdev
10:50:57 --- quit: aalm (Ping timeout: 240 seconds)
10:50:57 --- quit: variable (Quit: /dev/null is full)
10:55:46 --- join: aalm (~aalm@37-219-119-194.nat.bb.dnainternet.fi) joined #osdev
10:56:00 --- join: AverageJ0e (~joe@ip98-167-200-207.ph.ph.cox.net) joined #osdev
11:02:07 --- quit: ALowther (Remote host closed the connection)
11:04:34 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
11:13:40 --- join: baschdel (~baschdel@2a01:5c0:10:3d11:bca2:7797:3876:4668) joined #osdev
11:14:06 --- join: zeus1 (~zeus@197.239.35.229) joined #osdev
11:20:21 --- quit: KernelBloomer (Quit: Leaving)
11:22:03 --- join: KernelBloomer (~SASLExter@gateway/tor-sasl/kernelbloomer) joined #osdev
11:22:08 <lachlan_s> How to schedulers generally know when to wake up a process that's sleeping?
11:22:22 <lachlan_s> Do they have to iterate through all of them?
11:23:36 <llmm_> simplest one: take the first process of the ready queue
11:24:20 <llmm_> ah, you mean sleeping
11:24:28 <lachlan_s> Yeah
11:24:31 --- quit: KernelBloomer (Remote host closed the connection)
11:25:25 <llmm_> maybe they can be ordered by end of wait time
11:25:59 <llmm_> and test the first one
11:26:24 --- quit: m3nt4L (Remote host closed the connection)
11:27:22 <lachlan_s> I suppose you could use a btree for that
11:28:50 <lachlan_s> geist: how does the fuchsia scheduler know when to wake up a thread?
11:29:29 <lkurusa> rb-tree
11:29:43 --- quit: bauen1 (Ping timeout: 248 seconds)
11:29:53 <lkurusa> (not fuchsia, i don’t know fuchsia, but my gut suggests rb-tree for tracking processes)
11:30:54 <lachlan_s> I think fuchsia maintains a linked ready queue of threads
11:31:12 <lachlan_s> As far as I know, there's no global thread pool
11:31:38 --- join: bauen1 (~bauen1@ipbcc18c77.dynamic.kabel-deutschland.de) joined #osdev
11:34:14 --- quit: ALowther (Remote host closed the connection)
11:35:49 --- join: oaken-source (~oaken-sou@p5DDB52B4.dip0.t-ipconnect.de) joined #osdev
11:37:13 --- join: awang_ (~awang@cpe-98-31-27-190.columbus.res.rr.com) joined #osdev
11:37:43 --- quit: JonRob (Ping timeout: 268 seconds)
11:38:57 --- quit: bauen1 (Ping timeout: 268 seconds)
11:39:13 --- join: JonRob (~jon@host165-120-84-36.range165-120.btcentralplus.com) joined #osdev
11:42:34 --- join: daniele_athome (~daniele_a@5.170.121.43) joined #osdev
11:47:26 --- join: Mistell (~james@unaffiliated/magejames) joined #osdev
11:52:28 --- quit: AverageJ0e (Ping timeout: 255 seconds)
11:56:15 --- join: bauen1 (~bauen1@ipbcc18c77.dynamic.kabel-deutschland.de) joined #osdev
12:00:24 <geist> lachlan_s: depends. it's fairly simple
12:00:52 <geist> either the thread was woken up because another one (or in a few cases an irq handler) triggered a blocking primtiive the thread was blocked on
12:00:58 <geist> that's the most common case
12:01:20 <geist> or, the thread was sleeping and a timer irq went off, ran a wakeup handler, and then the thread was put back in the run queue
12:01:30 --- quit: light2yellow (Quit: light2yellow)
12:01:32 <geist> or the thread was blocking on something with a timeout, and then the path is fairly similar to the sleeping case
12:02:10 <geist> so the run queue(s) themselves are only for threads that are ready to run. when they're not running they're tracked on the thing they're blocked on
12:02:33 <geist> or in the case of sleeping, no where at all, a timer event 'owns' the thread essentially until it wakes up or is interrupted
12:03:45 <geist> in pretty much all cases it's fairly symmetric: the thread has a linked list node (queue_node) that is used either when the thread is in a run queue or blocked on a wait_queue (which will be at the bottom of a mutex or event or futex or whatnot)
12:04:05 <geist> so from the threads's point of view it's just sitting in a queue, the queue just changes. so all scheduler decisions like this are O(1)
12:04:13 <geist> never ever do you iterate through all threads in the system
12:05:08 <geist> occasionally you'll find old literate about classic unix or whatnot that called for periodically (say once a second) iterating over all threads in the system, recomputing their time slices, putting them in a run queue, etc
12:05:27 <geist> nothing does that aosrt of O(N) stuff anymore, but it probably made sense back when N was something like 10
12:05:34 --- quit: elevated (Quit: bye)
12:06:35 <geist> now that modern machines have thousands or tens of thousands of threads, it doesn't make sense to do any of that
12:07:22 --- join: manwar (~manwar@41.37.73.39) joined #osdev
12:07:38 --- join: variable (~variable@freebsd/developer/variable) joined #osdev
12:10:07 --- quit: manwar (Client Quit)
12:13:34 --- join: attah (~attah@h-155-4-135-114.NA.cust.bahnhof.se) joined #osdev
12:17:40 --- join: elevated (~elevated@unaffiliated/elevated) joined #osdev
12:19:01 --- join: unixpickle (~alex@c-24-5-86-101.hsd1.ca.comcast.net) joined #osdev
12:22:32 --- quit: elevated (Ping timeout: 256 seconds)
12:24:47 --- join: lldd_ (~atrapado@unaffiliated/atrapado) joined #osdev
12:26:06 <lachlan_s> geist: Thanks
12:26:14 <lachlan_s> So, there's no global list of threads?
12:26:20 <lachlan_s> How do thread ids work?
12:28:50 --- join: drakonis (~drakonis@unaffiliated/drakonis) joined #osdev
12:29:14 --- join: elevated (~elevated@unaffiliated/elevated) joined #osdev
12:29:36 --- quit: EvanCarroll (Remote host closed the connection)
12:30:00 --- quit: Mistell (Quit: WeeChat 2.1)
12:30:10 --- join: EvanCarroll (~ecarroll@172.58.100.104) joined #osdev
12:31:30 --- join: isd (~isd@pool-71-174-32-198.bstnma.east.verizon.net) joined #osdev
12:34:21 --- join: KernelBloomer (~SASLExter@gateway/tor-sasl/kernelbloomer) joined #osdev
12:36:55 --- quit: KernelBloomer (Remote host closed the connection)
12:37:31 <geist> they dont, fuchsia doesn't really have a mechanism to look up thread by id
12:37:47 <geist> but if it did, there'd probablyu be a bigass hash table for that
12:38:15 <geist> note that objects can be in multiple data structures. so it's possible for all the threads to exist in a hash table to help you look up by ID for certain apis, but the scheduler doesn't have to use it at all
12:38:27 <geist> and you generally wouldn't want to, since a perfectly constant time hash table is hard
12:39:12 <geist> _mjg_: what was that github with the syscall benchmark?
12:39:31 --- quit: quc (Read error: No route to host)
12:39:35 <_mjg_> was not a github mate, https://svnweb.freebsd.org/base/head/tools/tools/syscall_timing/syscall_timing.c
12:39:37 <bslsk05> ​svnweb.freebsd.org: [base] Log of /head/tools/tools/syscall_timing/syscall_timing.c
12:39:49 <geist> so... how does one run that on linux?
12:39:50 <_mjg_> if this one turns out to be non functional, i can link an older release
12:39:50 --- quit: awang_ (Ping timeout: 250 seconds)
12:40:03 <geist> i thought the whole point was to have a generic app to do this
12:40:03 <_mjg_> should be ez, cc -o syscall_timing syscall_timing.c
12:40:04 <_mjg_> done
12:40:12 <geist> should be, have you tested this?
12:40:33 <_mjg_> now that you asked, ye, but i see i forgot to commit the fix: #include <time.h> for linux
12:41:20 <geist> i was hoping to run it on other machines too with differing levels of linux and freebsd, for lulz
12:42:02 <_mjg_> what's the lowest place you are starting with?
12:42:12 <geist> as far as it'll go
12:42:24 <_mjg_> solid
12:42:44 <geist> actually i was more interested in running on modern machines, and arm boxes, mips machines, etc
12:43:34 --- quit: t3h_n3rd (Read error: Connection reset by peer)
12:43:49 --- quit: isd (Ping timeout: 240 seconds)
12:44:06 <_mjg_> if going for somewhat modern you may want to invest into will-it-scale instead
12:44:28 <geist> hmm, so adding time.h yields a problem with getresuid, asprintf, and SHM_*
12:44:34 <geist> on linux 18.04 that is
12:44:34 <_mjg_> i have a rhel5 handy
12:44:41 <_mjg_> will get a version compilable on this
12:44:45 <_mjg_> should be old enough
12:44:48 <_mjg_> 5 mins
12:44:59 <lachlan_s> geist: So, to resume a thread, what do you pass into the syscall?
12:45:12 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
12:45:12 <geist> the handle to it
12:45:22 --- quit: jrheaton (Remote host closed the connection)
12:45:43 <geist> everything in fuchsia/zircon is handle based. the id is almost irrelevant, though there is a unique 64bit id assigned to all kernel objects, mostly for debugging purposes
12:45:46 <lachlan_s> Ah, so it's just a normal object then?
12:45:51 <geist> since no API actually uses the ID
12:45:57 --- quit: EvanCarroll (Remote host closed the connection)
12:46:00 <geist> yes, handles to threads are just like handles to any other object
12:46:17 <lachlan_s> Oh, cool
12:47:02 --- join: jrheaton (~jrheaton@162-235-12-210.lightspeed.bcvloh.sbcglobal.net) joined #osdev
12:48:02 <radens> geist: there's a classic problem where when you're adding a page to a page table you need to map the page first so you can initialize it, and set the bits inside of it. One solution is fractal paging. Another is to have a dedicated pool of physical memory reserved for the purpose. Another is to have a third level PT mapped in the kernel data space, and then you map a your new page table into a reserved slot to
12:48:04 <geist> hmm, so the SHM_* stuff doesn't seems to exist on linux, can probably just disable that part of the test
12:48:04 <radens> zero it and add any entries, and then unmap it. All of these solutions seem kind of inelegant to me, but do you have any particular recommendations?
12:48:26 --- quit: jrheaton (Client Quit)
12:48:30 <geist> radens: what architecture are you on?
12:48:39 <radens> aarch64
12:48:55 <geist> i usually just map all of usable DRAM into the kernel using 1GB pages
12:48:58 <radens> but it also applies to amd64 and armv7 I think.
12:49:07 <geist> then ou always have a direct, simple way to manipulate pages
12:49:11 <geist> it's what linux does too
12:49:22 <lachlan_s> geist: So, when a thread is in a wait queue, the queue contains the global handle for that thread?
12:49:27 <radens> geist: and you're not concerned with the security issue if an attacker has an arbitrary RW?
12:49:34 <_mjg_> geist: https://people.freebsd.org/~mjg/.junk/syscall_timing.c
12:49:35 <bslsk05> ​people.freebsd.org <no title>
12:49:40 <geist> radens: i am. but the world is full of compromises
12:49:47 <geist> lachlan_s: nope. the pointer to the thread structure
12:49:51 --- quit: ALowther (Ping timeout: 256 seconds)
12:49:53 <geist> handle is purely a user space facing concept
12:49:58 <lachlan_s> Ah
12:49:58 <_mjg_> geist: on old linux compile with gcc -lrt -o sycall_timing syscall_timing.c
12:50:01 <drakonis> oh its a you
12:50:02 <geist> kk
12:50:06 --- join: quc (~quc@host-89-230-160-119.dynamic.mm.pl) joined #osdev
12:50:31 <lachlan_s> So, how do you avoid the thread structure being freed while the queue contains the pointer?
12:51:06 <geist> via ref counting and being carefl
12:51:18 <geist> well, more specifically via state tracking
12:51:36 <geist> the thread itself runs through one of 6 states (READY, RUN, BLOCKED, SLEEPINg, etc)
12:51:45 <geist> and so based on the state you know if it's in a queue or not
12:51:59 <geist> so you never ever free the thread until you've successfully parked it (DEAD state)
12:52:21 <lachlan_s> Oh, I see
12:52:29 <geist> thread teardown is actually fairly tricky, you have to be careful to step things through the right stages
12:52:30 --- join: isd (~isd@pool-71-174-32-198.bstnma.east.verizon.net) joined #osdev
12:52:36 <lachlan_s> So, it's the job of whatever finally puts it in a dead state to free it
12:52:53 <geist> _mjg_: is there a file next to it that it's using? I'm getting some bad refs to memset_orig and tbzero_*
12:53:08 <geist> that smells like some extra test code you were fiddling with
12:53:17 <geist> lachlan_s: basically, yes
12:53:54 <lachlan_s> Cool, cool!
12:54:13 <_mjg_> geist: eh, sorry. copied the wrong file used for other benchs, see https://people.freebsd.org/~mjg/.junk/syscall_timing2.c
12:54:14 <bslsk05> ​people.freebsd.org <no title>
12:54:23 <lachlan_s> Right now, I maintain a single global list of threads and pass around indexes of that list, but I think i'll move it to this pseudo-reference counting method
12:54:27 --- join: awang_ (~awang@rrcs-24-106-163-46.central.biz.rr.com) joined #osdev
12:54:47 <_mjg_> have to stop doing 3 things at once
12:54:55 <geist> _mjg_: same
12:55:00 <geist> so which test are you interested in running?
12:55:30 --- quit: drakonis (Read error: Connection reset by peer)
12:56:13 <geist> some of these in linux may be vdso calls, so dunno which one is a good representative of a deep dive
12:56:23 <_mjg_> predominantly getuid, but if you want to reach higher linuxes you may want to just getppid
12:56:27 <_mjg_> yea
12:56:32 <_mjg_> getppid is defo not cacheable
12:57:01 <geist> so for example on this nehalem i get for time of ~1, 8605041 for getuid and 8281791 for getppid
12:57:05 <_mjg_> that said, i expect fbsd perf to blow goats
12:57:08 <geist> which actually indicates that the getppid isn't much slower
12:57:25 <geist> if one were vdso and one wasn't i'd expect to have them be off by a fair amount
12:57:42 <_mjg_> i got a nehalem here as well
12:57:46 <_mjg_> what's your freq
12:57:50 <geist> 2.26
12:57:59 <geist> it's a xeon E5520
12:58:06 <_mjg_> i'm sitting at 3.07GHz
12:58:22 <_mjg_> getuid 0 1.000003328 10676172 0.000000093
12:58:24 <lachlan_s> geist: Does fuchsia have an idle thread that it defaults to?
12:58:28 <geist> lachlan_s: yes
12:58:34 <geist> one per core
12:58:46 <geist> _mjg_: that's about right, yours seems to be about the right ratio faster
12:58:59 <lachlan_s> Is there a faux kernel thread?
12:59:08 <lachlan_s> That it creates in order to switch from?
12:59:12 --- quit: nur (Quit: Leaving)
12:59:26 <geist> lachlan_s: it's not faux, it's a real thread, just has the priority set to 0 with affinity locked to the core
12:59:53 <lachlan_s> Does it do any work?
13:00:07 <geist> basically just sits in an idle loop
13:00:14 <geist> for (;;) arch_idle();
13:00:21 --- join: nur (~hussein@slb-97-111.tm.net.my) joined #osdev
13:00:37 <geist> excepppt on x86 we have it do a monitor/mwait, but that's a pure optimization to avoid a full IPI to wake up a core
13:00:56 <geist> but it's implemented inside arch_idle() i believe
13:01:20 <lachlan_s> So, most of the time, it's the same as the idle thread?
13:01:36 <geist> failed to parse that sentence
13:02:33 <lachlan_s> Sorry, let me try that again
13:02:49 --- quit: eremitah (Ping timeout: 268 seconds)
13:03:08 <lachlan_s> So, switching to the kernel thread is just wasted cpu time?
13:03:19 --- quit: zeus1 (Ping timeout: 240 seconds)
13:03:22 <geist> compared to what?
13:03:33 <geist> that's a fairly weighted question, what are you trying to get to?
13:04:43 <lachlan_s> I'm not really trying to get anywhere. Just trying to understand how the scheduler on fuchsia works
13:04:46 <geist> note that in all my kernel designs, switching to a kernel thread is the same mechanism. the scheduler doesn't care
13:04:56 <geist> yes but your question has an implicit 'compared to something else'
13:05:04 <geist> as in waste of time, vs doing what? not doing it?
13:05:08 <geist> what would it do otherwise?
13:05:08 --- join: dbittman_ (~dbittman@2601:647:ca00:1651:b26e:bfff:fe31:5ba2) joined #osdev
13:05:33 <lachlan_s> Like executing a regular thread.
13:05:34 <geist> if what you're getting at is 'the idle thread isn't a real one so why do a full switch to it' then that's a complicated question
13:06:07 <geist> like i said, the scheduler/context switch code doesn't care. it's only doing a kernel context switch, so switching between user space and between kernel threads is precisely the same amount of work
13:06:25 <geist> why does the idle thread get it's own thread and stack and whatnot? simplicity, symmetric design
13:06:44 <_mjg_> linux has a hack where kernel thread -> kernel thread retains the address space of whoever was on cpu
13:06:47 <geist> you *could* do a special case for the idle where you park the cpu in a funny, stackless state, but i dont see the real advantage of that
13:07:02 <lachlan_s> Okay, thanks!
13:07:06 <geist> the main thing you want to do though is unload the stack of whatever you were switching from
13:07:24 <geist> so you want to at least get the SP to not point at the old thread anymoire, since it may be destroyed or running on another core shortly
13:07:29 --- join: eremitah (~int@unaffiliated/eremitah) joined #osdev
13:07:57 <geist> so it's simpler to just create an idle thread that runs at priority 0, so that way the scheduler has no special case. it *always* finds a thread to run, or that's a fatal, kernel-is-screwed state
13:08:22 <_mjg_> it is also easier to account who is on cpu
13:08:35 <geist> _mjg_: yeah i think we have that too, though it's dangerous because you may end up leaving the cpu pointing at an aspace you want to tear down
13:08:59 <geist> _mjg_: exactly. the idleness of a core is literally the inverse of the amount of time spent running the idle thread bound to that core
13:09:17 <geist> or the busyness
13:09:35 <geist> _mjg_: i'll build a little doc here as long as my attention does not wane.... what other tests would you like to see run?
13:09:40 --- quit: rain1 (Quit: Leaving)
13:09:43 <geist> besides getuid/getppid?
13:10:12 <geist> a run on this p3 500 with 18.04 i get 1387060 and 1593552
13:10:20 <geist> interesting that getppid is actually *faster*
13:10:31 <_mjg_> is this repeatable?
13:10:35 <geist> 100%
13:10:42 <_mjg_> huh
13:10:57 <geist> biggest difference is it's running x86-32 kernel, so who knows
13:11:04 <geist> maybe the dispatch there, vdso works differently
13:11:21 <_mjg_> i should know, but i don't - is this cpu affected by meltdown et al?
13:11:24 <geist> oh and it wouldn't be using syscall by definition. probably sysenter
13:11:41 <_mjg_> as in, can you tru booting with nopti?
13:12:03 --- quit: eremitah (Ping timeout: 246 seconds)
13:12:11 --- join: rain1 (~rain1@unaffiliated/rain1) joined #osdev
13:12:24 <geist> kk, though i have no evidence that the nopti patches have made it to x86-32 yet. i dont see any references in the dmesgf about using user page table switch
13:12:38 <geist> it shows cpu_meltdown and specture 1 and 2 in /proc/cpuinfo
13:12:50 <geist> but normally you can see the signature in dmesg that it's doing the full kernel isolation
13:13:20 <geist> "Kernel/User page tables isolation: disabled on command line." is what i have on the nehalem, because i am running nopti
13:14:30 <geist> do yo h ave nopti on your nehalem too?
13:15:13 <_mjg_> don't tell anyone but it is running 3.13.11-100.fc19.x86_64
13:15:28 <_mjg_> i'm afraid to upgrade
13:15:37 <geist> so pre pti, which makes sense, because it has a similar speed signature as my nehalem
13:15:50 <geist> most of my other spare linux boxes are zen so they're unaffected
13:16:17 <_mjg_> btw https://svnweb.freebsd.org/base?view=revision&revision=333241
13:16:19 <bslsk05> ​svnweb.freebsd.org: [base] Revision 333241
13:16:21 <Ameisen> I'm actually curious now if there's a performance penalty to denormals in software implementations
13:16:28 <Ameisen> atm I detect... nothing.
13:16:35 <Ameisen> I just do a trivial conversion, no checks.
13:16:48 --- join: zeus1 (~zeus@197.239.5.46) joined #osdev
13:17:01 <geist> _mjg_: https://docs.google.com/document/d/e/2PACX-1vSV7fuj-KFgqmRbKg98izD8Hgqp7I8w5Pdvi6hEbICxMGzxXk0OB3aHajeyKoY_l6SFWT7E4yLYZ0-u/pub is my doc in progress. will fill things in as i get it
13:17:02 <bslsk05> ​docs.google.com: Syscall_timing
13:17:15 <_mjg_> nice, thanks
13:17:26 <geist> well, https://docs.google.com/document/d/1RYGlcyrOnroLeoWXN_Qg8tI01SwdGvaWxyrH-tBWsns/edit?usp=sharing for the real one if you want
13:17:27 <bslsk05> ​docs.google.com: Syscall_timing - Google Docs
13:17:32 --- join: eremitah (~int@unaffiliated/eremitah) joined #osdev
13:17:46 <_mjg_> let me know if you will get around to benching a *recent* freebsd kernel
13:17:49 <_mjg_> 32 bit i presume
13:18:06 <_mjg_> if so, patch in some unfucking did for the 64 bit base
13:18:14 <_mjg_> i will*
13:18:19 --- quit: oaken-source (Ping timeout: 240 seconds)
13:18:25 --- quit: awang_ (Ping timeout: 255 seconds)
13:18:34 --- quit: bauen1 (Remote host closed the connection)
13:20:47 --- join: pounce (~pounce@c-24-11-27-195.hsd1.ut.comcast.net) joined #osdev
13:24:10 <geist> arm machine seems to be in the same ballpark
13:25:30 <_mjg_> so i got almost identical getuid and getppid perf on my box
13:25:46 <geist> good, good baseline
13:26:14 <_mjg_> i *suspect* there is some vdso fuckery in your case which falls back to calling the kernel
13:26:23 <geist> in exactly which case?
13:26:24 <_mjg_> but the extra branches screw things up
13:26:29 <_mjg_> getuid slower than getppid
13:26:54 <geist> there are already multiple situations here: it's a pentium 3, 32bit, and it wont have syscall
13:27:05 <geist> so it's sort of apples/oranges there
13:27:30 <_mjg_> my core2 also has almost identical getuid and getppid
13:27:39 <_mjg_> just tryint to make sense of a case where getuid is slower than getppid on the same box
13:27:42 <_mjg_> sounds odd
13:27:47 <geist> wait, look at the numbers again. actually in most cases the getppid is faster
13:27:48 <_mjg_> if anything it should be the opposite
13:27:49 <geist> even on arm
13:28:04 <geist> it's the p3 32bit where it's slower (like you'd expect)
13:28:11 <geist> so i'm suspecting all of these are vdso
13:28:16 <geist> or all syscall. but not a mix
13:28:28 <_mjg_> getppid cannot be cached, getuid can be
13:28:43 <geist> you say that but we should confirm before spending too much mroe tmie here
13:28:44 <_mjg_> so getppid should be the same or slower
13:28:51 <geist> otherwise we'll just end up benchmarking the vdso
13:29:08 <_mjg_> you can't catche getppid because the parent can change at any time
13:29:15 <geist> agreed. but confirmation is required
13:29:26 --- quit: lldd_ (Quit: Leaving)
13:29:34 <geist> since the numbers are so close, there's no way one of them is a syscall and one isn't
13:29:35 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
13:29:38 <_mjg_> well that's easy to check, you can attach yourself with systemtap to getppid entry point
13:29:40 <geist> it should skew the numbers by a lot
13:29:49 <geist> how about you do it since yuo know this better
13:29:53 <_mjg_> sure
13:31:30 <_mjg_> anyway, since the numbers are close but getuid is visibly slower, i suspect there are some vdso crappers in usersapce which end up calling the sycall anyway
13:31:33 <_mjg_> but add overhead
13:32:00 <geist> re-read the numbers please. getuid is not visibly slower on all of the 64bit stuff
13:32:04 --- join: JusticeEX (~justiceex@pool-98-113-143-43.nycmny.fios.verizon.net) joined #osdev
13:32:10 <geist> only on the 32bit machine, but that's a different arch so we can look at it separately
13:32:12 <_mjg_> i know, i mean the 32 biot case
13:32:21 <geist> sure, but even in the 64bit case they're very close
13:33:49 <geist> i'm going to guess that getuid is not a vdso call, though you think it is
13:34:01 <geist> last i looked at the vdso stuff in linux, it was fairly limited
13:34:13 <_mjg_> test[30564] 0xffffffffb00b4d80 : sys_getppid+0x0/0x30 [kernel] 0xffffffffb0803932 : entry_SYSCALL_64_fastpath+0x1a/0xa4 [kernel] (inexact)
13:34:18 <_mjg_> getppid is most definitely a syscall
13:34:26 --- quit: ALowther (Ping timeout: 260 seconds)
13:34:28 <geist> so now do the same for getuid
13:34:38 <_mjg_> it should be the same since it has the same perf
13:34:47 <_mjg_> it should be the same since it has the same perf on this box
13:34:50 <geist> should be and personally verified are two different things
13:35:04 <geist> look before i dump potentially hours into this i want to know precisely what i'm measuring
13:35:14 <geist> otherwise tables will be flipped
13:35:18 <_mjg_> legit inquiery, no argument
13:35:28 <_mjg_> test[30959] 0xffffffffb00b4db0 : sys_getuid+0x0/0x30 [kernel] 0xffffffffb0803932 : entry_SYSCALL_64_fastpath+0x1a/0xa4 [kernel] (inexact)
13:35:58 <geist> from poking around in the source, it looks like gettimeofday is definitely a vdso call
13:36:02 <geist> or at least has some nonsense around it
13:37:05 <_mjg_> still, i have not heard of anyone caching getppid (not only why would you do it, but how)
13:37:36 --- quit: Arcaelyx (Quit: My MacBook has gone to sleep. ZZZzzz…)
13:38:33 <_mjg_> *getpid* is definitely vdso-ed
13:38:42 --- quit: attah (Quit: Leaving)
13:38:48 <_mjg_> 186200852 ops
13:39:11 <_mjg_> 11659890 ops for getppid
13:39:25 <geist> yah, gettimeofday too
13:39:36 <_mjg_> gettimeofday is the standard in thsi regard
13:39:43 <_mjg_> i somehow thought getuid also got the treatment
13:39:56 <_mjg_> perhaps i confused the state of a certain flamewar with what went in
13:40:26 <geist> maybe its because you can setuid, so it can change?
13:40:44 <_mjg_> glibc has an extreme hacks for this
13:40:44 <geist> though seems that still you can just read a variable the kernel publishes for this thread/process
13:41:03 <_mjg_> there is no concept of process creds, onlly thread creds
13:41:24 <_mjg_> so when you setuid in a multithreaded proc, glibc makes all the threads also do the call
13:41:30 <geist> oh ugh!
13:41:32 <_mjg_> ye
13:41:41 <_mjg_> with this in place adding caching would not be a big deal
13:41:43 <geist> that sounds like a linux specific thing, does posix say it works that way?
13:41:52 <_mjg_> but arguably getudi is not really abused anywhere as much as getpid
13:42:27 <_mjg_> afair posix has no concept of per-thread credentials. provided nothing fails, the behaviour is as if you had proc creds
13:42:44 <_mjg_> so it is conformant, although ugly as shit
13:42:48 <geist> i guess i should reboot this box and try it with PTI. theoretically they could have left the getuid stuff on the shared side of the PTI bits
13:42:53 <_mjg_> this is basically a side effect of nobody cleaning up the old "threading" hack
13:42:59 <geist> maybe the 'fast' syscalls do that
13:43:18 <geist> yeah the clone threading stuff in linux is pretty ugly
13:43:25 <_mjg_> everyone was doing it
13:43:31 <_mjg_> bsds too
13:43:38 <_mjg_> well chances are solaris did not, that i don't know
13:43:51 <geist> right, solaris i generalyl consider the gold standard of that stuff on unix
13:44:00 <geist> though i have no real basis to back it up, it was probaby just as terrible
13:44:23 <_mjg_> btw
13:44:23 <_mjg_> probe kernel.function("sys_getuid") { printf("%s[%d]\n", execname(), tid()); print_backtrace();
13:44:28 <_mjg_> }
13:44:28 <geist> and the BSDs didn't get true user space threads for a long time, probably just did whatever linux did maybe
13:44:51 <geist> is 'probe' a command line?
13:45:06 <_mjg_> that's what you put into a file fed to systemtap
13:45:25 <geist> ah
13:45:25 <_mjg_> sudo stap getppid.stap
13:45:27 <_mjg_> et voila
13:45:37 <_mjg_> although it may require a painful setup on non-red hat boxen
13:45:38 --- quit: spare (Quit: leaving)
13:46:06 <geist> seems like ktrace would show it too
13:46:37 <geist> but anyway, we've established that both of those are syscalls, so the fact that on 32bit the ppid is slightly faster is probably just some internal 32bit detail
13:47:33 <_mjg_> on fbsd or os x you would dtrace -n 'syscall::getppid:entry { @[execname,stack()] = count(); }'
13:47:47 <_mjg_> ktrace is kind of wonky
13:48:18 <geist> anyway, what would be the math here... 500mhz at 1321322 cycles per 1 second
13:48:30 <_mjg_> anyway i read a lot of illumos/solaris, while the code is a pleasant read in general, i got over initial thoughts of greatness
13:48:58 <_mjg_> it may be it was better than anything else at the time
13:49:02 <geist> so that's about 378 cycles/call
13:49:40 <geist> on the nehalem it's about.... 265 cycles/call
13:50:07 --- join: fp7 (~fp7@unaffiliated/fp7) joined #osdev
13:50:15 <_mjg_> sounds legit. have you tried the 32 bit binary on it?
13:50:24 <geist> and on the threadripper is about 206 cycles
13:50:34 <geist> good idea. lets see
13:50:53 <geist> slower by 20% or so
13:50:57 <geist> 6852170
13:51:08 <geist> vs 8527704
13:54:34 <geist> the threadripper is a little fudgey since the speed of the cpu floats around a little bit, but when just one core is spooled up it seems to stabilize around 4.1Ghz
13:55:21 --- quit: fp7 (Quit: .)
13:56:25 <_mjg_> oh right, embarassing perhaps but i don't know how to disable it on linux
13:56:43 --- quit: JusticeEX (Quit: Changing server)
13:57:15 <_mjg_> so i got a freebsd vm running on that nehalem box
13:57:16 <geist> not sure you can that easily,it's a feature. you just have to generally have a idle machine and try to just spool up one core
13:57:33 --- join: fp7 (~fp7@unaffiliated/fp7) joined #osdev
13:57:39 <geist> yah theoretically the VM should be about as quick, since it's just a ring transition
13:57:50 <geist> i have a freebsd vm on that nehalem box too
13:57:52 <_mjg_> i don't have a super apples:apples comparison since this is bare metal vs vm, but should be fine
13:57:55 <_mjg_> anyway
13:58:05 <_mjg_> fbsd has a little bit of an edge in my test
13:58:10 <_mjg_> which i find surprising
13:58:22 <_mjg_> (note: this is very fresh development version, the release kernel is slow as shit)
13:58:40 <geist> the bsds generally use a vdso like thing called the commpage, iirc
13:58:49 <geist> i know mac does, i worked on the ARM version of it when i was at apple
13:59:05 <_mjg_> there is a shared page for gettimeofday, but nothing else afair
13:59:09 --- quit: variable (Quit: /dev/null is full)
13:59:21 <geist> the syscall vector was through there too
13:59:41 <geist> but mostly for the same reasons: provide a cpu specific fast mechanism
14:00:01 <geist> i remember for some reason on PPC they had pointers to fast memcpies and whatnot that the kernel knew about
14:00:12 --- quit: fp7 (Remote host closed the connection)
14:00:24 <_mjg_> anyway i got: getppid periteration linux 0.000000095 freebsd 0.000000092
14:00:31 <_mjg_> so almost the same with slight edge to fbsd
14:00:33 <geist> though *really* on anything but x86-32 there is usually only one defacto or official syscall instruction, so you dont need that much fallback nonsense
14:00:49 --- join: variable (~variable@freebsd/developer/variable) joined #osdev
14:00:58 <geist> so that's probably just number of instructions in their path
14:01:01 <_mjg_> i suspect they mostly did it to have an official route for all syscalls
14:01:07 <geist> right
14:01:17 <geist> we enforce it on fuchsia actually. you *must* make a call via the vdso
14:01:24 <geist> all other return addresses are invalid
14:01:36 <_mjg_> i'm surprised linux is doing faster here. freebsd has very serious pessimisations
14:01:56 <_mjg_> like no joke, fucking function calls through pointers and whatnot
14:02:00 <geist> sometimes it surprises you, turns out the part that you think matters is not that important
14:02:12 <geist> doubleplus so if it's all hot in the branch predictor
14:02:18 <_mjg_> i actually spent time profiling this, there is about 3% lost on this fact
14:02:24 <_mjg_> i did not profile on linux
14:02:40 <Brnocrist> if I remember correctly freebsd implemented fastcall (syscall/sysenter) very later, at least 2/3 years after linux
14:02:49 <_mjg_> plausible
14:03:44 <geist> but then the linux one probably gets rewritten every 3 weeks because someone figured out a way to make it harder to read and shave off a quarter cycle on some gigantic 1k core box
14:04:23 <_mjg_> looks even worse for getuid: http://dpaste.com/3AT5JQF
14:04:24 <bslsk05> ​dpaste.com: dpaste: 3AT5JQF
14:04:37 <geist> also make sure you're not seeing a kpti vs no kpti thing there
14:04:44 <geist> as in linux with kpti vs a freebsd without
14:04:47 <_mjg_> the cpu_set/fetch suckers ae the avoidable calls
14:04:58 <_mjg_> i already posted the kernel
14:05:04 <_mjg_> it predates meltdown discovery by 4 years
14:05:32 <_mjg_> so unless someone snuck in a fix, it's fine 8)
14:05:34 --- quit: glauxosdever (Quit: leaving)
14:06:05 <geist> yeah reminds me, at some point i'm going to have to rethink the cpu vs thread pointer on arm
14:06:28 <geist> someone at work had a fairly good set of reasons why i should switch it around (tpidr points to curr cpu, x18 points to current thread)
14:07:22 <geist> a lot of it boils down to which of the two do you use more often, which can be synthesized from the other, and which ones need to be idempotent
14:07:25 <_mjg_> sigh
14:07:29 <_mjg_> access(path, O_RDONLY);
14:07:30 <_mjg_> close(fd);
14:07:33 <_mjg_> solid benchmark
14:07:53 <geist> i put the curr cpu pointer in x18 because then i can access it via a single instruction (ldr/str reg, [x18, #offset]) and thus it has the same property as gs:
14:08:48 <geist> and so if the cpu gets context switched in the middle there, but... really the number of cases where you want to do something against the curr cpu structure outside of some sort of preemption disable is pretty rare
14:08:57 --- quit: variable (Quit: /dev/null is full)
14:09:00 <geist> and if yuo did you can still be careful to cache the pointer so at worst it's just a stale cpu
14:09:43 <geist> and x18 has the properly that gs has in x86 where it gets trashed by user space, so on the first instruction of an exception you need to get it back somehow (and save the old x18)
14:09:44 <_mjg_> is not that fatal though?
14:10:19 <geist> depends on what you're doing. if you're, for example, bumping a counter for debugging purposes you can argue that at most it's just a stale bump
14:10:22 <_mjg_> unless you have a lock or use atomics for whatever cpu-local you are modifying, this sounds like a bug
14:10:28 <_mjg_> ah ok, ye
14:10:49 <geist> technically you could have a cpu-going-offline scenario where the acgtual structure becomes stale
14:11:10 <geist> but it's currently allocated out of a per cpu aligned array of structures, so in general the structure wont go away, even if the cpu gets shutdown
14:11:11 --- join: variable (~variable@freebsd/developer/variable) joined #osdev
14:12:00 <geist> but the main use of the current cpu structure is getting the curr cpu number, and that is also intrisnsically out of date the instant you read it unless preemption is disabled, so same rules apply anyway
14:12:18 <_mjg_> what do you do with the number?
14:12:31 --- join: awang_ (~awang@cpe-98-31-27-190.columbus.res.rr.com) joined #osdev
14:12:33 <geist> there are some scheduler related stuff, but that's all down inside locks
14:12:46 <_mjg_> ah, that's more like it:
14:13:08 <_mjg_> access: linux 0.000000726 vs freebsd 0.000003073
14:13:09 <_mjg_> :)
14:14:14 <_mjg_> (this however is predominantly zfs just being slow single-threaded)
14:14:24 <geist> yeah that uses lots of layers of stuff
14:14:30 <_mjg_> 6.6 kernel _sx_xlock zfs_freebsd_access:2.1 zfs_zaccess:1.0 ...
14:14:31 <_mjg_> 6.3 kernel _sx_xunlock zfs_freebsd_access:1.7 ...
14:14:41 <_mjg_> top 2
14:14:45 <geist> access i'm a little surprised doesn't terminate int he dir cache actually
14:15:01 <geist> have to think about that, but i figured you might be able to store the access bits in it too, and avoid diving into the fs
14:15:04 <_mjg_> you always have to do permission checking
14:15:07 <_mjg_> there is no i/o done
14:15:25 <geist> well sure, but i mean not even entering the fs if it's fully cached in the dir cache in the vfs
14:15:26 <_mjg_> this is where you run into trouble if there is anything fancy going on
14:15:37 <_mjg_> like extended attributes, acls and so on
14:15:42 <geist> yeah guess so
14:15:46 <_mjg_> however
14:15:56 <_mjg_> the common case of the dir just being readable no problem can be hacked
14:16:13 <_mjg_> i was thinking of doing that, would avoid A LOT of work
14:16:24 <geist> so i have to say i'm a bit dissapoint that the cortex-a72 box is slower than the nehalem
14:16:27 <_mjg_> dragonflybsd has a hack of this sort
14:16:30 <geist> i figured it's syscall path would be similar
14:17:02 <geist> the instruction itself is certainly darn simpler and no kpti
14:17:31 <geist> though.... arm does funnel the syscall through a single synchronous exception vector
14:17:48 <geist> so you do have to test early and have a fast path for the syscalls, which does complicate things a bit
14:18:00 <geist> it's hard to have a dedicated register save/restore for syscalls
14:19:22 --- quit: variable (Quit: Found 1 in /dev/zero)
14:20:36 --- join: Shamar (~giacomote@unaffiliated/giacomotesio) joined #osdev
14:21:32 --- quit: baschdel (Ping timeout: 256 seconds)
14:21:36 <geist> https://github.com/torvalds/linux/blob/master/arch/arm64/kernel/entry.S#L631 for example
14:21:39 <bslsk05> ​github.com: linux/entry.S at master · torvalds/linux · GitHub
14:22:04 <geist> that's after you've saved the state and whatnot, you have to now basically have a switch statement deciding why the sync exception handled. it's pretty and clean, but harder to do efficiently
14:22:21 <geist> the b.eq to el0_svc is the actual syscall path
14:24:49 --- quit: daniele_athome (Ping timeout: 240 seconds)
14:27:01 <pounce> What does the write-through paging bit do?
14:28:36 --- join: bemeurer (~bemeurer@104.200.129.61) joined #osdev
14:28:46 <bcos> pounce: Causes the cacheing method to be "write-through" :-)
14:29:29 <bcos> (specifically; reads come from cache and cause cache misses like normal, but writes always update the underlying RAM and don't just update the cache)
14:30:02 <bcos> (essentially, the write goes through the cache to RAM)
14:30:10 --- join: Arcaelyx (~Arcaelyx@rrcs-184-75-117-179.nyc.biz.rr.com) joined #osdev
14:30:24 --- quit: Arcaelyx (Client Quit)
14:30:39 <pounce> if the bit is unset do they update the cache too?
14:30:44 <pounce> I mean set*
14:30:49 <bcos> Yes
14:32:15 <geist> write back is the usual mode
14:32:33 <geist> as in writes go to the cache and the cache may eventually write it back to memory over time
14:32:36 <pounce> Oh, is there any benefit to write through then?
14:33:03 <pounce> This may seem silly, but do atomics write straight to ram then?
14:33:41 <_mjg_> no
14:34:16 <geist> write through you might use to map a framebuffer or whatnot
14:34:25 <geist> where when you write to it you want it to flush more or less right now
14:35:03 <pounce> ah, gotcha
14:35:07 <_mjg_> in ccNUMA (and that's what you got) cpus talk with each other and in particular a modified atomic can be served directly from the cache of another cpu
14:35:30 <_mjg_> (well it may be you have only one node, but if you got more than 1, see above)
14:35:43 <pounce> huh
14:36:01 <geist> and even that, hrough all the levels of caches the cpus are coherent, so atomics live in cache space just the same as any other memory access to cached memory
14:36:16 <bcos> pounce: Cache coherency protocols ensure that CPU can atomically update cache (and not RAM)
14:36:24 --- quit: bemeurer (Quit: WeeChat 2.1)
14:36:33 --- join: bemeurer (~bemeurer@104.200.129.61) joined #osdev
14:36:48 <geist> it's so uch the case that on ARM you *must* use cached memory for an atomic
14:37:05 <geist> uncached memory (via page table mappings that say dont cache this) will immediatley fault f you try to use an atomic
14:37:28 <geist> unclear what happens on x86 if you try that. most likely it'll magically work
14:37:30 --- quit: bemeurer (Client Quit)
14:37:35 <_mjg_> geist: do you know perhaps what are the perf penalties for false sharing on arm?
14:37:39 --- join: bemeurer (~bemeurer@104.200.129.61) joined #osdev
14:38:09 <geist> unknown to me
14:38:20 <bcos> For 80x86, if you do an atomic on "uncached" then it (conceptually) locks the bus (not sure what that actually means for modern systems with links and not buses)
14:38:24 <geist> seems like it'd be highly core specific depending o how their caches are laid out
14:38:47 <geist> bcos: yeah, on say an early machine like a 386 the presence of a cache wasn't even mandated at all
14:38:49 <pounce> computers are weird
14:39:10 <geist> in fact i wouldn't be surprised if the WT bits and whatnot that pounce was talking about showed up in 486
14:39:42 * bcos can imagine some horrific corner-cases - e.g. atomic access that's misaligned across page boundaries where half is "write-back" and the other half isn't..
14:40:16 <_mjg_> that should generate a "go fuck yourself" exception
14:40:19 <geist> yah that's why in arm the atomics i believe need to be naturally aligned. avoids that page/cache line crossing problem
14:40:35 <pounce> "fuck-off, osdever"
14:40:40 <geist> since clearly it's the already existing cache coherency mechanism that actually implements the atomics
14:41:23 <pounce> what are links? (instead of buses)
14:43:27 <bcos> pounce: Rather than having "N things" sharing a common bus, you have a direct links that aren't shared
14:43:48 --- quit: unixpickle (Quit: My MacBook has gone to sleep. ZZZzzz…)
14:45:35 <bcos> It used to be (for 80x86) you'd have a "front side bus" shared by all the CPUs, plus PCI host controller/s, plus other stuff (maybe graphics). That all died - replaced by quickpath/hyper-transport and DMI links.
14:45:55 <bcos> Same for PCI - the old "PCI conventional" was shared bus, and PCI-E is serial links
14:46:23 <bcos> (actually, "serial links; in parallel")
14:46:51 <brynet> sortie: Did you ever look at the newer POSIX test suites at all for Sortix? The license agreement is pretty terrible.. but would the results would be interesting.
14:46:54 <brynet> sortie: http://get.posixcertified.ieee.org/docs/testsuites.html
14:46:55 <bslsk05> ​get.posixcertified.ieee.org: POSIX Test suites
14:47:37 <sortie> brynet: Nope
14:47:49 <brynet> Not that you care share them, of course. Or claim to have every used them.. *sigh*
14:47:54 <sortie> brynet: What's the problem with the license agreement?
14:48:09 <sortie> Wait.. the results are NDA and you can't say you've used it?
14:48:29 <brynet> pretty much.. on that page
14:48:47 <sortie> ... So every POSIX OS might have used this and I wouldn't know?
14:49:11 <brynet> sucks because of course you would fix bugs pointed out by running this
14:49:19 --- quit: Kimundi__ (Ping timeout: 240 seconds)
14:49:44 <sortie> Wait --
14:49:49 <sortie> Does it tell me what the bugs are at least?
14:49:49 <brynet> An OpenBSD ported just integrated an older open source suite from sourceforge, but.. seems quite outdated.
14:50:03 <_mjg_> at this point i would give up on this
14:50:11 <brynet> dunno, have to first jump through hoops
14:50:13 <_mjg_> the de facto standard is unfortunately whatever linux is doing
14:50:21 <brynet> Also: US Fax machine.
14:50:24 <_mjg_> the good news is that there is a decent test suite from them: ltp
14:51:37 <brynet> sortie: I guess getting POSIX certified would be a good idicator of "Oh hey, they ran the thing!"
14:51:49 <brynet> :P
14:52:11 <_mjg_> is there any value in getting posix cert to begin with?
14:52:12 <sortie> _mjg_: https://sortix.org/os-test/ ← I try to know a thing or two about what's actually de-facto
14:52:13 <bslsk05> ​sortix.org: os-test
14:52:23 <brynet> :)
14:52:42 <_mjg_> nice
14:52:43 --- quit: jp (Read error: Connection reset by peer)
14:53:10 <sortie> _mjg_: Not really. I mean if the open group was actually open and the free systems actually got certified, and POSIX was followed, that would be great.
14:53:30 <_mjg_> oh haha, opening a dir for reading
14:53:35 <_mjg_> old stuff
14:54:12 <sortie> There's big value in systems being interoperable and following standards
14:54:25 <sortie> But honestly getting certified is just a fancy sticker on the product
14:54:37 <_mjg_> no argument here, i don't like the linux monoculture
14:54:54 <_mjg_> i'm just saying the posix standard is probably not useful enough for real world progs
14:54:59 <_mjg_> classic example: poll
14:55:05 <_mjg_> (or select for that matter)
14:55:13 <sortie> poll(2) is POSIX
14:55:18 <_mjg_> it is
14:55:19 <_mjg_> and it sucks
14:55:21 <sortie> select is obsolecent
14:55:24 <brynet> I just found the idea of the test suite more required to pass for certification more interesting than certification itself-- but it's no good.
14:55:26 <pounce> I like poll :O
14:55:37 <sortie> Me too
14:55:40 <_mjg_> if you want perf, you go for kqueue, epoll or whatever else is on your platform
14:55:42 <sortie> _mjg_: Well opening a directory for reading is perfectly reasonable
14:55:44 <_mjg_> you don't use poll
14:56:00 <_mjg_> which brings me back to ^^
14:56:03 <sortie> _mjg_: The main use is to pass it to openat, or convert it into a DIR with fdopendir and then readdir
14:56:22 <_mjg_> there is a flag to open it and not for reading
14:56:27 <sortie> Honestly kqueue and epoll is a non-portable mess
14:56:32 <_mjg_> it is
14:56:33 <pounce> I wrote an event loop using poll for a project I work on and I loved it
14:56:36 <_mjg_> and posix does not offer an equivalent
14:56:45 <_mjg_> (probably rightfullyl so too though)
14:56:45 <pounce> what's bad with its performance?
14:56:47 --- join: jp (ayy@youlosethega.me) joined #osdev
14:56:54 <sortie> Nor should POSIX until systems figure this out
14:57:18 <sortie> pounce: Problem is that the pollfd's are uploaded to the kernel over and over again, that's O(N) for each iteration
14:57:37 <sortie> kqueue and epoll lets you upload the pollfd once and then change the subscription only when it changes
14:57:40 <_mjg_> and rescanned
14:57:46 <sortie> ^
14:57:56 <pounce> ah, gotcha
14:58:01 <_mjg_> try this https://people.freebsd.org/~jlemon/papers/kqueue.pdf
14:58:02 --- quit: jp (Read error: Connection reset by peer)
14:58:06 <bslsk05> ​people.freebsd.org <no title>
14:58:39 <pounce> that hasn't been a problem for this project bc few fds, but I can see that as a huge issue for big event-driven programs
14:58:41 <_mjg_> not that either of epoll or kqueue are the panacea, but they suck much less than poll
14:59:12 --- quit: rorx (Read error: Connection reset by peer)
14:59:42 <_mjg_> bottom line is, a non-trivial real world prog which wants to stick to posix is going to run int osevere performance issues
14:59:56 <pounce> also the maintainer is adamantly POSI
15:00:04 <_mjg_> even if apis are compatible there are too many internals quirks
15:00:15 --- quit: asdafdsasd (Quit: asdafdsasd)
15:00:18 --- join: immibis (~chatzilla@222-155-160-32-fibre.bb.spark.co.nz) joined #osdev
15:00:28 <sortie> I look forward to implementing kqueue in Sortix
15:00:29 <sortie> niht
15:01:35 --- join: rorx (~rorx@tserver.truestep.com) joined #osdev
15:02:10 --- quit: sortie (Quit: Leaving)
15:02:14 <_mjg_> i can only imagine the flamewar originating from an attempt to standarize the use of huge pages
15:02:33 <_mjg_> freebsd for instance deliberatelly does *not* expose a way to force the use
15:03:42 <brynet> non-blocking kqueue is also interesting, as you can poll on a kqueue fd, heh.
15:04:55 --- quit: SM0TVI (Ping timeout: 268 seconds)
15:04:55 --- quit: olsner (Ping timeout: 268 seconds)
15:05:17 --- join: jp (ayy@youlosethega.me) joined #osdev
15:05:23 --- quit: JonRob (Quit: Lost terminal)
15:05:34 --- quit: bemeurer (Quit: WeeChat 2.1)
15:05:44 --- join: bemeurer (~bemeurer@104.200.129.61) joined #osdev
15:07:50 --- quit: bemeurer (Client Quit)
15:08:00 --- join: bemeurer (~bemeurer@104.200.129.61) joined #osdev
15:08:24 --- quit: bemeurer (Client Quit)
15:11:19 --- join: olsner (~salparot@c213-89-109-20.bredband.comhem.se) joined #osdev
15:12:29 --- join: SM0TVI (~sm0tvi@unaffiliated/sm0tvi) joined #osdev
15:13:15 <radens> Is there an easy way to get the demangled names out of the qemu -d logs?
15:14:06 <geist> do you have an example?
15:14:24 <geist> if it's c++ mangled names, then generally piping it through c++filt will fix it
15:15:45 <pounce> if it's c++ mangled names then it's your penance for using c++
15:20:19 <aalm> hah
15:23:51 --- quit: dennis95 (Quit: Leaving)
15:25:59 <matviy> Hello! I'm trying to return from long mode to real mode on x86. The first thing I'm doing is creating a new GDT at somewhere around 0x80000 with a null and basic 64K descriptors, and LGDT'ing it.
15:26:22 --- quit: xenos1984 (Quit: Leaving.)
15:26:26 <matviy> But when i push the cs,ss values onto the stack and IRETQ to load the CS, my code jumps somewhere else
15:27:08 <matviy> The way I understand, LGDT requires a GDT at a physical address, but since I'm creating a GDT in long mode at 0x80000 w/ paging enable, I have no guarantee that my GDT is at physical address 0x80000
15:27:31 <matviy> Unless i re-page and identity-map that area
15:27:33 <matviy> Does that make sense?
15:27:36 --- join: unixpickle (~alex@c-24-5-86-101.hsd1.ca.comcast.net) joined #osdev
15:27:56 --- join: bemeurer (~bemeurer@104.200.129.61) joined #osdev
15:29:21 <lachlan_s> Rewrote the scheduler again!
15:29:26 <lachlan_s> Hopefully last time for a while
15:31:39 <pounce> I have written and deleted my scheduler twice now
15:32:06 <lkurusa> what’s a comput... scheduler? :-)
15:33:22 <lachlan_s> A computer is a thing that connects to a serial of tubes
15:33:27 <lachlan_s> s/serial/series
15:34:19 --- quit: quc (Ping timeout: 265 seconds)
15:35:35 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
15:38:24 <lachlan_s> pounce: Didn't know your os had a scheduler yet
15:40:18 --- quit: ALowther (Ping timeout: 276 seconds)
15:42:25 --- quit: awang_ (Ping timeout: 255 seconds)
15:48:36 --- quit: immibis (Ping timeout: 260 seconds)
15:49:02 <geist> matviy: yep
15:49:21 <geist> you gotta know the physical layout of what you're switching to since when ou drop paging you're back into physical mode again
15:53:04 <lachlan_s> Are there any registers that are the same value for all cpus?
15:58:00 <aalm> what
15:59:31 <aalm> lachlan_s, for what?
16:02:45 --- join: chao-tic (~chao@203.97.21.86) joined #osdev
16:07:07 <lachlan_s> aalm: Like, for storing data that can be accessed by any executing cpu without actually knowing where the specific location in memory is.
16:07:24 --- quit: rorx (Read error: Connection reset by peer)
16:11:12 --- quit: Shamar (Quit: Lost terminal)
16:13:38 <izabera> how many systems other than linux let you execute different processes in the same address space?
16:15:53 <aalm> DOS?:p
16:16:12 <izabera> at the same time
16:16:14 <geist> yah windows 3.11
16:16:28 <booyah> izabera: DOS, with TSR
16:16:36 <booyah> terminate state resident, oldschoool bitcheees
16:16:38 <geist> though insome sence both dos and windows 3 had separate address spaces, you just were supposed to use the segments that were given you
16:16:44 <izabera> ok but seriously, modern systems
16:16:45 <booyah> *stay
16:16:47 <geist> you could just go tear around and break things though
16:16:54 <booyah> izabera: ok, OpenDOS? (:
16:17:01 <geist> many embedded systems are single address space
16:17:19 <geist> mostly because the hardware doesn't support hardware to build separate ones
16:17:24 <geist> microcontroller class stuff
16:17:27 <izabera> ok let me clarify
16:17:28 <aalm> mmu<3
16:17:51 <izabera> MODERN systems, with MULTIPLE processes, with SEPARATE address spaces by default
16:18:06 <geist> then you just described precisely somethign that doesn't do what you awnt
16:18:08 <geist> so no
16:18:24 <izabera> linux lets you do that if you want
16:18:33 <geist> i suppose, it is likely to be a really bad idea
16:18:39 <geist> i dont know if anyone actually does that
16:18:54 <geist> perhaps the bsds do, i dunno how they implement clone
16:19:10 <izabera> does anyone even use this?
16:19:20 <bcos> izabera: Linux allows that?
16:19:21 <geist> i have no idea
16:19:35 <geist> i guess it does, based on your idea of what a 'process' is
16:19:48 <izabera> bcos: clone with CLONE_VM without CLONE_THREAD
16:19:50 <geist> i honestly dont know to the extent that it fully does
16:20:02 <geist> what about the file table, do you get a new one in that case?
16:20:07 <bcos> izabera: Cloning an address space is not the same as sharing an address space
16:20:16 <izabera> geist: you can decide if you want to share it or not
16:20:27 <geist> ah that's right, CLONE_VM would make a new copy, wouildnt it?
16:20:42 <geist> well, virtual copy. copy on write. that's basically fork()
16:20:48 <izabera> no?
16:21:05 <geist> well, you're apparently the expert. i dunno
16:21:14 <izabera> it's poorly named
16:21:17 <izabera> but it's the same
16:21:19 <izabera> not a copy
16:21:43 --- quit: mrush (Ping timeout: 264 seconds)
16:21:53 <bcos> Ah - I think I understand the confusion now. Linux has a funky tasking model, where "everything" is a task and some tasks (processes) have different virtual address spaces and other tasks (threads) share a virtual address space, and their internal "clone" is a schizophrenic mess that needs to work for both
16:22:07 <izabera> that's a good description
16:22:23 <geist> we thought about it a bit for fuchsia, since creating the process and creating the address space could be separate operations
16:22:34 <geist> a previous (unreleased) closed source os i wrote did exactly that
16:22:49 <geist> but we talked about it a bit and couldn't find a good reason to let anyone do that, so we decided not to
16:22:59 * bcos prefers a "process is just a container (with shared resources, like the virtual address space) that must contain at least one thread" model
16:23:12 <geist> yah tat's precisely what fuchsia does
16:23:14 <bcos> ..where scheduler mostly doesn't care about processes and only schedules threads
16:23:30 --- join: mrush (~user@64.72.212.39) joined #osdev
16:23:30 <geist> a process itself is an empty container, you must add a thread and map something in it to actually be useful
16:23:33 * bcos nods - much cleaner IMHO. Sadly, Linux is Unix and threads didn't exist in the 1960s
16:23:53 --- nick: mrush -> Guest29616
16:24:04 <lachlan_s> Yeah, in nebulet, processes are distinct objects and contain 1 or more threads.
16:24:18 <geist> i was going to have it be one more abstraction: a process needs to be bound to an address space and at least one thread to be 'running'
16:24:26 <geist> but that would allow binding of a single address space to multiepl processes
16:24:30 <geist> but then. ehhh, not useful
16:25:26 <geist> but we do have essentially a handle to the address space as a distinct thing froma process
16:25:38 <geist> it's just when you call process_create it gives you two handles: one for the process and another for the address space
16:25:52 <lachlan_s> Plus, fuchsia allows chopping up an address space and handing it to children, right?
16:26:06 <geist> correct
16:26:21 <geist> actually that's what you get a handle to: a range of the address space that is the entire aspace
16:26:26 <geist> you are assumed to subdivide it yourself
16:26:40 --- join: rorx (~rorx@tserver.truestep.com) joined #osdev
16:27:15 <lachlan_s> Unfortunately, because of the way wasm will work in the future, nebulet has to support sharing the entire address space with any number of other processes.
16:27:43 <geist> that's generally not a problem, as long as it's a 1:N relationship
16:27:53 <geist> having subranges of one address space mapped into another one is where it gets out of hand
16:28:09 <geist> conceptually you just have an aspace object that the process owns a ref to
16:28:22 <lachlan_s> No, I mean, a process supports sharing it's own entire address space
16:28:23 <geist> and when the cpu context switches it checks to see what aspace it's switching from and to, and does what it needs to
16:28:44 <lachlan_s> The sandboxed aspace
16:32:33 --- join: Philpax (~Philpax@144.132.100.253) joined #osdev
16:58:04 --- join: Pseudonym73 (~Pseudonym@e-nat-unimelb-128-250-0-54.uniaccess.unimelb.edu.au) joined #osdev
16:58:51 --- quit: zeus1 (Ping timeout: 246 seconds)
17:00:32 --- quit: atk (Quit: Well this is unexpected.)
17:00:47 --- join: atk (~Arch-TK@ircpuzzles/staff/Arch-TK) joined #osdev
17:06:35 --- join: Rodya_ (~Rodya_@64.62.224.29) joined #osdev
17:21:02 --- join: awang_ (~awang@cpe-98-31-27-190.columbus.res.rr.com) joined #osdev
17:40:19 --- quit: awang_ (Ping timeout: 264 seconds)
17:42:28 --- join: AverageJ0e (~joe@ip98-167-200-207.ph.ph.cox.net) joined #osdev
17:42:48 --- join: zeus1 (~zeus@197.239.5.46) joined #osdev
17:44:19 --- quit: Belxjander (Ping timeout: 240 seconds)
17:48:07 --- quit: chao-tic (Ping timeout: 264 seconds)
17:51:32 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
17:54:26 --- join: sh3iny (~sh3iny@2601:141:201:4328:a53e:395c:5567:4a2e) joined #osdev
17:56:28 --- quit: isd (Quit: Leaving.)
17:57:40 --- quit: banisterfiend (Quit: My MacBook has gone to sleep. ZZZzzz…)
17:58:33 --- join: promach__ (~promach@bb219-74-174-136.singnet.com.sg) joined #osdev
17:59:51 --- join: banisterfiend (~banister@ruby/staff/banisterfiend) joined #osdev
18:04:36 <_mjg_> oh interesting: So at this point, I may as well try to live up to my reputation. I have written up series for the other two defunct processors that were at one point officially supported by Windows NT: The MIPS and the PowerPC.
18:04:42 <_mjg_> https://blogs.msdn.microsoft.com/oldnewthing/20170901-00/?p=96925
18:04:44 <bslsk05> ​blogs.msdn.microsoft.com: I guess I may as well confess that I wrote the Itanium information in the Debugging Tools for Windows package – The Old New Thing
18:04:48 <_mjg_> i thought windows was x86 and itanium
18:07:49 --- join: awang_ (~awang@cpe-98-31-27-190.columbus.res.rr.com) joined #osdev
18:08:18 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
18:16:32 --- quit: Belxjander (Ping timeout: 250 seconds)
18:18:02 <geist> _mjg_: initkally NT was mips, alpha, ppc, x86
18:18:09 <geist> and supposedly i860, but that never shipped
18:18:29 <geist> 3.51 for sure, and I believe NT 4.0 shipped on most of those still
18:19:04 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
18:19:23 <_mjg_> solid
18:20:11 <_mjg_> so i had a look at getuid
18:20:19 <_mjg_> turns out it is way slower than you would think
18:20:38 <_mjg_> return from_kuid_munged(current_user_ns(), current_uid());
18:21:27 <_mjg_> it's basically a minor func call and branch fest
18:21:45 --- join: qeos|2 (~qeos@ppp91-79-242-242.pppoe.mtu-net.ru) joined #osdev
18:21:58 <_mjg_> in comparison getppid is simpler, but it still is not just a read
18:22:48 --- quit: awang_ (Ping timeout: 256 seconds)
18:23:07 <_mjg_> chances are solid the 32 bit cpu has a worse time dealing with it
18:23:54 --- quit: zeus1 (Ping timeout: 250 seconds)
18:24:19 --- quit: qeos (Ping timeout: 240 seconds)
18:26:27 --- quit: nj0rd (Ping timeout: 256 seconds)
18:27:00 <geist> yah. slow machine, less branch predict, much sad
18:27:38 <geist> remimds me, i should build a custom kernel for it with a bunch of the probably not useful nonsense turned off
18:28:45 --- join: MaryJaneInChain (~user@223.255.127.32) joined #osdev
18:29:35 <geist> or, probably more specifically, just put it away
18:30:22 <_mjg_> go retro and build mplayer
18:30:30 <_mjg_> i don't know any benchmarks for that sucker though
18:30:50 <geist> i did toss together a little hello world boot from floppy thing yesterday
18:31:00 <geist> i should polish on that and pyut it in github
18:32:05 <_mjg_> when you alloc memory in fuchsia, do you pass FOO_ZERO or whatever flag to have it zeroed out?
18:32:12 <_mjg_> or perhaps you explicitly memset?
18:32:15 <geist> in user space it's intrinsically zeroed
18:32:39 <geist> usual rules, new anonymous memory is full of zeros, because otherwise you'd havce to leak whatever was there
18:32:47 <_mjg_> ye, ye
18:32:53 <_mjg_> i mean malloc equivalent in the kernel
18:32:56 <_mjg_> i don't know what you got
18:33:03 <lachlan_s> Is there anyway to force physical memory to zero without memsetting it?
18:33:22 <geist> malloc in the kernel does not zero it, if you alloc a page out of the PMM it does not zro it. working on the next phase of the pmm and will probably add that as a flag
18:33:23 <lachlan_s> Without having to iterate over all of it, that is
18:33:23 <_mjg_> typically ram has random shit from previous bot
18:33:45 <geist> correct. so no
18:33:48 <_mjg_> if you don't zero "by hand", the content is unspecified
18:34:00 <geist> even from cold boot dram usually comes up with garbage in it
18:34:19 --- quit: Belxjander (Ping timeout: 240 seconds)
18:34:21 <_mjg_> it comes in handy, freebsd uses it to retain dmesg across reboots
18:34:51 <geist> yep
18:35:02 --- quit: promach__ (Quit: WeeChat 2.1)
18:35:23 <geist> and it seems no modern os does the whole background thread zeroing things, so i leave it up to the allocator
18:35:39 <_mjg_> xen is doing for vms
18:35:44 <geist> 95% of the allocations are either anonymous mmeory when demand faulting something in or a Copy on write
18:35:47 <_mjg_> kind of makes sense :>
18:35:56 <geist> yah for hypervisors you have to
18:36:18 <_mjg_> now i'm curious if bhyve does
18:36:23 <_mjg_> would not be shocked if it did not
18:36:33 <geist> that would be a terrible security leak
18:36:43 <_mjg_> not the first one
18:36:46 <_mjg_> or last
18:37:21 --- quit: Rodya_ (Quit: Leaving...)
18:37:45 <geist> reminds me, i should benchmark the new clzero thing for ryzen
18:38:00 <geist> probably will be like dc zva (on arm64) where it doesn't appear to be faster than a memset, except it doesn't dirty the cache
18:38:38 <_mjg_> dillon@dfly benchmarked it recently for pagezero purposes
18:38:41 <geist> so across the entire system it might allow for slightly better cache
18:38:43 <geist> yeah?
18:38:53 <_mjg_> it zeroes way faster, but the non-termporal stores result in saturating the bus
18:39:00 <_mjg_> so concurrent execution goes to shit
18:39:05 <geist> ah interesting
18:39:07 <_mjg_> linux sticks to rep stosq
18:39:50 <_mjg_> ... for amd
18:39:59 <geist> ah interesting
18:40:12 <_mjg_> also i'm not that confident clzero for pagezero is a good idea to begin with, even if it did not run into the saturation problem
18:40:21 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
18:40:29 <geist> interesting thing that arm's official memset actually uses dc zva
18:40:31 <_mjg_> you will end up accessing a lot of the page anyway
18:40:37 <geist> which i thought was slightly alarming
18:41:01 <_mjg_> and if you happen to have a page from the wrong node, you are just adding yourself latency
18:41:52 <_mjg_> few years back freebsd was using non temporal stores for zeroing, whacking this in favor of simple rep stos saved few % of buildkernel time on a 2 node box
18:41:56 <geist> https://git.linaro.org/toolchain/cortex-strings.git/tree/src/aarch64/memset.S is some pretty tricky shit
18:41:57 <bslsk05> ​git.linaro.org: memset.S\aarch64\src - cortex-strings.git - Optimized string routines for ARM and AArch64
18:42:00 <_mjg_> granted the kernel new very little about numa at the time
18:42:42 <_mjg_> that said, i was negatively surprised to learn it uses nt stores
18:43:05 <_mjg_> i guess the intent was to zero out big areas which are not expected to fit into the cache anyway
18:43:07 <geist> interesting., i'd think maybe the nt store would be useful if it's on the other node
18:43:14 <geist> but i guess it's worse because now it saturates the link
18:43:46 <_mjg_> it would just add latency on top of cc trafic
18:44:00 <geist> and i'd assume that most architectures (i know arm does and i'm sure x86does) can detect a straight cache line fill and not do a read to populate it
18:44:25 <geist> so the assumption is the cache aligned parts of the fills just allocate a new line in the local cache
18:44:44 <_mjg_> afair it does generate some misses, but perhaps depends on the microarch
18:44:55 <geist> might take a cycle or two to prime it
18:45:21 <geist> was looking at cortex-a53 prefetch behavior the other day. apparently it takes aabout 2 cache lines of linear reads before it starts to figure that you're going to need a prefetch
18:45:27 <_mjg_> but tl;dr the only os zeroing with nt stores today that i know of is openbsd 8)
18:45:31 <geist> then it starts running ahead, keeps up to 5 prefetches in flight
18:45:35 <_mjg_> and perhaps they are not the perf model
18:45:53 <geist> well, fuchsia does on arm, but i will need to bench that at some point and decide what to do there
18:46:35 <_mjg_> (actually netbsd may also be in the camp, not sure)
18:47:54 * bcos wonders if/when Intel will implement AMD's new CLZERO instruction
18:48:03 <geist> like many things, if it's a scalability issue it's hard to get too involved in the microarch details when you have bigger fish to fry
18:48:14 <geist> at the end of the day everything matters, but you focus on the big stuff first
18:48:36 <geist> fuchsia is full of scalability problems, as you've seen, but there are only so many hours of the day so i have to keep focused on the big issues
18:49:02 <geist> and iot's easy (and fun) to get side tracked in minutae, but not necessarilly the best use of your time
18:49:18 <geist> so i allow myself a certain amount of minutae per unit of real work
18:49:27 <_mjg_> :)
18:49:50 <geist> ie, make the scheduler understand hyperthreading, then you have permission to benchmark clzero :)
18:49:53 <_mjg_> one stupid inefficiency a day would be a chill little goal
18:49:59 <geist> yah
18:50:14 <_mjg_> taking any microbenchmark really makes you sad when you take a look at the profile on the bsds
18:50:20 <geist> in theory we have 'performance friday' but for the most part everyone just checks in their half baked patches and goes home
18:50:27 <_mjg_> :)
18:50:52 <_mjg_> i think you should get at least one person who is anal about arch details
18:50:58 <geist> working on darwin at apple gave me a little different perspective on some of this stuff
18:51:17 <geist> at the end of the day what matters is what people see (if you're building something with a gui, and is primarily targetted towards users)
18:51:55 <geist> linux, great, it can own the server/unix market, but not every little detail and little performance tweak is always required
18:52:12 <geist> and there are serious downsides to writing complex, hard to read, microptimized code
18:52:31 <geist> perhaps linux has enough monkeys to plow through it, but when you're a smaller team you have to pick your battles
18:52:31 <_mjg_> squizing every last cycle out is a waste of time in most cases
18:52:52 <geist> right, and it's feasible if you have thousands of people each wanting to get their tiny microptimization in. hence they infinite monkeys
18:52:55 <_mjg_> the thing is most real world code has just stupid pessimizations all around
18:52:59 <_mjg_> if you actually take a look
18:53:02 <geist> great, but that's not how real projects are run with finite resources
18:53:24 <_mjg_> trivial sample: someone relocks in a loop
18:53:28 <_mjg_> instead of hoisting the lock up
18:53:41 <_mjg_> or malloc/free
18:54:04 <geist> right, i want to focus on the big picture, low hanging algorithmic fruit first
18:54:27 <geist> if you sit down and write a microbenchmark you'll find something every time, and then you'll wonder why no one noticed it, etc
18:54:29 <_mjg_> i suspect you could get a decent intern to microbench and hunt for blatant issues of the sort
18:54:47 <_mjg_> could be a nice intro to kernel programming
18:54:57 <lachlan_s> And, compilers are quite good these days and can often figure out how to squeeze out that extra performance
18:55:17 <_mjg_> not if the code you got actively prohibits them
18:55:20 --- quit: tacco| ()
18:55:23 * _mjg_ points at bsd
18:55:26 <geist> yah and even then we have to be careful, but 9/10 times you'll find some detail that may look like it's worth optimizing, but really it's an edge case of a larger thing and the larger thing needs to be redone
18:55:35 <geist> but... perfect is the enemy of good
18:56:03 <_mjg_> is there a real world prog for fuchsia yet?
18:56:21 <geist> you can run the ui and stuff, but no not really
18:56:54 <geist> par tof my challenge is to build a generic foundation that runs a more and more specific thing as you go higher up
18:57:19 <geist> so, sometmies folks just want feature A like that, but i have to consider how A fits in an N year timeframe, assuming that the base stuff gets scaled in some different way
18:57:36 <geist> ie, i want zircon to be a usable system outside of our end goal
18:57:47 <geist> and no we can't talk about what it is
18:58:19 --- join: awang_ (~awang@98.31.27.190) joined #osdev
18:58:25 <lachlan_s> As anyone tried to get chrome running on fuchsia yet?
18:58:36 <geist> oh yeah absolutely
18:59:16 <geist> chrome is actually highly portable
18:59:27 <geist> at least as much as modern gui software is nowadays. since it runs on mac, linux, windows
18:59:51 <_mjg_> one day i'm going to drop all the os-level stuff and write something in node.js
19:00:10 <geist> someone here was working on doing stuff in wasm
19:00:15 <lachlan_s> It's fun for like 30 minutes, and then you get bored
19:00:20 <geist> which i guess is like something like that
19:00:24 <lachlan_s> Sorta
19:00:37 <lachlan_s> I feel like wasm is still pretty low-level
19:00:58 <_mjg_> ye the point is to not have to manage memory by hand
19:01:09 <_mjg_> there are languages where you can open, read and close the file in one line
19:01:13 <_mjg_> pretty cool
19:01:27 <lachlan_s> Gets rid of the fun in it
19:01:43 <_mjg_> i don't have much fun error checking
19:02:00 <geist> i was fiddling around with quickly bootstrapping a 386 os yesterday, and was thinking it'd be fun to use a pure segmentation user space for lulz
19:02:15 <geist> then i realized that it'll be a pain to configure a compiler to deal with the different segments
19:03:32 <lachlan_s> Travis, did you ever play around with the native client?
19:04:46 <geist> me?
19:04:55 <lachlan_s> Ye
19:04:59 <geist> native client of what?
19:05:09 <lachlan_s> Nacl
19:05:15 <geist> no. not at all
19:05:50 <lachlan_s> They apparently used segmentation for sandboxing
19:06:07 <geist> oh sure, but only on 32bit i assume
19:06:17 <lachlan_s> Yep
19:06:32 --- join: zeus1 (~zeus@197.239.5.46) joined #osdev
19:06:35 <froggey> openwatcom supports far pointers, just sayin'
19:07:14 <geist> froggey: yeah was thinking about that, but does it generate 32bit protected mode?
19:07:21 <froggey> yep
19:07:25 <geist> as in 32bit protected mode segmentation based mode
19:07:56 <ryoshu> woohoo!
19:07:57 <geist> i was thinking it'd be fun to do something unix v7 style. as in each oprogram had a text/data/bss segment + a stack segment + a heap segment
19:08:14 <ryoshu> the exploit is done so I can detach from this channel
19:08:15 <geist> or maybe text/data/bss/heap + brk() to extend taht segment, + a stack
19:08:33 <geist> oh great, we helped you sploit?
19:08:37 <ryoshu> yes
19:08:39 <geist> are we at least going to get credit on the CVE?
19:08:43 <froggey> I'm not sure if it supports running with cs!=ss!=ds
19:08:49 <ryoshu> no
19:08:51 --- join: variable (~variable@freebsd/developer/variable) joined #osdev
19:08:51 <rain1> its not OS dev but can i ask help on it here? someone wrote their own C compler and libc in C and i cant get it working
19:09:02 <ryoshu> geist it has been committed
19:09:08 <lachlan_s> ryoshu: what's the vuln?
19:09:13 <geist> ah, you sploited it so you could fix it? do tell!
19:09:47 <geist> froggey: right, that's what i mean. when all of those have to be == each other that's the actual definition of flat memory model
19:10:00 <geist> as in the compiler emits code assuming that pointers are interchangeable between those
19:12:03 * bcos wouldn't be too surprised if (e.g.) the code GCC generates can handle a "harvard architecture-like" separation between CS and SS/DS/ES
19:12:20 <bcos> - most code doesn't self modify
19:12:21 <froggey> *maybe*? the manual mentions small & large memory models for 32-bit code. I guess small is flat & large is far pointers for everyone
19:12:31 <ryoshu> cvs -d anoncvs@anoncvs.netbsd.org:/cvsroot rdiff -u -r1.2 -r1.3 src/tests/lib/libc/sys/t_ptrace_amd64_wait.h
19:12:33 <ryoshu> it's here
19:12:38 --- quit: sh3iny (Quit: WeeChat 2.0.1)
19:12:49 <ryoshu> I got here feedback on ljmp
19:13:09 * geist blinks, remembering that people actually use cvs still
19:13:57 <ryoshu> "Verify mitigation for CVE-2018-8897 (POP SS debug exception)"
19:14:07 <geist> got it. cool
19:14:49 --- quit: zeus1 (Ping timeout: 240 seconds)
19:15:02 * klange blinks out of existence at the thought of people still using cvs
19:15:07 <lkurusa> wat
19:15:27 <ryoshu> cvs isn't bad
19:15:36 <lachlan_s> I had to use cvs at the saas job I had last summer
19:15:41 <lachlan_s> Wasn't awful
19:15:48 <lachlan_s> Git is way better though
19:16:23 <ryoshu> I find bashing tools as unproductive discussion
19:16:27 <ryoshu> it does the job
19:16:45 <_mjg_> $ uname -s
19:16:51 <_mjg_> 4.2BSD
19:16:54 <_mjg_> $uname -m
19:16:55 <_mjg_> vax
19:17:01 <_mjg_> works for me
19:17:20 <ryoshu> vax is good
19:18:22 <klange> if I didn't live in a tiny apartment in Tokyo I would love to have some old hardware like a vax
19:18:47 <ryoshu> I have three vax computers, but I'm burdened by amd64 work
19:18:49 <froggey> geist: ok, yes. openwatcom supports a memory model (large) where all pointers (code & data) are far pointers and should work with a non-flat segment setup
19:18:52 * bcos suspects there'd be about the same amount of living space
19:19:39 <bcos> - probably hard to get good plumbing (bathroom, kitchen) inside a VAX though..
19:19:53 <bcos> :-)
19:20:00 <klange> Depends on the model.
19:20:57 <_mjg_> ryoshu: now that's nice
19:21:00 <ryoshu> there were military models, big like a panzer car with a computer inside
19:21:14 <klange> That said, if you're living int he vax, you'd probably have to remove all the hardware.
19:21:36 <klange> Back to reality, a late-model vaxstation would be a nice thing to own.
19:21:38 <bcos> Just replace the hardware with smaller modern parts ;-)
19:22:10 <ryoshu> netbsd devs produced usb for vax, so we can use wifi
19:22:21 <ryoshu> and other devices
19:22:41 <klange> On that note, I would love an old Cray case.
19:22:45 <klange> To use as a couch.
19:22:59 <ryoshu> cray used to borrow netbsd's tcp/ip stack
19:23:04 <_mjg_> i would use sunfire as a hair dryer
19:23:09 --- join: Arcaelyx (~Arcaelyx@2604:2000:f14a:2500:ccf4:ef50:f79e:84de) joined #osdev
19:23:13 <ryoshu> but they are now defunct, switched to linux
19:23:43 <ryoshu> similar to sgi
19:24:23 <_mjg_> ya know, i may try a joke for one day
19:24:38 <_mjg_> boot a super old $whatever-unix in a vm and fullscreen the sucker
19:24:43 --- join: chao-tic (~chao@203.97.21.86) joined #osdev
19:24:54 <_mjg_> as long as it has means to at least telnet out, i can work from that
19:25:05 <klange> Psh.
19:25:11 <_mjg_> just telnet back into the host and ssh from there
19:25:21 <_mjg_> the laptop is basically a terminal multiplexer
19:25:29 <ryoshu> https://github.com/NetBSD/src/commit/909b78fedc3e6eb86752f9311c8ee1a4a9c85cd8
19:25:30 <klange> You want a fun joke, you get a VT220, you hook it up to the serial on your workstation, you attach *that* to an old Unix running in a VM.
19:25:31 <bslsk05> ​github.com: Add a new ptrace(2) ATF exploit for: CVE-2018-8897 (POP SS debug exce… · NetBSD/src@909b78f · GitHub
19:25:47 <ryoshu> it has been mirrored for github/git users
19:25:51 <_mjg_> klange: i also kind of still want to get work done
19:25:53 <_mjg_> 8)
19:25:56 <ryoshu> thanks again
19:26:18 <_mjg_> if i wanted to go useless, i would go with a windows desktop
19:26:54 --- part: ryoshu left #osdev
19:26:58 <klange> Back when I was in uni, which mind you was only ~6-7 years ago, we had an ACM student chapter with a full office in the CS building, and we were often fielding questions from fellow students due to our prominent location, so we had a helpdesk set up...
19:27:33 <klange> On April 1st we would swap out the monitor and keyboard for a VT220.
19:28:05 <klange> http://i.imgur.com/2MhE2.jpg
19:29:14 <_mjg_> solid
19:29:25 <lachlan_s> klange: There's an acm chapter at the uni I'm going to next year. What is it exactly?
19:30:24 <klange> ACM is an international association probably most known for the digital library of research papers and the various awards and events they run. For a Uni student chapter, it's likely going to be the computer nerds club.
19:31:24 <klange> Ours acted as an umbrella organization for several different smaller student organizations with more focused agendas. We managed the shared facilities and infrastructure and hosted a tech conference, while actual meetings and projects tended to happen in the smaller groups.
19:31:38 <lachlan_s> Ah, cool
19:31:51 <klange> Following the ACM terminology, we called the smaller groups SIGs
19:31:58 <klange> (Special Interest Groups)
19:32:01 <lachlan_s> The don't have a sigops chapter, but they're apparently open to creating one
19:32:51 <klange> Our uni's LUG was part of ACM, we had a SIGOps, we had SIGs for embedded systems, SIGs for games, SIGs for music...
19:34:04 <klange> I miss that aspect of college.
19:58:27 --- quit: Belxjander (Ping timeout: 240 seconds)
19:59:05 --- join: zeus1 (~zeus@197.239.5.46) joined #osdev
20:00:45 --- quit: Pseudonym73 (Remote host closed the connection)
20:01:19 --- join: Pseudonym73 (~Pseudonym@e-nat-unimelb-128-250-0-54.uniaccess.unimelb.edu.au) joined #osdev
20:03:26 <izabera> how upset would you be
20:03:30 <izabera> if you used a function in a library
20:03:35 <izabera> and this function forked
20:03:47 <izabera> and did some work IN THE PARENT and returned to you in the child
20:03:53 <izabera> ?
20:04:23 --- quit: zeus1 (Ping timeout: 248 seconds)
20:05:07 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
20:06:07 --- quit: Pseudonym73 (Ping timeout: 264 seconds)
20:11:33 --- quit: MaryJaneInChain (Remote host closed the connection)
20:18:06 --- quit: Belxjander (Ping timeout: 260 seconds)
20:18:41 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
20:19:19 --- quit: bork (Ping timeout: 248 seconds)
20:21:14 --- join: bork (ayy@cpe-76-173-133-37.hawaii.res.rr.com) joined #osdev
20:22:36 --- quit: ALowther (Remote host closed the connection)
20:26:58 --- quit: chao-tic (Ping timeout: 250 seconds)
20:28:23 --- quit: bork (Ping timeout: 248 seconds)
20:31:00 --- join: bork (ayy@cpe-76-173-133-37.hawaii.res.rr.com) joined #osdev
20:32:22 --- quit: Belxjander (Ping timeout: 268 seconds)
20:33:12 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
20:36:08 --- join: chao-tic (~chao@203.97.21.86) joined #osdev
20:36:12 <pounce> So would I use the `G` page table flag for my kernel?
20:36:19 --- join: MaryJaneInChain (~user@223.255.127.32) joined #osdev
20:36:39 <Mutabah> Usually
20:36:52 <Mutabah> You use that bit for anything that's mapped into every address space
20:36:59 <Mutabah> (saves TLB flushes)
20:37:02 <bcos> Use 'G' for all pages that are the same in all virtual addresses that a CPU uses
20:37:20 <pounce> nice, I was wondering about how to do that
20:37:20 <bcos> *in all virtual address spaces
20:37:30 <Mutabah> Just make sure to not use it on pages that actually do differ
20:37:40 <Mutabah> because that'll lead to incorrect behavior
20:38:40 <bcos> ^ this implies that (e.g.) if you have a "per CPU" area of kernel space, you can use 'G' even though it's different for different CPUs; and that if you have something (VSDO?, shared library?) that's in user-space and not in kernel then that could be eligable for 'G' too
20:39:06 <pounce> (I assume this will stay across TLB flushes too?)
20:39:38 --- join: zmzm (~zmzm@gateway/tor-sasl/zmzm) joined #osdev
20:39:40 --- nick: BronzeBeard -> zorbon
20:40:03 <bcos> 'G' means it doesn't get flushed when CR3 is loaded (when changing to a different virtual address space). To flush a TLB entry for a page marked with 'G' you need to INVLPG or diddle CR4
20:40:16 --- nick: zorbon -> BronzeBeard
20:40:23 <pounce> kk
20:40:31 --- join: ctrlshftn (~ctrlshftn@unaffiliated/ctrlshftn) joined #osdev
20:40:45 --- quit: Shikadi (Ping timeout: 256 seconds)
20:41:16 --- quit: Griwes (Ping timeout: 250 seconds)
20:41:39 <pounce> bcos: what if I do 1) mark userspace lib `G` in process 1, 2) switch to process 2, 3) process 2 accesses the addresses for the userspace lib
20:41:55 <pounce> will process 2 be able to access the library even though I didn't map it into its address space?
20:42:32 <bcos> Yes or no - it will be able to access it if the translation happens to be cached in TLB
20:42:46 <bcos> ..but you wouldn't use 'G' for that case
20:43:02 <pounce> hmm. Doesn't seem like a good idea to have U&G then
20:43:30 <bcos> ..and if you're worried about meltdown (implement "kernel page table isolation") you might not want to use 'G' for anything
20:44:14 <pounce> I've been wondering about how kernel page table isolation is done
20:44:20 --- join: Pseudonym73 (~Pseudonym@e-nat-unimelb-128-250-0-54.uniaccess.unimelb.edu.au) joined #osdev
20:44:42 <pounce> (I'll still do it for simplicity though, I'm not worried about meltdown/spectre when I'm probably vulnerable to all hardware vulns now)
20:44:43 <Mutabah> There's a very small bounce pad that's global mapped, which switches page tables to one that includes the full kernel when there's a kernel entry
20:45:01 --- join: Griwes (~griwes@znc.reaver-project.org) joined #osdev
20:45:04 <bcos> In theory if you have 75 processes that all used a C standard library mapped at the same virtual address, plus 5 processes that don't use the library; then you could use 'G' for 75 processes and then diddle CR4 when you switch to one of the processes that don't use it
20:45:33 <bcos> ...which might help performance on average
20:45:46 <Mutabah> Or just do an invlpg on entry to the non-using ones
20:45:49 <bcos> Hrm. Would probably also make ASLR hard
20:46:02 <pounce> Mutabah: so would the kernel change tables on every entry?
20:46:29 <pounce> (I assume the bounce-pad is all kernel entry points)
20:46:42 <Mutabah> pounce: No? I mean run invlpg over the libc area when you switch to a process that doesn't have it mapped
20:46:49 <Mutabah> (which will clear the TLB for those locations)
20:46:57 <pounce> I mean for kernel page table isolation
20:47:08 <Mutabah> Oh, yes, for KPTI that's what happens
20:47:26 <Mutabah> I assume there's two PDs/PML4s per process
20:47:39 <pounce> that's terrible ;w;
20:47:41 <Mutabah> One for user mode, and another that's switched to on entry to kernel mode
20:48:00 --- join: zeus1 (~zeus@197.239.5.46) joined #osdev
20:48:24 <bcos> For KPTI each process would have 2 virtual address spaces - one with the process and the small bounce-pad (but not the rest of the kernel), and one with all the process, bounce-pad and the rest of kernel. The bounce-pad would switch between these.
20:48:59 <bcos> ..which is why PCID helps a lot
20:49:45 <pounce> (PCID?)
20:49:48 <bcos> Although that's not the only way... Can have one virtual address space per process, then give the kernel its own (full) virtual address space
20:49:57 --- quit: ctrlshftn (Read error: Connection reset by peer)
20:50:20 <bcos> "Process Context ID" - a little tag the CPU uses to figure out which TLB entry belongs to which process, so that it doesn't flush everything when CR3 is changed
20:51:02 <pounce> so you just give the kernel a different PCID?
20:51:05 <bcos> (TLB can contain entries for many virtual address spaces, where CPU will ignore the ones that aren't for the current virtual address space)
20:51:35 <bcos> (and where hopefully when you switch to a virtual address space its TLB entries (with its ID) are still in the TLB so you avoid lots of TLB misses)
20:52:45 <bcos> For "2 virtual address spaces per process" you'd use two IDs for each process (and none for kernel alone)
20:53:06 <bcos> For "1 virtual address spaces per process plus one for kernel alone" you'd use one ID for each process and one for kernel alone
20:53:33 <bcos> ..the former is a lot easier to retro-fit onto existing kernels
20:53:43 <bcos> ..the latter might be a lot more efficient
20:54:09 <bcos> (or might not - depends on too many things)
20:55:22 --- quit: zeus1 (Ping timeout: 256 seconds)
20:57:18 --- join: Goplat (~Goplat@reactos/developer/Goplat) joined #osdev
20:58:34 <pounce> I thought that most kernels were like the later, with a kernelspace shared between processes and userspaces separate
20:59:05 <Mutabah> Nah, most of the time the kernel is always present in the upper half/quater of the address space
20:59:26 <pounce> that's what I do with mine
20:59:36 <bcos> Most are like the former; mostly because all the kernel's code is designed for the "kernel can directly access things in user-space" assumption (because kernels without KPTI put kernel in all virtual address spaces)
21:00:18 --- quit: freakazoid0223 (Quit: Few women admit their age. Few men act theirs.)
21:00:28 <bcos> ..that's why it's easier to retro-fit the "2 address spaces per process" - you don't need to search through and modify milions of lines of kernel code
21:00:48 <pounce> ooh makes sense now
21:01:13 <pounce> that seems harder to keep up though, because each userspace page must be mapped twice
21:01:46 <bcos> You'd just plug the same PDPTs into both PML4s
21:02:13 <pounce> BTW does 32bit have no-execute? Because I don't see a flag on the wiki and on x64 it's the last bit
21:02:18 <bcos> (and do extra TLB invalidations)
21:02:41 <bcos> 32-bit with PAE can have no-execute; but 32-bit with "plain 32-bit paging" can't
21:03:14 <pounce> ah, gotcha
21:06:09 --- join: S_Gautam (3bb6f9ab@gateway/web/cgi-irc/kiwiirc.com/ip.59.182.249.171) joined #osdev
21:08:13 --- join: oaken-source (~oaken-sou@p5DDB6424.dip0.t-ipconnect.de) joined #osdev
21:11:31 --- quit: unixpickle (Quit: My MacBook has gone to sleep. ZZZzzz…)
21:13:45 --- join: norvic (~norvic@47.149.126.110) joined #osdev
21:14:16 --- quit: S_Gautam (Changing host)
21:14:16 --- join: S_Gautam (3bb6f9ab@unaffiliated/gautams) joined #osdev
21:14:16 --- quit: S_Gautam (Changing host)
21:14:16 --- join: S_Gautam (3bb6f9ab@gateway/web/cgi-irc/kiwiirc.com/ip.59.182.249.171) joined #osdev
21:14:42 --- join: jrheaton (~jrheaton@162-235-12-210.lightspeed.bcvloh.sbcglobal.net) joined #osdev
21:16:55 --- quit: Pseudonym73 (Remote host closed the connection)
21:17:28 --- join: Pseudonym73 (~Pseudonym@e-nat-unimelb-128-250-0-54.uniaccess.unimelb.edu.au) joined #osdev
21:21:41 --- join: unixpickle (~alex@c-24-5-86-101.hsd1.ca.comcast.net) joined #osdev
21:22:16 --- quit: Pseudonym73 (Ping timeout: 260 seconds)
21:23:57 --- quit: Belxjander (Ping timeout: 240 seconds)
21:24:48 --- quit: earlz (Ping timeout: 276 seconds)
21:25:08 --- nick: variable -> constant
21:25:16 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
21:26:13 <pounce> I've gone into another documentation frenzy -w-
21:30:40 --- quit: unixpickle (Quit: My MacBook has gone to sleep. ZZZzzz…)
21:31:53 --- join: unixpickle (~alex@c-24-5-86-101.hsd1.ca.comcast.net) joined #osdev
21:31:57 --- quit: bemeurer (Ping timeout: 276 seconds)
21:33:16 --- join: bemeurer (~bemeurer@2600:8802:5300:bb90:863a:4bff:fe06:c8b2) joined #osdev
21:33:46 --- join: ALowther (~alowther@68.200.236.134) joined #osdev
21:34:15 --- quit: hmmmm (Remote host closed the connection)
21:37:34 --- join: hmmmm (~sdfgsf@pool-72-79-167-28.sctnpa.east.verizon.net) joined #osdev
21:38:15 --- quit: ALowther (Ping timeout: 248 seconds)
21:40:01 --- join: zeus1 (~zeus@197.239.5.46) joined #osdev
21:42:29 --- quit: jrheaton (Quit: My MacBook has gone to sleep. ZZZzzz…)
21:42:39 --- quit: unixpickle (Quit: My MacBook has gone to sleep. ZZZzzz…)
21:43:35 --- join: jrheaton (~jrheaton@162-235-12-210.lightspeed.bcvloh.sbcglobal.net) joined #osdev
21:44:04 --- quit: Oxyd76 (Ping timeout: 265 seconds)
21:44:06 --- quit: MarchHare (Ping timeout: 250 seconds)
21:44:43 --- join: elderK (~k@122-61-112-9-adsl.bb.spark.co.nz) joined #osdev
21:44:43 --- quit: elderK (Changing host)
21:44:43 --- join: elderK (~k@pdpc/supporter/active/elderk) joined #osdev
21:46:49 --- quit: climjark_ (Ping timeout: 240 seconds)
21:47:19 --- quit: robert_ (Ping timeout: 248 seconds)
21:48:46 --- join: climjark_ (~climjark@c-24-0-220-123.hsd1.nj.comcast.net) joined #osdev
21:53:47 --- join: S_Gautam56 (3bb6feb7@gateway/web/cgi-irc/kiwiirc.com/ip.59.182.254.183) joined #osdev
21:54:02 --- quit: latentprion (Quit: Lost terminal)
21:54:05 --- quit: S_Gautam56 (Changing host)
21:54:05 --- join: S_Gautam56 (3bb6feb7@unaffiliated/gautams) joined #osdev
21:54:05 --- quit: S_Gautam56 (Changing host)
21:54:05 --- join: S_Gautam56 (3bb6feb7@gateway/web/cgi-irc/kiwiirc.com/ip.59.182.254.183) joined #osdev
21:56:31 --- quit: S_Gautam (Ping timeout: 264 seconds)
21:56:46 --- nick: S_Gautam56 -> S_Gautam
21:57:46 --- join: Oxyd76 (~quassel@5.18.99.0) joined #osdev
21:57:51 --- quit: Belxjander (Ping timeout: 260 seconds)
21:58:10 --- join: ACE_Recliner (~ACE_Recli@c-73-18-225-48.hsd1.mi.comcast.net) joined #osdev
21:58:45 --- join: xenos1984 (~xenos1984@2001:bb8:2002:200:6651:6ff:fe53:a120) joined #osdev
21:58:53 <pounce> I need to make a wiki account...
21:59:23 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
21:59:32 --- join: Pseudonym73 (~Pseudonym@e-nat-unimelb-128-250-0-54.uniaccess.unimelb.edu.au) joined #osdev
22:01:49 --- join: latentprion (~latentpri@sel4/latentprion) joined #osdev
22:04:19 --- quit: zeus1 (Ping timeout: 264 seconds)
22:05:11 * geist just soldered more vram on a video card
22:05:16 <geist> just like in the old days
22:05:42 <latentprion> a modern video card?
22:05:46 <geist> oh no
22:05:53 <latentprion> aww :(
22:06:05 <geist> 386 machine, had 256KB vram but 6 more solder pads for more dram
22:06:16 <geist> so i just soldered 2 more chips on and sure enough it how has 512KB
22:09:04 --- quit: lachlan_s (Quit: Connection closed for inactivity)
22:09:13 --- quit: jrheaton (Quit: My MacBook has gone to sleep. ZZZzzz…)
22:09:19 --- quit: MaryJaneInChain (Remote host closed the connection)
22:09:36 --- join: MaryJaneInChain (~user@223.255.127.32) joined #osdev
22:09:59 --- quit: Pseudonym73 (Remote host closed the connection)
22:12:26 --- quit: Belxjander (Ping timeout: 260 seconds)
22:13:22 <_mjg_> geist: solid
22:13:47 <_mjg_> i may do an atari ram upgrade for funzies
22:13:48 --- quit: chao-tic (Quit: WeeChat 1.0.1)
22:13:48 --- quit: hohum (Remote host closed the connection)
22:18:23 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
22:19:06 --- quit: climjark_ (Ping timeout: 256 seconds)
22:19:55 --- join: [ddrpa]amet (~amethyste@61.130.55.86) joined #osdev
22:20:43 <geist> not sure there's any value to going tro the full 1MB, since i'm fairly certan the ramdac on this card can't really handle it
22:20:59 <geist> all attempts to go into modes higher than 640x480 have resulted in a bad sync up
22:21:07 <geist> but now i can do 640x480x256!
22:21:15 <geist> the future isnow
22:22:07 <S_Gautam> I think God said 640x480x16 was the future.
22:22:16 <S_Gautam> But congrats, geist.
22:22:36 <geist> well, it's also a bit of a downside: windows 3.1 probably runs slower now that i put it in x256 mode
22:22:42 <geist> instead of plain old default 640x480x16
22:23:16 --- join: quc (~quc@host-89-230-160-119.dynamic.mm.pl) joined #osdev
22:23:21 --- join: BreakingFree (~libre@kite.riseup.net) joined #osdev
22:23:35 <S_Gautam> Wait how??
22:23:46 <S_Gautam> the total buffer size in bytes is: 78643200
22:24:18 <geist> math is hard!
22:24:24 <S_Gautam> i mean
22:24:32 <S_Gautam> How can you access that much memory in RM?
22:24:46 <S_Gautam> Windows 3.1 ran in real mode, right?
22:24:53 --- quit: constant (Quit: /dev/null is full)
22:24:53 <geist> no
22:25:07 <Mutabah> It was post ?himem
22:25:11 <geist> it's very much protected mode. by 3.11 it required 386
22:25:21 <geist> but earlier windows required 286 protected mode, i believe
22:25:23 <S_Gautam> oh
22:25:32 <geist> at least by 3.0, since nobody used it before that
22:25:55 <S_Gautam> so.. how could it run DOS programs so nicely? virtual8086?
22:26:18 <geist> yep. i believe to be able to concurrently run any dos apps required 386 mode
22:26:24 <geist> i can't test that because i'm using 3.11 here, and it was 386 only
22:26:29 <S_Gautam> NTVDM is v8086 but runs DOS programs like crap
22:26:39 <geist> but i vaguely remember there being a 286 and 386 open
22:26:54 <pikhq> NTVDM is v8086 with NT implementing DOS.
22:27:01 <pikhq> Win3.1 is v8086 *running DOS*.
22:27:21 <geist> fun fact: if you load emm386 at boot and set up extended and expanded memory, it drops you back in dos in v86 mode
22:27:47 <geist> ie, the dos machine is intrinsically running in v86, because the dpmi stuff left behind is sitting up in 386 protected mode, using paging to implemented expanded memory
22:27:49 <S_Gautam> pikhq, oic
22:28:26 <S_Gautam> wait, DPMI doesn't work in v86, correct?
22:28:35 <S_Gautam> that would be weird.
22:28:44 <geist> oh it absolultely does, you just get out of your little real mode box then
22:28:58 <geist> you switch into protected mode if you make the appropriate dpmi calls, and thus you are free
22:29:03 <S_Gautam> Protected Mode - >Virtual 8086 Real Mode -> Virtual 80386 Protected Mode?
22:29:12 <S_Gautam> Oh
22:29:27 <geist> i read through the dpmi api the other day, pretty slick (for the day)
22:29:49 <geist> basically you request one or more protected mode segments and can basically ask it to switch you into protected mode (16 or 32)
22:29:52 <S_Gautam> So, you basically take control of the system after jumping to protected mode from v8086? (Unless the host has implemented something against jumping to 32-bit mode)
22:30:10 <geist> and then it acts as the PMM essentially, because you can ask and return segments of physical memory
22:30:36 <geist> with dos you're always in control. the v86 stuff left after emm386 has nothing to do with protecting the system from your code
22:30:54 <geist> it's about providing a 8086 real mode environment while being able to use protected mode segmentation and whatnot behind the scenes
22:31:25 <geist> well, 386 paging in that case, so it can fiddle around mapping crap into dos upper memory blocks in the 640k hole
22:33:04 <geist> i learned most of this the other day after piecing together what himem.sys does, emm386, dpmi and then fiddling with some cpu benchmarking and system info dos utilities that basically describe exactly what's going on
22:33:31 <geist> oph and with regards to how does the vga card 'see' this much memory? i read the Tseng Labs ET4000 manual the other day (the vga chipset on this)
22:33:51 <geist> a) you can page in pieces of the larger up to 1MB vid ram into the 64k hole at 0xa0000...0xb0000
22:34:06 <geist> and... b) you can map all of 1MB at 15MB physically, which would require protected mode to get to
22:34:35 <geist> hence the 15-16MB memory hole you sometimes see mentioned. that was apparently a common thing to do before PCI came along
22:38:17 --- join: grouse (~grouse@83-233-9-2.cust.bredband2.com) joined #osdev
22:38:54 --- quit: pounce (Quit: WeeChat 2.1)
22:48:54 --- join: zeus1 (~zeus@197.239.5.46) joined #osdev
22:52:06 --- quit: awang_ (Ping timeout: 260 seconds)
22:58:31 --- quit: bemeurer (Quit: WeeChat 2.1)
23:00:05 --- join: NaNkeen (~nankeen@203.188.234.181) joined #osdev
23:09:00 --- join: MarchHare (~Starwulf@mo-184-5-204-232.dhcp.embarqhsd.net) joined #osdev
23:11:34 --- join: sortie (~sortie@static-5-186-55-44.ip.fibianet.dk) joined #osdev
23:12:18 --- quit: behalebabo (Ping timeout: 246 seconds)
23:12:25 --- quit: BreakingFree (Ping timeout: 255 seconds)
23:14:54 --- join: BreakingFree (~libre@199.254.238.216) joined #osdev
23:15:27 --- quit: Belxjander (Ping timeout: 246 seconds)
23:20:59 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
23:21:21 --- quit: BreakingFree (Remote host closed the connection)
23:21:58 --- quit: Belxjander (Max SendQ exceeded)
23:22:15 --- join: Belxjander (~Belxjande@sourcemage/Mage/Abh-Elementalist) joined #osdev
23:22:27 --- join: immibis (~chatzilla@222-155-160-32-fibre.bb.spark.co.nz) joined #osdev
23:31:10 --- quit: [ddrpa]amet (Quit: Leaving)
23:32:15 --- quit: dbittman_ (Ping timeout: 246 seconds)
23:38:07 --- join: behalebabo (~behalebab@unaffiliated/behalebabo) joined #osdev
23:43:21 --- join: BreakingFree (~libre@199.254.238.209) joined #osdev
23:43:32 --- join: Kimundi__ (~Kimundi@i577A9912.versanet.de) joined #osdev
23:46:21 --- quit: sortie (Quit: Leaving)
23:51:04 --- quit: AverageJ0e (Ping timeout: 250 seconds)
23:52:27 --- quit: zmzm (Quit: zmzm)
23:59:24 --- quit: MarchHare (Ping timeout: 256 seconds)
23:59:59 --- log: ended osdev/18.05.13