channel logs for 2004 - 2010 are archived at http://tunes.org/~nef/logs/old/ ·· can't be searched

#osdev2 = #osdev @ Libera from 23may2021 to present

#osdev @ OPN/FreeNode from 3apr2001 to 23may2021

all other channels are on OPN/FreeNode from 2004 to present

http://bespin.org/~qz/search/?view=1&c=osdev2&y=22&m=6&d=2

Thursday, 2 June 2022

00:59:00 <gog> hello from my new rig
00:59:00 <gog> :D
01:04:00 <klange> It's probably time for me to build a new one myself...
01:04:00 <klange> This poor old i5-6600K is getting up there in the years, I built it when I moved to Tokyo and that's now six years ago...
01:04:00 * vdamewood gives gog a rig-warming fishy
01:04:00 <klange> Laptops I have purchased in the interim are faster...
01:06:00 <zid> benchmarks or riot
01:06:00 <klange> I run benchmarks for my Python-alike regularly, so I can see the diferences.
01:07:00 <klange> Also build times for toaru
01:07:00 <klange> though macOS I had to cheat and implement thread-local storage myself for it to not be stupidly slow
01:08:00 <vdamewood> Python-alike?
01:08:00 <klange> https://kuroko-lang.github.io/
01:08:00 <bslsk05> ​kuroko-lang.github.io: Kuroko
01:09:00 <zid> not you, gog :P
01:09:00 <zid> new laptop = new benchmarks, only reason I get new hw
01:09:00 <zid> it's been fast enough for forever
01:09:00 <gog> i haven't done any benchmarks yet
01:10:00 <gog> i'm still getting used to the iso keyboard too
01:10:00 <gog> i've been having to use one at work for the last few weeks so i'm getting there
01:10:00 <zid> iso keyboard <3
01:10:00 * vdamewood stares at the fishy
01:10:00 <zid> Welcome to the real world, goggo
01:10:00 * gog chomps fishy
01:10:00 <zid> enjoy being able to type BOTH kinds of slash
01:11:00 <vdamewood> klange: I did import os and os.exit(5) and it crashed on me.
01:11:00 <klange> in the web repl?
01:11:00 <klange> you told emscripten to exit :D
01:11:00 <vdamewood> klange: Seriously though, the response to the exit looks totally normal.
01:13:00 <vdamewood> klange: Oh, the slides that tutorial() produces are neat. The Python REPL doesn't do anything that fancy.
01:13:00 <zid> is it as good as vimtutor!1!wq!
01:14:00 <klange> That's specific to the web repl and intended to to be a demonstration of using the wasm builds to use kuroko for frontend web scripting, but it's rather hacky since wasm doesn't have DOM access
01:15:00 <vdamewood> klange: Are integers machine int64s, (or 32s?) or are they bignums?
01:15:00 <klange> 48-bit due to NaN boxing constraints, with plans for bigints that have yet to come to fruition.
01:16:00 <vdamewood> Got it, one amd64 pointer.
01:16:00 <vdamewood> (not really. The pointers are still 64-bit, even if the high-order byte is redundant.)
01:17:00 <vdamewood> "Objects are more complicated values like..." FWIW, I think the word 'complex' is more appropriate here.
01:17:00 <klange> I've been really happy with how Kuroko has turned out. It's still got some quirks and the standard library is nearly non-existent, but it's served its original purpose quite well.
01:18:00 <vdamewood> It looks neat.
01:19:00 <vdamewood> It's certainly more of a programming language than my toy language, which only does expression evaluation.
01:20:00 <klange> I've gotten Kuroko to a very high degree of compatibility with Python, have managed to squeeze in some things I personally consider improvements on the syntax, and am doing well in benchmarks.
01:21:00 <klange> It's been much more rewarding than OSdev :)
01:21:00 <klange> Also, since it's embedded in my editor, I use it every day - including at work :)
01:21:00 <vdamewood> Are you shooting for a superset relationship?
01:22:00 <klange> More of a venn diagram with a tight overlap. I haven't gone for too much new syntax, and I'm racing to catch up with new functionality in Python - the new(ish, I'm like a year late to this) pattern matching stuff is neat but they made some decisions with it that are giving me headaches.
01:25:00 <vdamewood> I've been thinking of switching my build system to CMake... I think I'll give it a shot now.
01:26:00 <klys> devs who switch to cmake should partially document cmake in their repos
01:31:00 <vdamewood> klange: Given how customizable build system generators like CMake are, the build system should certainly be documented.
01:31:00 <vdamewood> err klys ^
01:31:00 <vdamewood> Damn... I have to type 3 letters to tab between you two.
01:31:00 <Mutabah> klange klys kollision
01:32:00 <klys> :)
01:32:00 <klys> what's new mutabah
01:37:00 <Mutabah> cold
01:37:00 <Mutabah> ... and background working on USB support
01:37:00 <Mutabah> Doing EHCI because I'm a sucker for punishment
01:38:00 <klys> oh yeah you were doing usb eh
01:39:00 <klys> breaking out usb into serial for use on a breadboard seems imaginable yet perhaps a bit heavy on the difficult side
01:40:00 <klys> actually parsing usb with circuity would be a job for some programmable logic I figure
01:41:00 <klys> most usb-serial relies on the in-circuit uart, like ftdi, though if you're going to put it back into usb then you can't really rely on those.
01:42:00 <klys> back into usb being, otg via a hub of sorts
01:42:00 <klys> which reminds me I busted a port on my older usb 3.0 hub by plugging the input into the output
01:44:00 <klys> meanwhile on the software side, attempting ehci must seem quite a bit different to someone who actually has the acpi stuff running
01:45:00 <klys> the usb stack is fairly intricate itself, from views of wireshark, etc.
02:05:00 <Mutabah> There's a recentish forum thread about EHCI. It's a mess
02:06:00 <Mutabah> Mostly with how it handles USB1 devices, but the controller layout is also a little odd
02:07:00 <klys> the ehci controller is probably the most standardized
02:13:00 <klys> I can see something here about interrupts not working
02:16:00 <klys> said "the mouse shuts off" because it isn't "truly a high speed device."
02:18:00 <klys> so he switched to writing an xhci driver and "found the...ehci controller" on a higher interrupt
02:26:00 <klys> I guess if you were making circuits the trick would be having multiple serial ports available to assign enumerated devices to
02:27:00 <klys> I'm also fairly interested in usb-otg, though still don't know much there yet.
02:29:00 <klys> this was cursorily interesting, https://en.wikipedia.org/wiki/USB_On-The-Go#Protocols
02:29:00 <bslsk05> ​en.wikipedia.org: USB On-The-Go - Wikipedia
02:32:00 <klys> > "The new OTG protocols cannot pass through a standard USB hub since they are based on electrical signaling via a dedicated wire."
02:40:00 <zid> over the gounter
04:02:00 <kingoffrance> "which reminds me I busted a port on my older usb 3.0 hub by plugging the input into the output" klys is on my ignore, thats him in a nutshell lol
04:41:00 <sonny> Isn't the `hlt` code redundant in this UEFI 'hello world'? <https://bpa.st/LJ6Q>
04:41:00 <bslsk05> ​bpa.st: View paste LJ6Q
04:43:00 <moon-child> sonny: it lets you do nothing faster
04:44:00 <moon-child> (or, rather, slower)
04:44:00 <moon-child> (depending on how you look at it)
04:44:00 <moon-child> it saves power
04:44:00 <sonny> ok
04:44:00 <kingoffrance> looks like it doesnt want it to return/exit
04:44:00 <sonny> for(;;) won't exit
04:45:00 <sonny> reading about hlt, seems like that by itself is sufficient
04:46:00 <moon-child> hlt will spuriously wake
04:46:00 <moon-child> in particular, when an interrupt is received
04:47:00 <sonny> oh ok
09:01:00 <jafarlihi> Is there any resource out there for creating lexer generators?
09:12:00 <klange> You may want to ask in #proglangdesign
14:32:00 <heat> re: for(;;) <-- this can be completely eliminated (and is if you're using clang) by the compiler
14:32:00 <heat> as it's UB
14:32:00 <zid> how is it UB exactly?
14:32:00 <mrvn> heat: for(;;);?
14:32:00 <heat> infinite loop
14:32:00 <zid> how is that UB?
14:33:00 <heat> it is
14:33:00 <mrvn> c++ foprbids infinite loops
14:33:00 <zid> That's a great answer heat and you know it
14:33:00 <zid> if you don't know just say so and I can try google it
14:33:00 <heat> mrvn, is it just C++?
14:33:00 <mrvn> ask google, no idea.
14:33:00 <zid> C11+ apparently says something about it
14:34:00 <zid> I don't use C11 so I'd have to go read it
14:34:00 <mrvn> Is clang actually optimizing that UB? gcc sure doesn't.
14:34:00 <heat> yes, clang does
14:34:00 <heat> It has gotten me a few times
14:34:00 <mrvn> DOes it insert UB2 opcode?
14:34:00 <zid> Okay, "An iteration statement ... does not access volatile blah blah ... may be assumed by the implementation to terminate."
14:34:00 <zid> so not UB
14:34:00 <zid> but elidable
14:34:00 <heat> mrvn, no
14:34:00 <heat> it's stupider
14:35:00 <heat> https://godbolt.org/z/73jMr81xn mrvn
14:35:00 <bslsk05> ​godbolt.org: Compiler Explorer
14:35:00 <heat> no warning no nothing
14:35:00 <mrvn> I guess it unrolls it infinite number of times and then it's just ";" :)
14:35:00 <zid> I'd honestly say clang is just fucking *wrong* here
14:35:00 <mrvn> totally.
14:35:00 <zid> In a very torvaldsian way, what the fuck is the point of that behavior
14:36:00 <zid> it just exists to make programmers look stupid and break things
14:36:00 <mrvn> But il-formed code may be translated how ever it wants
14:36:00 <mrvn> zid: The optiomizer will see "for (something; something; something) { statement with no affect; }" and eliminate the loop. That "something" is "" doesn't matter.
14:37:00 <heat> right
14:37:00 <heat> __asm__ (""); should work
14:37:00 <mrvn> since that is implicitly volatile, yes.
14:39:00 <zid> https://lkml.iu.edu/hypermail/linux/kernel/1510.3/02866.html
14:39:00 <bslsk05> ​lkml.iu.edu: Linux-Kernel Archive: Re: [GIT] Networking
14:39:00 <zid> basically this but replace what he's talking about with "clang eliding loops"
14:40:00 <zid> err wrong link whoops
14:40:00 <heat> pre-therapy linus had some bad vibes :|
14:40:00 <zid> I never had an issue with linus
14:41:00 <zid> There's a rant I am thinking of but I can't find it, oh well
14:41:00 <j`ey> cos he never had an issue with you :p
14:43:00 <zid> Mainly because I also think the only recourse to people making the same mistakes over and over if I can't fire them is to send them angry emails :P
14:43:00 <zid> including the C standar's comittee
14:51:00 <heat> btw I believe that clang's behavior is OK since your .text is padded with int3's
14:52:00 <heat> scary, but it should work I think
14:52:00 <mrvn> I think it's stupid but it's correct.
14:53:00 <heat> oh it's correct for sure since "hurr durr undefined behavior"
14:53:00 <mrvn> I like my binaries to fail/hang when I write bad code. Not have the compoiler magically fix it.
14:53:00 <heat> it doesn't seem to do this in C
14:53:00 <heat> :)
14:54:00 <zid> (it's not actual UB in C?)
14:54:00 <zid> er sorry badly worded
14:54:00 <mrvn> for(;;); is a simple way to quickly add a "breakpoint" in some code.
14:54:00 <zid> it's not actual UB in C, is it in C++?
14:54:00 <kingoffrance> confusion of tongues :)
14:54:00 <zid> It's just pointed out explicitly that it's elidable
14:54:00 <heat> yes it is
14:54:00 <heat> in C++ that is
14:54:00 <zid> C++ is weird
14:54:00 <mrvn> zid: you are not allowed to loop forever. Doing so is UB.
14:55:00 <heat> https://en.cppreference.com/w/cpp/language/ub <-- not a formal source, but it says so here
14:55:00 <bslsk05> ​en.cppreference.com: Undefined behavior - cppreference.com
14:55:00 <kingoffrance> hmm, on the macro scale that would imply binaries must be restarted every so often
14:55:00 <mrvn> I have no idea how they square that with signals and interrupts.
14:55:00 <kingoffrance> :)
14:55:00 <mrvn> alarm(1); for(;;); is perfectly fine code I feel.
14:55:00 <mrvn> kingoffrance: binaries tend to have observable side effects
14:56:00 <Andrew> An observable side effect is that your computer has became a spaceheater
14:56:00 <heat> mrvn, clang disagrees :)
14:56:00 <mrvn> for(;;); doesn't heat that much.
14:57:00 <heat> i agree
14:57:00 <heat> it doesn't me that much
14:57:00 <heat> however
14:57:00 <heat> damn inefficient
14:57:00 <mrvn> How do you actually write the above c++ conform?
14:57:00 <heat> __asm__(""); :)
14:57:00 <Andrew> ... Is anyone trying to do OS development in C++? Sorry I cleared my backlog
14:57:00 <\Test_User> __asm__("jmp $"); ought to do it quite well
14:57:00 <heat> now it's implementation defined!
14:57:00 <mrvn> that's not c++
14:57:00 <zid> A few people do microkernels and shit in it
14:58:00 <zid> where they don't have to deal with the C++ nonsense so much
14:58:00 <heat> \Test_User, jmp $ is bound to screw something up
14:58:00 <\Test_User> heat: for(;;); is the same infinite loop
14:58:00 <heat> I have a whole kernel in C++
14:58:00 <heat> \Test_User, assembly jumps in inline assembly is bound to fuck something up
14:58:00 <\Test_User> hmm fair enough
14:59:00 <heat> especially when __asm__(""); is enough
14:59:00 <Andrew> heat: Any reasons to use C++ over C (or is it you choosijg it because personal preference) for kernels?
14:59:00 <heat> RAII
14:59:00 <\Test_User> you mean for(;;) __asm__(""); ?
14:59:00 <heat> also helps out with OO programming
14:59:00 <zid> C++'s actually a huge pain in the butt in kernel code unless it's doing something *for* you that offsets it
14:59:00 <heat> \Test_User, yup
14:59:00 <\Test_User> k
15:00:00 <zid> like "I really really want to have no compile time encapsulation, I'm willing to implement the C++ runtime to aquire it"
15:00:00 <heat> Andrew, RAII is super useful, classes and structs and virtual methods are fantastic, templates are useful (in moderation)
15:00:00 <Andrew> zid: I agree
15:00:00 <heat> erm. it's not
15:00:00 <heat> it's exactly like C
15:00:00 <Andrew> heat: C has structs though, I generally find classes messey
15:00:00 <heat> you need like 7 simple functions for the whole ffreestanding C++ runtime
15:01:00 <sbalmos> global naming convention discipline gets to be a real PITA
15:01:00 <sbalmos> err s/convention/uniqueness/
15:01:00 <Andrew> Oh, that absolutely
15:01:00 <heat> Andrew, structs are OK but then you end up 1) passing the equivalent of a this pointer everywhere, manually; 2) reinvent the function pointer vtable wheel
15:01:00 <sbalmos> mm hmm
15:02:00 <heat> struct file_operations, struct address_space_operations, etc :)
15:02:00 <heat> linux is a great example of it
15:02:00 <heat> OO programming makes sense in *moderation*
15:02:00 <Andrew> I'm looking into Zig language for kernel development, it has namespaces and stuff but its standard library is disgustingly big
15:02:00 <heat> well, you're not supposed to use the standard library
15:03:00 <Andrew> Well
15:03:00 <Andrew> There are things that I might lke
15:03:00 <Andrew> for example sqrt()
15:03:00 <Andrew> But the stdlib contains stuff like JSON parsers, meh
15:04:00 <heat> you don't need sqrt in a kernel :)
15:04:00 <kingoffrance> IIRC the zig author used to hang in here, but i played ddr "drop the bomb" for great justice, and i think i scared him off :/
15:05:00 <Andrew> ?
15:05:00 <Andrew> heat: Fine lol
15:06:00 <heat> Andrew, right. anyway, C++ is fine for kernel development. Some aspects of the language are bad, yes (exceptions, some good portions of the standard library, the expectation of Boost everywhere) but you can mostly manage to escape all that for a kernel
15:06:00 <heat> does it compile as fast as C? no. but that's almost impossible to beat anyway
15:06:00 <heat> my C++ compiles relatively fast as it's lightweight
15:07:00 <Andrew> assembly "compiles" faster, but yeah that's not really compiling
15:07:00 <zid> C++ is as good as the amount of it you ignore
15:07:00 <mrvn> Andrew: just namespaces alone make C++ worth it
15:07:00 <heat> AND it's more maintainable and safe
15:07:00 <Andrew> Though compiling time isn't really important to me
15:07:00 * Andrew looks at Zig and feels a bit better
15:07:00 <heat> like RAII was almost the whole reason I switched, and is almost the whole reason why I use it
15:08:00 <heat> i'm not smart enough to manually manage resources correctly
15:08:00 <mrvn> RAII, constexpr, constevel, constinit. Invaluable.
15:08:00 <mrvn> templates add type safety to void* constructs in C
15:09:00 <heat> templates are good but they're very overused I feel like
15:09:00 <mrvn> In the STL? totally. concepts lessen that somewhat but it's ongoing.
15:09:00 <heat> also SFINAE, etc make me barf
15:10:00 <heat> and fucking recursive templates
15:10:00 <mrvn> that's much less needed in c++20 too
15:10:00 <heat> if you want something like std::tuple you do need recursive templates
15:11:00 <heat> IMO a big problem with C++ is that people prefer to build stuff on top of existing stuff instead of adding more features directly
15:11:00 <heat> leading to stupid shit like SFINAE
15:12:00 <heat> a lot of it really kills compile time
15:12:00 <heat> you end up with stupid late stage C++ stuff like LLVM
15:12:00 <mrvn> heat: why? parameter packs work great
15:12:00 <zid> late stage capitalism C++ is amazing
15:13:00 <zid> "I've provided printing for my object via a virtually inherited polymorphic left shift operator friend class! Look how clean my code is!"
15:13:00 <mrvn> only bad thing in std:tuple is std::tuple::get<int>()
15:13:00 <heat> zid, it's not a virtual method I'm sure
15:13:00 <zid> That's why I describe C++ as a meta-language for writing domain languages is, you write 40k lines of templates and operator overloads so that your main.cpp is 4 lines of BASIC
15:13:00 <heat> just a simple shift operator overload with two operators
15:14:00 <mrvn> zid: I tend to have a public: ostream & print(ostream &out) const;
15:14:00 <zid> way to miss the forest for the trees there heat :P
15:14:00 <heat> well. it's simpler than other languages
15:14:00 <heat> in C you need GNU extensions to do that
15:14:00 <mrvn> auto [a, b, c] = foo(); is greate too.
15:15:00 <zid> Ever implemented iostream in an hour?
15:15:00 <zid> ever implemented printf in an hour? :p
15:15:00 <mrvn> Too bad that still calls std::tuple<A, B, C>::~tuple()
15:15:00 <heat> i'm not defending iostream
15:15:00 <heat> it's bad
15:15:00 <mrvn> zid: rm -rf iostream, implement format in a week.
15:15:00 <heat> std::format and std::print are good though
15:16:00 <mrvn> I don't know how many c++ programs I have that still use printf().
15:16:00 <heat> most of mine do
15:17:00 <heat> <format> is not yet portable
15:17:00 <heat> and iostream is crap
15:17:00 <j`ey> how so?
15:17:00 <heat> which part
15:18:00 <mrvn> heat: you mean not all compilers have formatg yet?
15:18:00 <heat> mrvn, yes
15:18:00 <mrvn> saddly true. give it time.
15:18:00 <j`ey> heat: portable bit
15:18:00 <j`ey> oh ok
15:18:00 <heat> well, ^^
15:18:00 <j`ey> I was thinking architecure wise
15:18:00 <heat> one of {gcc, clang} doesn't have it yet
15:19:00 <heat> (the stupidest bit is that they only added std::string std::format(...); to C++20; if you want to print something, std::cout << std::format(...);)
15:20:00 <heat> std::print is afaik C++23
15:21:00 <heat> also my pet peeve for C++ is the networking proposal(s)
15:21:00 <heat> holy fucking shit the overengineering they're doing is astonishing
15:22:00 <heat> async everything because why not, everything is abstracted in little classes because why not
15:22:00 <zid> 50p says whatever it is they've proposed clashes wildly with every other C++ feature
15:22:00 <zid> and has to be used exclusively in a safe-subset manner
15:23:00 <heat> C++ (a language standardized since 1998) does not have networking in 2022
15:23:00 <heat> that's how overengineered it is
15:24:00 <heat> it's almost like BSD sockets aren't a portable thing
15:26:00 <mrvn> heat: other languages, including C, just spawn threads like there is no tomorrow for asyncIO. It's horrible.
15:26:00 <heat> io_uring?
15:26:00 <mrvn> heat: new and linux specific.
15:27:00 <heat> you can also *just do things yourself*
15:27:00 <heat> async networking will fallback on top of threads without io_uring or similar
15:27:00 <mrvn> heat: it should use select/poll/epoll and coroutines
15:28:00 <mrvn> or callable objects
15:30:00 <mrvn> Is there a c++ binding for liburing?
15:30:00 <heat> apparently https://github.com/CarterLi/liburing4cpp
15:30:00 <bslsk05> ​CarterLi/liburing4cpp - Modern C++ binding for liburing (io_uring) that features C++ coroutines support (12 forks/152 stargazers/MIT)
16:06:00 <_73> On Linux are syscalls scheduled with a higher priority then the user processes that requested the syscall?
16:06:00 <zid> pardon
16:07:00 <zid> you don't schedule syscalls, you schedule processes/threads
16:07:00 <_73> So I should say is a thread that is executing a syscall scheduled with a higher priority.
16:08:00 <zid> If you're clever, you can just wake them up when the thing that is blocking them resolves
16:08:00 <zid> but watch out for https://en.wikipedia.org/wiki/Thundering_herd_problem
16:08:00 <bslsk05> ​en.wikipedia.org: Thundering herd problem - Wikipedia
16:10:00 <_73> Ok the thundering herd problem makes sense.
16:11:00 <_73> I am wondering though if Linux by default schedules threads executing syscalls with a higher priority then a thread executing in user space.
16:11:00 <zid> why would it do that
16:11:00 <zid> surely that's the opposite of what you want
16:12:00 <zid> the 'user' thread has stuff to do, the blocked thread is obviously stuck with nothing to do
16:14:00 <zid> wait for whatever IRQ to happen that unblocks the blocking thread (data arrives from the network or hard drive to resolve the read() or such), and either wake that thread up, or just mark the data as 'arrived' and the next time you look at the threads, the user thread will have ran 'too much' and the blocked one will wake up because it is now able to
16:14:00 <_73> So when the syscall thread goes to do something that takes a while like a read/write it will be blocked, I see.
16:14:00 <zid> ever written.. user applications?
16:14:00 <zid> getchar(), read(), etc?
16:15:00 <_73> Yes
16:15:00 <zid> they don't just do while(1); if there's no data and your program fails
16:15:00 <mrvn> zid: the question is if, when an IRQ wakes up a process stuck in a syscall, the kernel switches to that process.
16:16:00 <mrvn> I do that depending on the remaining time slice of the process
16:16:00 <zid> Like, 90% of the programs running on my desktop are using 0 cpu, they're stuck blocking on some resource like a message or mouse click or whatever
16:17:00 <zid> no point running them if running them means they'd be broken cus the data they asked for isn't there
16:17:00 <_73> Why would a blocked thread scheduled with a high priority negativly effect system performance?
16:17:00 <mrvn> _73: while(true) getchar(); would DoS your system
16:18:00 <zid> 'poor system performance' in this case meaning, more cpu power was required than you had in order to keep things running in 'realtime'? They're not running, how are they using cpu time?
16:18:00 <zid> They're specifcally *not* doing a while(1)
16:18:00 <zid> They're just not running
16:18:00 <_73> mrvn: Ok this example will be useful for me to spend time considering
16:19:00 <\Test_User> "higher priority, not ignoring the blocking state and running them anyways"
16:19:00 <mrvn> _73: giving a process stuck in a syscall higher priority makes sense, until they keep running syscalls all thetime. Hence why I only prioritize them when they haven't exceeded their time slice.
16:20:00 <mrvn> _73: In luinux the behavior probably depends on whether you compiler your kernel as preemtive or not.
16:20:00 <Andrew> \Test_User: Speaking of that, are you going to have processes in your OS? And if so, preemptive or coorporative?
16:21:00 <_73> Ah I see, so surely the timeslice must be less time then say a disk read, right?
16:21:00 <mrvn> And the recommended setting is totaly contrary to what you might thing. Desktops, where you want faster reaction time, should not preempt, which should improve reaction time. It's totaly backwards in linux.
16:21:00 <\Test_User> Andrew: processes yes definitely, I like multithreading/etc too much to leave it out of mine :P
16:21:00 <\Test_User> preemptive vs cooperative I haven't looked at yet though so idk
16:21:00 <mrvn> _73: a disk read takes no time, that's just waiting for the disk.
16:21:00 <Andrew> If I have processes in my system it's definitely going to be coorperative
16:21:00 <_73> well I should say the time for a disk read to return
16:22:00 <mrvn> Andrew: Nobody has cooperative processes. Everybody has some measure of preemptive.
16:22:00 <j`ey> Andrew: if you dont have processes, how else is stuff run?
16:23:00 <zid> who needs processes when you can just launch your OS via one command line option per core
16:23:00 <zid> that tells it which code to run
16:24:00 <Andrew> j`ey: * processes in the traditional sense; "How else" is basically like how emacs works
16:24:00 <Andrew> zid: You launch OSes with command line options ... what
16:24:00 <j`ey> thta doesnt really explain anything
16:25:00 <zid> Andrew: That's how linux and windows both work
16:25:00 <zid> ??
16:25:00 <j`ey> Andrew: yes? like linux etc has command line options
16:25:00 <j`ey> root=/ etc
16:25:00 <Andrew> Multiboot stuff, ah
16:25:00 <zid> nothing to do with multiboot
16:25:00 <zid> both linux and windows take options, that's all there is to it
16:25:00 <Andrew> Alright - who passes the options?
16:25:00 <heat> the bootloader
16:26:00 <heat> even UEFI can pass you options
16:26:00 <zid> how do you think safe mode happens in windows
16:26:00 <zid> or debug mode
16:26:00 <heat> how you get them depends on the boot protocol of course
16:26:00 <zid> it's literally just /debug
16:26:00 <Andrew> I was thinking in very siple (MBR) works
16:26:00 <heat> the MBR code is part of the bootloader
16:26:00 <heat> not the kernel
16:27:00 <_73> With grub you can pass options by setting GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub
16:27:00 <Andrew> zid: I don't use Windows, sorry I'm not familiar with what safe mode is, is it some kernel mode for reduced functionality and better dtability?
16:27:00 <zid> my kernel options are root=/dev/sda1 fastboot net.ifnames=0
16:27:00 <zid> yep
16:27:00 <heat> Andrew, yes plus the whole system
16:27:00 <Andrew> Ah
16:27:00 <heat> lots of services won't start
16:27:00 <heat> debug mode is debug mode :)
16:28:00 <zid> and fwiw linux isn't even bootable with multiboot, it has its own custom 'linux boot protocol'
16:28:00 <heat> it's not even the linux boot protocol
16:28:00 <heat> it's the linux x86 boot protocol
16:28:00 <heat> every arch has its own
16:28:00 <zid> right, there are many like it, but this one is mine
16:28:00 <heat> it's ridiculously stupid but that's the way it is
16:28:00 <Andrew> Is there something that makes it that I can't directly jump form the MBR to the kernel but I have to use a bootloader? I don't really see the point here - or it could just be that I have a wider definition of the kernel
16:28:00 <heat> Andrew, you need to load the kernel
16:28:00 <zid> a master boot record is just a concept
16:29:00 <zid> what it does or doesn't do is up to what you put *in* it for it to do
16:29:00 <heat> loading a whole kernel in less than 512 bytes is a challenge and a half
16:29:00 <Andrew> Yes, I know that
16:29:00 <Andrew> Uh
16:29:00 <zid> The MBR is just a PC concept that the first sector on the 'boot device' loads to 7C000 and runs
16:29:00 <Andrew> I mean
16:29:00 <\Test_User> s/000/00/
16:29:00 <zid> nod
16:29:00 <Andrew> For modenr systems, the MBR loads the rest of the bootloader which loads ther kernel
16:29:00 <heat> yes
16:29:00 <Andrew> Yes, I know what the MBR is lol
16:29:00 <heat> (modern x86 systems)
16:29:00 <zid> Well you don't seem to
16:30:00 <Andrew> But, not sure why the rest of the bootloader is really necessary
16:30:00 <\Test_User> you'll typically need more than 512 bytes to do everything you want before leaving 16-bit mode and entering the kernel
16:30:00 <heat> embedded stuff may very well have a lot more functionality
16:30:00 <zid> because 510 bytes
16:30:00 <Andrew> As it's basically jump jumping to the kernel
16:30:00 <zid> is absolutely fuck all
16:30:00 <zid> My OS lives on an ext4 partition
16:30:00 <heat> Andrew, that's not it
16:30:00 <Andrew> \Test_User: Oh, yes
16:30:00 <\Test_User> Andrew: if you write a 16-bit kernel that still collects stuff from bios sure
16:30:00 <zid> good luck parsing and loading my kernel from ext4 in 510 bytes
16:30:00 <zid> it *has* to chainload more code
16:30:00 <Andrew> That makes the point
16:31:00 <_73> You need a bootloader for a splash screen :)
16:31:00 <Andrew> zid: *your* kernel :P
16:31:00 <zid> The code that it chainloads is *called* the bootloader
16:31:00 <zid> It's irrelevent for anything but loading and booting, and isn't part of the kernel
16:31:00 <zid> so it's called the bootloader, and is seperate to the kernel
16:31:00 <Andrew> Ah, got it
16:31:00 <heat> things you need to do in an x86 PC bootloader: get the memory map, get acpi data, get efi tables (if EFI), read disk, interpret the partition, read the kernel and any possible ramdisks and any possible config file, load the kernel into memory, maybe relocate it, set up boot protocol arguments, jump to it
16:31:00 <mrvn> Andrew: the MBR isn't used to start your kernel on UEFI
16:31:00 <Andrew> ... yes
16:32:00 <Andrew> Though, I don't use UEFI
16:32:00 <zid> In 510 bytes. Also those 510 bytes might also need to be a valid partition header
16:32:00 <heat> and then in ARM (sometimes/usually) the bootloader kinda fuses together with the firmware
16:32:00 <zid> which cuts you down to like 40 bytes or whatever
16:32:00 <mrvn> Andrew: The reason why you should use a bootloader is that you should not waste your time messing around in 16bit code or caring about bios vs. uefi.
16:32:00 <Andrew> mrvn: Yeah, get that point
16:32:00 <heat> in fact u-boot has its own UEFI implementation these days for Reasons(tm)
16:32:00 <Andrew> "valid partition header", what
16:33:00 <zid> We're loading code of a 'boot drive'
16:33:00 <zid> not a raw hard drive
16:33:00 <zid> it has partitions, filesystems
16:33:00 <zid> If the filesystem or partitioning scheme we're using says offset 7 on sector 0 has to be 'B'
16:33:00 <Andrew> Oh, I'm using raw hard drives, so yeah that doesn't really apply
16:33:00 <zid> then you have to code around that.
16:33:00 <heat> oh right! I forgot you need drivers for a bunch of stuff
16:33:00 <heat> GRUB doesn't use the BIOS for example
16:33:00 <j`ey> heat: instead of chainloading edk2 i guess
16:33:00 <Andrew> Ah, hard drive drivers, eww
16:33:00 <Andrew> and other drivers too, yup
16:34:00 <heat> j`ey, yup. although chainloading edk2 is the least trivial thing ever
16:34:00 <heat> UefiPayloadPkg has two separate chainloading solutions (coreboot and slim bootloader)
16:34:00 <heat> totally incompatible
16:34:00 <j`ey> I mean from uboot -> edk2, not edk2 -> something
16:34:00 <heat> there's no "generic" solution besides maybe ripping off slim bootloader's interface
16:34:00 <heat> yes.
16:35:00 <heat> UefiPayloadPkg = UEFI/edk2 as a payload
16:35:00 <j`ey> ohh
16:35:00 <heat> there's no generic payloading interface unfortunately
16:35:00 <zid> Andrew: understand why 'why a bootloader exists?' is a silly question now I hope? :p
16:35:00 <heat> I don't know how generic slim's interface is, but it's pretty Intel/UEFI-ish, based on HOBs and whatnot
16:36:00 <Andrew> zid: Yup, I forgot that normal kernels reside on filesystems
16:36:00 <zid> well at least you took something away, sigh
16:36:00 <heat> Andrew, also, networking!
16:37:00 <heat> GRUB is a small modular OS on its own
16:37:00 <\Test_User> Andrew: even without parsing that, just collecting everything you want for your kernel will take quite a bit of code
16:37:00 <Andrew> zid: Yeah and more of because not messing with 16-bit stuff in the kernel and atuff
16:37:00 <Andrew> \Test_User: "parsing that", ?
16:37:00 <\Test_User> filesystems/partitions
16:37:00 <\Test_User> well, everything you *probably want
16:38:00 <Andrew> \Test_User: I'm worried about hdd drivers, not partition/filesystems because no filesystems (no explicit distinction between primary and secondary memory)
16:38:00 <\Test_User> ...no files is just insane
16:39:00 <Andrew> I never said that I'm sane
16:39:00 <\Test_User> fair enough
16:39:00 <heat> that is very clear :D
16:39:00 <Andrew> lol
16:40:00 <heat> anyway booting is hard and in no way minimal
16:40:00 <heat> don't forget that the *processor that boots your processor* also has its own OS
16:41:00 <Andrew> Hmm, yes
16:41:00 <Andrew> At least for the computers with ME and similar things
16:41:00 <heat> like everything built in the last 10-15 years
16:42:00 <heat> relevant stuff
16:42:00 <Andrew> Probably a similar but still not-exactly-that way of doing things for my pre-ME intel x86s and the arms
16:42:00 <\Test_User> amd was later to adding their version
16:42:00 <Andrew> \Test_User: Yes, I know
16:42:00 <\Test_User> I was talking to heat with that one Andrew
16:42:00 <Andrew> ok
16:42:00 <heat> ARMs also have coprocessors
16:42:00 <_73> The "processor that boots your processor" is the BIOS correct?
16:42:00 <Andrew> heat: I'm mainly working with older hardware, i.e. ThinkPad x60s
16:42:00 <\Test_User> amd, not arm
16:42:00 <\Test_User> not to say they don't have it now
16:42:00 <mrvn> heat: anything but RPi doing that?
16:43:00 <heat> clever can stockholm-syndrome-tell-you all about the vc4 and how it boots the rpi
16:43:00 <\Test_User> but I'm saying they were later to doing it
16:43:00 <\Test_User> _73: no, the BIOS is running directly on your processor
16:43:00 <Andrew> yup
16:43:00 <mrvn> RPi doesn't have a coprocessor for the ARM, the ARM is the coprocessor for the rest of the system
16:43:00 <heat> mrvn, dunno, I don't have much ARM hardware
16:43:00 <j`ey> heat: :'(
16:43:00 <j`ey> heat: can i buy you an m1
16:43:00 <heat> send me sum j`ey
16:44:00 * Andrew looks into RISC-V and MMIX
16:44:00 <_73> \Test_User: Oh ok
16:44:00 <Andrew> MMIX code is the sanest code I've seen so far
16:44:00 <\Test_User> "code"
16:44:00 <Andrew> ?
16:44:00 <\Test_User> mmix is the hardware
16:44:00 <Andrew> Code for MMIX
16:45:00 <\Test_User> code for mmix can be written in multiple languages
16:45:00 <Andrew> .... code that natively runs on mmix = the freaking binaries = basically we write assembly for those
16:47:00 <\Test_User> I don't see how that's any more sane than the same for anything else; calling the arcitecture itself sane though fair enough
16:47:00 <heat> j`ey, the macbook pro i'm getting for work is still an i7 :|
16:47:00 <j`ey> heat: laaaame
16:48:00 <Andrew> MacBooks, meh
16:48:00 <heat> j`ey: send me the M1 together with the job offer
16:48:00 <heat> kthx
16:48:00 <j`ey> heat: heh
16:48:00 <clever> heat: *waves*
16:49:00 <heat> j`ey: I have a feeling they don't give out ARM macbooks for work because the software isn't built for ARM lol
16:49:00 <heat> which, fair enough, makes sense
16:50:00 <j`ey> change that!
16:50:00 <heat> give me an SBSA + SBBR compliant board and I will
16:51:00 <j`ey> i think ampere altra is
16:55:00 <Andrew> \Test_User: Where's your repo?
16:55:00 <\Test_User> for which
16:56:00 <Andrew> Architecture
16:56:00 <\Test_User> still just a few files of basic ideas on my local filesystem
16:57:00 <Andrew> When you extend them please share it in #os
16:57:00 <Andrew> Preferrably in a version-controlled repository
16:57:00 <\Test_User> ye
18:25:00 <heat> clever, what does the vc4 use as memory?
18:25:00 <mrvn> the first 1GB of ram
18:29:00 <heat> you need to configure that though
18:29:00 <heat> does it cache-as-ram before that?
18:29:00 <heat> and then can't the arm processor access it? isn't the vc4 supposed to be kinda a security boundary?
18:36:00 <geist> i dont think so, i think it's simply the graphics thing
18:36:00 <gamozo> I've never done NVMe dev before. How hard is it? Just a ring of DMA buffers and some basic commands?
18:37:00 <geist> oh hey heat is ready to go!
18:37:00 <heat> sup les noobs
18:37:00 <geist> answer is yeah basically. it's fairly sophisticated, but looks also like it's fairly straightforward
18:37:00 <heat> gamozo, sooooooooooooooooooo
18:37:00 <geist> and you can get the specs for free
18:37:00 <gamozo> Lets assume I want to only support 1 drive, not trying to be general purpose
18:37:00 <heat> lets establish a "queue" as a tx queue and rx queue
18:38:00 <gamozo> I recently got a fancy storage server and I am _not_ impressed with the performance I'm getting. I kinda wanna see what it can do if I control the whole stack
18:38:00 <heat> (the tx and rx queues are mostly ring buffers of commands)
18:38:00 <gamozo> I see
18:38:00 * geist senses heat cracking their fingers
18:38:00 <geist> (heat was just working on a driver)
18:38:00 <heat> an nvme controller has an admin queue, where you send it admin commands like identify, identify namespaces, create_sq, create_cq, blah blah
18:38:00 <heat> you get it. admin commands, no IO here
18:39:00 <heat> then you have IO queues that do all sorts of IO commands
18:39:00 <heat> READ and WRITE are the basic ones of course but there's a lot more fancyness in there if you so desire
18:40:00 <heat> your NVMe drive needs to support the admin queue (always there, no need to create it) and a single IO SQ/CQ pair (as far as I know)
18:40:00 <gamozo> Okay, doesn't sound too bad. Hmm
18:40:00 <heat> it tells you how many queues it supports and then you create them
18:41:00 <geist> yah the basic queue mechanism is pretty straightforward. and it can complete them out of order of course
18:41:00 <gamozo> There's no middle-man between the drive and the processor right? You are directly talking to the drive over PCI?
18:41:00 <heat> it also tells you the max queue size
18:41:00 <geist> i forget how you cause them to be in order aside from draining the queues or sticking some sort of barrier in it
18:41:00 <heat> gamozo, sí
18:41:00 <heat> also, no
18:41:00 <heat> it's complicated(tm)
18:42:00 <heat> NVMe itself has no transport, you can do it over fabrics, over PCIe, over something else they make up 5 years from now
18:42:00 <geist> basic nvme/pci is just that. theres some way to sream it over other things
18:42:00 <gamozo> Why does it always have to be complicated? :( I just want to say "giff bytes real fast" or "set bytes real fast"
18:42:00 <gamozo> Ahh, so NVMe is detached from the transport
18:42:00 <heat> then you have the transport specs that tell you how NVMe behaves over fabrics, pcie, blah blah
18:42:00 <heat> yes
18:42:00 <geist> well, i poersonally think nvme is much more straightforward than most, however it's also modern and sophisticated
18:42:00 <heat> yes, it's both
18:42:00 <geist> so it's not trying to be simple per se, but it's conceptually simple because it doesn't have any backwards compatibility
18:43:00 <gamozo> Yeah, that makes sense. I'm guessing it's all based on LBAs and queues, which to me is away easier than CHS and I/O ports :D
18:43:00 <geist> absolutely
18:43:00 <geist> and there's no link state or any of that nonsense
18:43:00 <heat> also, if you go through the spec, some parts have slightly different behavior depending on the transport, some things are only supported in a single transport, etc
18:43:00 <gamozo> Hmm... I really need to play around with this. I ordered some new 100gbe nics that I'm gonna be writing drivers for, and I really really wanna switch my storage server off of Linux
18:44:00 <heat> oh yeah, nvme-over-fabrics time
18:44:00 <gamozo> But maybe I need to just look at more complex solutions than NFSoRDMA + Linux
18:44:00 <geist> ooooooh 100gbe nics
18:44:00 <geist> do you have any docs for those? which ones are they?
18:44:00 <gamozo> Yes!
18:44:00 <gamozo> So I bought $3k worth of mellanox NICs, and now I'm dumping them for Intel NICs because they're open docs
18:44:00 <geist> i'm curious how a good 100gbe nic does its own stuff
18:44:00 <geist> wow is this for some work thing?
18:44:00 <gamozo> and they're identical to the same model of Intel drivers you've written for all other Intel NICs (in fact, my 1g/10g Intel NIC driver should "work nearly out of the box")
18:45:00 <gamozo> obviously, using the fancy features will require some work
18:45:00 <heat> obviously gamozo is just flexing
18:45:00 <gamozo> Nah, just a personal project :D
18:45:00 <geist> that's a little surprising. i would assume that >=10gbe would probably switch to something a bit more efficient
18:45:00 <gamozo> I like perf okay, and this is where I've gotten to now :D
18:45:00 <geist> though intel nics are pretty darn efficient
18:45:00 <heat> im mean tbf regular NICs are already pretty efficient
18:45:00 <geist> heh well, i just bought a new mobo for my ryzen server
18:45:00 <heat> ring buffer of stuff, lots of offloading, etc
18:46:00 <geist> heat: yeah i just was thinking that even more high end ones would need to offload even more stuff
18:46:00 <zid> https://www.servethehome.com/this-changes-networking-intel-ipu-hands-on-with-big-spring-canyon/
18:46:00 <bslsk05> ​www.servethehome.com: This Changes Networking Intel IPU Hands-on with Big Spring Canyon
18:46:00 <zid> Think my e1000 driver will work? :P
18:46:00 <heat> inb4 tcp stack offloading
18:46:00 <gamozo> https://www.intel.com/content/www/us/en/products/details/ethernet/800-controllers/e810-controllers/docs.html?s=Newest
18:46:00 <bslsk05> ​www.intel.com: Intel® Ethernet Controller E810 - Product Info, Docs, Support and...
18:46:00 <gamozo> The E810 datasheet has it all
18:46:00 <gamozo> it is nearly identical to the x540/earlier Intel datasheets. It's actually really cool
18:46:00 <geist> heat: basically
18:47:00 <gamozo> I'm pretty sure an e1000 driver would work as long as you maybe tweak some of the BAR offsets. But I think the same registers still exist
18:47:00 <geist> anyway if you're familiar with those sort of work queues nvme isn't that different
18:47:00 <zid> On a weird fpga network accelerator? that's confidence
18:47:00 <geist> though a bit more aggressive about scatter gather,etc
18:47:00 <gamozo> I have no idea what kind of PPS/throughtput you'd get without leveraging offloads. But honestly, if you have a super close-to-metal driver without much abstraction, you might be able to hit 100gbe? I don't think sequential is too hard
18:48:00 <geist> since the intel nic buffer management is actually *less* sophisticated than most nics i've seen in the past
18:48:00 <geist> gamozo: i think the design of your net stack almost instantly becomes the bottleneck
18:48:00 <heat> yeah
18:48:00 <heat> you're better off using dpdk
18:48:00 <gamozo> I now have all the hardware, NICs, switching, and drives to make my dream of "100gbe saturated with 4 KiB random reads". But now I'm really being held back by software (NFS, Linux kernel, filesystem design). I really wanna see what I can do!
18:49:00 <heat> although for instance cloudflare uses the linux kernel's stack so, who knows
18:49:00 <gamozo> I got an AMD box where all the NVMe drives are direct to the processor, no PCIe bridge or anything fancy. So in theory, I should be able to linearly scale with drives (but I am not getting that with Linux)
18:49:00 <heat> maybe the kernel's stack really is that optimized
18:49:00 <gamozo> Yeah, I haven't tried any crazy tuning
18:50:00 <heat> the big advantage of dpdk might just be that you can avoid user <-> kernel transitions
18:50:00 <gamozo> Out-of-the-box NFSoRDMA has been very very slow for random access. Totally fine for large file transfers, but I want to do things like "grep a AOSP source tree", and have it saturate hardware. It's the random/many small files that really seem to struggle
18:50:00 <geist> well, i mean i'm not going to try to talk you out of it, because at the end of the day this is a darn good learning experience
18:51:00 <heat> yeah
18:51:00 <gamozo> My dream goal is to not have any drives in my workstations, instead have them network mounted. But get great performance (read: native NVMe performance) for things like source tree greps (this is my common personal workflow)
18:51:00 <gamozo> For sure
18:51:00 <heat> realistically you'll fail, but you'll learn a lot
18:51:00 <geist> i dont quite have that level of disposible income and/or i'm a bit more realistic in my endeavors
18:51:00 <gamozo> That's what all this is about at the end of the day :)
18:51:00 <gamozo> I'm really just here to get every ounce of juice out of my hardware. Largely to know what is possible
18:51:00 <gamozo> Less so than to make a viable product
18:52:00 <geist> sure. and that's *totally* fun
18:52:00 <geist> as i was telling people, *real* driver and hardware people this is what gets them up in the morning
18:52:00 <geist> you write a driver because you want to make it work and work efficiently, and that's what gets you the dopamine hit
18:52:00 <geist> everyone has a different thing that really makes them happy
18:53:00 <gamozo> If it all works out maybe I'll try to convince a company to adopt it and maybe make a product :D I've already been really happy with my specialized hypervisor performance
18:53:00 <gamozo> Exactly!
18:53:00 <heat> hmm
18:53:00 <heat> geist, what's yours?
18:53:00 <heat> board bringup? cpu stuff?
18:53:00 <gamozo> geist gets the dopamine hit every time he sees a new bootloader using LK
18:53:00 <gamozo> :D
18:53:00 <heat> :D
18:53:00 <geist> gamozo: surprisingly no, honestly, or i'd have figured out a way to make moneies off it
18:54:00 <geist> i'm much more into doing things than getting it done
18:54:00 <geist> seeing Lk used in stuff is nice, but not a huge source of excitement
18:54:00 <gamozo> Awhh, that's fair though
18:54:00 <geist> i love doing deep cpu stuff. i really get a kic out of learning all the nitty gritty supervisor mode bits
18:55:00 <geist> eventually you learn most of an arch which is why i switch to other arches
18:55:00 <gamozo> What's your current arch of study?
18:55:00 <geist> i mostly spend my days thinking about arm and riscv
18:55:00 <geist> though i had recently done some bringup on 68k
18:55:00 <heat> i bet you get sad when thinking about x86
18:55:00 <geist> hmm? why so?
18:55:00 <gamozo> Good choices. I love RISC-V with a passion. I haven't touched the system side though, only user-land design
18:56:00 <heat> descriptor tables
18:56:00 <heat> weird compat stuff
18:56:00 <mrvn> I'm trying to convert between std::array and std::bitset: https://godbolt.org/z/MdeTr6xfs How does the compiler not see that this is just copying 16bit from A to B?
18:56:00 <bslsk05> ​godbolt.org: Compiler Explorer
18:56:00 <geist> heat: oh no. that's fun. almost the more gnarly the arch the more interesting it is
18:56:00 <sbalmos> SPARC? Alpha?
18:56:00 <geist> i just have been working with x86 for 30 years so there's not much interesting for me to personally learn about it anymore
18:56:00 <gamozo> mrvn: Oh yikes ahaha
18:57:00 <geist> i have owned a few of those, one day i'll go back to fiddling with
18:57:00 <mrvn> to_array2 is verry interesting
18:57:00 <heat> geist, ever done firmware?
18:57:00 <geist> sure
18:57:00 <mrvn> Why is the compiler doing a sign extend there by hand?
18:57:00 <heat> that's probably a fresh take on the CPU
18:57:00 <geist> but i also love writing kernels, or little blobs of free rnning code that have to deal with data over a long period of time
18:57:00 <heat> (well, to an extent maybe)
18:57:00 <geist> ie, kernels, filesystems, game engines, etc
18:58:00 <geist> that seems like intrinsically more interesting to me than run-once-and-stop sort of things: user space apps, even compilers/etc
18:58:00 <gamozo> I think my favorite (albeit 15 minute project) was flashing "my own BIOS" on an Intel system. All it did was `out` to the PIT to play a sound on the PC speaker. But it was surreal to press a power button on a fully featured Intel workstation and _instantly_ it beeped. I've never had my code execute so fast on a big computer like that
18:58:00 <geist> for no particular reason, just my preference
18:59:00 <geist> gamozo: yep! thats a hard thing to describe, but i still, 30 years later, get a kick out of seeing exactly that sort of thing
18:59:00 <geist> ie, building something out of nothing. being the first thing running on a thing, etc
18:59:00 <heat> gamozo, that's totally fun but how do you recove your system after that? if you don't have a backup BIOS
18:59:00 <heat> that's the thing that scares me
18:59:00 <heat> unless you have an SPI programmer (?)
18:59:00 <sbalmos> I got a kick out of firmware just because it was right at the hardware boundary to converting to electrons.
19:00:00 <zid> ME fixes all sins
19:00:00 <gamozo> heat: I actually uhh... desoldered my flash chip (8-pin flash), then crudly hot glued and soldered a DIP socket onto the motherboard, and then converted the surface mount bios flash to a DIP
19:00:00 <gamozo> and I could just pluck it in and out, would re-program it physically. So I could recover/restore any bit on it
19:00:00 <sbalmos> it's like when you /really/ understand that enabling an address line just energizes a wire to a set of transistors, and due to electron migration that's how a transistor lets through a signal, etc... build all that a billion+ times, and you somehow magically have logic
19:00:00 <gamozo> I think I still have that computer
19:00:00 <zid> a lot of my bios chips on mobos have been factory socketed
19:01:00 <heat> sbalmos, that's still pretty far from the firmware though
19:01:00 <geist> sbalmos: exactly
19:01:00 <heat> at least x86 fw
19:01:00 <zid> Lots of the little square holders where you pop the chip out from grabbing it
19:01:00 <geist> there's a hard to describe enjoyment out of working at that level
19:01:00 <zid> idk what they're called
19:01:00 <sbalmos> heat: ture
19:01:00 <geist> and i refuse to let people like heat and zid tell me it doesn't count :)
19:01:00 <sbalmos> heheheh
19:01:00 <zid> https://res.cloudinary.com/rsc/image/upload/b_rgb:FFFFFF,c_pad,dpr_1.0,f_auto,h_843,q_auto,w_1500/c_pad,h_843,w_1500/R0211231-10?pgw=1&pgwact=1
19:01:00 <zid> PLCC apparently
19:02:00 <geist> there's always a 'well ackchyually' person
19:02:00 <sbalmos> geist: it's why my degree is CSE. All the fun of hardware design and electrical eng without dual-majoring and having to deal with all that power electronics crap knowledge
19:02:00 <geist> CSE is computer engineeering?
19:02:00 <heat> geist, it's whats best described as "galaxy brain" time
19:02:00 <sbalmos> Comp Sci Eng, ya
19:02:00 <sbalmos> err rather more like Comp Sci & Eng
19:02:00 <heat> true nirvana of understanding
19:02:00 <zid> Fuck power electronics
19:02:00 <geist> ah. mine is a differnt name but basically the same thing: computer engineering, but as taught by the CS department
19:03:00 <sbalmos> huh interesting
19:03:00 <sbalmos> yeah I'd always been told CS is the software side, CE is the hardware side
19:03:00 <geist> they had a CS and EE track at the time. probably because the two departments hadn't yet decided who gets the degree
19:03:00 <sbalmos> zid: What's a few blown caps and resistors among friends? :D
19:04:00 <zid> power electronics are why HW dies
19:04:00 <zid> true fact
19:04:00 <sbalmos> heheheh
19:04:00 <geist> funny i was just looking up what to replace REFA caps with
19:04:00 <gamozo> Here's the fancy bios chip https://imgur.com/a/dHjeGsU
19:04:00 <bslsk05> ​imgur.com: Imgur: The magic of the Internet
19:04:00 <zid> does it need to be a gallery link
19:04:00 <zid> I'd have to find chrome
19:04:00 <geist> i see all thee videos of people doing it, but they never say what they replaed it with. i guess there's a modern equivalent of the same caps that just dont blow up
19:04:00 <mrvn> Anyone an idea how to make the array <> bitset conversion more compiler friendly?
19:04:00 <sbalmos> zid: yeah, don't ask how many 68HC12s and resistors I've /literally/ blown because I turned the knob on the bench PS one too many notches, and ended up putting 20V through it instead of 2V
19:05:00 <zid> sbalmos: come fix my monitor, it needs new power electronics
19:05:00 <sbalmos> gahd that was fun
19:05:00 <heat> mrvn, memcpy and pray? :P
19:06:00 <mrvn> heat: just acsting array and bitset to void*? urgs
19:06:00 <gamozo> heat: Yeah! I have a TL866 which can program pretty much anything. It's like $80 on ebay + like 30-40 adapters for various chips. It has increased my confidence a lot
19:06:00 <heat> i've slightly thought about getting open firmware on an old laptop but i'm just not qualified for it
19:06:00 <mrvn> clang -O3 actually manages to optimize to_array2 perfectly. Just from_array2 is crap.
19:07:00 <gamozo> I've lifted chips and firmware now off of pretty much all random things in my house. Printers, routers, my oscilloscope (eww runs Blackfin, hard to compile against)
19:07:00 <mrvn> movzx eax, word ptr [rdi] which I find quite impressive
19:07:00 <heat> 1) can't desolder; 2) can't solder; 3) can't debug stuff; 4) No idea what's different from the reference platforms
19:07:00 <mrvn> gamozo: have you found a use for any of those chips?
19:08:00 <heat> i also have a big fear of fucking things up and ending with a bricked machine
19:08:00 <gamozo> Uhh, so I had this dream plan to write an OS for my oscilloscope (common Rigol oscilloscope) and massively improve the UX/performance/remote data streaming. But when I found it ran blackfin I gave up
19:08:00 <heat> > write an OS for my oscilloscope
19:08:00 <gamozo> Mainly I just like to dump firmware and try to look for exploits and stuff :D
19:08:00 <heat> cursed
19:09:00 <mrvn> gamozo: play doom on your oscilloscope
19:09:00 <heat> gamozo, you should look at edk2
19:09:00 <gamozo> I drilled a hole in one of my printers and actually relocated the flash chip to an external socket. So you can hot swap the flash chip while the printer is fully assembled! :D
19:09:00 <heat> it is neither fuzzed nor sanitized
19:09:00 <heat> it's so :/
19:10:00 <gamozo> mrvn: Ahaha, I really wanted to do something like that. But Rust can't target Blackfin (well, LLVm can't). Didn't want to write C. Although, I think rust's GCC backend might be stable enough at this point to use
19:11:00 <mrvn> gamozo: no need to compile anything. There is a doom version that outputs sound and your oscilloscope then draws the screen.
19:11:00 <gamozo> heat: I've never audited edk2, that'd be fun :D
19:11:00 <gamozo> mrvn: Ooh I think I've seen that
19:11:00 <gamozo> (or something like it?)
19:11:00 <mrvn> https://www.youtube.com/watch?v=OU16lIx_pC8
19:11:00 <bslsk05> ​'Doom on an Oscilloscope (Tektronix 2220)' by MrSlehofer (00:05:10)
19:12:00 <sbalmos> gamozo: Did my college cooping with Keithley Instruments, doing firmware for some of their bench scopes and high-end stuff. Working in Wind River for the high end stuff, but the bench scopes we ended up embedding a Lua runtime to execute sampler scripts
19:12:00 <sbalmos> that was... interesting.
19:12:00 <sbalmos> can't remember what processor it ran though
19:12:00 <mrvn> Looks much better than quake on an oscilloscope too. Not sure if it's the game or the oscilloscope used.
19:12:00 <gamozo> Hngggg that's awesome! I just got a Keithley DMM and it's honsetly the nicest piece of instrument I've ever used
19:13:00 <gamozo> It's just so beautiful, great user interface, fantastic docs and programming interface
19:13:00 <sbalmos> ya
19:13:00 <sbalmos> which model?
19:13:00 <gamozo> DMM7510
19:13:00 <gamozo> I might get 1 or 2 more to add to my stack, right now I can only probe one thing. Would love to be able to do power analysis
19:13:00 <sbalmos> geez that rings a bell. I think... was it a DMM750 series? Have to go look it up
19:14:00 <gamozo> It's their main 7.5 digit multimeter model I think
19:14:00 <sbalmos> yeah
19:14:00 <gamozo> I bought it for doing hardware side-channel stuff where I care about super tiny changes in current, and needed good network data logging for long tests
19:15:00 <gamozo> I love it. My nerdy friends and I love standing across the room and rubbing our feet no the carpet and seeing the instrument visibly pick up the data in the graph
19:15:00 <heat> gamozo, last time someone enabled ASAN in edk2 it discovered a ridiculous amount of bugs
19:15:00 <gamozo> ahaha
19:15:00 <heat> it's... concerning
19:15:00 <gamozo> heat: ugh, kinda not surprised. Lots of firmware/embedded stuff is really new
19:15:00 <sbalmos> yup. my head hurt when I was first intro'd to the idea of a secondary signal ground probe when reading down in the pico/femtovolt range
19:15:00 <heat> i want to see if I can get proper fuzzing going this summer as a side-project
19:15:00 <gamozo> Sooooo I actually have a company that really wants me to start up their fuzzing team and really focus on being able to fuzz in weird places and embedded stuff. It sounds really fun
19:16:00 <geist> FUZZ TILL YOU DROP
19:16:00 <gamozo> Let me know if you have fuzzing questions. I normally do more one-off stuff, but I can always lend an ear
19:16:00 <heat> gamozo, edk2 is like 22 years old. not new.
19:16:00 <heat> it failed to adapt basically
19:16:00 <gamozo> Really!? I always thought it was newish. Huh
19:17:00 <heat> no lol
19:17:00 <heat> it's the first (and the main) UEFI implementation ever
19:17:00 <sbalmos> gamozo: gah! oh man, memory lane. TSB, the Test Script Builder. Yeah, my name's in a few of the contributor header comment blocks in there somewhere. LOL
19:17:00 <heat> vendors just branch out from edk2
19:17:00 <gamozo> sbalmos: That's awesome!
19:18:00 <gamozo> I've only recently gotten into UEFI. I've written one UEFI OS. I like it, largely as it just normalizes a lot of annoying parts of boot. But I bet it's an absolute hack under the hood ahaha
19:18:00 <heat> I had thought about using syzkaller to fuzz the APIs
19:18:00 <heat> well, it's not a hack
19:18:00 <gamozo> I did (read: followed someones instructions) an EDK2 mod for one of my ARM64 dev boards to get Windows booting on it
19:18:00 <heat> sometimes totally overengineered even
19:19:00 <heat> it just has poor testing
19:19:00 <sonny> gamozo: tldr boot process pls
19:19:00 <gamozo> Someone had a patch that injected QEMU into EDK2, such that you could run an x86 video BIOS on an arm64 board
19:19:00 <gamozo> AND IT WORKED
19:19:00 <heat> thats upstream
19:19:00 <gamozo> LITERALLY WINDOWS DESKTOP
19:19:00 <gamozo> oh really?
19:19:00 <gamozo> wow
19:19:00 <gamozo> cool
19:19:00 <heat> yes
19:19:00 <heat> hold on
19:19:00 <heat> https://github.com/tianocore/edk2-non-osi/tree/master/Emulator/X86EmulatorDxe
19:19:00 <bslsk05> ​github.com: edk2-non-osi/Emulator/X86EmulatorDxe at master · tianocore/edk2-non-osi · GitHub
19:19:00 <sbalmos> whoa
19:19:00 <gamozo> that's the one!
19:19:00 <j`ey> bit crazy aint it
19:19:00 <gamozo> yeah it was a patch set back then!
19:20:00 <sbalmos> that's nuts
19:20:00 <mrvn> I've booted Win95 on my Amiga.
19:20:00 <heat> you know how we could've avoided this?
19:20:00 <heat> efi bytecode
19:20:00 <heat> just sayin
19:20:00 <gamozo> I'm all for bytecode
19:20:00 <zid> you mean EfiByteCode
19:20:00 <j`ey> EFI_BYTE_CODE
19:20:00 <heat> gamozo, you don't understand. this is a real thing
19:20:00 <mrvn> and that was before emulators had good JIT.
19:21:00 <gamozo> Can someone give me the EFI GUID for the EfiByteCode?
19:21:00 <gamozo> ;)
19:21:00 <heat> gEfiByteCodeProtocolGuid
19:22:00 <gamozo> Honestly I'm a huge fan of ILs and bytecodes. I wish software shipped in a super-close-to-native format such that it could be converted when you install it. I know, I know, Android kinda does this (but very poorly)
19:22:00 <sbalmos> I just laugh/love how the FreeBSD bootloader is just Forth
19:22:00 <zid> I love bytecodes
19:22:00 <geist> yah re: running emulators to initialize video bios. i've seen various forms of that for various non x86 machines over the years
19:22:00 <gamozo> I like to believe that in the future you can just run software on any arch and it's just fine
19:22:00 <zid> As long as they're basically an arbitrary machine code, and not like "do complicated java thing"
19:22:00 <heat> ask intel for the EBC compiler
19:22:00 <geist> i used to have a PPC machine (Pegasos I think) that had a openfirmware based bios, but also did the emulator-to-initialize-vesa thing
19:22:00 <heat> OVMF has the EBC interpreter built in :)
19:23:00 <\Test_User> gamozo: sounds more like you want there to only be one arch in the future
19:23:00 <geist> of course. as does ACPI, etc etc
19:23:00 <\Test_User> the whole point of a different arch isn't to do... the exact same thing
19:23:00 <gamozo> geist: Hmm. I think my G4 also uses openfirmware? Whatever has the white background-black text EFI shell. That was wild
19:23:00 <geist> but my point is emulating a different arch because the rom is for it is an old silly tricky
19:23:00 <geist> yep. that's openfirmware
19:23:00 <heat> well an efi shell won't be openfirmware i think
19:23:00 <geist> that's correct
19:23:00 <gamozo> \Test_User: Not really. More just architecture-agnostic code. I don't care how many different ILs there are
19:24:00 <geist> also iirc one of the X servers does it even on x86. i think it emulates a 8086 to re-run the video bios or something on some state transition
19:24:00 <heat> yes they do that
19:24:00 <gamozo> My security dream is shipping users IL, and users will randomize structure layouts and code layouts, and then turn on execute-no-read
19:24:00 <gamozo> Would make exploitation really really hard
19:24:00 <\Test_User> different architectures have different formatting and features, if it's the same code for the same results is it really a different arcitecture
19:24:00 <gamozo> users meaning, the installer when you install the application
19:24:00 <geist> yah we have an RFC and design in place to enable --X for arm64 in some cases
19:25:00 <geist> it's a hairy mess though
19:25:00 <heat> theoretically you can also have it in x86 server CPUs
19:25:00 <heat> pkeys
19:25:00 <geist> ah yeah, and that's intel specific right?
19:25:00 <j`ey> --X?
19:25:00 <j`ey> ohh RWX, --X
19:25:00 <heat> geist, I believe so, at least for now
19:25:00 <geist> j`ey: execute only
19:25:00 <gamozo> \Test_User: It's mainly just getting people in the mindset that the same code they compiled will run in different environments, which would encourage future-portable code designs. It's less about the standardization, and more about getting developers to adopt coding styles that allow for their code to be easily shipped on many targets
19:25:00 <j`ey> (thought it was a compile flag lol)
19:25:00 <geist> ah haha. X-only
19:26:00 <gamozo> Eg. a new arch comes out and you just re-compile with clang and it just works (barring some low level details). Arguably, this is the point of libc or an OS, but I don't think they do these things very well anymore
19:26:00 <geist> you can do it on arm64 and i think riscv has the capability maybe? i should check, now that i am thinking about it
19:26:00 <geist> but alas, arm64 has a bug in it that makes it not useful until later cores
19:26:00 <heat> gamozo, EFI is done around those principles
19:26:00 <gamozo> The fact that porting applications from gcc to clang was a big deal (because of GNU extensions) is insanely tilting
19:26:00 <geist> one of the few straight up ARM spec bugs where someone forgot a detail
19:26:00 <heat> geist, yes it does, iirc there's a read bit separate from the valid bit
19:26:00 <gamozo> and to me, is a failure of our developer cycle :(
19:27:00 <geist> heat: yah i think you're right. and IIRC the riscv version of SMAP/SMEP/PAN is pretty heavy hammered but works
19:27:00 <\Test_User> gamozo: then that's simply a matter of the compiler's job, and fair enough
19:27:00 <geist> ie, with the bit set supervisor mode simply cant access anything marked user, period.
19:27:00 <heat> i thought it was fine
19:28:00 <geist> right. that's sort of the most straightforward and hard to screw up version you can think of
19:28:00 <gamozo> \Test_User: Honsetly, I think things like the ARM64 macbook have really helped this. I've been seeing a lot of applications (that ship binaries) now shipping with ARM64 on both Windows and OSX because of it. Which is nifty
19:28:00 <heat> apparently I do indeed do execute only on riscv
19:28:00 <heat> nice
19:29:00 <geist> yep, just saw the same thing, i was trying to remember what the combination was for 'next table pointer' but it's ---
19:29:00 <geist> so that makes sense
19:29:00 * kingoffrance "the fact that porting applications from gcc to clang was a big deal (because of GNU extensions)" looks at big ball of c89 code. looks at gamazo. are you SURE you want to go there? <no> <no> "places where men fear to tread" -- frogurt guy
19:29:00 <geist> they burned a bit for the valid bit but that's nice for other reasons
19:30:00 <heat> what if --- was invalid and they had a separate huge page bit?
19:30:00 <heat> I guess you'd still lose a bit but that sounds a bit saner
19:30:00 <heat> at least more... normal?
19:31:00 <geist> well one way they could have done it is used 3 bits, 000 is not valid and all the other combinations are RWX and one other invalid/not useful version of that is a next table pointer
19:31:00 <mrvn> kingoffrance: what GNU extension doesn't clang have?
19:31:00 <geist> but, having a separate bit 0 being V is useful for OSes that want to put stuff in an invalid entry, since they then have 31 or 63 bits to stuff what they want
19:31:00 <kingoffrance> mrvn: ask crazy person gamazo :)
19:32:00 <heat> mrvn, a bunch tbh
19:32:00 <heat> also it ignores a lot of gcc options
19:32:00 <heat> like straight up ignores
19:32:00 <heat> -fcheck-new for instance
19:32:00 <geist> iirc arm64 has i think 2 bits at the bottom, which is kinda a waste: something like valid + this is a terminal or non terminal entry
19:33:00 <heat> also if anyone really wants clang -fcheck-new, ping https://reviews.llvm.org/D125272 :)
19:33:00 <bslsk05> ​reviews.llvm.org: ⚙ D125272 [clang] Add -fcheck-new support
19:33:00 <geist> so in that regard the riscv format is a bit more compact since you only need 4 bits to describe all of valid + permissions + pointer to next page table
19:33:00 <heat> upstream is being very slow... yay open source
19:33:00 <geist> but of course what it *doesnt* le tyou do is have heirarchial permissions, like arm and x86 have
19:33:00 <geist> ie, having permissions on an inner page table node that take precendence on anything downstream of it
19:34:00 <j`ey> heat: is that for work?
19:34:00 <heat> i think hierarchical perms is a bit stupid
19:34:00 <heat> j`ey, no. just something i really want
19:34:00 <geist> they're great for safety purposes
19:34:00 <mrvn> heat: sounds like a bogus option
19:34:00 <heat> my kernel has been accidentally compiling without nullptr checks in clang builds *silently*
19:35:00 <heat> you know, clang used to warn you about this
19:35:00 <heat> but *they explicitly ignored it*
19:35:00 <heat> this is a literal security risk
19:35:00 <mrvn> heat: The check seems to be only for operator new that claims to throw but then returns nullptr instead
19:36:00 <heat> geist, why? don't you just fail early on bad perms?
19:36:00 <heat> mrvn, yes
19:36:00 <mrvn> heat: so fix your damn new
19:36:00 <clever> heat: the VPU can access the lower 1gig of memory, and the firmware places itself at the top $gpu_mem mb of that memory
19:36:00 <heat> how about
19:36:00 <heat> no
19:36:00 <clever> heat: the hw is capable of blocking access from the arm, but that is not enabled on the rpi firmware
19:36:00 * geist waits for the c++ discussion to die down
19:36:00 <heat> new (std::noexcept) blah looks horrible
19:37:00 <mrvn> heat: give me a compiler flag that gives a warning/error when new claims to throw but returns nullptr. That would be usefull.
19:37:00 <heat> ...
19:37:00 <j`ey> heat: reply 'patches welcome'
19:37:00 <geist> thats actually not always true
19:38:00 <geist> patches welcome that line up with the pre-decided notions of how things should work are welcome
19:38:00 <mrvn> heat: sure does look horrible but it's the right thing.
19:38:00 <j`ey> geist: hehe
19:38:00 <geist> heh i only say that because as a trite answer it's frequently not really true
19:38:00 <sbalmos> void* bwahahaha = new (0x800000)EvilPointerClass();
19:38:00 <geist> it's based on the idea that if you put forth energy any project will take what you give them
19:38:00 <heat> mrvn, so. there's this option, it's useful, people find it useful, i want it, its a security risk if you don't have it
19:39:00 <j`ey> geist: patches are welcome... to be rejected maybe :P
19:39:00 <mrvn> heat: you should call it -fnew-nothrow
19:40:00 <heat> .....
19:40:00 <mrvn> heat: and then all the std::bad_alloc throws should return nullptr
19:41:00 <heat> j`ey, i'm ok with a rejection, but a silent upstream is just frustrating as hell
19:42:00 <mrvn> heat: I would love to have new/constructor behavior like you have with exceptions (stack unwinding etc) but return nullptr instead.
19:42:00 <j`ey> yeah
19:42:00 <j`ey> hopefully a few more pings will be enough...
19:43:00 <sbalmos> Captain Ramius asked for one ping only though
19:44:00 <heat> anyway geist how are hierarchical perms good for security?
19:44:00 <mrvn> while(sleep(1m)) { if fixed then break else ping }
19:45:00 <mrvn> sbalmos: newest gcc will warn about potential nullptr + offset access for that
19:45:00 <mrvn> (annoying bug)
19:46:00 <sonny> heat seems like the straigtfoward way to get RBAC
19:47:00 <heat> hm?
19:47:00 <heat> we're talking about page tables
19:50:00 <mykernel> extern void f(uint8_t a[12]); // how do i write to this array in assembly
19:51:00 <mrvn> mykernel: you write to it in C, compile and look at the objdump -d output
19:51:00 <mrvn> and then you just keep the C version because doing it in asm is stupid.
19:51:00 <heat> mykernel, it's passed as a pointer
19:51:00 <heat> so you write to it normally, like a pointer
19:52:00 <mrvn> mov.b #42, (a0)
19:52:00 <heat> like mov $0xdead, (%rdi)
19:52:00 <jafarlihi> How do browsers do rendering? Can something like GTK be used to create a respectable browser?
19:52:00 <heat> they use tons of opengl, vulkan, d3d
19:52:00 <heat> not gtk
19:52:00 <mykernel> ty
19:53:00 <sbalmos> don't forget the JS+DOM engine
19:53:00 <heat> gtk draws the normal UI widgets, window decorations, but not the HTML
19:53:00 <heat> mykernel, np
19:54:00 <jafarlihi> heat: Will I be able to figure out how if I read a book or two on OpenGL and Vulkan?
19:54:00 <sonny> page tables have permission bits (on intel64), but it seems like there's other stuff on other arches
19:54:00 <GeDaMo> https://browser.engineering/index.html
19:54:00 <bslsk05> ​browser.engineering: Web Browser Engineering
19:55:00 <heat> jafarlihi, no. you can figure out how to use opengl or vulkan, but browser rendering is totally non trivial
19:55:00 <sonny> jafarlihi you have to make an 'html engine' or something
19:55:00 <sonny> like webkit
19:55:00 <jafarlihi> GeDaMo: He uses tkinter lol
19:55:00 <heat> have you noticed how there are only really 2 or 3 web engines?
19:55:00 <GeDaMo> The principles should be the same
19:56:00 <sbalmos> heat: no love for lynx? :)
19:56:00 <heat> soz lynx
19:56:00 <sonny> what html version does lynx support?
19:57:00 <jafarlihi> Is making a passable web engine harder than making a passable OS?
19:57:00 <heat> not only html, but you need css too
19:57:00 <heat> fuck do I know
19:57:00 <sbalmos> might as well be
19:57:00 <heat> it's hard
19:58:00 <heat> is it harder? dunno
19:58:00 <sbalmos> hell, seeing as you can run OSs inside a web browser...
19:58:00 <sonny> jafarlihi depends what html+css version you support
19:58:00 <heat> and javascript!
19:58:00 <sonny> oh yeah
19:58:00 <heat> and if you care about performance, conformance, compatibility
19:58:00 <sonny> m$ couldn't keep up a modern one, so it seems pretty hard
19:59:00 <heat> haha m dollarsign get it
19:59:00 <GeDaMo> https://drewdevault.com/2020/03/18/Reckless-limitless-scope.html
19:59:00 <bslsk05> ​drewdevault.com: The reckless, infinite scope of web browsers
19:59:00 <GeDaMo> https://platform.html5.org/
19:59:00 <bslsk05> ​platform.html5.org: The Web Platform: Browser technologies
20:00:00 <sonny> and they have all the reason to want to have their own engine too
20:00:00 <sonny> well, parts of it seems to have survived
20:00:00 <heat> GeDaMo, that's not entirely true
20:00:00 <heat> serenityOS people are writing their own for instance
20:01:00 * kingoffrance smells a phone does everything but make calls, browser does everything but browse joke
20:01:00 <GeDaMo> Yeah, you don't have to implement everything in order to have a useful browser
20:02:00 <sonny> hmm edge webview2 might be chrome, idk
20:02:00 <sbalmos> Edge is based off Chromium nowadays
20:03:00 <sonny> I don't know what a browser is supposed to be tbh
20:04:00 <sonny> maybe that is the real problem
20:04:00 <GeDaMo> It's a file viewer
20:04:00 * sonny thinks
20:04:00 <GeDaMo> It reads files from the Internet and displays them
20:05:00 <sonny> if that's what it is, I'm really disapointed
20:05:00 <heat> yes that's mostly it
20:05:00 <GeDaMo> Why?
20:05:00 <jafarlihi> Do you also think about software and gitting gud from sun up to sun down yet still feel like a total noob?
20:05:00 <heat> file:// is a URI too
20:06:00 <sonny> GeDaMo: so instead of a preview pane we get some additonal formats that aren't that nice
20:07:00 <heat> jafarlihi, you've got the wrong priorities :)
20:07:00 <heat> focus on one thing at a time
20:07:00 <heat> you can't galaxy brain your way through
20:07:00 <sonny> GeDaMo: so search isn't the primary part?
20:07:00 <GeDaMo> What search?
20:07:00 <heat> browsers don't search
20:07:00 <sonny> huh
20:07:00 <heat> well, tbh even I can search
20:08:00 <GeDaMo> Web search is a database back end with a very simple front end
20:08:00 <heat> https://google.com/?q=osdev
20:08:00 <sonny> so you have to know where you are, then you can move around?
20:08:00 <GeDaMo> But that's a website, not a brwoser thing
20:09:00 <GeDaMo> The browser can search within webpages it's displaying but that's not complex
20:09:00 <sonny> GeDaMo: just seems like a lot of additonal complexity that could be offloaded to various applications
20:09:00 <GeDaMo> Every program expands to become an operating system :P
20:09:00 <sonny> that's why I'm disappointed
20:10:00 <heat> let me pipe chrome to grep
20:10:00 <GeDaMo> You could see a web browser as a frontend to multiple format viewers
20:10:00 <sonny> slightly related, I was considered a file explorer with the concept of space, and also a file system that uses a coordinate system
20:10:00 <GeDaMo> Although it's not usually implemented like that
20:11:00 <GeDaMo> heat: you could pipe curl to grep :P
20:11:00 <heat> can I pipe links to grep though?
20:11:00 <heat> that's the real question
20:11:00 <GeDaMo> sonny: https://en.wikipedia.org/wiki/Fsn_(file_manager)
20:11:00 <bslsk05> ​en.wikipedia.org: fsn (file manager) - Wikipedia
20:11:00 <sonny> :O
20:12:00 <sonny> GeDaMo: yeah, pretty much this
20:12:00 <sonny> but I was gonna make it a little bit more decorated and game-like
20:12:00 <GeDaMo> https://www.cs.unm.edu/~dlchao/flake/doom/chi/chi.html
20:12:00 <bslsk05> ​www.cs.unm.edu: Doom as an Interface for Process Management
20:13:00 <sonny> wow, what a cool paper/project
20:13:00 <heat> someone got paid to write that?
20:13:00 <geist> nice gig
20:14:00 <sonny> I like interfaces like "General Magic's MagicCap"
20:15:00 <GeDaMo> https://en.wikipedia.org/wiki/Skeuomorph#Virtual_examples
20:15:00 <bslsk05> ​en.wikipedia.org: Skeuomorph - Wikipedia
20:15:00 <geist> does that let you pilot your mecha with more efficiency?
20:15:00 <sonny> "Less than a week after the initial version of the code was written, the project's website was attracting tens of thousands of visitors per day." -- That's a lot, especially since this paper looks old
20:15:00 <geist> you can frag the Zentraedi invaders with more effectiveness
20:15:00 <heat> "Competitive video games and operating system development as an interface for eternal anger issues"
20:16:00 <sonny> apple bring Forstall back pls
20:17:00 <heat> intel bring itanium back pls
20:17:00 <sonny> heat: are you prepared to write the compiler?
20:17:00 <heat> it's not the arch we deserve but it's the arch we need
20:18:00 <kingoffrance> re: doom it makes sense, would you trust a sysadmin who is not a bofh :)
20:18:00 <sonny> oh I learned some risc architectures have micro-ops
20:18:00 <heat> sonny, backport gcc 10's itanium support
20:18:00 <sonny> what a surprise
20:19:00 <sonny> gcc 10 had itanium support!?
20:19:00 <heat> yes
20:20:00 <sonny> so I guess itanium is the arch we need if it doesn't have micro-ops
20:20:00 <sonny> heat: to think there's still servers running that stuff
20:20:00 <heat> it must have
20:20:00 <heat> itanium is a perfectly fine architecture
20:20:00 <heat> very different
20:21:00 <sonny> EPIC
20:21:00 <heat> no that's amd
20:21:00 <heat> because of itanium we got UEFI
20:21:00 <heat> and the C++ abi
20:21:00 <GeDaMo> https://www.cnet.com/tech/tech-industry/intel-hp-unveil-epic-technology/
20:21:00 <bslsk05> ​www.cnet.com: Intel, HP unveil EPIC technology - CNET
20:22:00 <sonny> Does UEFI have an abi?
20:22:00 <heat> yes
20:22:00 <sonny> Is it C or C++?
20:22:00 <heat> it's the MS ABI for x64 for all intents and purposes
20:22:00 <sonny> oh
20:22:00 <heat> C
20:23:00 <heat> we have a gsoc student that's going to try to add rust to it
20:23:00 <sonny> what a weird collab to introduce secure boot
20:23:00 <heat> huh?
20:23:00 <sonny> I thought uefi and secureboot are from one result?
20:24:00 <heat> i'm not sure i'm getting you
20:24:00 <sonny> microsoft and intel teamed up to make the spec
20:24:00 <heat> microsoft and intel teamed up to make the original spec
20:25:00 <heat> microsoft, intel and a lot of other vendors team up regularly to make newer specs and work on EDK2, the reference UEFI implementation
20:25:00 <heat> secure boot is an *optional* feature
20:26:00 <sonny> were windows servers still popular back then?
20:28:00 <heat> idk
20:28:00 <heat> but UEFI has nothing to do with servers
20:29:00 <heat> it attempted to fix BIOS development
20:29:00 <heat> it partially worked
20:29:00 <sonny> servers seem to be mostly intel64
20:29:00 <heat> I can write a UEFI driver and compile it with GCC and you can load it on your visual studio-compiled firmware, made in 2012
20:29:00 <heat> it still works
20:29:00 <sonny> or x86
20:30:00 <sonny> wait
20:30:00 <sonny> visual studio compiled firmware!?
20:30:00 <j`ey> yep
20:30:00 <heat> a lot of it is compiled in visual studio
20:30:00 <sonny> I thought they only got clang recently? has it always worked with other compilers?
20:31:00 <heat> intel is a pretty solid user of windows and vs
20:31:00 <heat> sonny, no. it's been there for a good while afaik
20:31:00 <sonny> oh wow
20:32:00 <heat> edk2 technically supports a whole lot of visual studios, clang3.5+, clang3.8+, gcc 4.8+, gcc 5+, xcode
20:32:00 <sonny> I was under the impression visual studio is only for microsoft apps
20:32:00 <heat> visual studio is a C/C++ compiler like any other
20:34:00 <sonny> sure but consider how they support it
20:34:00 <sonny> well, the more you know
20:34:00 <mrvn> if you call a compiler that lacks C99 support a C compiler.
20:34:00 <mrvn> or did they finally add stdint.h?
20:35:00 <sonny> it supports c11 and c17 now
20:35:00 <mrvn> sonny: all of it or just the parts c++ covers?
20:35:00 <GeDaMo> I think variable length arrays was the stumbling block
20:36:00 <sonny> not sure, they had some lame excuse about not supporting VLAs
20:36:00 <sonny> made me wonder if it had anything to do with the architecture of their C++ compiler
20:36:00 <mrvn> sonny: they are pretty horrible to deal with
20:36:00 <GeDaMo> https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving-in-msvc/
20:36:00 <bslsk05> ​devblogs.microsoft.com: C11 and C17 Standard Support Arriving in MSVC - C++ Team Blog
20:36:00 <heat> mrvn, EDK2 is C89
20:36:00 <sonny> masm official docs aren't great
20:36:00 <mrvn> heat: and three guesses why
20:37:00 <heat> portability and visual studio
20:37:00 <mrvn> s/and/to/
20:37:00 <heat> no.
20:37:00 <sonny> still not sure how to do custom linking stuff with visual studio, I don't think it's possible
20:38:00 <heat> the subset of C EDK2 uses is crafted to be portable
20:38:00 <heat> no intrinsics for instance
20:38:00 <mrvn> sonny: I think structures with VLA should have a size without tail padding.
20:38:00 <heat> sonny, how do you think EDK2 links?
20:38:00 <sonny> linkerscript?
20:39:00 <heat> no
20:39:00 <heat> well, not with visual studio
20:40:00 <sonny> ok, time for me to read docs again
20:41:00 <sonny> oh, is it using module-definition files?
20:52:00 <jafarlihi> I can't make OS worth shit. What do you suggest one make instead of OS to get skills before attempting OS again?
20:52:00 <sbalmos> anything?
20:52:00 <jafarlihi> Browsers too hard, games - I hate games
20:54:00 <zid> anything
20:54:00 <zid> rewrite all of binutils
20:54:00 <zid> rewrite all of coreutils
20:54:00 <zid> make a 3D visualizer for toast
20:54:00 <zid> play TIS-100, play spacechem, play exapunks
20:54:00 <sbalmos> write a bytecode interpreter
20:54:00 <kingoffrance> "to make gold we must have gold" i agree with klange, stubbornness/spite goes a long way :)
20:54:00 <zid> write an emulator for gameboy
20:55:00 <kingoffrance> tell zid you want to continue his psx port
20:56:00 <jafarlihi> Would x86 assembler be too hard?
20:57:00 <zid> It's non-trivial if you've never thought about parsing before
20:58:00 <zid> parsing takes a little bit of thinking to not suck at
20:58:00 <zid> You could work through K&R2 that does lots of writing crummy parsers :p
20:58:00 <gamozo> I did an x86 assembler, it's probably more of a pain than OS-dev, not sure if it's really worth it as a learning experience :P
20:59:00 <jafarlihi> zid: What is K&R2?
20:59:00 <gamozo> Honestly I'd say a disassembler is a better way to learn, cause you're working more on the architecture/x86 side, than the parsing side
20:59:00 <zid> The C book, kernighan and ritchie second edition
20:59:00 <jafarlihi> Read it long time ago
20:59:00 <GeDaMo> https://en.wikipedia.org/wiki/The_C_Programming_Language
20:59:00 <bslsk05> ​en.wikipedia.org: The C Programming Language - Wikipedia
20:59:00 <zid> did you do all the excercises?
20:59:00 <jafarlihi> No
20:59:00 <jafarlihi> Should I?
20:59:00 <zid> do you want to learn to program, or just read about programming?
21:00:00 <zid> It's a skill, you have to practice.
21:00:00 <zid> You don't get good at drawing animals by watching people on youtube do it, you just learn *how* they do it.
21:03:00 <heat> yes stubbornness is great
21:03:00 <heat> that's how you know you like something
21:03:00 <jafarlihi> K&R exercises are too easy
21:04:00 <zid> Do them then
21:04:00 <jafarlihi> I'll try x86 assembler
21:04:00 <zid> shouldn't take you long
21:04:00 <heat> it's like you ignore every advice
21:04:00 <jafarlihi> I wrote this lexer 3 years ago: https://github.com/jafarlihi/am01-assembler/blob/master/sources/lex.c
21:04:00 <bslsk05> ​github.com: am01-assembler/lex.c at master · jafarlihi/am01-assembler · GitHub
21:04:00 <zid> He just likes to hear himself talk
21:06:00 <sonny> lol
21:07:00 <sonny> a memory allocator is a good exercise
21:07:00 <heat> i tell you to focus on something and you go off and look at something else
21:07:00 <heat> focus on something
21:08:00 <heat> you're not galaxy braining here, there's no way to get a good idea of how a whole operating system works if you haven't written any of it yet
21:08:00 <heat> theory is useless
21:08:00 <geist> heat: yeah there's a whole architype for this one
21:08:00 <geist> have seen this sort of person come and go on the channel
21:08:00 <sonny> heat: worked for me :P
21:09:00 <gamozo> I mean, if a lot of os developers are stubborn, does that not mean stubbornness is the key to os dev?
21:09:00 <geist> knowing some theory is useful, but it is some tiny slice of the pie. it doesn't *do* anything on its own
21:09:00 <geist> you have to do
21:09:00 <heat> gamozo, yes
21:09:00 <geist> because it's a unrewarding hard slog at first, so you have to be stubborn and stick with it
21:09:00 <geist> at least thats one aspect of it
21:09:00 <heat> i got into it clueless and I stubborned my way through it
21:10:00 <geist> if you keep flipping around tyring to find a simple thing to do you wont get started
21:10:00 <geist> or if you constantly prepare to do the thing but dont just sit down and plow through it you also wont get anywhere
21:11:00 <geist> however a stubborn osdev failure mode i've also seen folks here do over the years is essentially refuse to acknoledge that there is anything to help them, or rely on anything preexisting
21:11:00 <geist> specifically "i'll start from writing a bootloader and fill in everything from there"
21:11:00 <geist> or even worse "i'll start by writing my own programming language" or building own hardware, etc
21:11:00 <sonny> xD
21:12:00 <sonny> I making my own programming language
21:12:00 <geist> or falls into "i'll make the best bootloader ever and *finish* it before going to the next step"
21:12:00 <zid> Yea bootloader pergatory is pretty common
21:12:00 <geist> that one i've seen many times. 10 years later they're still trying to build the best bootloader ever
21:12:00 <zid> It's not even a single factor that causes it either
21:12:00 <zid> Usually it's that they just have no idea what a kernel is or does, but a bootloader was the first 'step'
21:12:00 <geist> right, yeah i'm not saying it's a failure mode to be stubborn, but it's a warning sign
21:13:00 <zid> so they just make bootloaders all day instead, thinking they're progressing
21:13:00 <geist> right the stubbornness is not listening to advice that it's not worth your time, etc. they usually retort with "yeah but i want to understand *everything*"
21:13:00 <heat> well, it's a valid point
21:13:00 <heat> *if* you're competent enough to write a bootloader
21:14:00 <kingoffrance> there used to be a dolphin logo with an anchor. it meant you discuss/debate ...and then execute :)
21:14:00 <zid> I remember when I couldn't have written a bootloader, because I didn't understand mmio or anything so once interrupts were gone I was lost
21:14:00 <zid> err a kernel
21:14:00 <zid> I could imagine sitting there writing bootloaders all day
21:14:00 <geist> and it's not that there's something to be learned there, but it's a purgatory that is easy to fall into, because you end up immediately lost in the trees instead of seeing the forest
21:14:00 <zid> Thankfully instead I just put 'osdev' as a concept away in my mind for when I was more experienced
21:14:00 <zid> did some gameboy and stuff
21:15:00 <geist> yah i remember when you were gameboying here
21:15:00 <zid> Yea I came back to gaemboy, was really fun
21:15:00 <jafarlihi> That's what I wanna do too but I haven't found my gameboy
21:15:00 <zid> I ended up being the reference implementation for a few things on the gbdev discord
21:15:00 <geist> woot
21:15:00 <zid> because they were doing a project to convert the die shots at the time
21:16:00 <zid> so a bunch of new bugs and behaviors were being found
21:16:00 <geist> i remember being fairly excited about it because it was someone here actally doing something with ARM
21:16:00 <geist> back before it became kinda defacto the thing other than x86
21:16:00 <sbalmos> go emulate one of the old TI calculators with the amber digits that you had to hold the calculator at an angle to read
21:17:00 <jafarlihi> We don't use TI calcs in third world
21:17:00 <geist> yahs omeone here was doing a bunch of z80 work for their calc
21:17:00 <geist> had some sort of DOS like thing on it
21:17:00 <geist> (also a fun way to anger a gameboy original person: call it z80)
21:17:00 <zid> It's A SHARP SOMETHING SOMETHING
21:17:00 <mrvn> zBoy?
21:17:00 <sbalmos> heh
21:17:00 <zid> I don't even remember what it is
21:18:00 <geist> zid: wait you were doing z80 gameboy or arm gameboy? i was thinking it was arm GBA stuff
21:18:00 <mrvn> geist: avr gameboy
21:18:00 <zid> gameboy advance
21:18:00 <zid> is the arm7tdmi one
21:18:00 <zid> NDS is arm7tdmi + ARM9
21:18:00 <zid> I was doing gameboy
21:18:00 <zid> I've fiddled with gba though
21:18:00 <geist> yah, okay it was you then. there was someone here that was doing a z80 gameboy stuff and would get upset if you called it z80
21:18:00 <zid> I have a linker script and stuff
21:18:00 <zid> I don't get upset
21:19:00 <zid> I don't even remember what the damn thing is
21:19:00 <geist> anyway, fun times. i kinda wanted to hack on GBA, but too many other things to hack on
21:19:00 <zid> https://izik1.github.io/gbops/
21:19:00 <bslsk05> ​izik1.github.io: gbops - The Game Boy opcode table
21:20:00 <zid> It's one of those, whatever one of those is
21:20:00 <geist> yah it's some modified thing. not quite a z80 but it some superset of 8080 but etc etc
21:20:00 <zid> yea it's sort of inbetween
21:20:00 <geist> kinda a venn diagram where it overlaps with z80 somewhat which itself is an 8080 extension
21:20:00 <zid> it's the z80 encoding, but with 8080's actual features
21:20:00 <zid> no shadow regs etc
21:21:00 <zid> It's either a stripped z80, or a built up 8080 in the style of z80
21:21:00 <geist> yah honestly i'm not a huge fan of z80 or 8080. it feels a lot like all the bad parts about x86 except older and less functional
21:21:00 <geist> but that's cause i didn't grow up with one and thus learn to love it, etc
21:22:00 <zid> It's super clean for a little handheld though
21:22:00 <geist> always been a pit partial to motorola style stuff, including 6502
21:22:00 <zid> It's packed with nice bit operations via CB, and has auto-incrementing loads and stores etc
21:22:00 <geist> yah
21:22:00 <jafarlihi> NASM is 85 kLOC
21:22:00 <zid> and given that 99% of what you do is twiddle bits and memcpy tiles around
21:22:00 <zid> It's super tight and nice
21:22:00 <geist> z80 is definitely much nicer to deal with than 8080 at least
21:22:00 <zid> 6502 is a fucking nightmare though, so crippled you have to software implement really basic things
21:23:00 <zid> which was the alternative at the time
21:23:00 <geist> yah that's why i say i thik it's just whatever you grew up with
21:23:00 <geist> but in that whole line 6809 is super nice and pretty
21:23:00 <zid> what's the weird 16bit extension one the snes used
21:23:00 <zid> where they use the 6502 mode like thumb
21:23:00 <geist> 65c816
21:23:00 <zid> 650812823C16
21:23:00 <zid> that's the one
21:24:00 <geist> it's a mode switch basically, extnds the addressing out to 24 bits by adding some 8 bit extension reigsters and a few addressing modes that let you do long branches and whatnot
21:24:00 <geist> and then extending the registers out to 16 bits
21:24:00 <zid> yea I know about it
21:24:00 <zid> That's how I brought it up
21:24:00 <zid> I just couldn't remember the name of the cpu
21:24:00 <geist> yah apple IIgs also had one too
21:25:00 <zid> It's sorta what turned me off REing any SNES stuff
21:25:00 <geist> not sure there was any other desktop computer like things with it, since even the IIgs was wayy too late
21:25:00 <zid> because you can't diassemble it properly (it's not like you can do advanced static analysis with IDA)
21:25:00 <zid> and I hate 6502
21:26:00 <geist> I do like that it has a proper address extension, though you end up with some bank switching nonsense as part of your runtime to deal with it
21:26:00 <zid> gameboy uses bank switching but people just treat it like x86 segmentation
21:27:00 <zid> 05:FFFF is just bank 5
21:27:00 <geist> ah some hardware regs to set it?
21:27:00 <geist> some latch to extend, etc
21:27:00 <zid> not even that, the gameboy itself is unaware, but the cartridges respond to writes to the ROM by bank switching to that value's bank
21:28:00 <geist> so question then is how does the bank switching logic deal with interrupts?
21:28:00 <geist> or does it not bank the rom, and the first level vector is there, etc?
21:28:00 <zid> Easy, not all of ROM gets remapped
21:28:00 <zid> 8k is fixed, 8k swaps
21:28:00 <geist> ah and the fixed part is probably where the interrupt handlers and/or stack is located
21:28:00 <zid> so you put all your 'game' in the 8k and swap the data and level bytecode around
21:28:00 <zid> yup, z80's the same I think, there's a vector at 0, C, 14, etc or something
21:28:00 <geist> right yeah okay. makes sense. was pondering another spin on my 6809 board and adding some sort of banking logic for that
21:28:00 <zid> enough space for a cli jmp or whatever
21:29:00 <geist> yah
21:29:00 <clever> ive heard that some games used the rom mapper for animations
21:29:00 <zid> RST 0, RST 8 etc opcodes exist to use them for SWIs if you want but nobody really bothers
21:29:00 <clever> by swapping the rom banks, you can change the graphics for every sprite at once
21:29:00 <zid> You can on NES but it's trickier on gameboy
21:29:00 <geist> yep, CP/M had this whole complex scheme for that
21:29:00 <zid> NES has bankable video memory
21:30:00 <zid> gameboy just has bankable 'rom window'
21:30:00 <geist> re: rom mapper animations, there are some great vids on the topic. if you ever saw a nes game with some sort of parallax scrolling in the background (and not just at the top or bottom) it was almost certainly doing some rom mapper swizzling of background tiles
21:30:00 <zid> NES is super weird you do like, a serial link to the 'ppu' via an address register and a data register. That probably has a name.
21:31:00 <zid> gameboy just has the video memory in the address space instead. For NES you have to byte-stuff it into a single mmio reg.
21:31:00 <geist> there's a great youtube line of vids ive been casually watching lately that go into all the details of this for all sorts of consoles, literally just watched one the other day on precisely that topic
21:31:00 <clever> zid: nes also has a dedicated address space for the video rom on the cart, in its own rom chip
21:31:00 <zid> yes, hence being able to bank switch the video memory
21:31:00 <zid> because the PPU reads it over the same bus, so the cart can interpose
21:32:00 <zid> and do whatever response it wants
21:32:00 <geist> RetroGameMechanicsExplained is the channel. good stuff
21:32:00 <clever> very
21:32:00 <clever> was about to link one of his things
21:33:00 <zid> yea I've seen a couple of those vids I think, windwaker's water may have been him
21:33:00 <zid> 3D effects baffle me
21:33:00 <zid> graphics program is a huge dark art
21:33:00 <zid> mming*
21:33:00 <clever> https://www.youtube.com/watch?v=aF1Yw_wu2cM
21:33:00 <bslsk05> ​'Pokémon Sprite Decompression Explained' by Retro Game Mechanics Explained (00:34:04)
21:33:00 <clever> step by step, of how pokemon compresses its sprites
21:33:00 <zid> oh yea he's the pokermans guy
21:33:00 <zid> and why missingno looks so fucky
21:33:00 <clever> yep
21:35:00 <zid> geist: To expand on that a little, even regular hardware scrolling has to be controlled by the cart to some degree.
21:35:00 <clever> https://www.youtube.com/watch?v=-U76YvWdnZM
21:35:00 <bslsk05> ​'Memory Mapping - Super Nintendo Entertainment System Features Pt. 09' by Retro Game Mechanics Explained (00:21:56)
21:35:00 <zid> because it needs enough CHR ROM to be able to to it without corrupting the edges
21:35:00 <geist> yah
21:35:00 <zid> SMB is broken in that regard
21:36:00 <zid> The NES has a special 'the rightmost column of tiles is garbage' mode too
21:36:00 <zid> to allow left to right scrollers to function without special mappers
21:36:00 <clever> and overscan usually hid that
21:37:00 <clever> oh, there was an insane tas run of mario, let me find it
21:38:00 <clever> https://www.youtube.com/watch?v=jnZ2NNYySuE&t=24s
21:38:00 <bslsk05> ​'Super Mario World: Arbitrary Code Injection At AGDQ 2014, Performed Live' by invalidation (00:06:07)
21:38:00 <clever> yeah, thats it
21:38:00 <zid> yea I erm, worked on the early prototypes of tasbot
21:38:00 <zid> dwango calls me out at the first gdq it was presented
21:38:00 <clever> nice
21:40:00 <clever> ah wait no, this isnt the video i was thinking of
21:40:00 <geist> oh gosh that reminds me of DWANGO
21:40:00 <geist> the gaming dialup thing in the 90s
21:40:00 <clever> this is a different code injection
21:40:00 <geist> had it in Houston
21:40:00 <geist> https://en.wikipedia.org/wiki/DWANGO
21:40:00 <bslsk05> ​en.wikipedia.org: DWANGO - Wikipedia
21:40:00 <clever> https://youtu.be/eM8Z9e-WoFs?t=394 THIS one!
21:40:00 <bslsk05> ​'TASBot plays Super Mario Bros. 3 by Lord Tom - Awesome Games Done Quick 2016 - Part 155' by Games Done Quick (00:19:06)
21:40:00 <zid> the smw injections are the 'fun' ones
21:40:00 <clever> this tas is just insane
21:41:00 <clever> zid: and i can see the rightmost column being corrupted in this capture
21:41:00 <zid> yup :D
21:42:00 <moon-child> ahaha cute
21:42:00 <clever> but coming up, its going to get insane
21:42:00 <mjg_> is that the credit warp/
21:43:00 <clever> mjg_: more like rewriting the entire game
21:43:00 <zid> ACE got popular a while back
21:43:00 <zid> You can do it in zelda.nes which is super surprising, by carefully organizing a bunch of ghosts' x and y positions then executing the OAM as machine code
21:43:00 <mjg_> the crazy thing is that there are people who can pull it off
21:43:00 <mjg_> pixel and frame perfect inputs for minutes
21:43:00 <clever> mjg_: 9:53 is where it starts to go off the rails
21:43:00 <zid> the smb3 ACE stuff is trivial
21:44:00 <mjg_> ye, that's the wrong warp
21:44:00 <mjg_> i mean end credits warp
21:44:00 <clever> mjg_: no, look at 10:26
21:44:00 <clever> he is using sudo in mario....
21:44:00 <mjg_> lul it did not see that bit coming
21:44:00 <mjg_> nice
21:44:00 <clever> he didnt just wrong-warp, he injected custom code, and executed a custom app
21:45:00 <mjg_> normally people "just" finish the game with this
21:45:00 <zid> It basically memcpys from the multitap to memory for a bit
21:45:00 <clever> zid: yep
21:45:00 <zid> that's the 'payload' you build by manipulating OAM
21:45:00 <mjg_> ye, i give it B- for effort
21:45:00 <clever> 11:30 now watch this....
21:45:00 <zid> In OoT it's way easier because you can just execute your name
21:45:00 <zid> which gives you a free ROP style gadget
21:46:00 <zid> I know how the ACE actually works in OoT if anyone cares :P
21:47:00 <clever> 13:25, and now, the fixed auto-scroller
21:47:00 <clever> is going backwards....
21:50:00 <geist> neat
21:50:00 <mjg_> ooh man https://www.youtube.com/watch?v=T2SatL2mx3I
21:50:00 <bslsk05> ​'Fallout Anthology by tomatoangus in 2:16:21 - AGDQ2020' by Games Done Quick (02:33:37)
21:50:00 <clever> geist: and at ~16mins, they pass the controller to a human, with all of the hax still enabled, lol
21:51:00 <geist> so the TL;DR gist of theses runs is using only controller input and bugs in the code it ends up keying in machine code and/or modifying existing stuff and hacking the game?
21:51:00 <zid> yup
21:51:00 <zid> The OoT one I know how to explain *please let me*
21:51:00 <geist> sweet
21:51:00 <clever> geist: the credits warp stuff, is basically an exploit that causes it to treat sprite xy coords as opcodes
21:51:00 <geist> yah i saw some technical teardown of one of these, though i think in another game, but it was interesting
21:51:00 <mjg_> https://www.youtube.com/watch?v=Y5fnNIGvMAg
21:51:00 <bslsk05> ​'Controversial New Glitch Destroys Ocarina of Time World Record!' by Karl Jobst (00:16:58)
21:52:00 <zid> karl jobst is a nazi btw
21:52:00 <clever> and if you position sprites at the right locations, you can key in a jump to the credits routine
21:52:00 <geist> ah and hence why it's tossing a bunch of shells around to make sure the sprites are in the right spot at the time it enters it
21:52:00 <clever> yep
21:52:00 <geist> i think the glitch speed run of zelda BoTW is pretty great too, or at least one i watched
21:52:00 <geist> lots of slingshotting yourself across the map into the lap of something
21:52:00 <zid> are they still shield flying everywhere
21:52:00 <zid> sounds like it
21:52:00 <geist> oh i dunno, that was just the one i watched. probably be more elegant ones now
21:53:00 <zid> If they were flinging themselves across the map the answer is just yes
21:53:00 <clever> there was another one for super mario, let me find it
21:53:00 <zid> They used to use the timestop power and hit random logs a bunch of times and go like.. 100 meters at a time. Then they found out the equip system is mental and it slows down time when you hold the item menu up etc
21:53:00 <zid> and found a way to get really dumb acceleration by falling and shielding
21:53:00 <clever> https://www.youtube.com/watch?v=Dx1me5kpfuM
21:53:00 <bslsk05> ​'SNES Mouse Mod for SMW Jailbreak' by SethBling (00:05:44)
21:53:00 <clever> https://www.youtube.com/watch?v=Ixu8tn__91E
21:53:00 <bslsk05> ​'Jailbreaking Super Mario World to Install a Hex Editor & Mod Loader' by SethBling (00:10:25)
21:54:00 <clever> the 2nd one, is about corrupting your save-game in such a way, that it adds a hex editor to the game
21:54:00 <geist> ah yeah that's the one i thik i had seen
21:54:00 <clever> which then lets you key in a 512 byte "mod", that gets executed on every frame
21:54:00 <zid> pokemon's ACE is like 20 y ears old now and basically uses a hex editor
21:54:00 <clever> zid: skip to 1:39 in the 2nd vid, and youll see the hex editor
21:54:00 <zid> the missingno glitch gives you 255 of like every item
21:55:00 <zid> so you can just swap items in your inventory and toss certain amounts of them
21:55:00 <zid> and now you have a huge block of perfectly controlled RAM
21:55:00 <zid> because it's just item_id,count tuples
21:59:00 <clever> i also remember generating custom cheat codes in emulators
21:59:00 <clever> basically, you just input something like your current HP, and it runs a search for every memory location the number appears in
22:00:00 <clever> then you change your HP, and repeat that search, to see what locations still match up
22:00:00 <clever> and repeat until it only finds 1 location, that is the global var for HP, hard-code it to whatever you want
22:00:00 <zid> yea that's really bog standard
22:00:00 <zid> there were physical devices for it before emulators came about
22:00:00 <zid> gamegenie, gameshark, etc
22:01:00 <zid> gamegenie was interposed between the cart and cart slot
22:01:00 <clever> those cant run the search, only force a certain ROM addr to return a certain value
22:01:00 <zid> They absolutely can
22:01:00 <clever> while i think the emulator stuff, is instead forcing the RAM to return a fixed value
22:01:00 <zid> The cart is what the cpu is running, and the game genie interposes
22:01:00 <zid> it can just suddenly start returning the machine code for a ram scna
22:01:00 <clever> i think the gamegenie/gameshark can only use pre-designed codes, and you would need an emulator or disassembly to know what to even put into that code
22:02:00 <zid> nope
22:02:00 <clever> ive never seen such a gui in mine
22:02:00 <clever> but maybe it was just an undocumented key combo
22:02:00 <zid> exploder had loads of rad stuff
22:02:00 <zid> it even had a TIM viewer to view all the texture files on the disk and stuff
22:03:00 <zid> explorer*
22:10:00 <zid> https://godbolt.org/z/1Ychdjofo I am bored.
22:10:00 <bslsk05> ​godbolt.org: Compiler Explorer
22:14:00 <mrvn> How is that giving only one warning? <source>:3:5: warning: label 'http' defined but not used [-Wunused-label]
22:16:00 <mrvn> zid: only works with >= c11
22:17:00 <mrvn> (or whenever they added //)
22:17:00 <klange> // comments are in c99
22:18:00 <zid> well spotted
22:26:00 <mykernel> https://pastebin.com/fRnLvuhm
22:26:00 <bslsk05> ​pastebin.com: cpuid.s:.global cpuid_vendorcpuid_vendor: pushal mov $0x0, % - Pastebin.com
22:26:00 <mykernel> can someone help, i dont know how to get data from asm to c
22:26:00 <heat> that's not how calling conventions work
22:26:00 <zid> yea 100%
22:26:00 <zid> https://en.wikipedia.org/wiki/X86_calling_conventions
22:26:00 <bslsk05> ​en.wikipedia.org: x86 calling conventions - Wikipedia
22:27:00 <mrvn> it that 32bit or 64bit?
22:27:00 <heat> 1) why pushal?; 2) your first argument is not %edi. I used %rdi as an example because it's the *x86_64* sysv calling convention
22:27:00 <mykernel> 32bit
22:27:00 <mykernel> because if i dont do pushal kernel_main fucks up
22:27:00 <mrvn> mykernel: why don't you use inline asm?
22:27:00 <heat> mrvn, that's besides the point
22:28:00 <heat> mykernel: know your calling convention
22:28:00 <heat> hint: it's called the i386 system v abi
22:32:00 <mrvn> mykernel: #include <cpuid.h> /usr/lib/gcc/x86_64-linux-gnu/10/include/cpuid.h
22:35:00 <mykernel> ik that exists
22:35:00 <heat> that's not the point
22:35:00 <heat> learn your calling convention
22:35:00 <heat> then fix that
22:58:00 <mykernel> I FUCKING FIXED IT
22:58:00 <mykernel> thanks for pointing out how to do it instead of doing my work <3
22:58:00 <mykernel> i mean pointing to correct documentation
22:59:00 <heat> np
22:59:00 <heat> :)
22:59:00 <zid> happy to do so as long as you read it :D
22:59:00 <zid> we get a lot of "omg help me" "Okay read this" "omg help me"
23:00:00 <mykernel> it is really hard starting out
23:00:00 <heat> now, the proper way to do this would be to use some inline assembly because a full assembly function is way too hardcore for this
23:00:00 <zid> actually gcc has a cpuid intrinsic
23:01:00 <zid> https://github.com/gcc-mirror/gcc/blob/master/gcc/config/i386/cpuid.h
23:01:00 <bslsk05> ​github.com: gcc/cpuid.h at master · gcc-mirror/gcc · GitHub
23:01:00 <heat> yes
23:01:00 <heat> it's inline assembly done for you
23:01:00 <zid> inline assembly's pretty rough if you can barely do regular assembly though to be fair
23:02:00 <heat> fair
23:02:00 <zid> https://gcc.gnu.org/onlinedocs/gcc/Using-Assembly-Language-with-C.html#Using-Assembly-Language-with-C If you're interested, mykernel
23:02:00 <bslsk05> ​gcc.gnu.org: Using Assembly Language with C (Using the GNU Compiler Collection (GCC))
23:19:00 <mrvn> I recommend doing inline asm for a single opcode only and then combined them if needed.
23:21:00 <mrvn> And if you need a whole function then write it in C, copy what the compiler produces into a .S file and optimize.