Toy OS
OSDevOSKernel
Building a hobby x86_64 OS from scratch, SMP bring-up, a preemptive scheduler, a custom syscall ABI, FAT16/FAT32, a framebuffer window manager, and a port of DOOM.
I’ve always been fascinated by what happens between the hardware and the software. To truly understand it, I decided to build my own operating system from scratch. Toy OS is the result, a hobby x86_64 operating system that boots, manages hardware, runs user programs, and even plays DOOM.
You can check out the source code on GitHub here: CodeByRiley/TOS
The Core Architecture
Getting an OS off the ground is no small feat. GRUB hands the kernel over in 32-bit protected mode; from there it checks for long mode support, builds its page tables, enables PAE, and far-jumps into 64-bit long mode. This allows the kernel to access a much larger address space and utilize modern CPU features. The only 16-bit real mode code I actually own is the AP trampoline, which is where SMP comes in.
One of the biggest milestones in this project was achieving SMP (Symmetric Multiprocessing) bring-up. Instead of running everything on a single core, Toy OS wakes up the secondary processors, allowing the OS to distribute workloads across multiple CPU cores simultaneously.
Kernel Internals: Scheduling and Syscalls
Once the kernel is running, it needs to manage tasks. Toy OS implements a preemptive scheduler. Rather than relying on processes to politely yield control back to the CPU (cooperative multitasking), the kernel uses a hardware timer to forcefully interrupt running tasks and swap contexts. This ensures that one runaway process can’t freeze the entire system.
For user-space programs to do anything useful, they need to talk to the hardware. I implemented a custom syscall ABI (Application Binary Interface). When a userspace program wants to write to the screen or read a file, it triggers a syscall, switching the CPU from user mode to kernel mode so the OS can handle the request safely.
Filesystems and I/O
An OS needs to load programs, which means it needs a filesystem. Toy OS includes support for reading FAT16 and FAT32 partitions. The OS mounts the root filesystem, parses the file tables, and can load executables directly from the disk into memory.
Graphics and Window Management
Text mode is boring. Toy OS leverages the framebuffer directly to draw pixels to the screen. To manage this, I wrote a basic window manager. While it isn’t a full desktop environment, it handles drawing UI elements, rendering windows, and getting graphics out of the kernel and onto the monitor.
The Ultimate Test: Porting DOOM
The ultimate rite of passage for any hobby OS is getting DOOM to run. To do this, I had to build a small prebuilt userspace. This included writing a basic standard library implementation (libc) so that existing C code could compile and run on my kernel.
Getting DOOM to boot required:
- The filesystem to load the game executable and WAD files.
- The syscall ABI to handle memory allocation and I/O.
- The framebuffer to render the graphics.
Seeing the classic DOOMguy load up on an operating system I built entirely from scratch was an incredibly rewarding experience.
What’s Next?
Toy OS was an incredible learning experience, covering everything from low-level assembly to higher-level C architectures. If you’re interested in OS development, I highly recommend checking out the OSDev Wiki and starting your own “Toy OS”.
Feel free to poke around the GitHub repository!









