$ git commit -m "$(cat <<'EOF'
Phase 6: tiny compiler + bytecode VM
Adds a second language path alongside the Phase 5 interpreter: a
statically-scoped, int-only C-like language compiled to a small stack-
machine bytecode (docs/bytecode-isa.md) and executed by an in-kernel
VM. Reuses the shared lexer (kernel/lang/lexer.c) and the scripting
language's generic AST node type (kernel/lang/script/ast.h) - the
compiler's own parser (kernel/lang/compiler/parser.c) builds the same
tree shape minus string literals, since a full second AST would just
duplicate kernel/lang/script/ast.h for no benefit.
codegen.c does a two-pass compile: register all top-level function
names first (so forward/recursive calls resolve), then compile each
function body and the top-level statements into a flat bytecode
buffer plus a function table (entry address/param count/local count).
Every compiled expression leaves exactly one value on the VM's value
stack, including assignment (`<value> DUP STORE`), so callers can
treat all expressions uniformly.
vm.c executes the resulting .tbc file: separate value stack, a
locals stack sliced into per-call frames (supporting real recursion -
verified via a recursive fib(10) in the test fixture), a call stack
of {return_pc, locals_base}, and a fixed globals array. Functions
cannot see globals or an enclosing scope, matching the Phase 5
interpreter's "no closures" rule for consistency between the two
languages.
Shell gains `build <src> <out>` (parse + codegen, writes a .tbc file)
and `exec <file>` (runs it through the VM). tests/hello.tc exercises
arithmetic, a plain function call, recursion, and a while loop end-to-
end via tools/run_tests.sh.
EOF
)"[main 3c37ed4] Phase 6: tiny compiler + bytecode VM 13 files changed, 1248 insertions(+), 5 deletions(-) create mode 100644 docs/bytecode-isa.md create mode 100644 kernel/lang/bytecode.h create mode 100644 kernel/lang/compiler/codegen.c create mode 100644 kernel/lang/compiler/codegen.h create mode 100644 kernel/lang/compiler/parser.c create mode 100644 kernel/lang/compiler/parser.h create mode 100644 kernel/lang/vm.c create mode 100644 kernel/lang/vm.h create mode 100644 tests/expected/hello.tc.txt create mode 100644 tests/hello.tc
{
"taskId": "7",
"status": "completed"
}Updated task #7 status
{
"taskId": "8",
"status": "in_progress"
}Updated task #8 status
# tsh scripting language
The dynamic, tree-walking counterpart to the [tiny compiler](bytecode-isa.md).
Run with the shell's `run <file>` command; see `tests/hello.tsh` for a
worked example.
## Syntax
C-like statements, no semicolons after blocks:
```
x = 2 + 3 * 4;
print("x =", x);
func add(a, b) {
return a + b;
}
print("add(3,4) =", add(3, 4));
i = 0;
while (i < 3) {
print("loop", i);
i = i + 1;
}
```
- Types: 32-bit integers and strings (`"..."`). No arrays, no booleans -
`0`/empty-string are falsy, everything else truthy.
- Variables: no declarations: first assignment creates them. All
variables in a script share one global scope, *except* inside a
function body, which gets its own isolated local scope - functions
cannot see the caller's variables or each other's (no closures).
- Control flow: `if (...) {...} else {...}` (including `else if` chains),
`while (...) {...}`.
- Functions: `func name(params) {...}`, `return expr;` (or bare `return;`
/ falling off the end, both give `0`).
- Operators: `+ - * / %`, `== != < <= > >=`, `&& ||` (short-circuit),
unary `- !`. `+` on any string operand concatenates (numbers are
stringified).
- Comments: `# to end of line`.
## Builtins
| call | effect |
|---|---|
| `print(a, b, ...)` | writes args space-separated + newline to console and serial |
| `fread(name)` | returns a file's contents as a string, or `""` if missing |
| `fwrite(name, contents)` | creates/overwrites a tinyfs file, returns `1`/`0` |
| `flist()` | prints every file name (one per line), returns the count |
## Implementation
`kernel/lang/lexer.c` (shared with the compiler) tokenizes;
`kernel/lang/script/parser.c` builds an AST (`ast.h`) via recursive
descent; `kernel/lang/script/eval.c` tree-walks it directly - there is no
intermediate bytecode, unlike the compiled language. AST nodes and string
literals are allocated from the kernel heap and never freed; fine for a
single `run` of a short script, since the heap resets on reboot.
File created successfully at: /home/martin/workspace/prototypes/boot-to-claude/docs/scripting-language.md (file state is current in your context — no need to Read it back)
$ find . -path ./build -prune -o -path ./.git -prune -o -type f -print | sort
./.gitignore ./Makefile ./boot/boot.S ./boot/stage2.S ./docs/bytecode-isa.md ./docs/scripting-language.md ./docs/tinyfs.md ./kernel/ata.c ./kernel/ata.h ./kernel/console.c ./kernel/console.h ./kernel/entry.S ./kernel/fs/tinyfs.c ./kernel/fs/tinyfs.h ./kernel/fs/tinyfs_disk.c ./kernel/fs/tinyfs_disk.h ./kernel/gdt.c ./kernel/gdt.h ./kernel/idt.c ./kernel/idt.h ./kernel/idt_stubs.S ./kernel/keyboard.c ./kernel/keyboard.h ./kernel/kmain.c ./kernel/lang/bytecode.h ./kernel/lang/compiler/codegen.c ./kernel/lang/compiler/codegen.h ./kernel/lang/compiler/parser.c ./kernel/lang/compiler/parser.h ./kernel/lang/lexer.c ./kernel/lang/lexer.h ./kernel/lang/script/ast.c ./kernel/lang/script/ast.h ./kernel/lang/script/eval.c ./kernel/lang/script/eval.h ./kernel/lang/script/parser.c ./kernel/lang/script/parser.h ./kernel/lang/vm.c ./kernel/lang/vm.h ./kernel/link.ld ./kernel/memory.c ./kernel/memory.h ./kernel/selftest.c ./kernel/selftest.h ./kernel/serial.c ./kernel/serial.h ./kernel/shell/shell.c ./kernel/shell/shell.h ./kernel/string.c ./kernel/string.h ./tests/expected/hello.tc.txt ./tests/expected/hello.tsh.txt ./tests/hello.tc ./tests/hello.tsh ./tools/mkfs_tinyfs.c ./tools/qemu_type.py ./tools/run_tests.sh
# boot-to-claude
A from-scratch, single-tasking x86 OS for QEMU/KVM: a custom two-stage
boot loader, a monolithic kernel with its own tiny filesystem (tinyfs), an
interactive shell, a dynamic scripting language, and a small C-like
compiler with its own bytecode VM.
Every line of OS/filesystem/language source in this repo is hand-written
for this project - `nasm`, `gcc`, `ld`, `objcopy`, and `qemu` are used
purely as build/run infrastructure, the same way any project uses a
compiler.
## Quickstart
```
make # builds build/disk.img
make run # boots it in QEMU (headless, serial on stdio)
```
`make run` boots into the shell:
```
tiny-os shell. type 'help' for commands.
> ls
hello.txt 15
hello.tsh 281
hello.tc 206
> cat hello.txt
Hello, tinyfs!
> run hello.tsh
hello from tinyfs script
x = 14
...
> build hello.tc hello.tbc
build: ok
> exec hello.tbc
14
7
55
10
```
That one session exercises every layer: tinyfs (`ls`/`cat`), the
scripting language (`run`, including its own `fread`/`fwrite` calls into
tinyfs), and the compiler + bytecode VM (`build` + `exec`, including a
recursive function call). `tools/run_tests.sh` automates exactly this
against a headless QEMU instance - see [Testing](#testing).
Requires `nasm`, `gcc` (with `-m32` support), `ld`, `objcopy`, `dd`, and
`qemu-system-i386` on the host. No cross-compiler needed.
## Architecture
```
BIOS -> boot/boot.S (MBR, 512B) real mode
-> boot/stage2.S real mode -> protected mode
-> kernel/entry.S -> kmain.c 32-bit protected mode, flat, no paging
gdt.c / idt.c kernel-owned GDT; IDT covers CPU
exceptions only - no PIC/IRQs, ever
console.c / serial.c VGA text + COM1, dual-output throughout
memory.c free-list kmalloc/kfree over a 1MB heap
ata.c -> fs/tinyfs.c polled ATA PIO -> tinyfs filesystem
keyboard.c -> shell/shell.c polled PS/2 -> command REPL
lang/lexer.c (shared)
-> lang/script/{parser,ast,eval}.c scripting language (tree-walking)
-> lang/compiler/{parser,codegen}.c compiler -> lang/vm.c (bytecode VM)
```
Deliberately single-tasking: no scheduler, no preemption, no interrupts
enabled after boot, no process/syscall boundary. The shell and both
language runtimes execute directly in kernel context. This keeps the OS
layer small so the interesting work is in the filesystem and language
implementations - see [`docs/`](docs/) for their designs:
- [`docs/tinyfs.md`](docs/tinyfs.md) - on-disk filesystem format
- [`docs/scripting-language.md`](docs/scripting-language.md) - the `tsh` language
- [`docs/bytecode-isa.md`](docs/bytecode-isa.md) - the compiled language + VM ISA
## Repo layout
```
boot/ stage1 (MBR) + stage2 boot loaders
kernel/ the kernel itself (see architecture diagram above)
kernel/fs/ tinyfs (shared: freestanding in the kernel, native in mkfs_tinyfs)
kernel/lang/ shared lexer + the two language implementations
kernel/shell/ interactive command REPL
tools/ host-side build/test infrastructure (not OS code):
mkfs_tinyfs.c formats + seeds build/disk.img at build time
run_tests.sh headless QEMU test harness
qemu_type.py scripts keystrokes into QEMU via its HMP monitor
tests/ fixtures (*.tsh, *.tc) + tests/expected/ serial-output fixtures
docs/ design docs, listed above
```
## Testing
There's no hardware and no display in CI, so the serial port is the test
oracle throughout: `kernel/selftest.c` runs a boot-time self-test (string
ops, the allocator including coalescing, and a tinyfs create/write/read/
delete roundtrip) and prints `SELFTEST PASS`/`FAIL`.
```
tools/run_tests.sh
```
boots `build/disk.img` headless (`-display none -serial file:...`), waits
for `SELFTEST PASS`, then drives the shell by scripting keystrokes into
QEMU's HMP monitor over a unix socket (`tools/qemu_type.py sendkey ...`)
- the only way to feed the kernel's polled PS/2 driver without real
hardware - and asserts on the resulting serial output for every shell
command, `run hello.tsh`, and `build`+`exec` on `hello.tc`.
## Boot budget
The kernel image is capped at a fixed number of disk sectors (`boot/
stage2.S`'s `KERNEL_SECTORS`, currently 512 = 256KB), read via BIOS
INT13h in bounded chunks (SeaBIOS silently misbehaves on very large
single extended-read requests). `kernel/fs/tinyfs.h`'s
`TINYFS_FS_START_LBA` reserves exactly that much room before the
filesystem region begins. The Makefile fails the build loudly if the
kernel image ever grows past its budget - if you hit that, bump
`KERNEL_SECTORS`, the Makefile's `KERNEL_MAX_SECTORS`, and
`TINYFS_FS_START_LBA` together.
File created successfully at: /home/martin/workspace/prototypes/boot-to-claude/README.md (file state is current in your context — no need to Read it back)
$ cd /home/martin/workspace/prototypes/boot-to-claude rm -f build/mon.sock build/serial.log timeout 8 qemu-system-i386 -drive file=build/disk.img,format=raw -display none -serial file:build/serial.log -monitor unix:build/mon.sock,server,nowait -no-reboot & QPID=$! for i in $(seq 1 25); do [ -S build/mon.sock ] && break; sleep 0.2; done sleep 0.3 python3 tools/qemu_type.py build/mon.sock $'ls\n' sleep 0.3 python3 tools/qemu_type.py build/mon.sock $'cat hello.txt\n' sleep 0.3 wait $QPID 2>/dev/null cat build/serial.log
qemu-system-i386: terminating on signal 15 from pid 2113803 (timeout) tiny-os: kernel entered selftest: running SELFTEST PASS tiny-os shell. type 'help' for commands. > ls hello.txt 15 hello.tsh 281 hello.tc 272 script_out.txt 17 hello.tbc 184 > cat hello.txt Hello, tinyfs! >
$ cd /home/martin/workspace/prototypes/boot-to-claude make clean >/dev/null && make >/dev/null 2>&1 rm -f build/mon.sock build/serial.log timeout 6 qemu-system-i386 -drive file=build/disk.img,format=raw -display none -serial file:build/serial.log -monitor unix:build/mon.sock,server,nowait -no-reboot & QPID=$! for i in $(seq 1 25); do [ -S build/mon.sock ] && break; sleep 0.2; done sleep 0.3 python3 tools/qemu_type.py build/mon.sock $'ls\n' sleep 0.3 python3 tools/qemu_type.py build/mon.sock $'cat hello.txt\n' sleep 0.3 wait $QPID 2>/dev/null cat build/serial.log
qemu-system-i386: terminating on signal 15 from pid 2114108 (timeout) tiny-os: kernel entered selftest: running SELFTEST PASS tiny-os shell. type 'help' for commands. > ls hello.txt 15 hello.tsh 281 hello.tc 272 > cat hello.txt Hello, tinyfs! >
- > ls - hello.txt 15 - hello.tsh 281 - hello.tc 206 - > cat hello.txt + > ls + hello.txt 15 + hello.tsh 281 + hello.tc 272 + > cat hello.txt
The file /home/martin/workspace/prototypes/boot-to-claude/README.md has been updated successfully. (file state is current in your context — no need to Read it back)
$ cd /home/martin/workspace/prototypes/boot-to-claude pgrep -af qemu-system-i386 || echo "no stray qemu" make clean >/dev/null && ./tools/run_tests.sh 2>&1 | tail -5
2114264 /bin/bash -c source /home/martin/.claude/shell-snapshots/snapshot-bash-1784307599835-hlkzgk.sh 2>/dev/null || true && shopt -u extglob 2>/dev/null || true && eval 'cd /home/martin/workspace/prototypes/boot-to-claude pgrep -af qemu-system-i386 || echo "no stray qemu" make clean >/dev/null && ./tools/run_tests.sh 2>&1 | tail -5' < /dev/null && pwd -P >| /tmp/claude-383a-cwd ld: warning: build/kernel.elf has a LOAD segment with RWX permissions run_tests: PASS qemu-system-i386: terminating on signal 15 from pid 2114269 (bash)
$ cd /home/martin/workspace/prototypes/boot-to-claude git status --short echo "---" git status --short --ignored | grep '^!!' | head -20 echo "---clean check for stray untracked non-ignored files---" git clean -ndx
?? README.md ?? docs/scripting-language.md --- !! build/ ---clean check for stray untracked non-ignored files--- Would remove README.md Would remove build/ Would remove docs/scripting-language.md
$ cd /home/martin/workspace/prototypes/boot-to-claude git add README.md docs/scripting-language.md git status --short
A README.md A docs/scripting-language.md