Testing¶
Kod has built-in syntax for unit tests: test "description" { ... } blocks and an assert <expr> statement. kod test runs them.
A first test¶
// arithmetic_test.kod
test "addition works" {
assert 2 + 2 == 4
}
test "negative arithmetic" {
assert -1 + 1 == 0
assert -5 - 5 == -10
}
assert <expr>¶
If the expression evaluates to false, the current test is marked failed and the source text of the expression is captured in the failure message — no need to type f"got {x}, want {y}":
Multiple asserts per test continue running — they accumulate failure messages but don't short-circuit, so a single test surfaces every failing condition at once.
Outside a test block, assert panics with the same message (exit 1). Same syntax, intent matches the surrounding context — like Rust's always-on assert!.
fail(msg)¶
For failures that aren't a boolean check, call the builtin fail:
test "map lookup hits" {
mut m: Map[str, int64] = Map[str, int64]()
m.set("a", 1)
match m.get("a") {
Some(v) -> assert v == 1
none -> fail("missing key after set")
}
}
fail is a builtin — no import needed. It works from any function a test calls, not only inside the test block. Outside kod test there is no test to fail, so fail panics with its message, as assert does outside a test.
Discovery¶
kod test <file>builds the file and every transitively imported module that containstestblocks, then runs them all in one binary.kod test <dir>walks the directory for*_test.kodand builds them all into one binary, with no temporary module. Each file keeps its own type names (see Modules), so two test files may both declare a helpertype Pt, and two test files may share a file name (a/dup_test.kodandb/dup_test.kod).kod test .walks the entire project.- The walk skips a file with a path component below the directory that starts with
.(worktrees under.claude/, editor files, scratch files such as.kod_compile_test.kod) and abuilddirectory directly below it. (The test suite's own finder skips the dot-directories and a top-levelbuild/, but not dot-files.) Only files count: a directory namedx_test.kodis walked, not run. A match that is not a readable file (a link to nothing) stops the run withkod test: cannot read <path>, since a test skipped without a word reads as one that passed.kod test <file>takes the file named, whatever its path.
The aggregate exit code is 0 if every test passed, non-zero otherwise.
Each test in a process of its own¶
Each test runs in a child process of the test binary, made with fork just
before it, so a test that panics or crashes fails alone: its panic message (and
backtrace) or the signal that killed it is printed under it, it reports FAIL,
and the tests after it run as usual. A test starts from the state the program
had before the first test, not where the test before it left it: what one test
stores in a module-level variable, the next one doesn't see.
stdlib/foo_test.kod
ok one (1.20 ms)
panic: index 5 out of bounds for array of length 1
backtrace:
#0 test "boom" (stdlib/foo_test.kod:7)
FAIL boom (1.35 ms)
killed by SIGSEGV
FAIL bad pointer (985 μs)
ok three (1.30 ms)
A backtrace names a test's frame test "<its name>" and ends there. Starting
the process costs about a millisecond, which the time shown includes.
Running code in a process of its own¶
testing.run(body) runs the function body in a child process and gives back
what that process did, so a test can check what a program prints, reads or
exits with, and that something panics:
import "io"
import "testing"
test "indexing past the end panics" {
let r = testing.run(func() {
let xs = [1]
print_int(xs[5])
})
assert r.status == 1
assert r.stderr.contains("panic: index 5 out of bounds for array of length 1")
}
test "print ends the line" {
let r = testing.run(func() { print("hi") })
assert r.stdout == "hi\n"
}
test "read_stdin_line reads one line" {
let r = testing.run(func() { print(io.read_stdin_line()) }, stdin: "a\nb\n")
assert r.stdout == "a\n"
}
The result is a testing.Ran:
| Field | Meaning |
|---|---|
status |
the exit status: 0 when body returned, 1 when it panicked, 2 when it called fail; -1 when a signal ended it. body is a function, not a test block, so an assert in it panics when it fails |
signal |
the number of the signal that ended it (SIGSEGV is 11), else 0 |
stdout, stderr |
everything it wrote to them, through print or C's putchar alike |
stdin: is what the child reads from standard input (nothing by default). The
child starts from the state of the test at the call, and what it changes
(variables it captured, module variables) stays in the child. Its output goes
to files rather than pipes, so it can write any amount.
Output¶
kod test replaces itself with the test binary, so the output appears as the tests run, and the exit code is the binary's. The reporter flushes stdout around every ok/FAIL line and the file header, and before a failure's detail goes to stderr, so the lines stay in order when stdout and stderr are the same pipe or log. What a test prints with print is flushed with them, so it comes out before that test's result line. A panic (an index out of bounds, a failed must) flushes stdout before its message too.
When stdout is a terminal, ok is green, FAIL red, and the time each test took
dim; the summary line is green when all passed and red when any failed. The same
NO_COLOR and TERM=dumb turn this off as for errors.
Hermetic tests¶
A test needs nothing the repository does not have, and no network: a test of code that talks to a server starts a server of its own on the loopback interface. A test that needs the internet does not belong among the *_test.kod files that kod test <dir> and the test suite run. A module under test may declare C files and libraries (see Link inputs); kod test compiles and links them like kod build does, and a kod:require file that is missing stops the build, naming the line that asks for it.
Layout convention¶
Test files live next to the code they test, Go-style:
The naming convention *_test.kod makes them auto-discoverable by kod test <dir> without showing up as importable modules.
How it works (briefly)¶
- The parser turns
test "..." { ... }into aDecl.Testcarrying a synthesizedFuncDecl. - The codegen emits each test as a regular function (
<prefix>$__test_N) and, per module, a<prefix>$__run_testsdispatcher.<prefix>is described in Modules. - The runner is a Kod module,
stdlib/testing.kod, whichkod testlinks into every test program (andkod buildinto any program with atestblock, since the block's code is linked even though nothing runs it). Its counters are module-level variables. kod testbuilds a runtime_mainthat calls every test-bearing module's__run_testsin turn, then the runner'ssummaryonce at the end.fail/assertset a per-test "failed" flag the dispatcher reads after each test returns; that flag drives theok/FAILline and the aggregate count.
Testing the compiler itself¶
make test rebuilds the toolchain, then builds and runs the test runner, tools/run_tests.kod, which runs every *_test.kod in the repository that contains test blocks through kod test <file> (skipping dot-directories and the top-level build/); the files of tests/unit run together, as one kod test tests/unit. A file passes when kod test exits 0, its summary line says all passed, and it ran at least as many tests as the file has test blocks. Among them:
- Example programs (
tests/examples_test.kod): every othertests/**/*.kod. Trailing// expected output:,// expected errors:,// expected stderr:,// expected status:and// expected mode:comments say what should happen. Each one is compiled and run;mode: skipopts out, for a module other fixtures import. - Self-hosting (
tests/selfhost_test.kod): the fixed-point check described in Self-hosting.
The tests run programs through the native tools, build/apps/kod/kod and build/stage1/sh_kodc.
The runner runs as many files at once as the machine has cores (-j N for another number), the slow ones first. Files that build the same program, into the same build/apps/<name>, share a group, and two of a group never run at the same time; the lists of groups, slow files and serial files are at the top of tools/run_tests.kod. The serial files (those that rebuild or replace the kod driver, write into the project root, or run make on the real tree) run afterwards, one at a time. Each file has 120 seconds: one still running then is killed with everything it started, and fails. The runner prints a line per file as it ends, then the whole output of each file that failed and a summary, and exits 1 if any failed. Given files or directories, it runs only those:
$ make -s stage1 kod && build/apps/kod/kod build tools/run_tests.kod
$ build/apps/run_tests/run_tests -j 4 stdlib/ tests/lsp_test.kod
make gates is what to run before a commit: make test, then make test again with the runtime built with -DRC_POISON, which overwrites freed memory so a use after free shows up. It refuses to run when there is a link.local.cfg, since it writes its own for the poisoned pass, and removes it again however the pass ends.