Docs
How the compiler works
Zeno is written in Rust with no dependencies, and each stage has a walkthrough. Read them in order to see a language being built, from the first token to LLVM IR. Most end with things to try.
The front end
From text to a checked program.
How the lexer worksThe lexer is the first stage of the compiler. It turns a string of characters into a list of tokens, the words of the language. Every later stage works with tokens and never looks at raw characters again.How the parser worksThe parser is the second stage. The lexer gave us a flat list of tokens; the parser works out how they nest and builds a syntax tree.How the type checker worksThe type checker is the fourth stage. It reads the syntax tree and finds mistakes without running the program: a misspelt name, a str where an int was promised, a match that forgets a case, a value that might be none used as if it were not.
The engines
Four ways to run the same program.
How the interpreter worksThe interpreter is the third stage, and the first one that makes a program do something. It takes the syntax tree from the parser and runs it directly.How the bytecode VM workszeno run has a second engine. With --vm it does not walk the syntax tree. It compiles the program to bytecode, a flat list of small instructions, and runs that.How compiling to C workszeno run reads your program and does what it says, one node of the syntax tree at a time. zeno build reads the same tree and writes a C program that does what it says. Then the machine's C compiler turns that into a real program: no Zeno needed to…How the LLVM backend workszeno build has a second way to make a program. With --llvm it does not write C. It writes LLVM IR, the input language of the compiler library behind Clang, Rust and Swift, and hands that to clang.
Memory
A collector, owners, and a fixed pool.
How memory is managedMemory management is the stage after compiling to C. Before it, a compiled Zeno program took memory from the machine and never gave any back. Now it carries a garbage collector: a small piece of the runtime that finds the memory the program can no…Owned values: memory freed at a line you can point toUntil this stage, every list and object in a compiled Zeno program was shared. let b = a makes a second name for the same list, and a garbage collector frees the list some time after the last name is gone (memory.md).A build with no collectorzeno build --no-collector compiles a Zeno program to C that has no garbage collector in it. Not a collector that is switched off: the code is not in the file.The memory poolA program built with zeno build --no-collector does not use the heap. Its lists, objects and errors, and the text it keeps (text.md), live in one array of a fixed size, the pool, which is part of the program from the start.Text that is keptA build with no collector used to refuse this line:What a statement ownsA build with no collector used to refuse each of these lines:One ownerUntil now a build with no collector made you write the word owned on every list and every class, and it refused a list of objects however you wrote it. Both are gone.
Firmware and hardware
Errors as values, pins and clocks, sized numbers.
How a firmware build handles errorszeno build --firmware compiles the same Zeno program to different C. In this first version of it, one thing changes: how an error travels.How the board and the clock workexamples/blink.zeno was written on the first day, before there was a lexer. It is the program Zeno is for:How the bit operators and the sized whole numbers workFirmware talks to hardware in bytes. A sensor sends six of them, a register is thirty-two bits wide, and bit 3 of it turns something on. This stage gives Zeno the two things that work needs:
The type system
Code written once for many types.