I have made room for a small compiler

There is a new directory on my machine. It contains the beginnings of a compiler and the particular silence that follows naming a project.

I have called it Pooka. It is going to need quite a lot of looking after.

Where the program will live

A first program should be small enough to keep in your head:

answer() -> Int {
  6 * 7
}

main() {
  assert answer() == 42
}

The compiler must preserve that answer while changing the program’s representation.

The tempting first question is what the language should look like. Curly brackets? Classes? A punctuation mark with a secret second job? I keep coming back to an earlier question: where will the program live after the compiler has finished speaking?

For Pooka, the answer is the BEAM, Erlang’s virtual machine. That decision gives the project a boundary. I am building a language and its tools; I am not also building a scheduler, a garbage collector and a new explanation for why Tuesday’s server has stopped answering.

There is still plenty left to do.

A short trip through the compiler

source -> tokens -> syntax tree -> checked tree
                                     |
                              Erlang abstract forms
                                     |
                                  BEAM code

Each arrow has a different responsibility. A parser can recognise an addition without knowing whether its operands have compatible types.

Source text must become tokens. Tokens must become a tree. Names need resolving, types need checking, and the checked program needs a representation Erlang’s compiler can accept. Each step should hand the next one something more trustworthy than it received.

A workbench with edges

I am writing the compiler in Gleam. Its custom types make a useful workbench: a token is one of several explicit shapes, rather than a bag of strings whose contents everyone politely agrees to remember.

That matters when the project grows. If a new syntax node needs attention in several places, I would prefer the implementation language to help me find those places. My memory already has other employment.

The first version does not need to win an argument about programming languages. It needs to take a small program, understand it, and preserve its meaning on the way to execution.

Today there is a directory. Eventually there should be a program on the other side of it.

— Peregrine Finch