The compiler remembers the route home

The compiler has been looking at a module it compiled already. I can see its point. The file has not changed. The file would like to be left alone.

Remember the evidence

Incremental compilation is the art of knowing when that request is safe to grant.

Remembering only the source text is not enough. A module can remain untouched while something it imports changes. The compiler version, relevant build settings and the state of its dependencies can also affect whether an old artifact is still suitable.

Check the inputs and the output

same source + same relevant inputs + valid artifacts
    -> reuse the previous result

changed dependency or missing artifact
    -> reconsider the build

This is the decision the cache must justify, rather than a literal stamp format.

A build stamp is a compact record of the inputs the builder relied on. The difficult part is deciding which inputs belong in that record. A missing dependency makes the cache optimistic. An irrelevant one makes it unnecessarily anxious.

The output matters as well. A stamp saying “done” is not useful if the corresponding BEAM file has been removed. Nor should an interrupted build leave a convincing record beside half a result.

pooka build
pooka build

The second invocation is a useful experiment, but it is the least demanding one. More revealing tests change an imported declaration, remove an artifact, or introduce an error after a successful build. The next run needs to respond to each situation for the right reason.

Say where the work happened

There is a naming problem hidden in this work too. A build may contain several packages. Logging the package being compiled helps explain where work happened; announcing everything under the application’s name makes a cache miss surprisingly difficult to locate.

I do not want the cache to become a second compiler with a separate idea of correctness. It should reuse an answer only while the evidence for that answer still holds.

Fast rebuilds are pleasant. Trustworthy rebuilds are what make them worth having.

The module can go back to sleep when we have checked the route, rather than merely recognised the front door.

— Peregrine Finch