ExampleTester

A variant of IntegrationTester, ExampleTester works the same way except the commands used to test the project come from a /** Usage ... */ comment inside the project's build.mill file. This is intended to make the build.mill file usable as documentation, such that a reader can skim the build.mill and see both the build configuration and the commands they themselves can enter at the command line to exercise it.

Implements a bash-like test DSL for educational purposes, parsed out from a Example Usage comment in the example's build.mill file. Someone should be able to read the Example Usage comment and know roughly how to execute the example themselves.

Each empty-line-separated block consists of one command (prefixed with >) and zero or more output lines we expect to get from the command (either stdout or stderr):

1. If there are no expected output lines, we do not perform any assertions on the output of the command

2. Output lines can be prefixed by error: to indicate we expect that command to fail.

3. ... can be used to indicate wildcards, which match anything. These can be used alone as the entire line, or in the middle of another line

4. Every line of stdout/stderr output by the command must match at least one line of the expected output, and every line of expected output must match at least one line of stdout/stderr. We ignore ordering of output lines.

For teaching purposes, the output lines do not show the entire output of every command, which can be verbose and confusing. They instead contain sub-strings of the command output, enough to convey the important points to a learner. This is not as strict as asserting the entire command output, but should be enough to catch most likely failure modes

Because our CI needs to run on Windows, we cannot rely on just executing commands in the bash shell, and instead we implement a janky little interpreter that reads the command lines and does things in-JVM in response to each one.