Writing Mill Plugins
This example demonstrates how to write and test Mill plugin, and publish it to Sonatype’s Maven Central so it can be used by other developers over the internet via import $ivy.
Project Configuration
package build
import mill._, scalalib._, publish._
import mill.main.BuildInfo.millVersion
object myplugin extends ScalaModule with PublishModule {
def scalaVersion = "2.13.8"
def ivyDeps = Agg(ivy"com.lihaoyi:mill-dist:$millVersion")
// Testing Config
object test extends ScalaTests with TestModule.Utest{
def ivyDeps = Agg(ivy"com.lihaoyi::mill-testkit:$millVersion")
def forkEnv = Map("MILL_EXECUTABLE_PATH" -> millExecutable.assembly().path.toString)
object millExecutable extends JavaModule{
def ivyDeps = Agg(ivy"com.lihaoyi:mill-dist:$millVersion")
def mainClass = Some("mill.runner.client.MillClientMain")
def resources = Task {
val p = Task.dest / "mill/local-test-overrides" / s"com.lihaoyi-${myplugin.artifactId()}"
os.write(p, myplugin.localClasspath().map(_.path).mkString("\n"), createFolders = true)
Seq(PathRef(Task.dest))
}
}
}
// Publishing Config
def publishVersion = "0.0.1"
def pomSettings = PomSettings(
description = "Line Count Mill Plugin",
organization = "com.lihaoyi",
url = "https://github.com/lihaoyi/myplugin",
licenses = Seq(License.MIT),
versionControl = VersionControl.github("lihaoyi", "myplugin"),
developers = Seq(Developer("lihaoyi", "Li Haoyi", "https://github.com/lihaoyi"))
)
}
Mill plugins are fundamentally just JVM libraries that depend on Mill. Any vanilla JVM
library (whether written in Java or Scala) can be used in a Mill build.mill
file via
import $ivy
, but Mill plugins tend to integrate with Mill by defining a
trait for modules in your build.mill
to inherit from.
The above build.mill
file sets up a object myplugin extends ScalaModule
not just to
compile your Mill plugin project, but also to run automated tests using mill-testkit
,
and to configure it for publishing to Maven Central via PublishModule
.
It looks like any other Scala project, except for a few things to take note:
-
A dependency on
com.lihaoyi:mill-dist:$millVersion
-
A test dependency on
com.lihaoyi::mill-testkit:$millVersion
-
An
object millExecutable
that adds some resources to the publishedmill-dist
assembly to configure it to rewireimport $ivy
to instead use the local compiled classfiles for testing.
Plugin Implementation
Like any other trait
, a Mill plugin’s traits modules allow you to:
-
Add additional tasks to an existing module
-
Override existing tasks, possibly referencing the old task via
super
-
Define abstract tasks that the final module must implement
In this example, we define a LineCountJavaModule
that does all of the above:
it defines an abstract def lineCountResourceFileName
task, it adds an additional
def lineCount
task, and it overrides the def resources
:
package myplugin
import mill._
/**
* Example Mill plugin trait that adds a `line-count.txt`
* to the resources of your `JavaModule`
*/
trait LineCountJavaModule extends mill.javalib.JavaModule{
/** Name of the file containing the line count that we create in the resource path */
def lineCountResourceFileName: T[String]
/** Total number of lines in module source files */
def lineCount = Task {
allSourceFiles().map(f => os.read.lines(f.path).size).sum
}
/** Generate resources using lineCount of sources */
override def resources = Task {
os.write(Task.dest / lineCountResourceFileName(), "" + lineCount())
super.resources() ++ Seq(PathRef(Task.dest))
}
}
This is a synthetic example, but it serves to illustrate how Mill plugins are typically defined. The plugin can be compiled via:
> ./mill myplugin.compile
compiling 1 Scala source...
Mill provides the mill-testkit
library to make it easy for you to test your Mill
plugin. The example project above has set up tests that can be run via the normal .test
command, as shown below:
> ./mill myplugin.test
+ myplugin.UnitTests.unit...
+ myplugin.IntegrationTests.integration...
+ myplugin.ExampleTests.example...
...
mill-testkit
is the same set of helpers that Mill uses internally for its
own testing, and covers three approaches:
Unit Tests
These are tests that run in-process, with the Mill build.mill
defined as a TestBaseModule
,
and using a UnitTester
to run its tasks and inspect their output. UnitTester
is provided
a path to a folder on disk containing the files that are to be built with the given TestBaseModule
,
and can be used to evaluate tasks (by direct reference or by string-selector) and inspect
the results in-memory:
package myplugin
import mill.testkit.{TestBaseModule, UnitTester}
import utest._
object UnitTests extends TestSuite {
def tests: Tests = Tests {
test("unit") {
object build extends TestBaseModule with LineCountJavaModule {
def lineCountResourceFileName = "line-count.txt"
}
val resourceFolder = os.Path(sys.env("MILL_TEST_RESOURCE_DIR"))
UnitTester(build, resourceFolder / "unit-test-project").scoped { eval =>
// Evaluating tasks by direct reference
val Right(result) = eval(build.resources)
assert(
result.value.exists(pathref =>
os.exists(pathref.path / "line-count.txt") &&
os.read(pathref.path / "line-count.txt") == "17"
)
)
// Evaluating tasks by passing in their Mill selector
val Right(result2) = eval("resources")
val Seq(pathrefs: Seq[mill.api.PathRef]) = result2.value
assert(
pathrefs.exists(pathref =>
os.exists(pathref.path / "line-count.txt") &&
os.read(pathref.path / "line-count.txt") == "17"
)
)
}
}
}
}
package foo;
public class Foo {
public static String getLineCount(){
try{
return new String(
Foo.class.getClassLoader().getResourceAsStream("line-count.txt").readAllBytes()
);
}catch(java.io.IOException e){ return null; }
}
static String lineCount = getLineCount();
public static void main(String[] args) throws Exception {
System.out.println("Line Count: " + lineCount);
}
}
Mill Unit tests are good for exercising most kinds of business logic in Mill plugins. Their main limitation is that they do not exercise the Mill subprocess-launch and bootstrap process, but that should not be a concern for most Mill plugins.
Integration Tests
Integration tests are one step up from Unit tests: they are significantly slower to run due
to running Mill in a subprocess, but are able to exercise the end-to-end lifecycle of a Mill
command. Unlike unit tests which define a TestRootModule
in-memory as part of the test code,
Mill’s integration tests rely on a build.mill
that is processed and compiled as part of the
test initialization, and can only perform assertions on the four things that are returned from
any subprocess:
-
.isSuccess: Boolean
, whether or the Mill subprocess returned with exit code 0 -
.out: String
, the standard output captured by the Mill process -
.err: String
, the standard error captured by the Mill process -
Any files that are generated on disk. In particular, files generated by tasks can be fetched via the
tester.out("…").*
APIs to be read as JSON strings (via.text
), parsedujson.Value
ASTs (.json
), or parsed into a typed Scala value (.value[T]
)
package myplugin
import mill.testkit.IntegrationTester
import utest._
object IntegrationTests extends TestSuite {
println("initializing myplugin.IntegrationTest")
def tests: Tests = Tests {
println("initializing myplugin.IntegrationTest.tests")
test("integration") {
pprint.log(sys.env("MILL_EXECUTABLE_PATH"))
val resourceFolder = os.Path(sys.env("MILL_TEST_RESOURCE_DIR"))
val tester = new IntegrationTester(
clientServerMode = true,
workspaceSourcePath = resourceFolder / "integration-test-project",
millExecutable = os.Path(sys.env("MILL_EXECUTABLE_PATH"))
)
val res1 = tester.eval("run")
assert(res1.isSuccess)
assert(res1.err.contains("compiling 1 Java source")) // compiling the `build.mill`
assert(res1.out.contains("Line Count: 17"))
assert(tester.out("lineCount").value[Int] == 17)
val res2 = tester.eval("run") // No need to recompile when nothing changed
assert(!res2.err.contains("compiling 1 Java source"))
assert(res2.out.contains("Line Count: 17"))
assert(tester.out("lineCount").value[Int] == 17)
tester.modifyFile(tester.workspacePath / "src/foo/Foo.java", _ + "\n")
val res3 = tester.eval("run") // Additional newline forces recompile and increases line count
assert(res3.err.contains("compiling 1 Java source"))
assert(res3.out.contains("Line Count: 18"))
assert(tester.out("lineCount").value[Int] == 18)
}
}
}
package foo;
public class Foo {
public static String getLineCount(){
try{
return new String(
Foo.class.getClassLoader().getResourceAsStream("line-count.txt").readAllBytes()
);
}catch(java.io.IOException e){ return null; }
}
static String lineCount = getLineCount();
public static void main(String[] args) throws Exception {
System.out.println("Line Count: " + lineCount);
}
}
package build
import $ivy.`com.lihaoyi::myplugin:0.0.1`
import mill._, myplugin._
object `package` extends RootModule with LineCountJavaModule{
def lineCountResourceFileName = "line-count.txt"
}
Integration tests are generally used sparingly, but they are handy for scenarios where your Mill plugin logic prints to standard output or standard error, and you want to assert that the printed output is as expected.
Example Tests
Example tests are a variant of the integration tests mentioned above, but instead of
having the testing logic defined as part of the test suite in a .scala
file, the testing
logic is instead defined as a /** Usage */
comment in the build.mill
. These tests are
a great way of documenting expected usage of your plugin: a user can glance at a single
file to see how the plugin is imported (via import $ivy
) how it is used (e.g. by being
extended by a module) and what commands they can run exercising the plugin and the resultant
output they should expect:
package myplugin
import mill.testkit.ExampleTester
import utest._
object ExampleTests extends TestSuite {
def tests: Tests = Tests {
test("example") {
val resourceFolder = os.Path(sys.env("MILL_TEST_RESOURCE_DIR"))
ExampleTester.run(
clientServerMode = true,
workspaceSourcePath = resourceFolder / "example-test-project",
millExecutable = os.Path(sys.env("MILL_EXECUTABLE_PATH"))
)
}
}
}
package foo;
public class Foo {
public static String getLineCount(){
try{
return new String(
Foo.class.getClassLoader().getResourceAsStream("line-count.txt").readAllBytes()
);
}catch(java.io.IOException e){ return null; }
}
static String lineCount = getLineCount();
public static void main(String[] args) throws Exception {
System.out.println("Line Count: " + lineCount);
}
}
package build
import $ivy.`com.lihaoyi::myplugin:0.0.1`
import mill._, myplugin._
object `package` extends RootModule with LineCountJavaModule{
def lineCountResourceFileName = "line-count.txt"
}
/** Usage
> ./mill run
Line Count: 17
...
> printf "\n" >> src/foo/Foo.java
> ./mill run
Line Count: 18
...
*/
The /** Usage */
comment is of the following format:
-
Each line prefixed with
>
is a command that is to be run -
Following lines after commands are expected output, until the next blank line
-
If the command is expected to fail, the following lines should be prefixed by `error: `
-
Expected output lines can contain
…
wildcards to match against parts of the output which are unstable or unnecessary for someone reading through thebuild.mill
. -
A
…
wildcard on its own line can be used to match against any number of additional lines of output
The line-matching for example tests is intentionally fuzzy: it does not assert that the
ordering of the lines printed by the command matches the ordering given, as long as every
line printed is given and every line given is printed. …
wildcards intentionally add
additional fuzziness to the matching. The point of example tests is not to match
character-for-character exactly what the output must be, but to match on the "important"
parts of the output while simultaneously emphasizing these important parts to someone who
may be reading the build.mill
Example tests are similar to integration tests in that they exercise the full Mill bootstrapping
process, and are thus much slower and more expensive to run than unit tests. However, it is
usually a good idea to have at least one example test for your Mill plugin, so a user who
wishes to use it can take the build.mill
and associated files and immediately have a Mill
build that is runnable using your plugin, along with a list of commands they can run and
what output they should expect.
Publishing
> sed -i.bak 's/0.0.1/0.0.2/g' build.mill
> ./mill myplugin.publishLocal
Publishing Artifact(com.lihaoyi,myplugin_2.13,0.0.2) to ivy repo...
Mill plugins are JVM libraries like any other library written in Java or Scala. Thus they
are published the same way: by extending PublishModule
and defining the module’s publishVersion
and pomSettings
. Once done, you can publish the plugin locally via publishLocal
below,
or to Maven Central via `mill.scalalib.public.PublishModule/`for other developers to
use.