Testing Java Projects
This page will discuss common topics around working with test suites using the Mill build tool
Defining Unit Test Suites
package build
import mill._, javalib._
object foo extends JavaModule {
object test extends JavaTests {
def testFramework = "com.novocode.junit.JUnitFramework"
def ivyDeps = Agg(
ivy"com.novocode:junit-interface:0.11",
ivy"org.mockito:mockito-core:4.6.1"
)
}
}
This build defines a single module with a test suite, configured to use
"JUnit" as the testing framework, along with Mockito. Test suites are themselves
JavaModule
s, nested within the enclosing module,
and have all the normal tasks like
foo.test.compile
available to run, but with an additional .test
task
that runs the tests. You can also run the test suite directly, in which case
it will run the .test
task as the default task for that module.
> mill foo.compile
compiling 1 ... source...
> mill foo.test.compile
compiling 1 ... source...
> mill foo.test.test
...foo.FooTests...hello ...
...foo.FooTests...world ...
> mill foo.test # same as above, `.test` is the default task for the `test` module
...foo.FooTests...hello ...
...foo.FooTests...world ...
> mill foo.test.testOnly foo.FooTests # explicitly select the test class you wish to run
...foo.FooTests...hello ...
...foo.FooTests...world ...
For convenience, you can also use one of the predefined test frameworks:
-
TestModule.Junit4
-
TestModule.Junit5
-
TestModule.TestNg
-
TestModule.Munit
-
TestModule.ScalaTest
-
TestModule.Specs2
-
TestModule.Utest
-
TestModule.ZioTest
object bar extends JavaModule {
object test extends JavaTests with TestModule.Junit4{
def ivyDeps = super.ivyDeps() ++ Agg(
ivy"org.mockito:mockito-core:4.6.1"
)
}
}
> mill bar.test
...bar.BarTests...hello ...
...bar.BarTests...world ...
You can also select multiple test suites in one command using Mill’s Task Query Syntax
> mill __.test
...bar.BarTests...hello ...
...bar.BarTests...world ...
...foo.FooTests...hello ...
...foo.FooTests...world ...
...
Mill provides three ways of running tests
-
foo.test.test
: runs tests in a subprocess in an emptysandbox/
folder. -
foo.test.testCached
: runs the tests in an emptysandbox/
folder and caches the results if successful. Also allows multiple test modules to be run in parallel e.g. viamill __.testCached
-
foo.test.testLocal
: runs tests in an isolated classloader as part of the main Mill process. This can be faster than.test
, but is less flexible (e.g. you cannot passforkEnv
) and more prone to interference (due to sharing thesandbox/
folder provided by the Mill process)
> mill bar.test.test
> mill bar.test.testCached
> mill bar.test.testLocal
Mill provides three ways of running tests
-
foo.test.test
: runs tests in a subprocess in an emptysandbox/
folder, andforkArg
andforkEnv
can be overridden to pass JVM flags & environment variables. -
foo.test.testCached
: runs the tests in an emptysandbox/
folder and caches the results if successful. Also allows multiple test modules to be run in parallel e.g. viamill __.testCached
. Again,forkEnv
andforkArgs
can be configured. -
foo.test.testLocal
: runs tests in an isolated classloader as part of the main Mill process. This can be faster than.test
, but is less flexible (e.g. you cannot passforkEnv
) and more prone to interference (due to sharing thesandbox/
folder provided by the Mill process)
> mill bar.test.test
> mill bar.test.testCached
> mill bar.test.testLocal
Note that Mill runs tests with the working directory set to an empty sandbox/
folder by default.
Tests can access files from their resource directory via the environment variable
MILL_TEST_RESOURCE_DIR
which provides the path to the resource folder, and additional
paths can be provided to test via forkEnv
. See
Classpath and Filesystem Resources
for more details.
If you want to pass any arguments to the test framework, simply put them after
foo.test
in the command line. e.g. uTest
lets you pass in a selector to decide which test to run, which in Mill would be:
> mill bar.test bar.BarTests.hello
...bar.BarTests...hello ...
Test Dependencies
Mill has no |
You might be used to test-scoped dependencies from other build tools like
Maven, Gradle or sbt. As test modules in Mill are just regular modules,
there is no special need for a dedicated test-scope. You can use ivyDeps
and runIvyDeps
to declare dependencies in test modules, and test modules
can use their moduleDeps
to also depend on each other
package build
import mill._, javalib._
object qux extends JavaModule {
def moduleDeps = Seq(baz)
object test extends JavaTests with TestModule.Junit4 {
def moduleDeps = super.moduleDeps ++ Seq(baz.test)
def ivyDeps = super.ivyDeps() ++ Agg(ivy"com.google.guava:guava:33.3.0-jre")
}
}
object baz extends JavaModule {
object test extends JavaTests with TestModule.Junit4 {
def ivyDeps = super.ivyDeps() ++ Agg(ivy"com.google.guava:guava:33.3.0-jre")
}
}
In this example, not only does qux
depend on baz
, but we also make
qux.test
depend on baz.test
. That lets qux.test
make use of the
BazTestUtils
class that baz.test
defines, allowing us to re-use this
test helper throughout multiple modules' test suites
> ./mill qux.test
Using BazTestUtils.bazAssertEquals
... qux.QuxTests...simple ...
...
> ./mill baz.test
Using BazTestUtils.bazAssertEquals
... baz.BazTests...simple ...
...
Defining Integration Test Suites
You can also define test suites with different names other than test
. For example,
the build below defines a test suite with the name integration
, in addition
to that named test
.
package build
import mill._, javalib._
object qux extends JavaModule {
object test extends JavaTests with TestModule.Junit5
object integration extends JavaTests with TestModule.Junit5
}
The integration suite is just another regular test module within the parent JavaModule (This example also demonstrates using Junit 5 instead of Junit 4)
These two test modules will expect their sources to be in their respective foo/test
and
foo/integration
folder respectively
> mill 'qux.{test,integration}' # run both test suites
...qux.QuxTests...hello...
...qux.QuxTests...world...
...qux.QuxIntegrationTests...helloworld...
> mill __.integration.testCached # run all integration test suites
Test Grouping
Test Grouping is an opt-in feature that allows you to take a single
test module and group the test classes such that each group will
execute in parallel when you call test
. Test grouping is enabled
by overriding def testForkGrouping
, as shown below:
package build
import mill._, javalib._
object foo extends JavaModule {
object test extends JavaTests {
def testFramework = "com.novocode.junit.JUnitFramework"
def ivyDeps = Agg(
ivy"com.novocode:junit-interface:0.11",
ivy"org.mockito:mockito-core:4.6.1"
)
def testForkGrouping = discoveredTestClasses().grouped(1).toSeq
}
}
package foo;
import static org.junit.Assert.assertTrue;
import org.junit.Test;
public class HelloTests {
@Test
public void hello() throws Exception {
System.out.println("Testing Hello");
String result = new Foo().hello();
assertTrue(result.startsWith("Hello"));
Thread.sleep(1000);
System.out.println("Testing Hello Completed");
}
}
package foo;
import static org.junit.Assert.assertTrue;
import org.junit.Test;
public class WorldTests {
@Test
public void world() throws Exception {
System.out.println("Testing World");
String result = new Foo().hello();
assertTrue(result.endsWith("World"));
Thread.sleep(1000);
System.out.println("Testing World Completed");
}
}
In this example, we have one test module foo.test
, and two test classes
HelloTests
and WorldTests
. By default, all test classes in the same
module run sequentially in the same JVM, but with testForkGrouping
we can break up the
module and run each test class in parallel in separate JVMs, each with their own
separate sandbox
folder and .log
file:
> mill foo.test
> find out/foo/test/test.dest
...
out/foo/test/test.dest/foo.HelloTests.log
out/foo/test/test.dest/foo.HelloTests/sandbox
out/foo/test/test.dest/foo.WorldTests.log
out/foo/test/test.dest/foo.WorldTests/sandbox
out/foo/test/test.dest/test-report.xml
Test grouping allows you to run tests in parallel while keeping things deterministic and debuggable: parallel test groups will not write over each others files in their sandbox, and each one will have a separate set of logs that can be easily read without the others mixed in
In this example, def testForkGrouping = discoveredTestClasses().grouped(1).toSeq
assigns
each test class to its own group, running in its own JVM. This comes with some overhead
on a per-JVM basis, so if your test classes are numerous and small you may want to assign
multiple test classes per group. You can also configure testForkGrouping
to choose which
test classes you want to run together and which to run alone:
-
If some test classes are much slower than others, you may want to put the slow test classes each in its own group to reduce latency, while grouping multiple fast test classes together to reduce the per-group overhead of spinning up a separate JVM.
-
Some test classes may have global JVM-wide or filesystem side effects that means they have to run alone, while other test classes may be better-behaved and OK to run in a group
In general, testForkGrouping
leaves it up to you how you want to group your tests for
execution, based on the unique constraints of your test suite.