How Fast Does Java Compile?

To begin to understand the problem, lets consider the codebase of the popular Mockito project:

Mockito is a medium-sized Java project with a few dozen sub-modules and about ~100,000 lines of code. To give us a simple reproducible scenario, let’s consider the root mockito module with sources in src/main/java/, on which all the downstream module and tests depend on.

Mockito is built using Gradle. It’s not totally trivial to extract the compilation classpath from Gradle, but the following stackoverflow answer gives us some tips:

This gives us the following classpath:

Note that for this benchmark, all third-party dependencies have already been resolved and downloaded from Maven Central. We can thus simply reference the jars on disk directly, which we do above.

We can then pass this classpath into javac -cp, together with src/main/java/*/.java, to perform the compilation outside of Gradle using javac directly. Running this a few times gives us the timings below:

To give us an idea of how many lines of code we are compiling, we can run:

Combining this information, we find that 41601 lines of code compiled in ~1.29 seconds (taking the median of the three runs above) suggests that javac compiles about ~32,000 lines of code per second.

These benchmarks were run ad-hoc on my laptop, an M1 10-core Macbook Pro, with OpenJDK Corretto 17.0.6. The numbers would differ on different Java versions, hardware, operating systems, and filesystems. Nevertheless, the overall trend is strong enough that you should be able to reproduce the results despite variations in the benchmarking environment.

Compiling 32,000 lines of code per second is not bad. But it is nowhere near how fast the Java compiler can run. Any software experience with JVM experience would know the next obvious optimization for us to explore.

One issue with the above benchmark is that it uses javac as a sub-process. The Java compiler runs on the Java Virtual Machine, and like any JVM application, it has a slow startup time, takes time warming-up, but then has good steady-state performance. Running javac from the command line and compiling ~32,000 lines/sec is thus the worst possible performance you could get out of the Java compiler on this Java codebase.

To get good performance out of javac, like any other JVM application, we need to keep it long-lived so it has a chance to warm up. While running the javac in a long-lived Java program is not commonly taught, neither is it particularly difficult. Here is a complete Bench.java file that does this, repeatedly running java compilation in a loop where it has a chance to warm up, to emulate the long lived JVM process that a build tool like Mill may spawn and manage. We use the same MY_CLASSPATH and source files we saw earlier and print the output statistics to the terminal so we can see how fast Java compilation can occur once things have a chance to warm up:

Running this using java Bench.java in the Mockito repo root, eventually we see it settle on approximately the following numbers:

The codebase hasn’t changed - we are still compiling 41,601 lines of code - but now it only takes ~359ms. That tells us that using a long-lived warm Java compiler we can compile approximately 116,000 lines of Java a second on a single core.

Compiling 116,000 lines of Java per second is very fast. That means we should expect a million-line Java codebase to compile in about 9 seconds, on a single thread. That may seem surprisingly fast, and you may be forgiven if you find it hard to believe. As mentioned earlier, this number is expected to vary based on the codebase being compiled; could it be that Mockito-Core just happens to be a very simple Java module that compiles quickly?

To double-check our results, we can pick another codebase to run some ad-hoc benchmarks. For this I will use the Netty codebase:

Netty is a large-ish Java project: ~500,000 lines of code. Again, to pick a somewhat easily-reproducible benchmark, we want a decently-sized module that’s relatively standalone within the project: netty-common is a perfect fit. Again, we can use find | grep | xargs to see how many lines of code we are looking at:

Again, Maven doesn’t make it easy to show the classpath used to call javac ourselves, but the following stackoverflow answer gives us a hint in how to do so:

If you grep the output for -classpath, we see:

Again, we can export MY_CLASSPATH and start using javac from the command line:

Or programmatically using the Bench.java program we saw earlier:

Taking 285ms for a hot-in-memory compile of 29,712 lines of code, netty-common therefore compiles at ~104,000 lines/second.

Although the choice of project is arbitrary, Mockito-Core and Netty-Common are decent examples of Java code found "out in the wild". They aren’t synthetic fake codebases generated for the purpose of benchmarks, nor are they particularly unusual or idiosyncratic. They follow most Java best practices and adhere to many of the most common Java linters (although those were disabled for this performance benchmark). This is Java code that looks just like any Java code you may write in your own projects, and it effortlessless compiles at >100,000 lines/second.

Although the Java Compiler is blazing fast - compiling code at >100k lines/second and completing both Mockito-Core and Netty-Common in ~300ms - the experience of using typical Java build tools is nowhere near as snappy. Consider the benchmark of clean-compiling the Mockito-Core codebase using Gradle or Mill:

Or the benchmark of clean-compiling the Netty-Common codebase using Maven or Mill:

These benchmarks are run in similar conditions as those we saw earlier: ad-hoc on my M1 Macbook Pro, with the metadata and jars of all third-party dependencies already downloaded and cached locally. So the time we are seeing above is purely the Java compilation + the overhead of the build tool realizing it doesn’t need to do anything except compile the Java source code using the dependencies we already have on disk.

Tabulating this all together gives us the table we saw at the start of this page:

We explore the comparison between Gradle vs Mill or Maven vs Mill in more detail on their own dedicated pages. For this article, the important thing is not comparing the build tools against each other, but comparing the build tools against what how fast they could be if they just used the javac Java compiler directly. And it’s clear that compared to the actual work done by javac to actually compile your code, build tools add a frankly absurd amount of overhead ranging from ~4x for Mill to 15-16x for Maven and Gradle!

One thing worth calling out is that the overhead of the various build tools does not appear to go down in larger builds. This Clean Compile Single-Module benchmark we explored above only deals with compiling a single small module. But a similar Sequential Clean Compile benchmarks which compiles the entire Mockito and Netty projects on a single core shows similar numbers for the various build tools:

All of these are far below the 100,000 lines/s that we should expect from Java compilation, but they roughly line up with the numbers measured above. Again, these benchmarks are ad-hoc, on arbitrary hardware and JVM versions. They do include small amounts of other work, such as compiling C/C++ code in Netty or doing ad-hoc file operations in Mockito. However, most of the time is still spent in compilation, and this reinforces the early finding that build tools (especially older ones like Maven or Gradle) are indeed adding huge amounts of overhead on top of the extremely-fast Java compiler.

From this study we can see the paradox: the Java compiler is blazing fast, while Java build tools are dreadfully slow. Something that should compile in a fraction of a second using a warm javac takes several seconds (15-16x longer) to compile using Maven or Gradle. Mill does better, but even it adds 4x overhead and falls short of the snappiness you would expect from a compiler that takes ~0.3s to compile the 30-40kLOC Java codebases we experimented with.

These benchmarks were run ad-hoc and on my laptop on arbitrary codebases, and the details will obviously differ depending on environment and the code in question. Running it on an entire codebase, rather than a single module, will give different results. Nevertheless, the results are clear: "typical" Java code should compile at ~100,000 lines/second on a single thread. Anything less is purely build-tool overhead from Maven, Gradle, or Mill.

Build tools do a lot more than the Java compiler. They do dependency management, parallelism, caching and invalidation, and all sorts of other auxiliary tasks. But in the common case where someone edits code and then compiles it, and all your dependencies are already downloaded and cached locally, any time doing other things and not spent actually compiling Java is pure overhead. Checking for cache invalidation in shouldn’t take 15-16x as long as actually compiling your code. I mean it obviously does today, but it shouldn’t!

The Mill build tool goes to great lengths to try and minimize overhead, and already gets ~4x faster builds than Maven or Gradle on real-world projects like Mockito or Netty. But there still is a long way to go give Java developers the fast, snappy experience that the underlying Java platform can provide. If Java build and compile times are things you find important, you should try out Mill on your own projects and get involved in the effort!