Scripting on the JVM with Java, Scala, and Kotlin

Java libraries are mostly hosted on Sonatype Central, which is one of the largest and most comprehensive sets of third-party libraries of any programming language. This includes high-quality well-maintained libraries such as Jackson, Unirest, and PicoCLI that we used above. Other JVM languages like Kotlin or Scala host their libraries on the same platform, which allows them to interoperate and use each other’s libraries. If a library exists anywhere in the world, it is likely you will find a Java version of it on Sonatype Central for you to use.

Java is known for having great IDEs such as IntelliJ that make it much easier to navigate, understand, and work with your codebase. This is just as helpful for small scripts as it is for larger applications, as even in scripts you may be working with unfamiliar third-party libraries for which IDE assistance can be especially helpful.

Beyond the IDE, the JVM probably has the best tools of any programming ecosystem. From simple things like jstack to debug stuck processes, to Yourkit or JProfiler to investigate tricky performance issues. Many of these come in handy when working with small scripts, which like application code can get stuck or become slow and need investigation and debugging.

While people often neglect the performance of scripts and small programs, just as often you find people complaining that poorly-optimized scripts are slow and taking longer than they need to. The JVM not only provides excellent single-core performance that’s 10-20x faster than traditional scripting languages, it also allows you to easily spawn threads to parallelize your script across multiple cores. This makes it much easier to optimize and fix slow scripts on the JVM than when running on slower single-threaded runtimes like Node.js or Python

Scripts have a reputation for being fragile and easy to break. This is because they are often poorly tested, for a variety of reasons:

Writing your scripts in a JVM language like Java, Scala or Kotlin doesn’t solve the testing problems above, but it does give you one more tool to lean on: the compiler checking that all your method calls and parameter types are properly wired up. While this won’t catch every bug, it at least provides an additional guardrail helping prevent silly mistakes and nudging your scripts towards correctness.

The first thing you may notice is: the program above is pretty verbose! Although the Java language has gotten more concise over the years, with features like the var keyword that we make good use of in the script above, overall Java code tends to still be significantly more lines than a typical scripting language. While the code above seems fine, it might not seem elegant enough that you would choose to write your scripts in Java on the JVM rather than Python.

The next thing you might deal with writing scripts is how to run them: you need java, and you need a build tool like Maven

Installing Java and Maven is not a blocker. But it is certainly something that can be a lot more fiddly than expected, especially when you have to ensure consistency over a fleet of heterogeneous machines and ensure it stays consistent over time.

Once you have Java and Maven installed, you then need to configure Maven to build it. The script above has three third-party dependencies, so the Maven config looks something like this:

While this Maven config isn’t particularly complicated, it is quite a screenful of text we need to write or copy-paste from somewhere in order to get our small script running. Again, not a blocker, but more fiddly stuff to set up and maintain just to write our single-file Java program.

Lastly, we need to run our script with Maven. The command to do so is as follows:

The single-line command for running your Maven project has a whole bunch of footguns you need to avoid:

This may seem nitpicky - and it is! - but it’s just another papercut in making it annoying to run the JsonApiClient.java file we have already written.

None of these issues with Maven or Java are fatal. We’ve all been writing and running Java programs for decades by now, in my case since Java 4 in 2004. But the sum-total of all these papercuts is that it’s not pleasant to write small programs in Java. Even modern AI assistants have limited context windows and benefit from having concise code to work with, and don’t benefit from digging through verbose boilerplate, fiddling with installation problems, or deal with and awkward tooling that fills up their context windows.

And so small programs and scripts end up being written in Python or Bash or Javascript. Even though in theory Java or Scala or Kotlin could work, in practice nobody uses them for scripting.

The first thing we can do to simplify our script workflow is to use a lighter weight build tool such as JBang or Mill. These tools make it much easier to configure some third-party dependencies and compile/run a single file. For example, JBang lets you write //DEPS header comments and run the .java file directly from the command line:

The Mill build tool uses a similar YAML header syntax that looks a bit different but otherwise works basically the same way:

Such lightweight build tools make it much easier to run our small Java program:

However, even if building and running small scripts written in Java is convenient, writing and maintaining the code itself can be a pain due to the verbosity of the Java language and libraries that makes even simple programs take pages and pages of code. While Java’s verbosity may be fine - or even beneficial - for complicated application code, the same verbosity can get in the way of writing throwaway scripts.

But Java isn’t the only language on the JVM!

One option to consider is to write the script in Kotlin. Kotlin is much more syntactically concise than Java, which means much less code to write the same things overall. A version of the JsonApiClient.java translated to Kotlin is shown below, using the Kotlin Clikt library rather than PicoCLI:

The Kotlin program has about 1/3 fewer lines than the Java equivalent, and overall much less dense. Kotlin features like the properties used to define val depth and val start, ?, and .mapNotNull simplify the code substantially. This makes it possible to express the same program in a much less verbose syntax.

As Kotlin is also a JVM language, it comes with all the same benefits as writing scripts in Java, e.g. the excellent IDE support provided by editors such as IntelliJ:

This makes using Kotlin a great way to streamline the scripting experience on the JVM. Although the example above uses Mill as the build tool, Kotlin also supports its own scripting workflows, which are also used in Gradle and other projects.

Although scripts in Kotlin are markedly nicer to write and read than scripts written in Java, there is one more step further that we can take:

The last step to simplify scripting on the JVM is to write the script in Scala. Scala is yet another JVM language, like Kotlin. But unlike Kotlin, Scala has many script-focused libraries such as OS-Lib, MainArgs, Requests-Scala, uPickle, or PPrint that make it very convenient to write small script-like programs in Scala. The above JsonApiClient.kt translated to an equivalent JsonApiClient.scala is shown below:

What is notable about JsonApiClient.scala is how much less stuff there is to read, with about 1/2 the lines of code as JsonApiClient.kt and 1/3 the lines of code as the JsonApiClient.java:

In general, the Scala script we see above looks similar to any scripting language. It has code that specifies clearly the logical steps of accessing the Wikipedia API and performing the breadth-first search, but without the verbose machinery necessary to implement that logic in Kotlin or Java. And although the Scala program is much shorter and more concise than the Java program we started with, overall it still has all the benefits of running on the JVM: