Scala Single-File Scripts

Script Builtins

Mill Single-file Scala programs can make it more convenient to script simple command-line workflows interacting with files, subprocesses, and HTTP endpoints.

Mill Scala scripts come with a number of bundled libraries for these use cases. These are largely the same as the libraries available in Mill build files:

These libraries can make it more convenient to script simple command-line workflows interacting with files, subprocesses, and HTTP endpoints. For example, below is a simple script using these libraries to crawl wikipedia and save the crawl results to a file:

Crawler.scala (download, browse)

def fetchLinks(title: String): Seq[String] = {
  val resp = requests.get.stream(
    "https://en.wikipedia.org/w/api.php",
    params = Seq(
      "action" -> "query",
      "titles" -> title,
      "prop" -> "links",
      "format" -> "json"
    )
  )
  for {
    page <- ujson.read(resp)("query")("pages").obj.values.toSeq
    links <- page.obj.get("links").toSeq
    link <- links.arr
  } yield link("title").str
}

@mainargs.main
def main(startArticle: String, depth: Int) = {
  var seen = Set(startArticle)
  var current = Set(startArticle)
  for (i <- Range(0, depth)) {
    current = current.flatMap(fetchLinks(_)).filter(!seen.contains(_))
    seen = seen ++ current
  }

  pprint.log(seen)
  os.write(os.pwd / "fetched.json", upickle.stream(seen, indent = 4))
}

def main(args: Array[String]): Unit = mainargs.Parser(this).runOrExit(args)

> ./mill Crawler.scala --start-article singapore --depth 2
... seen: HashSet(
  "Calling code",
  "+65",
  "British Empire",
  "1st Parliament of Singapore",
...
)

> cat fetched.json
[
    "Calling code",
    "+65",
    "British Empire",
    "1st Parliament of Singapore",
    ...
]

These bundled libraries are only available in single-file scripts for convenience; in config-based modules or programmatic modules, you would need to add them to mvnDeps explicitly. You can also disable use of these libraries in scripts via //| includDefaultScriptMvnDeps: false. And as we saw earlier, you can include any other libraries you wish using //| mvnDeps.

While initially single-file Scala programs may be a bit more verbose than the equivalent Bash script containing cp or curl commands, as the script grows in complexity the value of IDE support, typechecking, and JVM libraries makes writing them in Scala an attractive proposition. This is especially true if you already have developers fluent in Scala which may not be as familiar with the intricacies of writing robust and maintainable Bash code.

Relative and Absolute Script moduleDeps

Mill single-file scripts can import each other via either relative or absolute imports. For example, given a bar/Bar.scala file such as below:

bar/Bar.scala (download, browse)

//| scalaVersion: 3.7.1
package bar

def generateHtml(text: String) = "<h1>" + text + "</h1>"

It can be imported via a ./Bar.scala import relative to its own enclosing folder (in this case bar/), as shown below where it is imported from the bar/BarTests.scala test suite in the same folder:

bar/BarTests.scala (download, browse)

//| extends: [mill.script.ScalaModule.Utest]
//| moduleDeps: [./Bar.scala]
//| mvnDeps:
//| - com.lihaoyi::utest:0.9.1
package bar
import utest.*
object BarTests extends TestSuite {
  def tests = Tests {
    assert(generateHtml("hello") == "<h1>hello</h1>")
  }
}

Or it can be imported via bar/Bar.scala absolute import, as shown below where it is imported from the foo/Foo.{language-exr}

foo/Foo.scala (download, browse)

//| moduleDeps: [bar/Bar.scala]
//| scalaVersion: 3.7.1

package foo

def main(args: Array[String]): Unit = println(bar.generateHtml(args(0)))

This examples can be exercised as follows:

> ./mill bar/Bar.scala:compile
> ./mill bar/BarTests.scala
> ./mill foo/Foo.scala --text hello

Custom Script Module Classes

By default, single-file Mill script modules inherit their behavior from the builtin mill.script.ScalaModule. However, you can also customize them to inherit from a custom Module class that you define as part of your meta-build in mill-build/src/. For example, if we want to add a resource file generated by processing the source file of the script, this can be done in a custom LineCountScalaModule as shown below:

Qux.scala (download, browse)

//| extends: [millbuild.LineCountScalaModule]
//| scalaVersion: 3.7.3
package qux

def getLineCount() = {
  scala.io.Source
    .fromResource("line-count.txt")
    .mkString
}

def main(args: Array[String]): Unit = {
  println(s"Line Count: ${getLineCount()}")
}

mill-build/src/LineCountScalaModule.scala (download, browse)

package millbuild
import mill.*, scalalib.*, script.*

class LineCountScalaModule(scriptConfig: ScriptModule.Config)
    extends mill.script.ScalaModule(scriptConfig) {

  /** Total number of lines in module source files */
  def lineCount = Task {
    allSourceFiles().map(f => os.read.lines(f.path).size).sum
  }

  /** Generate resources using lineCount of sources */
  override def resources = Task {
    os.write(Task.dest / "line-count.txt", "" + lineCount())
    super.resources() ++ Seq(PathRef(Task.dest))
  }
}

> ./mill Qux.scala
...
Line Count: 13

> ./mill show Qux.scala:lineCount
13

Your custom LineCountScalaModule must be a class take a mill.script.ScriptModule.Config as a parameter that is passed to the mill.script.ScalaModule. Custom script module classes allows you to customize the semantics of your Java, Scala, or Kotlin single-file script modules. If you have a large number of scripts with a similar configuration, or you need customizations that cannot be done in the YAML build header, placing these customizations in a custom script module class can let you centrally define the behavior and standardize it across all scripts that inherit it via extends.