Example: Typescript Support

This section walks through the process of adding support for a new programming language to Mill. We will be adding a small trait TypeScriptModule with the ability to resolve dependencies, typecheck local code, and optimize a final bundle.

The TypeScript integration here is not intended for production usage, but is instead meant for illustration purposes of the techniques typically used in implementing language toolchains.

Basic TypeScript Build Pipeline

This example demonstrates basic integration of Typescript compilation into a Mill build to compile Node.js apps. Mill does not come bundled with a Typescript integration, so here we begin setting one up from first principles using the NPM command line tool and package repository

Installing TypeScript

First, we need to use the npm CLI tool to install typescript and the @types/node library necessary for accessing Node.js APIs:

build.mill (download, browse)
package build
import mill._

def npmInstall = Task {
  os.call(("npm", "install", "--save-dev", "typescript@5.6.3", "@types/node@22.7.8"))
  PathRef(Task.dest)
}

The npmInstall task runs npm install to install TypeScript locally, following the TypeScript installation instructions. The os.call above by default runs inside the npmInstall task’s unique Task.dest output directory due to task sandboxing. Note that we use an explicit version on each of the modules to ensure the Task is reproducible. We then return a PathRef to the Task.dest so downstream tasks can make use of it.

Note that as written, the npmInstall task will never invalidate unless you change its code. This is what we should expect, since npmInstall has no upstream tasks it depends on and the versions of typescript and @types/node are fully specified in the task. This assumes that the npm package repository always returns the same artifacts for the given name and version, which is a reasonable assumption for most package repositories.

Defining our sources

Next, we define the sources of our Typescript build using a source task. Here sources refers to the source folder, and the subsequent allSources walks that folder and picks up all the individual typescript files within. This is a common pattern to give flexibility, e.g. someone can later easily override allSources to add additional filtering on exactly which files within the source root they wish to pick up.

def sources = Task.Source(millSourcePath / "src")
def allSources = Task {
  os.walk(sources().path).filter(_.ext == "ts").map(PathRef(_))
}

Compilation

Next, we define our compile task. This is again a relatively straightforward subprocess call invoking the typescript/bin/tsc executable within the node_modules folder from the upstream npmInstall task, passing it the sources, --outDir, --types, and --typeRoots Again we return a PathRef to the Task.dest folder we output the compiled JS files to

def compile = Task {
  os.call(
    (
      npmInstall().path / "node_modules/typescript/bin/tsc",
      allSources().map(_.path),
      "--outDir",
      Task.dest,
      "--typeRoots",
      npmInstall().path / "node_modules/@types"
    )
  )
  PathRef(Task.dest)
}

At this point, we have a minimal working build, with a build graph that looks like this:

G npmInstall npmInstall compile compile npmInstall->compile sources sources allSources allSources sources->allSources allSources->compile

Given an input file below, we can run mill compile and demonstrate it is installing typescript locally and using it to compile the .ts files in out src/ folder:

src/hello.ts (browse)
interface User {
    firstName: string
    lastName: string
    role: string
}

const user: User = {
    firstName: process.argv[2],
    lastName: process.argv[3],
    role: "Professor",
}

console.log("Hello " + user.name + " " + user.lastName)
> mill compile
error: .../src/hello.ts(...): error ... Property 'name' does not exist on type...

> sed -i.bak 's/user.name/user.firstName/g' src/hello.ts

> mill compile

> cat out/compile.dest/hello.js # Output is stripped of types and converted to javascript
var user = {
    firstName: process.argv[2],
    lastName: process.argv[3],
    role: "Professor",
};
console.log("Hello " + user.firstName + " " + user.lastName);

Running

The last step here is to allow the ability to run our compiled JavaScript file. To do this, we need a mainFileName task to tell Mill which file should be used as the program entrypoint, and a run command taking arguments that get used to call node along with the main Javascript file:

def mainFileName = Task { "hello.js" }
def run(args: mill.define.Args) = Task.Command {
  os.call(
    ("node", compile().path / mainFileName(), args.value),
    stdout = os.Inherit
  )
}

Note that we use stdout = os.Inherit since we want to display any output to the user, rather than capturing it for use in our command.

G npmInstall npmInstall compile compile npmInstall->compile run run compile->run sources sources allSources allSources sources->allSources allSources->compile mainFileName mainFileName mainFileName->run 
> mill run James Bond
Hello James Bond

So that’s a minimal example of implementing a single TypeScript to JavaScript build pipeline locally. Next, we will look at turning it into a TypeScriptModule that can be re-used

Re-usable TypeScriptModule

In this example, we will explore how to take the one-off typescript build pipeline we wrote above, and turn it into a re-usable TypeScriptModule.

To do this, we take all the code we wrote earlier and surround it with trait TypeScriptModule extends Module wrapper:

build.mill (download, browse)
package build
import mill._

trait TypeScriptModule extends Module {
  def npmInstall = Task {
    os.call(("npm", "install", "--save-dev", "typescript@5.6.3", "@types/node@22.7.8"))
    PathRef(Task.dest)
  }

  def sources = Task.Source(millSourcePath / "src")
  def allSources = Task {
    os.walk(sources().path).filter(_.ext == "ts").map(PathRef(_))
  }

  def compile = Task {
    val tsc = npmInstall().path / "node_modules/typescript/bin/tsc"
    val types = npmInstall().path / "node_modules/@types"
    os.call((tsc, allSources().map(_.path), "--outDir", Task.dest, "--typeRoots", types))
    PathRef(Task.dest)
  }

  def mainFileName = Task { s"${millSourcePath.last}.js" }
  def run(args: mill.define.Args) = Task.Command {
    val mainFile = compile().path / mainFileName()
    os.call(("node", mainFile, args.value), stdout = os.Inherit)
  }
}

We can then instantiate the module three times. Module can be adjacent or nested, as shown belo:

object foo extends TypeScriptModule {
  object bar extends TypeScriptModule
}
object qux extends TypeScriptModule
foo/src/foo.ts (browse)
interface User { firstName: string }

const user: User = { firstName: process.argv[2] }

console.log("Hello " + user.firstName + " Foo")
foo/bar/src/bar.ts (browse)
interface User { firstName: string }

const user: User = { firstName: process.argv[2] }

console.log("Hello " + user.firstName + " Bar")
qux/src/qux.ts (browse)
interface User { firstName: string }

const user: User = { firstName: process.argv[2] }

console.log("Hello " + user.firstName + " Qux")

And then invoke the .run method on each module from the command line:

> mill foo.run James
Hello James Foo

> mill foo.bar.run James
Hello James Bar

> mill qux.run James
Hello James Qux

At this point, we have multiple TypeScriptModules, with bar nested inside foo, but they are each independent and do not depend on one another.

G cluster_1 foo cluster_2 bar cluster_3 qux qux.npmInstall qux.npmInstall qux.compile qux.compile qux.npmInstall->qux.compile qux.run qux.run qux.compile->qux.run qux.sources qux.sources qux.allSources qux.allSources qux.sources->qux.allSources qux.allSources->qux.compile qux.mainFileName qux.mainFileName qux.mainFileName->qux.run bar.npmInstall bar.npmInstall bar.compile bar.compile bar.npmInstall->bar.compile bar.run bar.run bar.compile->bar.run bar.sources bar.sources bar.allSources bar.allSources bar.sources->bar.allSources bar.allSources->bar.compile bar.mainFileName bar.mainFileName bar.mainFileName->bar.run foo.npmInstall foo.npmInstall foo.compile foo.compile foo.npmInstall->foo.compile foo.run foo.run foo.compile->foo.run foo.sources foo.sources foo.allSources foo.allSources foo.sources->foo.allSources foo.allSources->foo.compile foo.mainFileName foo.mainFileName foo.mainFileName->foo.run

Next, we will look at how to wire them up using moduleDeps.

TypeScriptModule moduleDeps

This example extends TypeScriptModule to support moduleDeps.

  1. The def compile task is considerably fleshed out: rather than using command line flags, we generate a tsconfig.json file using the ujson.Obj/ujson.Arr JSON factory methods from the bundled uPickle library.

  2. def compile now returns two PathRef`s: one containing the `.js output to use in def run, and one containing the .d.ts output to use in downstream compiles.

  3. def moduleDeps is used to allow different TypeScriptModules to depend on each other, and we use Task.traverse to combine the upstream compiledDefinitions for use in compile, and compiledJavascript for use in run

build.mill (download, browse)
package build
import mill._

trait TypeScriptModule extends Module {
  def moduleDeps: Seq[TypeScriptModule] = Nil

  def npmInstall = Task {
    os.call(("npm", "install", "--save-dev", "typescript@5.6.3", "@types/node@22.7.8"))
    PathRef(Task.dest)
  }

  def sources = Task.Source(millSourcePath / "src")
  def allSources = Task { os.walk(sources().path).filter(_.ext == "ts").map(PathRef(_)) }

  def compile: T[(PathRef, PathRef)] = Task {

    val nodeTypes = npmInstall().path / "node_modules/@types"
    val javascriptOut = Task.dest / "javascript"
    val declarationsOut = Task.dest / "declarations"

    val upstreamPaths =
      for (((jsDir, dTsDir), mod) <- Task.traverse(moduleDeps)(_.compile)().zip(moduleDeps))
        yield (mod.millSourcePath.subRelativeTo(build.millSourcePath) + "/*", dTsDir.path)

    val allPaths = upstreamPaths ++ Seq("*" -> sources().path)

    os.write(
      Task.dest / "tsconfig.json",
      ujson.Obj(
        "compilerOptions" -> ujson.Obj(
          "outDir" -> javascriptOut.toString,
          "declaration" -> true,
          "declarationDir" -> declarationsOut.toString,
          "typeRoots" -> ujson.Arr(nodeTypes.toString),
          "paths" -> ujson.Obj.from(allPaths.map { case (k, v) => (k, ujson.Arr(s"$v/*")) })
        ),
        "files" -> allSources().map(_.path.toString)
      )
    )

    os.call(npmInstall().path / "node_modules/typescript/bin/tsc")
    (PathRef(javascriptOut), PathRef(declarationsOut))
  }

  def mainFileName = Task { s"${millSourcePath.last}.js" }
  def run(args: mill.define.Args) = Task.Command {

    val upstream = Task.traverse(moduleDeps)(_.compile)().zip(moduleDeps)
    for (((jsDir, tTsDir), mod) <- upstream) {
      os.copy(jsDir.path, Task.dest / mod.millSourcePath.subRelativeTo(build.millSourcePath))
    }
    val mainFile = compile()._1.path / mainFileName()
    os.call(
      ("node", mainFile, args.value),
      stdout = os.Inherit,
      env = Map("NODE_PATH" -> Seq(".", compile()._1.path).mkString(":"))
    )
  }
}

Note the use of Task.traverse(moduleDeps) in order to aggregate the compile output of the upstream modules, which is necessary both to configure the tsc TypeScript compiler in compile and also to set up the node working directory in run. This is a common pattern when defining language modules, whose module-level dependencies need to be translated into task-level dependencies

Again, we can instantiate TypeScriptModule three times, but now foo/src/foo.ts and foo/bar/src/bar.ts export their APIs which are then imported in qux/src/qux.ts:

object foo extends TypeScriptModule {
  object bar extends TypeScriptModule
}
object qux extends TypeScriptModule {
  def moduleDeps = Seq(foo, foo.bar)
}
foo/src/foo.ts (browse)
interface User {
    firstName: string
    lastName: string
    role: string
}

export {User}
foo/bar/src/bar.ts (browse)
const defaultRole = "Professor"
export {defaultRole}
qux/src/qux.ts (browse)
import {User} from "foo/foo.js"
import {defaultRole} from "foo/bar/bar.js"
const user: User = {
    firstName: process.argv[2],
    lastName: process.argv[3],
    role: defaultRole,
}

console.log("Hello " + user.firstName + " " + user.lastName + " " + user.role)

We can then invoke the qux.run method on each module from the command line:

> mill qux.run James Bond
Hello James Bond Professor

The dependency graph of tasks now looks like this, with the output of foo.compile and bar.compile now being fed into qux.compile (and ultimately qux.run):

G cluster_1 foo cluster_2 bar cluster_3 qux qux.npmInstall qux.npmInstall qux.compile qux.compile qux.npmInstall->qux.compile qux.run qux.run qux.compile->qux.run qux.sources qux.sources qux.allSources qux.allSources qux.sources->qux.allSources qux.allSources->qux.compile qux.mainFileName qux.mainFileName qux.mainFileName->qux.run bar.npmInstall bar.npmInstall bar.compile bar.compile bar.npmInstall->bar.compile bar.compile->qux.compile bar.run bar.run bar.compile->bar.run bar.sources bar.sources bar.allSources bar.allSources bar.sources->bar.allSources bar.allSources->bar.compile bar.mainFileName bar.mainFileName bar.mainFileName->bar.run foo.npmInstall foo.npmInstall foo.compile foo.compile foo.npmInstall->foo.compile foo.compile->qux.compile foo.run foo.run foo.compile->foo.run foo.sources foo.sources foo.allSources foo.allSources foo.sources->foo.allSources foo.allSources->foo.compile foo.mainFileName foo.mainFileName foo.mainFileName->foo.run

NPM dependencies and bundling

This example expands TypeScriptModule in two ways:

  1. Previously, def npmInstall was hardcoded to install typescript and @types/node, because that was what was needed to compile Typescript against the builtin Node.js APIs. In this example, we add a def npmDeps task, that is aggregated using Task.traverse into def transitiveNpmDeps, that are then included in the body of def npmInstall. The npmInstall destination folder in then used both in def compile to provide the tsc compiler and supporting installed type definitions, as well as in def run in order to provide the necessary files to the node runtime.

  2. We include esbuild@0.24.0 as part of our npm install, for use in a def bundle task that uses it to call esbuild to bundle our 3 TypeScriptModules into a single bundle.js file. The logic shared between def run and def bundle has been extracted into a def prepareRun task.

build.mill (download, browse)
package build
import mill._

trait TypeScriptModule extends Module {
  def moduleDeps: Seq[TypeScriptModule] = Nil

  def npmDeps: T[Seq[String]] = Task { Seq.empty[String] }

  def transitiveNpmDeps: T[Seq[String]] = Task {
    Task.traverse(moduleDeps)(_.npmDeps)().flatten ++ npmDeps()
  }

  def npmInstall = Task {
    os.call((
      "npm",
      "install",
      "--save-dev",
      "typescript@5.6.3",
      "@types/node@22.7.8",
      "esbuild@0.24.0",
      transitiveNpmDeps()
    ))
    PathRef(Task.dest)
  }

  def sources = Task.Source(millSourcePath / "src")
  def allSources = Task { os.walk(sources().path).filter(_.ext == "ts").map(PathRef(_)) }

  def compile: T[(PathRef, PathRef)] = Task {
    val nodeTypes = npmInstall().path / "node_modules/@types"
    val javascriptOut = Task.dest / "javascript"
    val declarationsOut = Task.dest / "declarations"

    val upstreamPaths =
      for (((jsDir, dTsDir), mod) <- Task.traverse(moduleDeps)(_.compile)().zip(moduleDeps))
        yield (mod.millSourcePath.subRelativeTo(build.millSourcePath) + "/*", dTsDir.path)

    val allPaths = upstreamPaths ++ Seq("*" -> sources().path, "*" -> npmInstall().path)

    os.write(
      Task.dest / "tsconfig.json",
      ujson.Obj(
        "compilerOptions" -> ujson.Obj(
          "outDir" -> javascriptOut.toString,
          "declaration" -> true,
          "declarationDir" -> declarationsOut.toString,
          "typeRoots" -> ujson.Arr(nodeTypes.toString),
          "paths" -> ujson.Obj.from(allPaths.map { case (k, v) => (k, ujson.Arr(s"$v/*")) })
        ),
        "files" -> allSources().map(_.path.toString)
      )
    )

    os.call((npmInstall().path / "node_modules/typescript/bin/tsc"))

    (PathRef(javascriptOut), PathRef(declarationsOut))
  }

  def mainFileName = Task { s"${millSourcePath.last}.js" }

  def prepareRun = Task.Anon {
    val upstream = Task.traverse(moduleDeps)(_.compile)().zip(moduleDeps)
    for (((jsDir, tTsDir), mod) <- upstream) {
      os.copy(jsDir.path, Task.dest / mod.millSourcePath.subRelativeTo(build.millSourcePath))
    }
    val mainFile = compile()._1.path / mainFileName()
    val env = Map("NODE_PATH" -> Seq(".", compile()._1.path, npmInstall().path).mkString(":"))
    (mainFile, env)
  }

  def run(args: mill.define.Args) = Task.Command {
    val (mainFile, env) = prepareRun()
    os.call(("node", mainFile, args.value), stdout = os.Inherit, env = env)
  }

  def bundle = Task {
    val (mainFile, env) = prepareRun()
    val esbuild = npmInstall().path / "node_modules/esbuild/bin/esbuild"
    val bundle = Task.dest / "bundle.js"
    os.call((esbuild, mainFile, "--bundle", s"--outfile=$bundle"), env = env)
    PathRef(bundle)
  }
}

object foo extends TypeScriptModule {
  object bar extends TypeScriptModule {
    def npmDeps = Seq("immutable@4.3.7")
  }
}
object qux extends TypeScriptModule {
  def moduleDeps = Seq(foo, foo.bar)
}

We can now not only invoke the qux.run to run the TypeScriptModule immediately using node, we can also use qux.bundle to generate a bundle.js file we can run standalone using node:

> mill qux.run James Bond prof
Hello James Bond Professor

> mill show qux.bundle
".../out/qux/bundle.dest/bundle.js"

> node out/qux/bundle.dest/bundle.js James Bond prof
Hello James Bond Professor

The final module tree and task graph is now as follows, with the additional npmDeps tasks upstream and the bundle tasks downstream:

G cluster_3 qux cluster_1 foo cluster_2 bar qux.npmInstall qux.npmInstall qux.compile qux.compile qux.npmInstall->qux.compile qux.run qux.run qux.compile->qux.run qux.bundle qux.bundle qux.compile->qux.bundle qux.sources qux.sources qux.allSources qux.allSources qux.sources->qux.allSources qux.allSources->qux.compile qux.mainFileName qux.mainFileName qux.mainFileName->qux.run qux.mainFileName->qux.bundle qux.npmDeps qux.npmDeps qux.npmDeps->qux.npmInstall bar.npmInstall bar.npmInstall bar.compile bar.compile bar.npmInstall->bar.compile bar.compile->qux.compile bar.run bar.run bar.compile->bar.run bar.bundle bar.bundle bar.compile->bar.bundle bar.sources bar.sources bar.allSources bar.allSources bar.sources->bar.allSources bar.allSources->bar.compile bar.mainFileName bar.mainFileName bar.mainFileName->bar.run bar.mainFileName->bar.bundle bar.npmDeps bar.npmDeps bar.npmDeps->qux.npmDeps bar.npmDeps->bar.npmInstall foo.npmInstall foo.npmInstall foo.compile foo.compile foo.npmInstall->foo.compile foo.compile->qux.compile foo.run foo.run foo.compile->foo.run foo.bundle foo.bundle foo.compile->foo.bundle foo.sources foo.sources foo.allSources foo.allSources foo.sources->foo.allSources foo.allSources->foo.compile foo.mainFileName foo.mainFileName foo.mainFileName->foo.run foo.mainFileName->foo.bundle foo.npmDeps foo.npmDeps foo.npmDeps->qux.npmDeps foo.npmDeps->foo.npmInstall

As mentioned earlier, the TypeScriptModule examples on this page are meant for demo purposes: to show what it looks like to add support in Mill for a new programming language toolchain. It would take significantly more work to flesh out the featureset and performance of TypeScriptModule to be usable in a real world build. But this should be enough to get you started working with Mill to add support to any language you need: whether it’s TypeScript or some other language, most programming language toolchains have similar concepts of compile, run, bundle, etc.

As mentioned, The PythonModule examples here demonstrate how to add support for a new language toolchain in Mill. A production-ready version would require more work to enhance features and performance.