Example: Python Support

This section demonstrates how to integrate Python support into Mill. We will define a simple PythonModule trait that can resolve dependencies, perform type checking on local code, and bundle an executable.

This integration is for educational purposes only, showcasing common technique used in building language toolchains, and is not intended for production use. The examples reuse `UvModule’s downloader so that uv does not need to be installed first.

Basic Python Build Pipeline

First, we define a pythonExe task that asks uv to create a Python virtual environment. uv also downloads the requested Python version when it is not already installed.

build.mill (download, browse)
package build
import mill.*
import mill.pythonlib.UvModule

Reuse Mill’s downloader so this build does not require uv on the PATH.

object uv extends UvModule

def pythonVersion: T[String] = Task { "3.12" }

def venv: T[PathRef] = Task {
  val path = Task.dest / "venv"
  os.call((uv.uvExe().path, "venv", "--python", pythonVersion(), path), stdout = os.Inherit)
  PathRef(path)
}

def pythonExe: T[PathRef] = Task {
  val bin = if (scala.util.Properties.isWin) "Scripts" else "bin"
  val executable = if (scala.util.Properties.isWin) "python.exe" else "python"
  PathRef(venv().path / bin / executable)
}

Defining our Sources

The sources task specifies the directory for Python source files (src folder).

def sources: T[PathRef] = Task.Source("src")

Type Checking

The typeCheck task verifies that the code in the main Python file passes type checks.

def typeCheck: T[Unit] = Task {
  os.call(
    (
      uv.uvExe().path,
      "tool",
      "run",
      "--from",
      "ty==0.0.84",
      "ty",
      "check",
      "--python",
      pythonExe().path,
      sources().path
    ),
    stdout = os.Inherit
  )
}

At this point, we have a minimal working build, with a build graph that looks like this:

G pythonExe pythonExe typeCheck typeCheck pythonExe->typeCheck sources sources sources->typeCheck

Here is the main.py file

src/main.py (download, browse)
import sys
def add(a: int, b: int) -> int: return a + b
def main() -> None: print("Hello, " + " ".join(sys.argv[1:]) + "!")
if __name__ == "__main__":
    main()
    print(add(5, 10)) # Error Example: add("5", 10) will cause a TypeError

Running

The mainFileName task defines the name of the main Python script (in this case main.py). The run task runs the main file with user-provided command-line arguments. It uses the virtual environment’s Python interpreter to execute the script, with output displayed in the console.

def mainFileName: T[String] = Task { "main.py" }
def run(args: mill.api.Args) = Task.Command {
  os.call(
    (pythonExe().path, sources().path / mainFileName(), args.value),
    stdout = os.Inherit
  )
}

Note that we use stdout = os.Inherit since we want to display any output to the user, rather than capturing it for use in our command.

G pythonExe pythonExe typeCheck typeCheck pythonExe->typeCheck run run pythonExe->run sources sources sources->typeCheck sources->run mainFileName mainFileName mainFileName->run

Note that like many optionally-typed languages, The run and typeCheck tasks are independent: you can run a Python program without needing to typecheck it first. This is different from compiled languages like Java, which require typechecking before execution

Below are commands that demonstrate the typechecking and running functionality of our pipeline:

> ./mill typeCheck
All checks passed!

> ./mill run Mill Python
Hello, Mill Python!
15

> sed -i.bak 's/print(add(5, 10))/print(addd(5, 10))/g' src/main.py

> ./mill typeCheck # if we make a typo in a method name, ty flags it
error: ...addd...

We have now completed a basic Python integration in Mill, as a pipeline of inter-related tasks. Next steps is to turn this one-off pipeline into a reusable PythonModule

Re-usable PythonModule

This example shows how to define a PythonModule trait for managing Python scripts within multiple Mill objects. PythonModule takes the one-off pipeline we defined earlier with sources, pythonExe, typeCheck, etc. and wraps it in a trait that can be re-used.

build.mill (download, browse)
package build
import mill.*
import mill.pythonlib.UvModule

object uv extends UvModule

trait PythonModule extends Module {

  def sources: T[PathRef] = Task.Source("src")
  def mainFileName: T[String] = Task { "main.py" }

  def pythonVersion: T[String] = Task { "3.12" }

  def venv: T[PathRef] = Task {
    val path = Task.dest / "venv"
    os.call((uv.uvExe().path, "venv", "--python", pythonVersion(), path), stdout = os.Inherit)
    PathRef(path)
  }

  def pythonExe: T[PathRef] = Task {
    val bin = if (scala.util.Properties.isWin) "Scripts" else "bin"
    val executable = if (scala.util.Properties.isWin) "python.exe" else "python"
    PathRef(venv().path / bin / executable)
  }

  def typeCheck: T[Unit] = Task {
    os.call(
      (
        uv.uvExe().path,
        "tool",
        "run",
        "--from",
        "ty==0.0.84",
        "ty",
        "check",
        "--python",
        pythonExe().path,
        sources().path
      ),
      stdout = os.Inherit
    )

  }

  def run(args: mill.api.Args) = Task.Command {
    os.call(
      (pythonExe().path, sources().path / mainFileName(), args.value),
      stdout = os.Inherit
    )
  }

}

Once the trait PythonModule, has been defined, we can re-use it in three separate objects below

object foo extends PythonModule {
  def mainFileName = "foo.py"
  object bar extends PythonModule {
    def mainFileName = "bar.py"
  }
}

object qux extends PythonModule {
  def mainFileName = "qux.py"
}

For this example, we have three different Python Scripts foo/src/foo.py, foo/bar/src/bar.py, qux/src/qux.py, one in each PythonModule. The following commands run each module and display their output:

> ./mill foo.run Mill
Hello, Mill Foo!

> ./mill foo.bar.run Mill
Hello, Mill Foo Bar!

> ./mill qux.run Mill
Hello, Mill Qux!

After this step, we have a build graph that looks like this:

G cluster_3 qux cluster_1 foo cluster_2 bar qux.pythonExe qux.pythonExe qux.typeCheck qux.typeCheck qux.pythonExe->qux.typeCheck qux.run qux.run qux.pythonExe->qux.run qux.sources qux.sources qux.sources->qux.typeCheck qux.sources->qux.run qux.mainFileName qux.mainFileName qux.mainFileName->qux.run bar.pythonExe bar.pythonExe bar.typeCheck bar.typeCheck bar.pythonExe->bar.typeCheck bar.run bar.run bar.pythonExe->bar.run bar.sources bar.sources bar.sources->bar.typeCheck bar.sources->bar.run bar.mainFileName bar.mainFileName bar.mainFileName->bar.run foo.pythonExe foo.pythonExe foo.typeCheck foo.typeCheck foo.pythonExe->foo.typeCheck foo.run foo.run foo.pythonExe->foo.run foo.sources foo.sources foo.sources->foo.typeCheck foo.sources->foo.run foo.mainFileName foo.mainFileName foo.mainFileName->foo.run

Right now, the three PythonModules are independent. Next we will look into how to allow them to depend on each other using moduleDeps.

PythonModule moduleDeps

This example shows how to add module dependencies to PythonModule, allowing them to depend on one another.

The main change is the addition of def moduleDeps to specify the inter-module dependencies. We then use Task.traverse to aggregate the sources of the upstream modules and make them available during typeCheck and run:

build.mill (download, browse)
package build
import mill.*
import mill.api.BuildCtx
import mill.pythonlib.UvModule

object uv extends UvModule

trait PythonModule extends Module {

  // List of module dependencies required by this module.
  def moduleDeps: Seq[PythonModule] = Nil

  def sources: T[PathRef] = Task.Source("src")
  def mainFileName: T[String] = Task { "main.py" }

  def pythonVersion: T[String] = Task { "3.12" }

  def venv: T[PathRef] = Task {
    val path = Task.dest / "venv"
    os.call((uv.uvExe().path, "venv", "--python", pythonVersion(), path), stdout = os.Inherit)
    PathRef(path)
  }

  def pythonExe: T[PathRef] = Task {
    val bin = if (scala.util.Properties.isWin) "Scripts" else "bin"
    val executable = if (scala.util.Properties.isWin) "python.exe" else "python"
    PathRef(venv().path / bin / executable)
  }

  def typeCheck: T[Unit] = Task {
    val upstreamTypeCheck = Task.traverse(moduleDeps)(_.typeCheck)()
    os.call(
      (
        uv.uvExe().path,
        "tool",
        "run",
        "--from",
        "ty==0.0.84",
        "ty",
        "check",
        "--python",
        pythonExe().path,
        "--extra-search-path",
        BuildCtx.workspaceRoot,
        sources().path
      ),
      stdout = os.Inherit
    )

  }

  def gatherScripts(upstream: Seq[(PathRef, PythonModule)]) = {
    for ((sourcesFolder, mod) <- upstream) {
      val destinationPath = os.pwd / mod.moduleDir.subRelativeTo(build.moduleDir)
      os.copy.over(sourcesFolder.path / os.up, destinationPath)
    }
  }

  def run(args: mill.api.Args) = Task.Command {
    gatherScripts(Task.traverse(moduleDeps)(_.sources)().zip(moduleDeps))

    os.call(
      (pythonExe().path, sources().path / mainFileName(), args.value),
      env = Map("PYTHONPATH" -> Task.dest.toString),
      stdout = os.Inherit
    )
  }

}

Now we can take the three modules defined earlier and wire them up: qux depends on foo and foo.bar, which export their APIs for use in qux.

object foo extends PythonModule {
  def mainFileName = "foo.py"
  object bar extends PythonModule {
    def mainFileName = "bar.py"
  }
}

object qux extends PythonModule {
  def mainFileName = "qux.py"
  def moduleDeps = Seq(foo, foo.bar)
}

For this example, we define the following three files, one in each module, that depend on one another:

foo/src/foo.py (download, browse)
import sys
def multiply(a: int, b: int) -> int: return a * b
foo/bar/src/bar.py (download, browse)
import sys
def add(a: int, b: int) -> int: return a + b
qux/src/qux.py (download, browse)
from foo.bar.src.bar import add
from foo.src.foo import multiply
import sys
def divide(a: int, b: int) -> float: return a/b
def main() -> None:
    x = int(sys.argv[1])
    y = int(sys.argv[2])
    print(f"Add: {x} + {y} = {add(x, y)} | Multiply: {x} * {y} = {multiply(x, y)} | Divide: {x} / {y} = {divide(x, y)}")
if __name__ == "__main__":
    main()
> ./mill qux.run 10 20
Add: 10 + 20 = 30 | Multiply: 10 * 20 = 200 | Divide: 10 / 20 = 0.5

Task dependency graph, showing foo and bar tasks feeding into qux:

G cluster_1 foo cluster_2 bar cluster_3 qux bar.pythonExe bar.pythonExe bar.typeCheck bar.typeCheck bar.pythonExe->bar.typeCheck bar.run bar.run bar.pythonExe->bar.run qux.typeCheck qux.typeCheck bar.typeCheck->qux.typeCheck bar.sources bar.sources bar.sources->bar.typeCheck bar.sources->bar.run qux.run qux.run bar.sources->qux.run bar.mainFileName bar.mainFileName bar.mainFileName->bar.run foo.pythonExe foo.pythonExe foo.typeCheck foo.typeCheck foo.pythonExe->foo.typeCheck foo.run foo.run foo.pythonExe->foo.run foo.typeCheck->qux.typeCheck foo.sources foo.sources foo.sources->foo.typeCheck foo.sources->foo.run foo.sources->qux.run foo.mainFileName foo.mainFileName foo.mainFileName->foo.run qux.pythonExe qux.pythonExe qux.pythonExe->qux.typeCheck qux.pythonExe->qux.run qux.sources qux.sources qux.sources->qux.typeCheck qux.sources->qux.run qux.mainFileName qux.mainFileName qux.mainFileName->qux.run

Next, we will add support for depending on external Python libraries from PyPI, and bundling via PEX

uv Dependencies and Bundling

This implementation extends PythonModule with these key Tasks:

  • pythonDeps: allows the user to define Python dependencies that uv installs. These are aggregated into transitivePythonDeps

  • bundle: Packages the module and dependencies into a self-contained native executable, making deployment easier.

build.mill (download, browse)
package build
import mill.*
import mill.api.BuildCtx
import mill.pythonlib.UvModule

object uv extends UvModule

trait PythonModule extends Module {
  def moduleDeps: Seq[PythonModule] = Nil
  def mainFileName: T[String] = Task { "main.py" }
  def sources: T[PathRef] = Task.Source("src")

  def pythonDeps: T[Seq[String]] = Task { Seq() }

  def transitivePythonDeps: T[Seq[String]] = Task {
    val upstreamDependencies = Task.traverse(moduleDeps)(_.transitivePythonDeps)().flatten
    pythonDeps() ++ upstreamDependencies
  }

  def pythonVersion: T[String] = Task { "3.12" }

  def venv: T[PathRef] = Task {
    val path = Task.dest / "venv"
    os.call((uv.uvExe().path, "venv", "--python", pythonVersion(), path), stdout = os.Inherit)
    val bin = if (scala.util.Properties.isWin) "Scripts" else "bin"
    val executable = if (scala.util.Properties.isWin) "python.exe" else "python"
    val python = path / bin / executable
    os.call(
      (uv.uvExe().path, "pip", "install", "--python", python, "--strict", transitivePythonDeps()),
      stdout = os.Inherit
    )
    PathRef(path)
  }

  def pythonExe: T[PathRef] = Task {
    val bin = if (scala.util.Properties.isWin) "Scripts" else "bin"
    val executable = if (scala.util.Properties.isWin) "python.exe" else "python"
    PathRef(venv().path / bin / executable)
  }

  def typeCheck: T[Unit] = Task {
    val upstreamTypeCheck = Task.traverse(moduleDeps)(_.typeCheck)()

    os.call(
      (
        uv.uvExe().path,
        "tool",
        "run",
        "--from",
        "ty==0.0.84",
        "ty",
        "check",
        "--python",
        pythonExe().path,
        "--extra-search-path",
        BuildCtx.workspaceRoot,
        sources().path
      ),
      stdout = os.Inherit,
      cwd = BuildCtx.workspaceRoot
    )
  }

  def gatherScripts(upstream: Seq[(PathRef, PythonModule)]) = {
    for ((sourcesFolder, mod) <- upstream) {
      val destinationPath = os.pwd / mod.moduleDir.subRelativeTo(build.moduleDir)
      BuildCtx.withFilesystemCheckerDisabled {
        os.copy.over(sourcesFolder.path / os.up, destinationPath)
      }
    }
  }

  def run(args: mill.api.Args) = Task.Command {
    gatherScripts(Task.traverse(moduleDeps)(_.sources)().zip(moduleDeps))

    os.call(
      (pythonExe().path, sources().path / mainFileName(), args.value),
      env = Map("PYTHONPATH" -> Task.dest.toString),
      stdout = os.Inherit
    )
  }

  /** Bundles the project into a self-contained native PEX SCIE executable. */
  def bundle = Task {
    gatherScripts(Task.traverse(moduleDeps)(_.sources)().zip(moduleDeps))

    val bundleFile = Task.dest / (if (scala.util.Properties.isWin) "bundle.exe" else "bundle")
    val sciePythonVersion = os.call(
      (
        pythonExe().path,
        "-c",
        "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')"
      )
    ).out.trim()
    os.call(
      (
        uv.uvExe().path,
        "tool",
        "run",
        "--from",
        "pex==2.103.4",
        "pex",
        transitivePythonDeps(),
        "--venv-repository",
        venv().path,
        "-D",
        Task.dest,
        "--exe",
        sources().path / mainFileName(),
        "--scie",
        "eager",
        "--scie-python-version",
        sciePythonVersion,
        "--scie-only",
        "-o",
        bundleFile
      ),
      env = Map("PYTHONPATH" -> Task.dest.toString),
      stdout = os.Inherit
    )

    PathRef(bundleFile)
  }

}

Note the use of Task.traverse(moduleDeps) in order to aggregate the upstream modules library dependencies and the typeCheck outputs.

Now, our three modules can define pythonDeps to be used at runtime:

object foo extends PythonModule {
  object bar extends PythonModule {
    def pythonDeps = Seq("pandas==2.2.3", "numpy==2.1.3")
  }
  def pythonDeps = Seq("numpy==2.1.3")
}

object qux extends PythonModule {
  def moduleDeps = Seq(foo, foo.bar)
}

To run the project and create a self-contained executable, use these commands:

> ./mill qux.run
Numpy : Sum: 150 | Pandas: Mean: 30.0, Max: 50

> ./mill show qux.bundle
"...$MILL_OUT/qux/bundle.dest/bundle"

> out/qux/bundle.dest/bundle # running the bundle outside of Mill
Numpy : Sum: 150 | Pandas: Mean: 30.0, Max: 50

This generates a PEX SCIE executable that packages all dependencies together with a portable Python interpreter and can run without a system Python.

The final module tree and task graph is now as follows, with the additional dependencies tasks with upstream and the bundle tasks downstream:

G cluster_3 qux cluster_1 foo cluster_2 bar qux.pythonDeps qux.pythonDeps qux.pythonExe qux.pythonExe qux.pythonDeps->qux.pythonExe qux.typeCheck qux.typeCheck qux.pythonExe->qux.typeCheck qux.run qux.run qux.pythonExe->qux.run qux.sources qux.sources qux.sources->qux.typeCheck qux.sources->qux.run qux.bundle qux.bundle qux.sources->qux.bundle qux.mainFileName qux.mainFileName qux.mainFileName->qux.run bar.pythonDeps bar.pythonDeps bar.pythonDeps->qux.pythonDeps bar.pythonExe bar.pythonExe bar.pythonDeps->bar.pythonExe bar.typeCheck bar.typeCheck bar.pythonExe->bar.typeCheck bar.run bar.run bar.pythonExe->bar.run bar.typeCheck->qux.typeCheck bar.sources bar.sources bar.sources->qux.run bar.sources->qux.bundle bar.sources->bar.typeCheck bar.sources->bar.run bar.bundle bar.bundle bar.sources->bar.bundle bar.mainFileName bar.mainFileName bar.mainFileName->bar.run foo.pythonDeps foo.pythonDeps foo.pythonDeps->qux.pythonDeps foo.pythonExe foo.pythonExe foo.pythonDeps->foo.pythonExe foo.typeCheck foo.typeCheck foo.pythonExe->foo.typeCheck foo.run foo.run foo.pythonExe->foo.run foo.typeCheck->qux.typeCheck foo.sources foo.sources foo.sources->qux.run foo.sources->qux.bundle foo.sources->foo.typeCheck foo.sources->foo.run foo.bundle foo.bundle foo.sources->foo.bundle foo.mainFileName foo.mainFileName foo.mainFileName->foo.run

As mentioned, The PythonModule examples here demonstrate how to add support for a new language toolchain in Mill. A production-ready version would require more work to enhance features and performance.