Python Library Dependencies

This page goes into more detail about configuring third party dependencies for `PythonModule`s.

Adding Dependencies

package build
import mill._, pythonlib._

object `package` extends RootModule with PythonModule {
  def pythonDeps = Seq(
    "numpy==2.1.2",
    "pandas~=2.2.3",
    "jinja2 @ https://github.com/pallets/jinja/releases/download/3.1.4/jinja2-3.1.4-py3-none-any.whl"
  )
}

You can define the pythonDeps field to add dependencies to your module, which will be installed via pip. Dependencies can include anything that pip understands, such as <package>==<version> constraints, or even direct references to wheels.

> ./mill run
[10 20 30 40 50]

Adding Dependencies via requirements.txt files

You can also read dependencies from requirements.txt files. This can be useful if you’re migrating an existing project to mill.

build.mill (download, browse)

package build
import mill._, pythonlib._

object `package` extends RootModule with PythonModule {
  def pythonRequirementFiles = Task.Sources {
    millSourcePath / "requirements.txt"
  }
}

> ./mill run
[10 20 30 40 50]

Unmanaged Wheels

In most scenarios you should rely on pythonDeps/moduleDeps and let Mill manage the downloading and caching of wheels for you. But in the rare case you receive a wheel or folder-full-of-wheels from somewhere and need to include it in your project, unmanagedWheels is the way to do it.

build.mill (download, browse)

package build
import mill._, pythonlib._

object `package` extends RootModule with PythonModule {
  def unmanagedWheels: T[Seq[PathRef]] = Task.Input {
    Seq.from(os.list(millSourcePath / "lib").map(PathRef(_)))
  }
}

You can override unmanagedWheels to point it at a wheel (.whl file) or source distribution (.tar.gz with a pyproject.toml file) you place on the filesystem, e.g. in the above snippet any files that happen to live in the lib/ folder.

> ./mill run
b'"Hello, world!"'

Downloading Unmanaged Wheels

You can also override unmanagedWheels to point it at wheels that you want to download from arbitrary URLs. requests.get comes from the Requests-Scala library, one of Mill’s Bundled Libraries.

build.mill (download, browse)

package build
import mill._, pythonlib._

object `package` extends RootModule with PythonModule {
  def unmanagedWheels = Task {
    val name = "jinja2-3.1.4-py3-none-any.whl"
    val url = s"https://github.com/pallets/jinja/releases/download/3.1.4/$name"
    os.write(Task.dest / name, requests.get.stream(url))
    Seq(PathRef(Task.dest / name))
  }
}

> ./mill run
Hello, world!

Tasks like unmanagedWheels and pythonDeps are cached, so your wheel is downloaded only once and re-used indefinitely after that. This is usually not a problem, because usually URLs follow the rule that Cool URIs don’t change, and so files downloaded from the same URL will always contain the same contents.

An unmanaged wheel downloaded via requests.get is still unmanaged: even though you downloaded it from somewhere, requests.get does not know how to pull in third party dependencies or de-duplicate different versions on the classpath. All the same caveats you need to worry about when dealing with unmanaged wheels apply here as well. In case you do want mill to take care of managing dependencies of a package which is not available on PyPI, you shouldn’t get that package in unmanagedWheels (like we did in the example above). Instead, you can declare the dependency as a regular pythonDep as a direct URL that pip understands.

Using Custom Package Indexes

By default, dependencies are resolved from the Python Package Index (PyPI), the standard package index for python projects. You can also add your own package indexes by overriding the indexes task in the module:

build.mill (download, browse)

package build
import mill._, pythonlib._

object foo extends PythonModule {

  def pythonDeps = Seq(
    "testpkg-jodersky==0.0.1" // a test package, only available on test.pypi.org
  )

  // override this task to add or replace the package indexes
  def indexes = super.indexes() ++ Seq("https://test.pypi.org/simple/")
}

Mill uses pip to find and install dependencies.

You can configure pip through its normal configuration files.

Private indexes

You can read up in more detail on how to configure pip to authenticate to private indexes. Here is an example which reads a package from an environment variable:

object bar extends PythonModule {
  def indexPassword = Task.Input { Task.env.apply("COMPANY_PASSWORD") }
  def indexes = Task {
    Seq(s"https://username:${indexPassword()}@pypi.company.com/simple")
  }
}

More advanced authentication techniques are available by configuring pip directly.

> ./mill foo.run
2

Debugging

In case anything goes wrong, or if you’re just curious, you can see what arguments mill passes to pip install by looking at the output of the pipInstallArgs task.

build.mill (download, browse)

package build
import mill._, pythonlib._

object `package` extends RootModule with PythonModule {
  def pythonDeps = Seq(
    "numpy==2.1.2",
    "pandas~=2.2.3",
    "jinja2 @ https://github.com/pallets/jinja/releases/download/3.1.4/jinja2-3.1.4-py3-none-any.whl"
  )

  def indexes = Seq("invalid_index")
}

> ./mill show pipInstallArgs
{
  "args": [
    "--index-url",
    "invalid_index",
    "mypy==1.13.0",
    "pex==2.24.1",
    "numpy==2.1.2",
    "pandas~=2.2.3",
    "jinja2 @ https://github.com/pallets/jinja/releases/download/3.1.4/jinja2-3.1.4-py3-none-any.whl"
  ],
  "sig": ...
}