Python Packaging & Publishing

This page will discuss common topics around publishing your Python projects for others to use.

All packaging and publishing functionality is defined in PublishModule. Start by extending it.

import mill._, pythonlib._

object `package` extends RootModule with PythonModule with PublishModule {

  // information about dependencies will be included in the published package
  def pythonDeps = Seq("jinja2==3.1.4")

  def publishMeta = PublishMeta(
    name = "testpkg-mill",
    description = "an example package",
    requiresPython = ">= 3.12",
    license = License.MIT,
    authors = Seq(Developer("John Doe", "jdoe@example.org"))
  )

  // the version under which the package will be published
  def publishVersion = "0.0.2"

}

You’ll need to define some metadata in the publishMeta and publishVersion tasks. This metadata is roughly equivalent to what you’d define in a pyproject.toml file.

You’ll also need to create a readme file, which will be bundled in the final package and serves as the landing page seen on PyPI. By default, Mill assumes a file starting with the string readme (in any capitalization), but you can override it to whatever you please.

The version of your package is not included in publishMeta, but rather in its own publishVersion task. This is done so that you can easily override the task to automate the version, such as deriving it from source control.

Building packages locally

You can build a source distribution or wheel by running the following tasks:

> mill show sdist
".../out/sdist.dest/dist/testpkg_mill-0.0.2.tar.gz"

> mill show wheel
".../out/wheel.dest/dist/testpkg_mill-0.0.2-py3-none-any.whl"

These files can then be pip-installed by other projects, or, if you’re using Mill, you can include them in your unmanagedWheels task. Usually however, you’d want to publish them to a package index such as PyPI or your organization’s internal package repository.

Uploading your packages to PyPI (or other repository)

Uploading your packages to PyPI can be done by running mill __.publish.

Mill uses twine to upload packages, and respects its configuration. You can also configure it with environment variables, prefixed with MILL_.

export MILL_TWINE_REPOSITORY_URL=https://test.pypi.org/legacy/
export MILL_TWINE_USERNAME=<username, not necessary for PyPI>
export MILL_TWINE_PASSWORD=<apitoken>
mill __.publish

Mill does not transitively upload all your packages, hence we recommended to use mill __.publish, instead of mill <module>.publish. While it’s technically possible to upload packages of individual Mill modules by calling their publish tasks separately, you’d usually want to ensure all your dependencies are also published.

Check before uploading

Twine has a nice feature to check your artifacts before uploading them. You can also do this with Mill, by running:

> mill __.checkPublish
... PASSED

Advanced Packaging

Behind the scenes, Mill delegates most Python packaging tasks to other tools, and only takes care of configuring them with information it has on your build.

By default, it will:

create a synthetic pyproject.toml file from its own metadata
use setuptools to package the module
first create a source distribution and then use that to build a wheel (instead of building a wheel directly)

While this should be sufficient for most projects, sometimes you need a little customization.

Customizing the `pyproject.toml` and other build files

If you’re happy to use a PEP-518-compliant pyproject.toml to describe how to package your published project, but would like some customization, you can amend or override the pyproject task with your own metadata.

You can also include additional files in the packaging process by adding them to buildFiles. You can then reference these in your pyproject.toml file.

The following example shows how to override the packaging process by providing a custom setup.py file.

build.mill (download, browse)

import mill._, pythonlib._

object `package` extends RootModule with PythonModule with PublishModule {

  def publishMeta = PublishMeta(
    name = "testpackage",
    description = "an example package",
    requiresPython = ">= 3.12",
    license = License.MIT,
    authors = Seq(Developer("John Doe", "jdoe@example.org"))
  )

  def publishVersion = "0.0.3"

  // you could also reference an existing setup.py file directly, e.g.
  // `def setup = Task.Source { millSourcePath / "setup.py" }`
  def setup = Task {
    val str =
      s"""#from setuptools import setup
          #
          #print("hello from custom setup.py!")
          #
          ## empty setup, defers to using values in pyproject.toml
          #setup()
          #""".stripMargin('#')
    os.write(Task.dest / "setup.py", str)
    PathRef(Task.dest / "setup.py")
  }

  override def buildFiles = Task {
    super.buildFiles() ++ Map("setup.py" -> setup())
  }

}

> mill sdist
...
hello from custom setup.py!
...

Changing the packaging process entirely

In case customizing of pyproject is too cumbersome, or you cannot use it for some reason, you can always override the sdist and wheel tasks with your own packaging implementation. Publishing with __.publish will still work as usual.