Java Packaging & Publishing

This page will discuss common topics around packaging and publishing your Java projects for others to use

Customizing the Assembly

build.mill (download, browse)
package build
import mill._, javalib._
import mill.javalib.Assembly._

object foo extends JavaModule {
  def moduleDeps = Seq(bar)
  def assemblyRules = Seq(
    // all application.conf files will be concatenated into single file
    Rule.Append("application.conf"),
    // all *.conf files will be concatenated into single file
    Rule.AppendPattern(".*\\.conf"),
    // all *.temp files will be excluded from a final jar
    Rule.ExcludePattern(".*\\.temp"),
    // the `shapeless` package will be relocated under the `shade` package
    Rule.Relocate("shapeless.**", "shade.shapless.@1")
  )
}

object bar extends JavaModule {}

When you make a runnable jar of your project with assembly command, you may want to exclude some files from a final jar (like signature files, and manifest files from library jars), and merge duplicated files (for instance reference.conf files from library dependencies).

By default mill excludes all *.sf, *.dsa, *.rsa, and META-INF/MANIFEST.MF files from assembly, and concatenates all reference.conf files. You can also define your own merge/exclude rules.

> ./mill foo.assembly

> unzip -p ./out/foo/assembly.dest/out.jar application.conf || true
Bar Application Conf
Foo Application Conf

> java -jar ./out/foo/assembly.dest/out.jar
Loaded application.conf from resources:...
...Foo Application Conf
...Bar Application Conf

Publishing Locally

build.mill (download, browse)
package build
import mill._, javalib._, publish._

object foo extends JavaModule with PublishModule {
  def publishVersion = "0.0.1"

  def pomSettings = PomSettings(
    description = "Hello",
    organization = "com.lihaoyi",
    url = "https://github.com/lihaoyi/example",
    licenses = Seq(License.MIT),
    versionControl = VersionControl.github("lihaoyi", "example"),
    developers = Seq(Developer("lihaoyi", "Li Haoyi", "https://github.com/lihaoyi"))
  )
}

This is an example JavaModule with added publishing capabilities via PublishModule. This requires that you define an additional publishVersion and pomSettings with the relevant metadata, and provides the .publishLocal and publishSigned tasks for publishing locally to the machine or to the central maven repository

> mill foo.publishLocal
Publishing Artifact(com.lihaoyi,foo,0.0.1) to ivy repo...

publishLocal publishes the artifacts to the ~/.ivy2/local folder on your machine, allowing them to be resolved by other projects and build tools. This is useful as a lightweight way of testing out the published artifacts, without the setup overhead and long latencies of publishing artifacts globally accessible to anyone in the world.

Checking API compatibility

Mill provides the ability to check API changes with the Revapi analysis and change tracking tool.

build.mill (download, browse)
package build
import mill._, javalib._, publish._, revapi._

object bar extends JavaModule with RevapiModule {
  def publishVersion = "0.0.1"

  def pomSettings = PomSettings(
    description = "Hello",
    organization = "com.lihaoyi",
    url = "https://github.com/lihaoyi/example",
    licenses = Seq(License.MIT),
    versionControl = VersionControl.github("lihaoyi", "example"),
    developers = Seq(Developer("lihaoyi", "Li Haoyi", "https://github.com/lihaoyi"))
  )

  override def revapiConfigFiles: T[Seq[PathRef]] =
    // add Revapi config JSON file(s)
    Task.Sources(millSourcePath / "conf/revapi.json")

  override def revapiClasspath: T[Agg[PathRef]] = T {
    // add folder containing logback.xml
    super.revapiClasspath() ++ Seq(PathRef(millSourcePath / "conf"))
  }
}

This example uses the revapi task, provided by the RevapiModule, to run an analysis on old and new archives of a module to identify incompatibilities.

For demonstration purposes, an archive, to compare against, is published locally. In real usage, the old version would be downloaded from the publish repository.
> mill bar.publishLocal
Publishing Artifact(com.lihaoyi,bar,0.0.1) to ivy repo...

> cp dev/src/Visibility.java bar/src/Visibility.java

> mill bar.revapi
Starting analysis
Analysis results
----------------
old: field Visibility.SuperClass.f @ Visibility.SubClass
new: <none>
java.field.removed: Field removed from class.
... BREAKING
old: field Visibility.f
new: field Visibility.f
java.field.visibilityReduced: Visibility was reduced from 'public' to 'protected'.
... BREAKING
Analysis took ...ms.
The revapi task does not fail if incompatibilities are reported. You should fix these, and verify by re-running revapi, before a release.

The revapi task returns the path to a directory that can be used to resolve the relative path to any extension configuration output.

[
  {
    "extension": "revapi.reporter.text",
    "configuration": {
      "minSeverity": "BREAKING",
      "output": "report.txt"
    }
  }
]

Publishing to Sonatype Maven Central

Once you’ve mixed in PublishModule, apart from publishing locally, you can also publish your project’s modules to maven central

GPG

If you’ve never created a keypair before that can be used to sign your artifacts you’ll need to do this. Sonatype’s GPG Documentation has the instructions on how to do this

Publishing Secrets

Mill uses the following environment variables as a way to pass the necessary secrets for publishing:

# The LHS and RHS of the User Token, accessible through the sonatype
# website `Profile` / `User Token` / `Access User Token`
export MILL_SONATYPE_USERNAME=...
export MILL_SONATYPE_PASSWORD=...

# The base-64 encoded PGP key, which can be encoded in the following way
# for each OS:
#
# MacOS or FreeBSD
# gpg --export-secret-key -a $LONG_ID | base64
#
# Ubuntu (assuming GNU base64)
# gpg --export-secret-key -a $LONG_ID | base64 -w0
#
# Arch
# gpg --export-secret-key -a $LONG_ID | base64 | sed -z 's;\n;;g'
#
# Windows
# gpg --export-secret-key -a %LONG_ID% | openssl base64
export MILL_PGP_SECRET_BASE64=...

# The passphrase associated with your PGP key
export MILL_PGP_PASSPHRASE=...

Publishing

You can publish all eligible modules in your Mill project using the default task of the External Module mill.scalalib.PublishModule:

mill mill.scalalib.PublishModule/

You can also specify individual modules you want to publish via a selector:

mill mill.scalalib.PublishModule/ foo.publishArtifacts

The default URL for publishing to sonatype’s Maven Central is oss.sonatype.org. Newer projects registered on sonatype may need to publish using s01.oss.sonatype.org. In that case, you can pass in a --sonatypeUri:

mill mill.scalalib.PublishModule/ \
        --sonatypeUri https://s01.oss.sonatype.org/service/local

This also allows you to publish to your own internal corporate sonatype deployment, by passing in --sonatypeUri example.company.com instead.

Since Feb. 2021 any new Sonatype accounts have been created on s01.oss.sonatype.org, so you’ll want to ensure you set the relevant URIs to match.

The symptom of using the "wrong" URL for publishing is typically a 403 error code, in response to the publish request.

Typically

Publishing Using Github Actions

To publish on Github Actions, you can use something like this:

# .github/workflows/publish-artifacts.yml
name: Publish Artifacts
on:
  push:
    tags:
      - '**'
  workflow_dispatch:
jobs:
  publish-artifacts:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-java@v3
        with:
          distribution: 'temurin'
          java-version: '17'
      - run: ./mill mill.scalalib.PublishModule/
        env:
          MILL_PGP_PASSPHRASE: ${{ secrets.MILL_PGP_PASSPHRASE }}
          MILL_PGP_SECRET_BASE64: ${{ secrets.MILL_PGP_SECRET_BASE64 }}
          MILL_SONATYPE_PASSWORD: ${{ secrets.MILL_SONATYPE_PASSWORD }}
          MILL_SONATYPE_USERNAME: ${{ secrets.MILL_SONATYPE_USERNAME }}

Where MILL_PGP_PASSPHRASE, MILL_PGP_SECRET_BASE64, MILL_SONATYPE_PASSWORD, and MILL_SONATYPE_USERNAME configured for the repository’s or organization’s Github Actions workflows. See Using Secrets in Github Actions for more details.

Non-Staging Releases (classic Maven uploads)

If the site does not support staging releases as oss.sonatype.org and s01.oss.sonatype.org do (for example, a self-hosted OSS nexus site), you can pass in the --stagingRelease false option to simply upload release artifacts to corresponding maven path under sonatypeUri instead of staging path.

mill mill.scalalib.PublishModule/ \
        foo.publishArtifacts \
        lihaoyi:$SONATYPE_PASSWORD \
        --sonatypeUri http://example.company.com/release \
        --stagingRelease false

Publishing to other repositories

While Sonatype Maven Central is the default publish repository for JVM ecosystem projects, there are also others that you can use. Mill supports these largely through contrib plugins:

This example illustrates how to use Mill to generate a runtime image using the jlink tool. Starting with JDK 9, jlink bundles Java app code with a stripped-down version of the JVM.

build.mill (download, browse)
package build
import mill._, javalib._
import mill.javalib.Assembly._
import mill.scalalib.JlinkModule

object foo extends JavaModule with JlinkModule {
  def jlinkModuleName: T[String] = T { "foo" }
  def jlinkModuleVersion: T[Option[String]] = T { Option("1.0") }
  def jlinkCompressLevel: T[String] = T { "2" }
}

Most of the work is done by the trait JlinkModule in two steps:

1.0. it uses the jmod tool to create a jlink.jmod file for the main Java module. The main Java module is typically the module containing the mainClass.

If your build file doesn’t explicitly specify a mainClass, JlinkModule will infer it from JavaModule, which is its parent trait. See Specifying the Main Class to learn more on how to influence the inference process. You can explicitly specify a mainClass like so in your build file:

def mainClass: T[Option[String]] = { Option("com.foo.app.Main") }

2.0. it then uses the jlink tool, to link the previously created jlink.jmod with a runtime image.

With respect to the jlinkCompressLevel option, on recent builds of OpenJDK and its descendants, jlink will accept [0, 1, 2] but it will issue a deprecation warning. Valid values on OpenJDK range between: ["zip-0" - "zip-9"].

The version of jlink that ships with the Oracle JDK will only accept [0, 1, 2] as valid values for compression, with 0 being "no compression" and 2 being "ZIP compression".

To use a specific JDK, first set your JAVA_HOME environment variable prior to running the build.

export JAVA_HOME=/Users/mac/.sdkman/candidates/java/17.0.9-oracle/

> mill foo.jlinkAppImage

> mill show foo.jlinkAppImage
".../out/foo/jlinkAppImage.dest/jlink-runtime"

> ./out/foo/jlinkAppImage.dest/jlink-runtime/bin/jlink
... foo.Bar main
INFO: Hello World!

Java Installers using jpackage

This example illustrates how to use Mill to generate a native package/installer using the jpackage tool.

build.mill (download, browse)
package build
import mill._, javalib._
import mill.javalib.Assembly._
import mill.scalalib.JpackageModule

object foo extends JavaModule with JpackageModule {
  def jpackageType = "app-image"

  def assemblyRules = Seq(
    // all application.conf files will be concatenated into single file
    Rule.Append("application.conf"),
    // all *.conf files will be concatenated into single file
    Rule.AppendPattern(".*\\.conf")
  )
}

JPMS (Java Platform Module System) is a modern distribution format that was designed to avoid several of the shortcomings of the ubiquitous JAR format, especially "JAR Hell".

A defining characteristic of module-based Java applications based on the JPMS format is that a module-info.java must be defined at the root of the module’s source file hierarchy. The module-info.java must explicitly list modules that it depends on, and also list packages that it exports, to make the integrity of these relationships easy to verify, both at compile-time and run-time.

Starting with version 14, the JDK ships with the jpackage tool which can assemble any module-based Java application into a native package/installer.

The above build file expects the following project layout:

build.mill
foo/
    src/
        Foo.java
        Bar.java

    module-info.java

The build defines a foo module that uses the trait JpackageModule.

The term Module is also used in Mill to refer to traits. This is not to be confused with Java app code structured as modules according to the JPMS format.

The JpackageModule trait will infer most of the options needed to assemble a native package/installer, but you can still customize its output. In our example, we specified:

def jpackageType = "pkg"

This tells jpackage to generate a .pkg, which is the native installer format on macOS. Valid values on macOS are: dmg, pkg and app-image.

jpackage doesn’t not support cross-targeting. Cross-targeting in this context means the jpackage binary shipped with a macOS JDK cannot be used to produce a native installer for another OS like Windows or Linux.
> mill foo.assembly

> mill show foo.assembly
".../out/foo/assembly.dest/out.jar"

> java -jar ./out/foo/assembly.dest/out.jar
INFO: Loaded application.conf from resources: Foo Application Conf
INFO: Hello World application started successfully

> mill foo.jpackageAppImage

> mill show foo.jpackageAppImage
".../out/foo/jpackageAppImage.dest/image"

On macOS, jpackageType accepts 3 values: "dmg" or "pkg" or "app-image" (default).

Setting def jpackageType = "dmg" will produce:

ls -l ./out/foo/jpackageAppImage.dest/image
... foo-1.0.dmg

Setting def jpackageType = "pkg" will produce:

ls -l ./out/foo/jpackageAppImage.dest/image
... foo-1.0.pkg

Setting def jpackageType = "app-image" will produce:

ls -l ./out/foo/jpackageAppImage.dest/image
... foo.app/
./out/foo/jpackageAppImage.dest/image/foo.app/Contents/MacOS/foo
... foo.Foo readConf
INFO: Loaded application.conf from resources: Foo Application Conf
... foo.Bar ...
INFO: Hello World application started successfully