Extending the build

Every chapter so far drove the stock pipeline: a layout auto-detects your modules, the default assembler wires the conventional compile/jar/test flow, and you configure it by choosing among the options it offers. This chapter is for the build that needs something the stock pipeline does not model - a preprocessing pass, a code-generation step, a bespoke packaging step, an unusual dependency wiring.

Almost always, the answer is a plugin: a build module, named in one line of a properties file, that joins a module of the stock build and adds to it, or runs once over everything the build produced. A plugin of every hook point adds to what the stock build produces, with one exception: a plugin of binary/transform rewrites the module's compiled classes, as a bytecode enhancer does. A build that must change anything else - one that redirects what the compiler reads, or wires a step between two stock ones - is an entry point of its own, and that comes last.

Adding plugins to the stock build

The plugins for the tools the earlier chapters describe - Checkstyle, the code generators, Kotlin, JReleaser and the others - are official plugins, and the build finds them by itself: each runs where a configuration folder holds its <name>.properties, or the tool's own conventional file, and a language wherever its sources are. Every one is switched off by jenesis.<part>.<tool>=false, named after the part of the build it belongs to, as jenesis.check.pmd=false or jenesis.language.kotlin=false.

Any other plugin is named by the project in jenesis.plugins.properties, beside jenesis.properties at its root, one per line:

greeting+binary/generated=./plugin
signing+artifact=demo.signing

The key is <name>+<hook point>. The name is the plugin's own, and holds neither / nor +. The hook point says where in the build the plugin joins, as the next section describes; a key without +<hook point> adds the plugin to the module build itself, and an unknown hook point is refused. The value says where the plugin comes from:

Value The plugin is
./<folder> a folder of the project, compiled from source on every build
<module name> resolved as module/<module name> whatever the project's layout: from the local module repository (~/.jenesis) first, then from the module repositories jenesis.module.uri names, the Jenesis Module Index by default
module/<module name> the same, spelled as a coordinate
maven/<groupId>/<artifactId> resolved from a Maven repository with the closure its POM declares

A coordinate is resolved by the repository its first segment names, and by that repository's resolver, as a dependency is anywhere else in the build. Its version comes from the pins unless the coordinate carries one, as maven/demo.plugin/demo.plugin/1.0.0. A value that is none of these, such as a path starting with ../, fails the build.

Any of them may end in @<provider>, which selects the provider annotated @BuildModuleName("<provider>") when the plugin provides several: signing+artifact=demo.signing@jarsigner. Without it, the plugin must provide exactly one unannotated provider.

A plugin is loaded as jpx runs a program, with the JDK's own class loaders. One named by a Maven coordinate loads with its whole closure on a class path of its own. One named by its module loads every jar that declares a module name into a module layer of the plugin's own, and any other jar on a class path beneath that layer, which its automatic modules read. Under -Djenesis.project.layout=modular every jar of such a plugin loads as a module, and a plugin named by a Maven coordinate is refused.

The file names its plugins under names of their own: a plugin it declares resolves and pins in the group plugin.<name>, and runs apart from the official plugins at every hook point, so it runs beside an official plugin of the same name and neither touches the other. Everything the project writes for it - the line, plugin-<name>.properties, its arguments and its selector - spells the name as the line writes it.

Hook points

A hook point is a place in the build graph that Jenesis leaves open for plugins. It is named by its path, as the build log shows it - binary/generated is the source generation of a module's binary build, stage/transform a place in stage - and it decides three things about a plugin: when it runs, what it is handed as input, and where what it writes goes. A plugin never inspects the build around it: it reads the folders its hook point hands it, and what it writes travels onwards by the same folder conventions the stock steps follow, so a sources/ tree written in binary/generated is compiled, and a maven/ folder written in stage/transform joins the staged Maven tree. Every hook point keeps its plugins in a namespace of its own - custom/, transform/, inspect/ or plugin/ - beside the stock steps and the official plugins, so a plugin may take any name. What a plugin writes is added to what the stock steps write, at every hook point but binary/transform, whose plugins rewrite the compiled classes.

Hook points come in two kinds. A module hook point runs a plugin once in every module of the project that configures it; a project hook point runs it once for the whole project. preprocess runs before any module is built and the other project hook points after every module is, so the module hook points sit between them; each table lists its hook points in the order the build reaches them.

A plugin of a module hook point runs as <hook point>/custom/<name> within the phase of each module that holds its hook point, as the build log shows it under build/maven/compose/module/ or build/modules/compose/module/ - build-module-app/produce/assemble/binary/generated/custom/<name> in the module's build, say, or verify-module-app/check/custom/<name> in its verify, where a plugin of binary/validate runs as validate/custom/<name>. A plugin with no hook point runs as custom/<name>, in the module's build itself. The plugins of observe, test and language sit as plugin/<name> beside the official observers, frameworks and languages:

Module hook point When a plugin runs, and what it is handed
check In the module's verify phase, beside the stock source checks, handed the module's sources and manifests.
format In verify, beside the stock formatters, handed the same.
compliance In verify, beside the licence and vulnerability checks, handed the manifests and the resolved dependencies.
binary/generated Before compilation, beside the stock generators; a sources/ tree it writes is compiled with the module, and a resources/ tree, such as a service file, joins the module's jar.
binary/compiled Beside the stock compilers, handed what they compile; what it writes below classes/ joins the module's jar, and a class that two compilers or plugins both write fails the build, naming both.
binary/transform After the compilers, handed the module's compiled classes first and the module's inputs after them; what it writes below classes/ replaces the class of that name. Several plugins run in the order the file names them, each handed what the one before wrote.
binary/validate In verify, beside the bytecode checks, handed the compiled classes.
binary Within the module's compile toolchain, handed what the toolchain reads.
artifact In verify, handed the module's jar with its dependencies.
observed With the module's tests, beside the stock observers such as coverage.
observe Attached to the JVM the module's tests run in, by a plugin that provides an ObservabilityFactory; its report runs as observed/observations/plugin/<name>, handed the tests' output.
test The framework the module's tests run on, from a plugin that provides a TestFrameworkFactory, consulted before the built-in frameworks.
language A language of its own, from a plugin that provides a LanguageFactory: compiled before or after javac, and documented, wherever a module holds a file of its extensions.
documentation/generate In the module's document phase, beside the stock documentation generators.
documentation In document, within the documentation build, handed the compiled module.
(none) In the module build itself, handed everything the module reads.
package In the module's package phase, handed its jar, its dependencies and the stock images; what it writes into packages/ is staged in stage/packages/.

A plugin of a project hook point runs under the goal it belongs to, at the same path in every project: build/preprocess/custom/plugin.<name>, build/postprocess/transform/plugin.<name> and build/postprocess/inspect/plugin.<name>, stage/transform/plugin.<name> and stage/inspect/plugin.<name>, release/custom/plugin.<name>, and plugin/<name>:

Project hook point When a plugin runs, and what it is handed
preprocess Before any module is built, handed only what it binds. It hands the build nothing, so it can stop the build early but never feed it.
postprocess/transform After every module is built, handed every module's inventory, in the order the file names them; adds files to the modules or to the whole project.
postprocess/inspect After the transforms, handed the same with what they added; fails the build by throwing.
stage/transform After the stock staging, handed every staged tree; what it writes under a tree's name joins that tree.
stage/inspect After the transforms of stage, handed the staged trees with what they added.
release With release, beside the stock release steps, only where its plugin-<name>.properties is found; handed everything staged, the project's root and its resolved metadata.
plugin Only when plugin/<name> is named, handed only what it binds; nothing waits for it and it waits for nothing.

The module hook points run while the module is built, so everything after them sees what they add; postprocess still belongs to build, so stage, export, release, pin, dependencies, ide and Execute.java never run past a failed inspection. stage runs only when a goal needs it, and export, release and plugin only when they are asked for.

Plugins of a module

A plugin in a module hook point reads what the module it joins reads, and its output belongs to that module like any of the stock steps': a plugin in binary/generated that writes a sources/ tree has it compiled with the project's own sources, and a plugin in artifact reads the jars the build produced.

It runs in a module only where plugin-<name>.properties is found, looked up like any other configuration file - in build.jenesis/, with the profiles first - so the same line can serve every module of a project and still run only in those that configure it. The file's values are handed to the plugin when it is created:

# build.jenesis/plugin-greeting.properties
greeting=Hello from a generated source

The plugin's dependencies are pinned like any other, in the module's declaration, under the dependency group plugin.<name>:

/**
 * @jenesis.pin plugin.greeting/module/org.json 20260522 SHA-256/...
 */

A plugin found in the local export is pinned and checked like one from a repository: build it again and its checksum changes, so the build refuses it until -Djenesis.dependency.pin=versions with pin records what the export holds.

Everything a plugin resolves itself is pinned in a group <group>:<purpose> of the group it is handed - plugin.<name> for a plugin the file declares, the plugin's name for an official one. The tool it runs lies in <group>:tool - plugin.<name>:tool, or xjc:tool/maven/org.glassfish.jaxb/jaxb-xjc for the official xjc plugin - and a closure that must not share versions with it in a group of its own, as kotlin:dokka for the Kotlin plugin's Dokka. Every pin a plugin needs therefore sorts under its group, and the build fails a plugin that resolves in a group of another name. A .properties file escapes the colon in a key, so pin writes such a pin there as xjc\:tool/maven/….

-Djenesis.plugin.<name>=false leaves out a plugin the file names, without editing the file, and -Djenesis.project.plugins=false leaves out every plugin the file names. The setting is a plugin's own: an official plugin of the same name is switched by jenesis.<part>.<tool> alone, so the two never share a setting.

A plugin hands the tests a file the same way: in the test module's binary/compiled, it writes the file below classes/, where only the tests' jar holds it, and the tests read it as a resource. Its plugin-<name>.properties sits where the test module's configuration is looked up, as src/test/build.jenesis/ in a pom.xml project. A framework that wants a file path in a system property instead gets one from a LauncherSessionListener of the tests, a JUnit service that copies the resource to a temporary file and sets the property before the first test runs.

A plugin in package can wrap or sign one of the images packaging.properties asks Jenesis to build - a runtime image, a jpackage image, a launcher - into a package of its own: an AppImage, an installer built with other tools, a distribution zip. A package whose name another packager, or jpackage, writes already fails the build rather than replacing it.

Handing a plugin files of the project

A plugin reads only what the build hands it, so a file of the project reaches it as an input. Among a plugin's values, a key starting with @ binds a file or folder instead of naming a value:

# build.jenesis/plugin-greeting.properties
greeting=Hello from a generated source
@templates=templates
@templates/legal/HEADER.txt=legal/HEADER.txt

@<input>=<path> binds the path into an input named <input>, which the plugin reads as the folder ../inputs/<input>. A target after the input's name, @<input>/<target>, places what is bound at that path inside the input rather than at its root, and one input takes one key per target, so it can gather several files and folders. A single file keeps its name unless a target renames it. Editing a bound file runs the plugin again. A value whose key really starts with @ is written with two: @@name=value hands the plugin @name=value.

In a module's plugin-<name>.properties, a path is resolved against the module's own folder, the one holding its module-info.java or pom.xml. A folder has to stay within the project, symbolic links included, and a plugin folder named as ./<folder> must lie within the project as well. A file may lie outside it, so that several projects share one configuration, and a path written as @<key> names one through the setting -Djenesis.variable.<key>. A folder that leaves the project, a path that does not exist, or one that shares a target with another binding of the same input fails the build.

A plugin runs code the project chooses - compiled from its own sources, or resolved by a module name it names - and every Jenesis that builds the project runs it, the installed jenesis included. Before you build a project whose plugins you have not reviewed, run the build in a container with -Djenesis.project.docker=true: the plugins then run inside the container and never on your machine, and the project cannot switch Docker off (see What runs on the host).

Plugins for the whole project

A plugin in a project hook point runs once for the whole project. The line in jenesis.plugins.properties switches it on by itself, as no plugin-<name>.properties applies to it:

# jenesis.plugins.properties
licence+preprocess=./licence
notice+postprocess/transform=./notice
audit+postprocess/inspect=./audit
checksums+stage/transform=./checksums
complete+stage/inspect=./complete
publish+export=./publish
announce+release=./announce
lines+plugin=./lines

Their names take no /, . or +, and a name used for a module hook point cannot also name one of them.

A preprocessor checks the project before anything is compiled: a licence header, a forbidden file, a policy on what the repository holds. A file every module needs comes from a plugin in binary/generated instead.

A transform adds files to the modules - a notice, a report, a signature of your own - and an inspection checks the result and fails the build when it is wrong. Transforms run in the order the file names them, each seeing what the ones before it added, and the inspections run after all of them.

A transform of stage writes into a folder named after a staged tree - maven/, modular/, packages/, project/ and the others under target/stage/ - and what it writes joins that tree where it stands, so export, release and their plugins take it like anything else staged: checksums or signatures beside every file, repository metadata, a manifest of what ships. A file the tree holds already fails the build rather than being replaced. An inspection of stage then checks the staged trees with what was added, and nothing is exported or released past one that fails.

A releaser delivers what was staged - to an internal repository, a bucket, a registry, a release page. Naming release/custom runs the releasers alone, without the steps Jenesis runs there itself. Such a step writes outside the build, so it overrides shouldRun to run every time it is selected, as the stock release steps do, rather than only when what it reads has changed. export has no hook point: it only installs what was staged into this machine's local repositories, and what it installs is shaped by the plugins of stage.

A plugin under plugin belongs to no goal and starts at once, as java build/jenesis/Make.java plugin/lines: a report over the project's own files, a task run by hand. What needs the built or staged result belongs in one of the hook points that follow them.

Configuring them

A plugin of the whole project reads its values from jenesis.plugins.arguments.properties beside jenesis.plugins.properties, one line per value as <plugin>.<key>, and from nowhere else - never from the command line:

# jenesis.plugins.arguments.properties
notice.holder=Example Corp.
notice.@legal=legal

The plugin notice then receives holder=Example Corp., and @legal binds the project's legal/ folder as its input, resolved against the project root. A line that names no declared plugin fails the build. A profile brings values of its own in jenesis.plugins.arguments-<profile>.properties, which win over the file without a profile, and of two active profiles the one named first wins. That is why the plugin files are named with dots: in a file name, a dash after a name always introduces a profile, as in jenesis-<profile>.properties.

Since the plugins in build run with every build, one that takes long is best switched off in jenesis.properties with jenesis.plugin.<name>=false and switched back on in the profile that ships. -Djenesis.project.plugins=false leaves out every plugin at once, those of the module hook points included.

A plugin of release

A plugin of release is the exception. It runs only where a configuration folder holds its plugin-<name>.properties, as a plugin of a module does, and it takes its values and binds its inputs from that file rather than from jenesis.plugins.arguments.properties, which refuses a line for it. Placed in the folder of a profile, such as build.jenesis/release/, the file keeps every build but a release from running the plugin.

Beside everything staged, such a plugin is handed two files that tell it where the project lies and what it is:

File Holds
root.properties root, the project's root folder, and stage.<tree>, the folder of each staged tree, such as stage.maven.
metadata.properties The project's metadata as the build resolved it, the version -Djenesis.project.version sets included.

A release tool runs in the project's checkout, which is why this is the one place a plugin is told where the project lies.

What a transform and an inspection see, and what they may add

Each of them is handed the inventory of every module: its jar, sources and documentation, its POM, and every dependency it resolved, each with the jar it resolved to. A transform adds to a module by writing files into its own output and naming them in an inventory.properties there, under the prefix the inventory gives the module (its build identity, such as module-sources):

module-sources.attachment.notice=notices/module-sources/NOTICE.txt

The key after the prefix is any key the module's inventory knows, read by the stages as the module's own:

  • <module>.attachment.<classifier> is staged beside the module's jar under that classifier: <artifact>-<version>-<classifier>.<extension> in the Maven tree, <module>-<classifier>.<extension> in the modular tree.
  • <module>.report.<name> is staged with the module's reports under stage/reports/.

What a transform adds is its author's responsibility. A module the build does not have fails the build, and so does a key two transforms add, or an attachment whose file name the build stages already.

What belongs to no module - an aggregated report, a site, a distribution of every module - a transform writes into a project/ folder of its own output, laid out as it sees fit. stage copies it as it stands into target/stage/project/output/, where export and releasers find it with the rest, and a file two transforms place at the same path fails the build. A transform that brings its own SBOM, say, attaches it as cyclonedx once -Djenesis.sbom.cyclonedx=false has switched off the stock one. An inspection reads the same inventories plus what the transforms added, and fails the build by throwing. It writes only into its own output: an inspection that changes a file it was handed fails the build as well.

Pinning them

These plugins belong to no module, so their pins live beside the file that names them, in jenesis.plugins.pin.properties, one line for each module in each plugin's closure:

plugin.audit/module/build.jenesis=<version> SHA-256/...
plugin.notice/module/build.jenesis=<version> SHA-256/...

pin writes the file and pins every plugin of the whole project the file names, including one a setting switches off, so a plugin that only a profile switches on is pinned all the same. pin runs last, after a dry release, so it also writes the tool a releaser resolves, such as the official JReleaser plugin's jreleaser\:tool/maven/org.jreleaser/jreleaser-tool-provider=<version> SHA-256/.... A releaser reads jenesis.release.publish from the environment it is handed and publishes nothing without it. Where a releaser does not run - its file sits in a profile pin was not given - pin keeps the lines that pin its tool as they stand. The plugins run with everything that builds, pin among it, so an inspection that fails stops pin too. This pins without running any plugin:

java -Djenesis.project.plugins=false build/jenesis/Make.java pin

Writing a build step

A build step is the unit of work introduced in Core concepts: it reads one or more input folders and writes into one fresh output folder. When you write your own, that is the shape you implement - a function handed its inputs and an output folder to fill:

CompletionStage<BuildStepResult> apply(Executor executor,
                                       BuildStepContext context,
                                       SequencedMap<String, BuildStepArgument> arguments);

The context gives you four folder slots:

  • next - the folder this run writes into. It is created fresh every time; your step writes here and nowhere else.
  • previous - the same step's output from the prior run, or null on a first run. You may read it to hard-link or copy unchanged files instead of regenerating them, but never write into it.
  • supplement - scratch space for intermediate files you do not want to publish in next.
  • persistent - state your step keeps for its own next run, such as an incremental compiler's caches. It is the folder the last successful run left, handed over whenever previous is, and empty otherwise: on a first run, after a failed run, when the step changed, or when the build cache supplied the last output. It is not part of the output, so dependents never see it, it is not cached and it does not change the step's key. Your step must write the same output with an empty folder as with a full one.

The arguments map carries one entry per predecessor you wired in. Each exposes the folder to read (argument.folder()) and a per-file change status - ADDED, ALTERED, REMOVED, or RETAINED - computed against the previous run. The default behaviour re-runs your step whenever any input changed; override shouldRun(...) if you want finer control.

A step can also say that its output is not worth sending to a cache server. Override shouldCacheRemotely() to return false when the output is large against what it costs to produce, or when it is re-derivable from something the machine already has. The output is still cached locally, so a rebuild on this machine still skips the work. See the build cache.

A step is expected to write the same bytes from the same inputs. A build checked with jenesis.project.reproduce names every step that does not, as described in Checking that a build reproduces.

Treat a step as a pure function of its input folders and its identity, its name in the graph and its serialised state: read from the argument folders, write to next, reach outside neither. That is what makes its output cacheable and safe to share between builds - the incremental engine relies on it.

Talk through folders, not step names

Steps compose by file and folder conventions, not by knowing who wired them. A step discovers what to read by looking for well-known paths inside each input folder - sources/ for Java sources, classes/ for compiled output, artifacts/ for jars - and writes its output under names its consumers look up the same way. Do not inspect the names of your predecessors to guess which input is which; read the folders. This is what lets you splice a custom step between two stock ones without either noticing.

The serialised-state rule

Core concepts flagged that a step re-runs when its serialised state changes, and left the details here. This is the one rule a custom step must get right.

Jenesis content-hashes each step's serialised form and folds that hash into its cache key. So a step re-runs when its inputs change or when its own configuration changes, and "configuration" means the values of its serialised fields. The practical rule follows directly:

Put every knob that should trigger a rebuild into a serialised field. A greeting to substitute, a flag, a target version - if changing it should re-run the step, it has to be a (non-transient) field, because that field's value is exactly what the cache hashes.

The flip side is the trap. Change detection keys off the step's serialised state, not its bytecode. If you change a step's logic - rewrite the body of its apply, fix a bug in a helper - without changing any serialised field, its hash is identical and Jenesis reuses the stale output. Example: a preprocess step whose substitution string lives in a field re-runs the moment you edit that string; but if you instead hard-code the string in the method body and edit it there, nothing re-runs until an input changes. Keep behaviour-affecting values in fields, or bump the step's serialVersionUID (below) when you have changed only code.

State must be serialisable

Because the step is serialised to be hashed, all of its state must be serialisable. This is checked on the first run, at hash time, not lazily. Two things make the common cases work:

  • A step written as a lambda serialises with what it captures, because BuildStep is itself Serializable. A lambda that uses only its parameters captures nothing; one that creates an anonymous class inside an instance method captures that instance, which then has to be Serializable too.
  • A lambda field serialises only if its declared type does. Declare the field as a serialisable functional interface, or cast the lambda to Function<…> & Serializable where you store it, and a lambda that closes over, say, a Path serialises cleanly. The stock steps do this at their constructors, which is why you can hand them a plain lambda.
  • A Path field is hashed by its string form, even though the JDK's Path is not itself Serializable. So Path-typed configuration is first-class.

Genuinely non-serialisable state - an open socket, a database handle, a live Context object - throws NotSerializableException at hash time, on the first run. That is deliberate: the error surfaces the bug immediately rather than silently breaking cache invalidation. If you see it, hold the serialisable description of the resource (a URL, a path, coordinates) as the field and open the resource inside apply, or mark truly incidental state transient so it never reaches the digest.

The hash also folds in the class's serialVersionUID, which is the lever for a code-only change. Without an explicit one, the JVM derives it from the class structure, so adding a field or changing a method signature already changes the hash - but editing a method body does not. Declare a serialVersionUID and bump it whenever you change a step's behaviour without touching its fields. With one declared, structural changes do not shift the hash on their own, so bumping it is your job.

Writing a plugin

A plugin is a named Java module that provides a BuildModuleFactory, which Jenesis discovers through that declaration alone:

module demo.plugin {
    requires build.jenesis;
    requires org.json;
    provides build.jenesis.BuildModuleFactory with demo.plugin.GreetingFactory;
}

The factory creates the BuildExecutorModule that adds the steps the plugin contributes, as a stock module does. It is handed the settings of the build, the values of its plugin-<name>.properties or its lines of jenesis.plugins.arguments.properties in the file's order, the Permits that bound the processes the build runs at once, its group - plugin.<name>, which everything it resolves lies under - and the build's repositories and resolvers:

public class GreetingFactory implements BuildModuleFactory {

    @Override
    public BuildExecutorModule create(Environment environment,
                                      SequencedMap<String, String> properties,
                                      Permits permits,
                                      String group,
                                      Map<String, Repository> repositories,
                                      Map<String, Resolver> resolvers) {
        return new GreetingModule(properties);
    }
}

A value arrives resolved: @<key> stands for the setting -Djenesis.variable.<key>, @<key>/<default> falls back to what follows the slash, and @@ writes a literal @. A value is never a path - a file the plugin reads is bound as an input, as Handing a plugin files of the project shows. A factory refuses a value it does not take, naming those it does, rather than dropping it.

The steps a plugin adds are handed their inputs as arguments. An input bound with @<input> arrives as the argument ../inputs/<input>, and a transform or an inspection finds each module's inventory as an inventory.properties in the folders of its arguments:

BuildStepArgument legal = arguments.get("../inputs/legal");

Each argument is keyed by its predecessor's path: argument.folder() is that predecessor's output, and argument.files() holds the checksum of each file in it. Beside the source folders, a plugin of a module hook point is handed the module's metadata.properties, with its version, name and the rest, and its module.properties, whose test key tells the tests of a pom.xml from its main code - so a step that belongs to one of them returns early on the other. The resolved dependencies arrive as a folder too, read with build.jenesis.step.Dependencies: Dependencies.paths(arguments, "main", "compile") lists the jars of one group and scope across every argument a step is handed, Dependencies.select(folder, "main", "compile") those of one folder, and Dependencies.all(folder) every one. What a step in binary/compiled writes as classes/ and as a manifest.mf is merged into the module's jar. A step in binary/transform finds the compiled classes in the first of its arguments, and what it writes below classes/ replaces the file of that name, while every file it does not write passes on as it was. A manifest.mf it writes replaces the one it was handed, so a plugin that writes one carries over the attributes that still hold, as Multi-Release while it keeps the versioned classes.

A plugin compiled from source lives in a project folder of its own, which carries an empty .jenesis.skip marker so the project's module discovery does not mistake it for a second project module. A published plugin is built and exported like any other module, and resolved by its module name or its Maven coordinate. Named by its module, it is found through its provides; on the class path a Maven coordinate loads it on, it is found through a META-INF/services entry for its factory, which its build writes from the provides when its configuration folder holds an empty classpath.properties, as A modular jar on the class path describes.

A plugin usually carries a different version of the Jenesis build API than the build running it. Each plugin is therefore loaded with class loaders of its own - into a ModuleLayer of its own where it loads as modules - and calls are bridged across the boundary, so the two copies never clash and a plugin may pin its own Jenesis version, as long as the API it uses lines up. A plugin named by its module or compiled from source is an explicit named module; its dependencies need not be, since a module layer admits automatic modules too.

A plugin compiled from source resolves its requires as a project's module-info.java does. A tool the module index does not serve, as one named by an Automatic-Module-Name alone, is required through an @jenesis.alias <module> <groupId>/<artifactId> line on the plugin's declaration, and @jenesis.exclude and @jenesis.pin lines apply to the plugin's dependencies as in a project. Every jar of the plugin's layer is a module of its own, so a jar that declares no module name at all, or two jars that share a package, cannot load: exclude it, or pin a version of it that names its module.

Observing the tests

A plugin of the hook point observe joins the JVM the tests run in, as a coverage agent does, and reports over what that run recorded. It provides an ObservabilityFactory rather than a BuildModuleFactory, is handed the same, and returns an Observation: the ObservabilityEngine that names the agent's coordinates and the JVM options of the test run, and the module that reads what the run left in its output.

public class LoadedFactory implements ObservabilityFactory {

    @Override
    public Observation create(Environment environment,
                              SequencedMap<String, String> properties,
                              Permits permits,
                              String group,
                              Map<String, Repository> repositories,
                              Map<String, Resolver> resolvers) {
        return new Observation(new ClassLoading(group),
                (executor, inherited) -> executor.addStep("loaded", new Loaded(prefix(properties)),
                        inherited.sequencedKeySet()));
    }
}

The engine is part of what decides whether the tests run again, so it holds its settings as serialisable fields, and a changed value runs the tests anew. It is named after the group the factory is handed, plugin.<name>, and its coordinates resolve with the test runtime in <group>:agent. The plugin is built before the tests, under observed/observers/plugin/<name>, and its report runs as observed/observations/plugin/<name>; it takes values but binds no files. Only a module that runs tests is observed.

Running the tests on a framework of your own

A plugin of the hook point test provides a TestFrameworkFactory, is handed the same, and returns a TestFramework: the runner the tests' JVM starts, the arguments it takes, and the modules that mark a test module as its own. A framework that writes a report can also return, from failed, the test methods that failed in a run, which jenesis.test.mode=failed and failedfirst run again the next time; one that returns nothing runs every test by class name. The frameworks a project declares are consulted first, in the order jenesis.plugins.properties names them, and the built-in JUnit Platform, JUnit 4 and TestNG after them, so the first framework a module of the tests marks runs them. A framework=<name> line in test.properties names a declared framework by its plugin name, as it names a built-in one.

A framework is part of what decides whether the tests run again, so it holds its settings as serialisable fields. It may delegate to a built-in one, as a framework that runs on the JUnit Platform but hands the tests a system property of its own does:

@Override
public Map<String, String> systemProperties() {
    Map<String, String> properties = new LinkedHashMap<>(platform.systemProperties());
    properties.put("demo.framework", name);
    return properties;
}

Compiling a language of your own

A plugin of the hook point language provides a LanguageFactory, is handed the same, and returns a Language: the file extensions it compiles, the module that compiles them, whether that runs before or after javac, and the module that documents them, or null. It needs no plugin-<name>.properties: it runs in every module that holds a file of its extensions, and takes its values from that file where one is found.

A language's Is handed, and writes
compiler before javac the module's sources and dependencies, and the classes of the compilers before it; javac compiles against the classes/ it writes.
compiler after javac the same, and the classes javac wrote.
documentation the module's sources; the javadoc/ it writes joins the module's documentation, at its root where the language is the module's only one and under javadoc/<name>/ beside others.

A file of the language's extensions is no resource, so it never reaches the jar. A pom.xml project compiles a folder beside src/main/java, such as src/main/kotlin, only where jenesis.maven.sources names it:

# jenesis.properties
jenesis.maven.sources=kotlin

A worked example: classes rewritten with Byte Buddy

A plugin is written against a library's own Java API. This one runs Byte Buddy's Plugin.Engine over the classes javac compiled, as Byte Buddy's Maven and Gradle plugins do, with two Byte Buddy plugins: ToStringPlugin, which Byte Buddy ships, and one written for the project. What it writes replaces the classes it was handed, so the jar, the tests and every later step see the rewritten ones.

The project names the plugin, the hook point it joins and the folder it is compiled from:

# jenesis.plugins.properties
enhance+binary/transform=./plugin

plugin/ holds the plugin module, with an empty .jenesis.skip beside its declaration. It requires the build API and Byte Buddy, and provides its factory:

module demo.plugin {
    requires build.jenesis;
    requires net.bytebuddy;
    provides build.jenesis.BuildModuleFactory with demo.plugin.EnhanceFactory;
}

The plugin runs in a module where plugin-enhance.properties is found:

# build.jenesis/plugin-enhance.properties
type=sample.Greeting
greeting=Hello from a method that Byte Buddy rewrote in a build plugin!

Its values reach the factory's create as a SequencedMap<String, String>, in the file's order, and the factory hands them to the EnhanceModule it returns. The module keeps what it needs, and its accept(BuildExecutor, SequencedMap<String, Path>) adds one step, a record of those two values, handed everything the hook point reads:

public EnhanceModule(SequencedMap<String, String> properties) {
    type = properties.getOrDefault("type", "sample.Greeting");
    greeting = properties.getOrDefault("greeting", "Hello from a rewritten method!");
}

@Override
public void accept(BuildExecutor executor, SequencedMap<String, Path> inherited) {
    executor.addStep("enhance", new Enhance(type, greeting), inherited.sequencedKeySet());
}

The step's apply(Executor, BuildStepContext, SequencedMap<String, BuildStepArgument>) collects the jars the module compiles against as Byte Buddy's class path, and runs the engine from the classes in its first argument into classes/ of its own output folder, context.next():

List<ClassFileLocator> classPath = new ArrayList<>();
for (BuildStepArgument argument : arguments.values()) {
    if (!argument.removed()) {
        for (Path jar : Dependencies.select(argument.folder(), "main", "compile")) {
            classPath.add(ClassFileLocator.ForJarFile.of(jar.toFile()));
        }
    }
}
try (ClassFileLocator locator = new ClassFileLocator.Compound(classPath)) {
    new Plugin.Engine.Default()
            .with(locator)
            .apply(arguments.firstEntry().getValue().folder().resolve(BuildStep.CLASSES).toFile(),
                    Files.createDirectories(context.next().resolve(BuildStep.CLASSES)).toFile(),
                    new Plugin.Factory.Simple(new ToStringPlugin()),
                    new Plugin.Factory.Simple(new Greeting(type, greeting)));
}
return CompletableFuture.completedStage(new BuildStepResult(true));

The class path is what lets Byte Buddy read the types a class refers to beyond the JDK. The module's own sample.Greeting carries @ToStringPlugin.Enhance, from a requires static net.bytebuddy; without the module's jars, Byte Buddy cannot resolve the annotation, and ToStringPlugin matches nothing. The second plugin, Greeting, is a net.bytebuddy.build.Plugin of the project's own that matches the type its properties name and rewrites its get() to return the configured text:

public boolean matches(TypeDescription target) {
    return target.getName().equals(type);
}

public DynamicType.Builder<?> apply(DynamicType.Builder<?> builder,
                                    TypeDescription typeDescription,
                                    ClassFileLocator classFileLocator) {
    return builder.method(named("get")).intercept(FixedValue.value(greeting));
}

The two values are the step's serialised state, so changing either in plugin-enhance.properties runs the step again, and so does a change to the classes it is handed. The application then prints what the rewritten classes return:

Hello from a method that Byte Buddy rewrote in a build plugin!
Greeting{audience=the build}

Byte Buddy is configured in code here rather than discovered. Its Maven and Gradle plugins run the Byte Buddy plugins they find in META-INF/net.bytebuddy/build.plugins or that the build file names, configured in XML or the build's DSL. The module above constructs the plugins it runs in Java instead, from the values of its own properties file, and runs nothing it did not name.

Byte Buddy resolves into the plugin's own module layer, and what the plugin rewrote refers to nothing but java.base, so it never reaches the running application. pin records the plugin's closure in the project's module-info.java, in the group plugin.enhance:

/**
 * @jenesis.pin plugin.enhance/module/build.jenesis <version> SHA-256/...
 * @jenesis.pin plugin.enhance/module/net.bytebuddy <version> SHA-256/...
 */

Forking a tool that cannot be a module

A tool that cannot be a module at all, as one with a class in the unnamed package, cannot load in the plugin's layer. The plugin's step forks it instead, and the project resolves it rather than the plugin. A module names it in a group of its own with @jenesis.plugin <group> maven/<groupId>/<artifactId> in its module-info.java, or a <!--jenesis.plugin <group> maven/<groupId>/<artifactId>--> comment in its pom.xml, and pins it as <group>/maven/<groupId>/<artifactId>. A module hook point such as binary/generated hands the jars over among the resolved dependencies, the input whose key ends in /dependencies/artifacts, where Dependencies.select(folder, "<group>", "plugin") lists them for the java -cp the step runs.

A step that forks java -cp <jars> <class or Launcher.java> <arguments> builds its handler with ProcessHandler.OfProcess.ofJavaMain() rather than ofJavaHome("bin/java"). Under jenesis.process.mode=daemon, which the build daemon sets and which is fork otherwise, its tool's main method then runs in a JVM kept warm for the next call with the same jars on its class path and the same options and environment, which compiles a .java launcher once. A jar is matched by the file rather than its path, so every module of a build that resolves the same tool shares that JVM. What one call prints is that call's output, a call that returns ends with 0, or with the int a main method declared to return one hands back, and one that calls System.exit ends with that code and takes its JVM with it. While the setting is not daemon, which also serves a tool whose static state must not outlive a call, every call runs in a JVM of its own, and a class's main method still ends the call as described: the code it returns is the code, and a thread the tool leaves running ends with the JVM. Under jenesis.process.mode=aot, which jenesis.make.mode=aot sets, that JVM starts from an ahead-of-time cache of the tool in .jenesis/aot/, trained on its first call. A command line of another shape - a module path, -jar, an argument file - is forked as it stands.

A handler marked shared(true), as a compiler's is, runs its calls side by side in one JVM in every mode, as long as jenesis.process.shared is not false: under fork and aot that JVM ends with the build, and under daemon it is kept for the next one. Each call's output is still its own, which the JVM routes by the thread that runs the call.

A tool whose main method calls System.exit on a call the build accepts takes its warm JVM with it every time. Run it through a driver instead: a class of your plugin with a public static int main(String[]) that calls the entry point the tool offers short of its main method and returns the code. Compile it against the tool with requires static, so your plugin's own dependencies do not include the tool, and put the folder that ProcessHandler.OfProcess.driver(<Driver>.class) writes ahead of the tool's jars on the class path. That folder holds the class files of the driver and the classes it nests alone, so the driver uses nothing else of your plugin, and it runs against the version of the tool the project pinned. A JVM runs one call at a time, because few tools allow more; a tool whose main method keeps its state per call - a compiler driven through its own API - asks for .shared(true) on the handler, and its calls then run side by side in one JVM, sharing its JIT, each printing only what it printed itself.

Hand such a step only the inputs it reads, executor.addStep(name, step, inputs) naming them among the keys of the map accept is handed, since each input is part of the step's key: a generator handed sources/ runs again on every edit. The keys are relative paths, as ../../../../../../../dependencies/artifacts, so pick an input by how its key ends.

Compiling native code

A native method is implemented in a library the module loads at run time, and the C side compiles against a JNI header that javac -h writes. A plugin at binary/compiled builds both with two steps:

  1. The build's own javac step, asked for the headers alone: new Javac(ProcessHandler.Factory.of()).headers(true). It compiles the module's sources as the module's compiler does, but hands on nothing but a headers/ folder, with one header for each class that declares a native method. The classes it compiles stay with the step, so they never join the jar a second time.
  2. A step of the plugin's own that forks the machine's C compiler, extending EnvironmentalProcessBuildStep. It puts the JDK's include/ folder, the folder for the operating system below it and headers/ on the include path. It writes the library below classes/, where it joins the module's jar.

The C sources reach the plugin as an input, bound with @c=<folder> in its plugin-<name>.properties, as in Handing a plugin files of the project. A library is built for the machine that builds it. A project that publishes one for several platforms builds each on its own machine and writes the library into a folder that its classifiers.properties names, instead of below classes/. Loading a library is a restricted method, so the application grants native access to the module that loads it, as Granting native access describes.

Writing an entry point of your own

A build that plugins cannot express - one that changes what the stock steps do, several builds compared, one staged build feeding the next, a graph with no project at all - is a program of its own, run as java build/Demo.java. Where it builds the project, it calls Make, which reads jenesis.properties, the profiles and the -Djenesis.* properties the JVM was started with, and returns what the build produced:

Make.Result staged = new Make("build.jenesis.Project").build("stage");
Path packages = staged.outputs().get("stage/packages");

Changing the stock build

An entry point that keeps the stock build but changes it builds the project from the settings, as Make does, and replaces its assembler - the callback that wires each module's compile/jar/test sub-graph - with one that wraps the stock InferredMultiProjectAssembler:

Environment environment = new Environment(Make.settings(Path.of(".")).keys());
InferredMultiProjectAssembler stock = InferredMultiProjectAssembler.ofEnvironment(environment);
Project project = Project.ofEnvironment(environment, Path.of("."))
        .assembler((descriptor, repositories, resolvers) -> stock
                .apply(descriptor, repositories, resolvers)
                .andThen("sign", (sub, inherited) -> sub.addStep("sign", (executor, context, arguments) -> {
                    // read the jars in each argument's folder, write their signatures into context.next()
                    return CompletableFuture.completedStage(new BuildStepResult(true));
                }, inherited.sequencedKeySet().stream()), AssemblyRequirement.of().after(AssemblyDescriptor.BUILD)));
System.exit(new Execution(project).execute(args));

andThen adds a phase to each module - here sign, registered as sign-<module> beside the stock build, verify and the others. Its AssemblyRequirement names what it waits for and is handed:

Requirement The phase is handed
after(phase) that phase of its own module, declared before it
upstream(phase) that phase of every module its module depends on, directly or not
tested(phase) that phase of every module that tests its module

andThen on a phase that is declared already extends it: the module's steps run after the phase's own, and the requirement adds to the phase's. A step that must run before the stock build, as a preprocessor of the sources, uses nest(AssemblyDescriptor.BUILD, "", build -> ...) instead: it hands over the stock phase as a module, and the module it returns adds its own step and then the stock one. Execution builds the project and runs the module that declares a main class, as build/jenesis/Execute.java does for the stock build.

The stock modules take additional steps and modules in code as well, through the same custom namespace a plugin is wired into:

InferredMultiProjectAssembler checked = stock.check(check -> check.custom("placeholders", (executor, context, arguments) -> {
    // fail when a source in any argument's folder still holds a ${ placeholder
    return CompletableFuture.completedStage(new BuildStepResult(true));
}));
  • custom(name, step) adds one step and custom(name, module) one module, after those added before; either refuses a name that is taken already.
  • custom(map) sets every added module at once, a SequencedMap<String, BuildExecutorModule> in the order they are wired.

The plugins of the whole project are a value of the project rather than of the assembler: plugins() answers the ProjectPlugins that jenesis.plugins.properties declared, and preprocess(name, step), transform(name, step), inspect(name, step), supplement(name, step) for stage/transform, audit(name, step) for stage/inspect, release(name, step) and goal(name, step) add one more, a step or a module, refusing a name that is taken already. Here ReleaseAudit is a BuildStep of the project that throws when a module's inventory lacks what every release must carry:

Project project = Project.ofEnvironment(environment, Path.of("."));
project = project.plugins(project.plugins().inspect("audit", new ReleaseAudit()));

A plugin that pin should pin also needs its resolution(), the part of an InternalModule or ExternalModule that resolves the plugin's closure without building it; ProjectPlugins takes those by name beside the plugins themselves.

Redirecting a module's inputs

An entry point can also change what the stock steps consume, because the module descriptor is immutable with a wither per property. Every reference accessor (sources, resources, manifests, dependencies, artifacts, content, coordinates, spdx) returns a SequencedSet<String>, so you can add or replace inputs in one line:

descriptor.sources("preprocess")   // stock compile reads the preprocess step's output, not sources/

That is the whole trick behind a preprocessing build: add a preprocess step that reads the module's sources/, rewrites it into its own output, then hand the stock assembler a descriptor whose sources() points at preprocess. javac, the jar step, and the tests all consume the transformed tree, and the rest of the build is untouched. Any pass that produces a sources/ tree - template expansion, code generation, licence-header stamping - fits the same shape.

Starting on the selected JDK

Such an entry point starts on whichever JDK runs it. To honour jenesis.toolchain.version, as Make.java does, ask Toolchain from build.jenesis: new Toolchain().home() answers the JDK the version selects, and launch(Demo.class, options, arguments) starts the entry point again on it and returns its exit code.

A file that names Project or the steps is compiled by the JDK's source launcher on every run, engine included. Compile it once when that matters - javac -d .jenesis/tool $(find build/ -name '*.java'), then java -cp .jenesis/tool build.Demo - and keep java build/Demo.java as the documented command.

Reusing the toolchain without Project

When the stock compile/jar/test flow fits but Project does not, call the convenience factory MavenProject.make (or ModularProject.make for a Java Module System project). It discovers the modules under a root, fills in sane defaults - a Maven Central repository, the right resolver, a digest - and leaves only the environment those defaults read their settings from and the assembler for you to supply:

Environment environment = new Environment(Make.settings(Path.of(".")).keys());
BuildExecutor root = BuildExecutor.of(Path.of("target"));
root.addModule("maven", MavenProject.make(environment,
        Path.of("."),
        (descriptor, repositories, resolvers) -> new InferredMultiProjectAssembler().apply(
                new ProjectModuleDescriptor(descriptor)            // the discovered module
                        .configuration(Path.of("."))               // its configuration folders
                        .pathPlacement(PathPlacement.CLASS_PATH),  // place dependencies on the class path
                repositories, resolvers)));
root.execute(args);

The descriptor runs the module's tests, attaches no sources or javadoc jar and pins leniently unless told otherwise; test(false), source(true), documentation(true) and pinning(Pinning.STRICT) change one of those each. A pom.xml module resolves its class paths as one graph, as Maven does: make reads jenesis.maven.unified from the environment it is handed, and a MavenProject built by its constructor resolves each class path on its own only after unified(false).

This is a middle ground: no layout, no goals, no Project, yet you did not wire every step by hand either. ModularProject.make is the modular counterpart; its convenience form builds pure modules (a modular jar, no generated POM). For full control - a custom repository, strict pinning, a different digest, or emitting a POM as well - switch to the longer make(...) overload that Project itself uses.

Wiring the graph by hand

When auto-detection is the wrong starting point entirely - a non-Java pipeline, code generation, a wildly custom graph - drop to the BuildExecutor primitives and build exactly the graph you want:

BuildExecutor root = BuildExecutor.of(Path.of("target"));
root.addSource("sources", Bind.asSources(), Path.of("sources"));
root.addStep("generate", new GenerateSource(), "sources"); // writes sources/sample/Generated.java
root.addStep("classes", new Javac(ProcessHandler.Factory.of()), "sources", "generate");
root.addStep("artifacts", new Jar(ProcessHandler.Factory.of(), Jar.Sort.CLASSES), "classes");
root.execute(args);

BuildExecutor.of(...) is the root and writes everything under target/. addSource binds a directory so changes to it invalidate downstream caches. addStep(name, step, predecessors…) chains a step whose arguments come from the named predecessors. execute runs the graph (or a selector's subtree), reusing cached outputs whose inputs are unchanged. The generate step above synthesises a Java source on the fly and Javac, which reads the sources/ of every predecessor, compiles it next to the hand-written ones. There is no phase lifecycle to fit into: a build is just steps wired to steps, and here you wire them yourself.

Running a build inside another program

An entry point of your own is still a process. A program that already runs - an IDE, a test harness, a server that builds what it serves - can run a build in its own JVM instead, through java.util.spi.ToolProvider, the JDK's interface for a tool that runs in-process the way javac and jlink do. build.jenesis publishes three, named after the commands they answer to:

Tool What it does What follows the settings
jenesis-make Builds the project Selectors, as on a command line
jenesis-exec Builds, then runs what it built The program's own arguments
jpx Runs a published program The target and jpx's options
StringWriter log = new StringWriter();
int code = ToolProvider.findFirst("jenesis-make").orElseThrow().run(
        new PrintWriter(log), new PrintWriter(log),
        "-Djenesis.project.version=1.0.0", "build");

The leading -Djenesis.* arguments configure that run and nothing else: they are never written to the JVM's properties, and the run never reads them from there, so two builds in one program can be configured differently and neither leaves anything behind. Everything the build prints arrives on the writers you passed, which the tool does not flush - they are yours, and an autoflushing PrintWriter already drains itself. An @<file> argument works here as on a command line.

A setting that would replace the process a build runs in cannot be honoured in-process, and is refused by name rather than ignored: jenesis.toolchain.version and jenesis.project.docker for all three, and jenesis.execute.docker for the program jenesis-exec runs. A -J option is refused as well, as the JDK's own tools refuse one: a tool runs in its caller's JVM and is handed no options for one, so the -J of Execute.java and of jpx is the commands' alone. The refusal arrives on err with a non-zero code rather than as an exception. jenesis-exec forks the program it runs, as its command does, so that program writes to the JVM's own streams while the build's output goes to the writers.

The tools are found by name when build.jenesis is a resolved module or a jar on the class path. Source mode registers no service, so a program there constructs new MakeTool(), new ExecuteTool() or new JpxTool() itself; the contract is the same.