Extending the build
Every chapter so far drove the stock pipeline: a layout auto-detects your modules, the default assembler wires the conventional compile/jar/test flow, and you configure it by choosing among the options it offers. This chapter is for the build that needs something the stock pipeline does not model - a preprocessing pass, a code-generation step, a bespoke packaging step, an unusual dependency wiring.
Almost always, the answer is a plugin: a build module, named in one line of a properties file, that joins a
module of the stock build and adds to it, or runs once over everything the build produced. A plugin of every
hook point adds to what the stock build produces, with one exception: a plugin of binary/ rewrites the
module's compiled classes, as a bytecode enhancer does. A build that must change anything else - one that
redirects what the compiler reads, or wires a step between two stock ones - is an entry point of its own, and that
comes last.
Adding plugins to the stock build
The plugins for the tools the earlier chapters describe - Checkstyle, the code generators, Kotlin, JReleaser and
the others - are official plugins, and the build finds them by itself: each runs where a configuration folder holds
its <name>., or the tool's own conventional file, and a language wherever its sources are. Every one
is switched off by jenesis., named after the part of the build it belongs to, as
jenesis. or jenesis..
Any other plugin is named by the project in jenesis., beside jenesis. at its root,
one per line:
greeting+binary/generated=./plugin
signing+artifact=demo.signing
The key is <name>+<hook point>. The name is the plugin's own, and holds neither / nor +. The hook
point says where in the build the plugin joins, as the next section describes; a key without +<hook point>
adds the plugin to the module build itself, and an unknown hook point is refused. The value says where the
plugin comes from:
| Value | The plugin is |
|---|---|
. |
a folder of the project, compiled from source on every build |
<module name> |
resolved as module/ whatever the project's layout: from the local module repository (~/) first, then from the module repositories jenesis. names, the Jenesis Module Index by default |
module/ |
the same, spelled as a coordinate |
maven/ |
resolved from a Maven repository with the closure its POM declares |
A coordinate is resolved by the repository its first segment names, and by that repository's resolver, as a
dependency is anywhere else in the build. Its version comes from the pins unless the coordinate carries one, as
maven/. A value that is none of these, such as a path starting with ., fails
the build.
Any of them may end in @<provider>, which selects the provider annotated @BuildModuleName("<provider>") when the
plugin provides several: signing+artifact=. Without it, the plugin must provide exactly
one unannotated provider.
A plugin is loaded as jpx runs a program, with the JDK's own class loaders. One named by a Maven coordinate
loads with its whole closure on a class path of its own. One named by its module loads every jar that declares a
module name into a module layer of the plugin's own, and any other jar on a class path beneath that layer, which
its automatic modules read. Under -Djenesis. every jar of such a plugin loads as a
module, and a plugin named by a Maven coordinate is refused.
The file names its plugins under names of their own: a plugin it declares resolves and pins in the group
plugin., and runs apart from the official plugins at every hook point, so it runs beside an official
plugin of the same name and neither touches the other. Everything the project writes for it - the line,
plugin-<name>., its arguments and its selector - spells the name as the line writes it.
Hook points
A hook point is a place in the build graph that Jenesis leaves open for plugins. It is named by its path, as
the build log shows it - binary/ is the source generation of a module's binary build,
stage/ a place in stage - and it decides three things about a plugin: when it runs, what it is handed
as input, and where what it writes goes. A plugin never inspects the build around it: it reads the folders its
hook point hands it, and what it writes travels onwards by the same folder conventions the stock steps follow,
so a sources/ tree written in binary/ is compiled, and a maven/ folder written in
stage/ joins the staged Maven tree. Every hook point keeps its plugins in a namespace of its own -
custom/, transform/, inspect/ or plugin/ - beside the stock steps and the official plugins, so a plugin
may take any name. What a
plugin writes is added to what the stock steps write, at every hook point but binary/, whose plugins
rewrite the compiled classes.
Hook points come in two kinds. A module hook point runs a plugin once in every module of the project that
configures it; a project hook point runs it once for the whole project. preprocess runs before any module is
built and the other project hook points after every module is, so the module hook points sit between them; each
table lists its hook points in the order the build reaches them.
A plugin of a module hook point runs as <hook point>/ within the phase of each module that holds its
hook point, as the build log shows it under build/ or build/ -
build-module-app/ in the module's build, say, or
verify-module-app/ in its verify, where a plugin of binary/ runs as
validate/. A plugin with no hook point runs as custom/, in the module's build itself. The
plugins of observe, test and language sit as plugin/ beside the official observers, frameworks and
languages:
| Module hook point | When a plugin runs, and what it is handed |
|---|---|
check |
In the module's verify phase, beside the stock source checks, handed the module's sources and manifests. |
format |
In verify, beside the stock formatters, handed the same. |
compliance |
In verify, beside the licence and vulnerability checks, handed the manifests and the resolved dependencies. |
binary/ |
Before compilation, beside the stock generators; a sources/ tree it writes is compiled with the module, and a resources/ tree, such as a service file, joins the module's jar. |
binary/ |
Beside the stock compilers, handed what they compile; what it writes below classes/ joins the module's jar, and a class that two compilers or plugins both write fails the build, naming both. |
binary/ |
After the compilers, handed the module's compiled classes first and the module's inputs after them; what it writes below classes/ replaces the class of that name. Several plugins run in the order the file names them, each handed what the one before wrote. |
binary/ |
In verify, beside the bytecode checks, handed the compiled classes. |
binary |
Within the module's compile toolchain, handed what the toolchain reads. |
artifact |
In verify, handed the module's jar with its dependencies. |
observed |
With the module's tests, beside the stock observers such as coverage. |
observe |
Attached to the JVM the module's tests run in, by a plugin that provides an ObservabilityFactory; its report runs as observed/, handed the tests' output. |
test |
The framework the module's tests run on, from a plugin that provides a TestFrameworkFactory, consulted before the built-in frameworks. |
language |
A language of its own, from a plugin that provides a LanguageFactory: compiled before or after javac, and documented, wherever a module holds a file of its extensions. |
documentation/ |
In the module's document phase, beside the stock documentation generators. |
documentation |
In document, within the documentation build, handed the compiled module. |
| (none) | In the module build itself, handed everything the module reads. |
package |
In the module's package phase, handed its jar, its dependencies and the stock images; what it writes into packages/ is staged in stage/. |
A plugin of a project hook point runs under the goal it belongs to, at the same path in every project:
build/, build/ and
build/, stage/ and stage/,
release/, and plugin/:
| Project hook point | When a plugin runs, and what it is handed |
|---|---|
preprocess |
Before any module is built, handed only what it binds. It hands the build nothing, so it can stop the build early but never feed it. |
postprocess/ |
After every module is built, handed every module's inventory, in the order the file names them; adds files to the modules or to the whole project. |
postprocess/ |
After the transforms, handed the same with what they added; fails the build by throwing. |
stage/ |
After the stock staging, handed every staged tree; what it writes under a tree's name joins that tree. |
stage/ |
After the transforms of stage, handed the staged trees with what they added. |
release |
With release, beside the stock release steps, only where its plugin-<name>. is found; handed everything staged, the project's root and its resolved metadata. |
plugin |
Only when plugin/ is named, handed only what it binds; nothing waits for it and it waits for nothing. |
The module hook points run while the module is built, so everything after them sees what they add;
postprocess still belongs to build, so stage, export, release, pin, dependencies, ide and
Execute. never run past a failed inspection. stage runs only when a goal needs it, and export,
release and plugin only when they are asked for.
Plugins of a module
A plugin in a module hook point reads what the module it joins reads, and its output belongs to that module like
any of the stock steps': a plugin in binary/ that writes a sources/ tree has it compiled with the
project's own sources, and a plugin in artifact reads the jars the build produced.
It runs in a module only where plugin-<name>. is found, looked up like any other
configuration file - in build., with the profiles first - so the same line can serve every module of
a project and still run only in those that configure it. The file's values are handed to the plugin when it
is created:
# build.jenesis/plugin-greeting.properties
greeting=Hello from a generated source
The plugin's dependencies are pinned like any other, in the module's declaration, under the dependency group
plugin.:
/**
* @jenesis.pin plugin.greeting/module/org.json 20260522 SHA-256/...
*/
A plugin found in the local export is pinned and checked like one from a repository: build it again and its
checksum changes, so the build refuses it until -Djenesis. with pin records what the
export holds.
Everything a plugin resolves itself is pinned in a group <group>:<purpose> of the group it is handed -
plugin. for a plugin the file declares, the plugin's name for an official one. The tool it runs lies in
<group>:tool - plugin., or xjc:tool/ for the official xjc
plugin - and a closure that must not share versions with it in a group of its own, as kotlin:dokka for the
Kotlin plugin's Dokka. Every pin a plugin needs therefore sorts under its group, and the build fails a plugin that
resolves in a group of another name. A . file escapes the colon in a key, so pin writes such a pin
there as xjc\:tool/.
-Djenesis. leaves out a plugin the file names, without editing the file, and
-Djenesis. leaves out every plugin the file names. The setting is a plugin's own: an
official plugin of the same name is switched by jenesis. alone, so the two never share a setting.
A plugin hands the tests a file the same way: in the test module's binary/, it writes the file below
classes/, where only the tests' jar holds it, and the tests read it as a resource. Its
plugin-<name>. sits where the test module's configuration is looked up, as src/
in a pom. project. A framework that wants a file path in a system property instead gets one from a
LauncherSessionListener of the tests, a JUnit service that copies the resource to a temporary file and sets the
property before the first test runs.
A plugin in package can wrap or sign one of the images packaging. asks Jenesis to build - a
runtime image, a jpackage image, a launcher - into a package of its own: an AppImage, an installer built with
other tools, a distribution zip. A package whose name another packager, or jpackage, writes already fails the
build rather than replacing it.
Handing a plugin files of the project
A plugin reads only what the build hands it, so a file of the project reaches it as an input. Among a
plugin's values, a key starting with @ binds a file or folder instead of naming a value:
# build.jenesis/plugin-greeting.properties
greeting=Hello from a generated source
@templates=templates
@templates/legal/HEADER.txt=legal/HEADER.txt
@<input>= binds the path into an input named <input>, which the plugin reads as the folder
.. A target after the input's name, @<input>/, places what is bound at that path
inside the input rather than at its root, and one input takes one key per target, so it can gather several files
and folders. A single file keeps its name unless a target renames it. Editing a bound file runs the plugin again.
A value whose key really starts with @ is written with two: @@name= hands the plugin @name=.
In a module's plugin-<name>., a path is resolved against the module's own folder, the one holding its
module-info. or pom.. A folder has
to stay within the project, symbolic links included, and a plugin folder named as . must lie within
the project as well. A file may lie outside it, so that several projects share one configuration, and a path
written as @<key> names one through the setting -Djenesis.. A folder that leaves the project, a
path that does not exist, or one that shares a target with another binding of the same input fails the build.
jenesis included. Before
you build a project whose plugins you have not reviewed, run the build in a container with
-Djenesis.project.docker=true: the plugins then run inside the container and never on your
machine, and the project cannot switch Docker off (see
What runs on the host).
Plugins for the whole project
A plugin in a project hook point runs once for the whole project. The line in jenesis.
switches it on by itself, as no plugin-<name>. applies to it:
# jenesis.plugins.properties
licence+preprocess=./licence
notice+postprocess/transform=./notice
audit+postprocess/inspect=./audit
checksums+stage/transform=./checksums
complete+stage/inspect=./complete
publish+export=./publish
announce+release=./announce
lines+plugin=./lines
Their names take no /, . or +, and a name used for a module hook point cannot also name one of them.
A preprocessor checks the project before anything is compiled: a licence header, a forbidden file, a policy
on what the repository holds. A file every module needs comes from a plugin in binary/ instead.
A transform adds files to the modules - a notice, a report, a signature of your own - and an inspection checks the result and fails the build when it is wrong. Transforms run in the order the file names them, each seeing what the ones before it added, and the inspections run after all of them.
A transform of stage writes into a folder named after a staged tree - maven/, modular/, packages/,
project/ and the others under target/ - and what it writes joins that tree where it stands, so
export, release and their plugins take it like anything else staged: checksums or
signatures beside every file, repository metadata, a manifest of what ships. A file the tree holds already fails
the build rather than being replaced. An inspection of stage then checks the staged trees with what was added,
and nothing is exported or released past one that fails.
A releaser delivers what was staged - to an internal repository, a bucket, a registry, a release page. Naming
release/ runs the releasers alone, without the steps Jenesis runs there itself. Such a step writes outside
the build, so it overrides shouldRun to run every time it is selected, as the stock release steps do, rather than
only when what it reads has changed. export has no hook point: it only installs what was staged into this
machine's local repositories, and what it installs is shaped by the plugins of stage.
A plugin under plugin belongs to no goal and starts at once, as
java build/: a report over the project's own files, a task run by hand. What
needs the built or staged result belongs in one of the hook points that follow them.
Configuring them
A plugin of the whole project reads its values from jenesis. beside
jenesis., one line per value as <plugin>., and from nowhere else - never from the
command line:
# jenesis.plugins.arguments.properties
notice.holder=Example Corp.
notice.@legal=legal
The plugin notice then receives holder=, and @legal binds the project's legal/ folder as its
input, resolved against the project root. A line that names no declared plugin fails the build. A profile brings
values of its own in jenesis., which win over the file without a
profile, and of two active profiles the one named first wins. That is why the plugin files are named with dots:
in a file name, a dash after a name always introduces a profile, as in jenesis-<profile>..
Since the plugins in build run with every build, one that takes long is best switched off in jenesis. with
jenesis. and switched back on in the profile that ships. -Djenesis.
leaves out every plugin at once, those of the module hook points included.
A plugin of release
A plugin of release is the exception. It runs only where a configuration folder holds its
plugin-<name>., as a plugin of a module does, and it takes its values and binds its inputs from
that file rather than from jenesis., which refuses a line for it. Placed in the
folder of a profile, such as build., the file keeps every build but a release from running the
plugin.
Beside everything staged, such a plugin is handed two files that tell it where the project lies and what it is:
| File | Holds |
|---|---|
root. |
root, the project's root folder, and stage., the folder of each staged tree, such as stage.. |
metadata. |
The project's metadata as the build resolved it, the version -Djenesis. sets included. |
A release tool runs in the project's checkout, which is why this is the one place a plugin is told where the project lies.
What a transform and an inspection see, and what they may add
Each of them is handed the inventory of every module: its jar, sources and documentation, its POM, and every
dependency it resolved, each with the jar it resolved to. A transform adds to a module by writing files into its
own output and naming them in an inventory. there, under the prefix the inventory gives the module
(its build identity, such as module-sources):
module-sources.attachment.notice=notices/module-sources/NOTICE.txt
The key after the prefix is any key the module's inventory knows, read by the stages as the module's own:
<module>.is staged beside the module's jar under that classifier:attachment. <classifier> <artifact>-<version>-<classifier>.in the Maven tree,<extension> <module>-<classifier>.in the modular tree.<extension> <module>.is staged with the module's reports underreport. <name> stage/.reports/
What a transform adds is its author's responsibility. A module the build does not have fails the build, and so does a key two transforms add, or an attachment whose file name the build stages already.
What belongs to no module - an aggregated report, a site, a distribution of every module - a transform writes into
a project/ folder of its own output, laid out as it sees fit. stage copies it as it stands into
target/, where export and releasers find it with the rest, and a file two transforms
place at the same path fails the build. A transform that brings
its own SBOM, say, attaches it as cyclonedx once -Djenesis. has switched off the stock
one. An inspection reads the same inventories plus what the transforms added, and fails the build by throwing.
It writes only into its own output: an inspection that changes a file it was handed fails the build as well.
Pinning them
These plugins belong to no module, so their pins live beside the file that names them, in
jenesis., one line for each module in each plugin's closure:
plugin.audit/module/build.jenesis=<version> SHA-256/...
plugin.notice/module/build.jenesis=<version> SHA-256/...
pin writes the file and pins every plugin of the whole project the file names, including one a setting
switches off, so a plugin that only a profile switches on is pinned all the same. pin runs last, after a dry
release, so it also writes the tool a releaser resolves, such as the official JReleaser plugin's
jreleaser\:tool/. A releaser reads
jenesis. from the environment it is handed and publishes nothing without it. Where a releaser
does not run - its file sits in a profile pin was not given - pin keeps the lines that pin its tool as they
stand. The plugins run with
everything that builds, pin among it, so an inspection that fails stops pin too. This pins without running
any plugin:
java -Djenesis.project.plugins=false build/jenesis/Make.java pin
Writing a build step
A build step is the unit of work introduced in Core concepts: it reads one or more input folders and writes into one fresh output folder. When you write your own, that is the shape you implement - a function handed its inputs and an output folder to fill:
CompletionStage<BuildStepResult> apply(Executor executor,
BuildStepContext context,
SequencedMap<String, BuildStepArgument> arguments);
The context gives you four folder slots:
next- the folder this run writes into. It is created fresh every time; your step writes here and nowhere else.previous- the same step's output from the prior run, ornullon a first run. You may read it to hard-link or copy unchanged files instead of regenerating them, but never write into it.supplement- scratch space for intermediate files you do not want to publish innext.persistent- state your step keeps for its own next run, such as an incremental compiler's caches. It is the folder the last successful run left, handed over wheneverpreviousis, and empty otherwise: on a first run, after a failed run, when the step changed, or when the build cache supplied the last output. It is not part of the output, so dependents never see it, it is not cached and it does not change the step's key. Your step must write the same output with an empty folder as with a full one.
The arguments map carries one entry per predecessor you wired in. Each exposes the folder to read
(argument.) and a per-file change status - ADDED, ALTERED, REMOVED, or RETAINED - computed
against the previous run. The default behaviour re-runs your step whenever any input changed; override
shouldRun(. if you want finer control.
A step can also say that its output is not worth sending to a cache server. Override shouldCacheRemotely()
to return false when the output is large against what it costs to produce, or when it is re-derivable from
something the machine already has. The output is still cached locally, so a rebuild on this machine still
skips the work. See the build cache.
A step is expected to write the same bytes from the same inputs. A build checked with
jenesis. names every step that does not, as described in
Checking that a build reproduces.
next, reach outside neither. That is what makes its output cacheable and safe to share
between builds - the incremental engine relies on it.
Talk through folders, not step names
Steps compose by file and folder conventions, not by knowing who wired them. A step discovers what to
read by looking for well-known paths inside each input folder - sources/ for Java sources, classes/ for
compiled output, artifacts/ for jars - and writes its output under names its consumers look up the same
way. Do not inspect the names of your predecessors to guess which input is which; read the folders. This is
what lets you splice a custom step between two stock ones without either noticing.
The serialised-state rule
Core concepts flagged that a step re-runs when its serialised state changes, and left the details here. This is the one rule a custom step must get right.
Jenesis content-hashes each step's serialised form and folds that hash into its cache key. So a step re-runs when its inputs change or when its own configuration changes, and "configuration" means the values of its serialised fields. The practical rule follows directly:
Put every knob that should trigger a rebuild into a serialised field. A greeting to substitute, a flag, a target version - if changing it should re-run the step, it has to be a (non-
transient) field, because that field's value is exactly what the cache hashes.
apply, fix a
bug in a helper - without changing any serialised field, its hash is identical and Jenesis reuses
the stale output. Example: a preprocess step whose substitution string lives in a
field re-runs the moment you edit that string; but if you instead hard-code the string in the method body
and edit it there, nothing re-runs until an input changes. Keep behaviour-affecting values in fields, or
bump the step's serialVersionUID (below) when you have changed only code.
State must be serialisable
Because the step is serialised to be hashed, all of its state must be serialisable. This is checked on the first run, at hash time, not lazily. Two things make the common cases work:
- A step written as a lambda serialises with what it captures, because
BuildStepis itselfSerializable. A lambda that uses only its parameters captures nothing; one that creates an anonymous class inside an instance method captures that instance, which then has to beSerializabletoo. - A lambda field serialises only if its declared type does. Declare the field as a serialisable
functional interface, or cast the lambda to
Function<…> &where you store it, and a lambda that closes over, say, aSerializable Pathserialises cleanly. The stock steps do this at their constructors, which is why you can hand them a plain lambda. - A
Pathfield is hashed by its string form, even though the JDK'sPathis not itselfSerializable. SoPath-typed configuration is first-class.
Genuinely non-serialisable state - an open socket, a database handle, a live Context object - throws
NotSerializableException at hash time, on the first run. That is deliberate: the error surfaces the bug
immediately rather than silently breaking cache invalidation. If you see it, hold the serialisable
description of the resource (a URL, a path, coordinates) as the field and open the resource inside
apply, or mark truly incidental state transient so it never reaches the digest.
serialVersionUID, which is the lever for a code-only change.
Without an explicit one, the JVM derives it from the class structure, so adding a field or changing a method
signature already changes the hash - but editing a method body does not. Declare a
serialVersionUID and bump it whenever you change a step's behaviour without touching its
fields. With one declared, structural changes do not shift the hash on their own, so bumping it is
your job.
Writing a plugin
A plugin is a named Java module that provides a BuildModuleFactory, which Jenesis discovers through that
declaration alone:
module demo.plugin {
requires build.jenesis;
requires org.json;
provides build.jenesis.BuildModuleFactory with demo.plugin.GreetingFactory;
}
The factory creates the BuildExecutorModule that adds the steps the plugin contributes, as a stock module does.
It is handed the settings of the build, the values of its plugin-<name>. or its lines of
jenesis. in the file's order, the Permits that bound the processes the build runs
at once, its group - plugin., which everything it resolves lies under - and the build's repositories and
resolvers:
public class GreetingFactory implements BuildModuleFactory {
@Override
public BuildExecutorModule create(Environment environment,
SequencedMap<String, String> properties,
Permits permits,
String group,
Map<String, Repository> repositories,
Map<String, Resolver> resolvers) {
return new GreetingModule(properties);
}
}
A value arrives resolved: @<key> stands for the setting -Djenesis., @<key>/ falls
back to what follows the slash, and @@ writes a literal @. A value is never a path - a file the plugin reads
is bound as an input, as Handing a plugin files of the project shows. A factory refuses a value it does not take, naming those it
does, rather than dropping it.
The steps a plugin adds are handed their inputs as arguments. An input bound with @<input> arrives as the
argument ., and a transform or an inspection finds each module's inventory as an
inventory. in the folders of its arguments:
BuildStepArgument legal = arguments.get("../inputs/legal");
Each argument is keyed by its predecessor's path: argument. is that predecessor's output, and
argument. holds the checksum of each file in it. Beside the source folders, a plugin of a module hook
point is handed the module's metadata., with its version, name and the rest, and its
module., whose test key tells the tests of a pom. from its main code - so a step that belongs
to one of them returns early on the other. The resolved dependencies arrive as a folder too, read with
build.: Dependencies. lists the jars of one group and
scope across every argument a step is handed, Dependencies. those of one folder,
and Dependencies. every one. What a step in binary/ writes as classes/ and as a
manifest. is merged into the module's jar. A step in binary/ finds the compiled classes in the
first of its arguments, and what it writes below classes/ replaces the file of that name, while every file it does not write passes on
as it was. A manifest. it writes replaces the one it was handed, so a plugin that writes one carries over the
attributes that still hold, as Multi-Release while it keeps the versioned classes.
A plugin compiled from source lives in a project folder of its own, which carries an empty .
marker so the project's module discovery does not mistake it for a second project module. A published plugin
is built and exported like any other module, and resolved by its module name or its Maven coordinate. Named by
its module, it is found through its provides; on the class path a Maven coordinate loads it on, it is found
through a META-INF/ entry for its factory, which its build writes from the provides when its
configuration folder holds an empty classpath., as
A modular jar on the class path describes.
ModuleLayer of its own where it loads
as modules - and calls are bridged across the boundary, so the two copies never clash and a plugin may pin its
own Jenesis version, as long as the API it uses lines up. A plugin named by its module or compiled from source
is an explicit named module; its dependencies need not be, since a module layer admits automatic
modules too.
A plugin compiled from source resolves its requires as a project's module-info. does. A tool the
module index does not serve, as one named by an Automatic-Module-Name alone, is required through an
@jenesis. line on the plugin's declaration, and @jenesis. and
@jenesis. lines apply to the plugin's dependencies as in a project. Every jar of the plugin's layer is a
module of its own, so a jar that declares no module name at all, or two jars that share a package, cannot load:
exclude it, or pin a version of it that names its module.
Observing the tests
A plugin of the hook point observe joins the JVM the tests run in, as a coverage agent does, and reports over what
that run recorded. It provides an ObservabilityFactory rather than a BuildModuleFactory, is handed the same,
and returns an Observation: the ObservabilityEngine that names the agent's coordinates and the JVM options of the
test run, and the module that reads what the run left in its output.
public class LoadedFactory implements ObservabilityFactory {
@Override
public Observation create(Environment environment,
SequencedMap<String, String> properties,
Permits permits,
String group,
Map<String, Repository> repositories,
Map<String, Resolver> resolvers) {
return new Observation(new ClassLoading(group),
(executor, inherited) -> executor.addStep("loaded", new Loaded(prefix(properties)),
inherited.sequencedKeySet()));
}
}
The engine is part of what decides whether the tests run again, so it holds its settings as serialisable fields,
and a changed value runs the tests anew. It is named after the group the factory is handed, plugin., and
its coordinates resolve with the test runtime in <group>:agent. The plugin is built before the tests, under
observed/, and its report runs as observed/; it takes values
but binds no files. Only a module that runs tests is observed.
Running the tests on a framework of your own
A plugin of the hook point test provides a TestFrameworkFactory, is handed the same, and returns a
TestFramework: the runner the tests' JVM starts, the arguments it takes, and the modules that mark a test module as
its own. A framework that writes a report can also return, from failed, the test methods that failed in a run,
which jenesis. and failedfirst run again the next time; one that returns nothing runs every test by class name. The frameworks a project declares are consulted first, in the order jenesis. names
them, and the built-in JUnit Platform, JUnit 4 and TestNG after them, so the first framework a module of the tests
marks runs them. A framework= line in test. names a declared framework by its plugin name, as it
names a built-in one.
A framework is part of what decides whether the tests run again, so it holds its settings as serialisable fields. It may delegate to a built-in one, as a framework that runs on the JUnit Platform but hands the tests a system property of its own does:
@Override
public Map<String, String> systemProperties() {
Map<String, String> properties = new LinkedHashMap<>(platform.systemProperties());
properties.put("demo.framework", name);
return properties;
}
Compiling a language of your own
A plugin of the hook point language provides a LanguageFactory, is handed the same, and returns a Language:
the file extensions it compiles, the module that compiles them, whether that runs before or after javac, and the
module that documents them, or null. It needs no plugin-<name>.: it runs in every module that holds a
file of its extensions, and takes its values from that file where one is found.
| A language's | Is handed, and writes |
|---|---|
| compiler before javac | the module's sources and dependencies, and the classes of the compilers before it; javac compiles against the classes/ it writes. |
| compiler after javac | the same, and the classes javac wrote. |
| documentation | the module's sources; the javadoc/ it writes joins the module's documentation, at its root where the language is the module's only one and under javadoc/ beside others. |
A file of the language's extensions is no resource, so it never reaches the jar. A pom. project compiles a
folder beside src/, such as src/, only where jenesis. names it:
# jenesis.properties
jenesis.maven.sources=kotlin
A worked example: classes rewritten with Byte Buddy
A plugin is written against a library's own Java API. This one runs Byte Buddy's
Plugin. over the classes javac compiled, as Byte Buddy's Maven and Gradle plugins do, with two Byte
Buddy plugins: ToStringPlugin, which Byte Buddy ships, and one written for the project. What it writes
replaces the classes it was handed, so the jar, the tests and every later step see the rewritten ones.
The project names the plugin, the hook point it joins and the folder it is compiled from:
# jenesis.plugins.properties
enhance+binary/transform=./plugin
plugin/ holds the plugin module, with an empty . beside its declaration. It requires the
build API and Byte Buddy, and provides its factory:
module demo.plugin {
requires build.jenesis;
requires net.bytebuddy;
provides build.jenesis.BuildModuleFactory with demo.plugin.EnhanceFactory;
}
The plugin runs in a module where plugin-enhance. is found:
# build.jenesis/plugin-enhance.properties
type=sample.Greeting
greeting=Hello from a method that Byte Buddy rewrote in a build plugin!
Its values reach the factory's create as a SequencedMap<String, String>, in the file's order, and the factory
hands them to the EnhanceModule it returns. The module keeps what it needs, and its
accept(BuildExecutor, SequencedMap<String, Path>) adds one step, a record of those two values, handed everything
the hook point reads:
public EnhanceModule(SequencedMap<String, String> properties) {
type = properties.getOrDefault("type", "sample.Greeting");
greeting = properties.getOrDefault("greeting", "Hello from a rewritten method!");
}
@Override
public void accept(BuildExecutor executor, SequencedMap<String, Path> inherited) {
executor.addStep("enhance", new Enhance(type, greeting), inherited.sequencedKeySet());
}
The step's apply(Executor, BuildStepContext, SequencedMap<String, BuildStepArgument>) collects the jars the
module compiles against as Byte Buddy's class path, and runs the engine from the classes in its first argument
into classes/ of its own output folder, context.:
List<ClassFileLocator> classPath = new ArrayList<>();
for (BuildStepArgument argument : arguments.values()) {
if (!argument.removed()) {
for (Path jar : Dependencies.select(argument.folder(), "main", "compile")) {
classPath.add(ClassFileLocator.ForJarFile.of(jar.toFile()));
}
}
}
try (ClassFileLocator locator = new ClassFileLocator.Compound(classPath)) {
new Plugin.Engine.Default()
.with(locator)
.apply(arguments.firstEntry().getValue().folder().resolve(BuildStep.CLASSES).toFile(),
Files.createDirectories(context.next().resolve(BuildStep.CLASSES)).toFile(),
new Plugin.Factory.Simple(new ToStringPlugin()),
new Plugin.Factory.Simple(new Greeting(type, greeting)));
}
return CompletableFuture.completedStage(new BuildStepResult(true));
The class path is what lets Byte Buddy read the types a class refers to beyond the JDK. The module's own
sample. carries @ToStringPlugin., from a requires static net.; without the
module's jars, Byte Buddy cannot resolve the annotation, and ToStringPlugin matches nothing. The second
plugin, Greeting, is a net. of the project's own that matches the type its properties
name and rewrites its get() to return the configured text:
public boolean matches(TypeDescription target) {
return target.getName().equals(type);
}
public DynamicType.Builder<?> apply(DynamicType.Builder<?> builder,
TypeDescription typeDescription,
ClassFileLocator classFileLocator) {
return builder.method(named("get")).intercept(FixedValue.value(greeting));
}
The two values are the step's serialised state, so changing either in plugin-enhance. runs the
step again, and so does a change to the classes it is handed. The application then prints what the rewritten
classes return:
Hello from a method that Byte Buddy rewrote in a build plugin!
Greeting{audience=the build}
Byte Buddy is configured in code here rather than discovered. Its Maven and Gradle plugins run the Byte Buddy
plugins they find in META-INF/ or that the build file names, configured in XML
or the build's DSL. The module above constructs the plugins it runs in Java instead, from the values of its
own properties file, and runs nothing it did not name.
Byte Buddy resolves into the plugin's own module layer, and what the plugin rewrote refers to nothing but
java., so it never reaches the running application. pin records the plugin's closure in the project's
module-info., in the group plugin.:
/**
* @jenesis.pin plugin.enhance/module/build.jenesis <version> SHA-256/...
* @jenesis.pin plugin.enhance/module/net.bytebuddy <version> SHA-256/...
*/
Forking a tool that cannot be a module
A tool that cannot be a module at all, as one with a class in the unnamed package, cannot load in the
plugin's layer. The plugin's step forks it instead, and the project resolves it rather than the plugin. A
module names it in a group of its own with @jenesis. in its
module-info., or a <!--jenesis. comment in its
pom., and pins it as <group>/. A module hook point such as
binary/ hands the jars over among the resolved dependencies, the input whose key ends in
/, where Dependencies. lists them for the
java -cp the step runs.
A step that forks java -cp <jars> <class or Launcher. builds its handler with
ProcessHandler. rather than ofJavaHome("bin/. Under jenesis., which the
build daemon sets and which is fork otherwise, its tool's main method then runs in a JVM kept warm for the next
call with the same jars on its class path and the same options and environment, which compiles a . launcher
once. A jar is matched by the file rather than its path, so every module of a build that
resolves the same tool shares that JVM. What one call prints is that call's output, a call that returns ends with 0, or with the int a main
method declared to return one hands back, and one that calls System. ends with that code and takes its JVM
with it. While the setting is not daemon, which also serves a tool whose static state must not outlive a call, every call
runs in a JVM of its own, and a class's main method still ends the call as described: the code it returns is the
code, and a thread the tool leaves running ends with the JVM. Under jenesis., which jenesis.
sets, that JVM starts from an ahead-of-time cache of the tool in ., trained on its first call. A command line of another
shape - a module path, -jar, an argument file - is forked as it stands.
A handler marked shared(true), as a compiler's is, runs its calls side by side in one JVM in every mode, as long as
jenesis. is not false: under fork and aot that JVM ends with the build, and under daemon it is
kept for the next one. Each call's output is still its own, which the JVM routes by the thread that runs the call.
A tool whose main method calls System. on a call the build accepts takes its warm JVM with it every time.
Run it through a driver instead: a class of your plugin with a public static int main(String[]) that calls the
entry point the tool offers short of its main method and returns the code. Compile it against the tool with
requires static, so your plugin's own dependencies do not include the tool, and put the folder that
ProcessHandler. writes ahead of the tool's jars on the class path. That folder
holds the class files of the driver and the classes it nests alone, so the driver uses nothing else of your plugin,
and it runs against the version of the tool the project pinned. A JVM runs one call at a time, because few tools allow more; a tool whose main
method keeps its state per call - a compiler driven through its own API - asks for . on the handler,
and its calls then run side by side in one JVM, sharing its JIT, each printing only what it printed itself.
Hand such a step only the inputs it reads, executor. naming them among the keys of
the map accept is handed, since each input is part of the step's key: a generator handed sources/ runs
again on every edit. The keys are relative paths, as ., so pick an
input by how its key ends.
Compiling native code
A native method is implemented in a library the module loads at run time, and the C side compiles against a
JNI header that javac -h writes. A plugin at binary/ builds both with two steps:
- The build's own
javacstep, asked for the headers alone:new Javac(ProcessHandler.. It compiles the module's sources as the module's compiler does, but hands on nothing but aFactory. of()). headers(true) headers/folder, with one header for each class that declares a native method. The classes it compiles stay with the step, so they never join the jar a second time. - A step of the plugin's own that forks the machine's C compiler, extending
EnvironmentalProcessBuildStep. It puts the JDK'sinclude/folder, the folder for the operating system below it andheaders/on the include path. It writes the library belowclasses/, where it joins the module's jar.
The C sources reach the plugin as an input, bound with @c= in its plugin-<name>., as in
Handing a plugin files of the project. A library is built for the
machine that builds it. A project that publishes one for several platforms builds each on its own machine and
writes the library into a folder that its
classifiers. names, instead of below classes/.
Loading a library is a restricted method, so the application grants native access to the module that loads it,
as Granting native access describes.
Writing an entry point of your own
A build that plugins cannot express - one that changes what the stock steps do, several builds compared, one
staged build feeding the next, a graph with no project at all - is a program of its own, run as
java build/. Where it builds the
project, it calls Make, which reads jenesis., the profiles and the -Djenesis. properties the
JVM was started with, and returns what the build produced:
Make.Result staged = new Make("build.jenesis.Project").build("stage");
Path packages = staged.outputs().get("stage/packages");
Changing the stock build
An entry point that keeps the stock build but changes it builds the project from the settings, as Make does,
and replaces its assembler - the callback that wires each module's compile/jar/test sub-graph - with one
that wraps the stock InferredMultiProjectAssembler:
Environment environment = new Environment(Make.settings(Path.of(".")).keys());
InferredMultiProjectAssembler stock = InferredMultiProjectAssembler.ofEnvironment(environment);
Project project = Project.ofEnvironment(environment, Path.of("."))
.assembler((descriptor, repositories, resolvers) -> stock
.apply(descriptor, repositories, resolvers)
.andThen("sign", (sub, inherited) -> sub.addStep("sign", (executor, context, arguments) -> {
// read the jars in each argument's folder, write their signatures into context.next()
return CompletableFuture.completedStage(new BuildStepResult(true));
}, inherited.sequencedKeySet().stream()), AssemblyRequirement.of().after(AssemblyDescriptor.BUILD)));
System.exit(new Execution(project).execute(args));
andThen adds a phase to each module - here sign, registered as sign-<module> beside the stock build,
verify and the others. Its AssemblyRequirement names what it waits for and is handed:
| Requirement | The phase is handed |
|---|---|
after(phase) |
that phase of its own module, declared before it |
upstream(phase) |
that phase of every module its module depends on, directly or not |
tested(phase) |
that phase of every module that tests its module |
andThen on a phase that is declared already extends it: the module's steps run after the phase's own, and the
requirement adds to the phase's. A step that must run before the stock build, as a preprocessor of the sources,
uses nest(AssemblyDescriptor. instead: it hands over the stock phase as a module, and the
module it returns adds its own step and then the stock one. Execution builds the project and runs the module that declares a main class,
as build/ does for the stock build.
The stock modules take additional steps and modules in code as well, through the same custom namespace a plugin is
wired into:
InferredMultiProjectAssembler checked = stock.check(check -> check.custom("placeholders", (executor, context, arguments) -> {
// fail when a source in any argument's folder still holds a ${ placeholder
return CompletableFuture.completedStage(new BuildStepResult(true));
}));
custom(name, step)adds one step andcustom(name, module)one module, after those added before; either refuses a name that is taken already.custom(map)sets every added module at once, aSequencedMap<String, BuildExecutorModule>in the order they are wired.
The plugins of the whole project are a value of the project rather than of the assembler: plugins()
answers the ProjectPlugins that jenesis. declared, and preprocess(name, step),
transform(name, step), inspect(name, step), supplement(name, step) for stage/,
audit(name, step) for stage/, release(name, step) and goal(name, step) add one more, a
step or a module, refusing a name that is taken already. Here
ReleaseAudit is a BuildStep of the project that throws when a module's inventory lacks what every release must
carry:
Project project = Project.ofEnvironment(environment, Path.of("."));
project = project.plugins(project.plugins().inspect("audit", new ReleaseAudit()));
A plugin that pin should pin also needs its resolution(), the part of an InternalModule or ExternalModule
that resolves the plugin's closure without building it; ProjectPlugins takes those by name beside the
plugins themselves.
Redirecting a module's inputs
An entry point can also change what the stock steps consume, because the module descriptor is immutable with a
wither per property. Every reference accessor (sources, resources, manifests, dependencies,
artifacts, content, coordinates, spdx) returns a SequencedSet<String>, so you can add or replace
inputs in one line:
descriptor.sources("preprocess") // stock compile reads the preprocess step's output, not sources/
That is the whole trick behind a preprocessing build: add a preprocess step that reads the module's
sources/, rewrites it into its own output, then hand the stock assembler a descriptor whose sources()
points at preprocess. javac, the jar step, and the tests all consume the transformed tree, and the rest
of the build is untouched. Any pass that produces a sources/ tree - template expansion, code generation,
licence-header stamping - fits the same shape.
Starting on the selected JDK
Such an entry point starts on whichever JDK runs it. To honour jenesis., as Make.
does, ask Toolchain from build.: new Toolchain(). answers the JDK the version selects, and
launch(Demo. starts the entry point again on it and returns its exit code.
A file that names Project or the steps is compiled by the JDK's source launcher on every run, engine
included. Compile it once when that matters - javac -d ., then
java -cp . - and keep java build/ as the documented command.
Reusing the toolchain without Project
When the stock compile/jar/test flow fits but Project does not, call the convenience factory
MavenProject. (or ModularProject. for a Java Module System project). It
discovers the modules under a root, fills in sane defaults - a Maven Central repository, the right resolver, a
digest - and leaves only the environment those defaults read their settings from and the assembler for you to
supply:
Environment environment = new Environment(Make.settings(Path.of(".")).keys());
BuildExecutor root = BuildExecutor.of(Path.of("target"));
root.addModule("maven", MavenProject.make(environment,
Path.of("."),
(descriptor, repositories, resolvers) -> new InferredMultiProjectAssembler().apply(
new ProjectModuleDescriptor(descriptor) // the discovered module
.configuration(Path.of(".")) // its configuration folders
.pathPlacement(PathPlacement.CLASS_PATH), // place dependencies on the class path
repositories, resolvers)));
root.execute(args);
The descriptor runs the module's tests, attaches no sources or javadoc jar and pins leniently unless told
otherwise; test(false), source(true), documentation(true) and pinning(Pinning. change one of
those each. A pom. module resolves its class paths as one graph, as Maven does: make reads
jenesis. from the environment it is handed, and a MavenProject built by its constructor resolves
each class path on its own only after unified(false).
This is a middle ground: no layout, no goals, no Project, yet you did not wire every step by hand either.
ModularProject. is the modular counterpart; its convenience form builds pure modules (a modular jar,
no generated POM). For full control - a custom repository, strict pinning, a different digest, or emitting a
POM as well - switch to the longer make(. overload that Project itself uses.
Wiring the graph by hand
When auto-detection is the wrong starting point entirely - a non-Java pipeline, code generation, a wildly
custom graph - drop to the BuildExecutor primitives and build exactly the graph you want:
BuildExecutor root = BuildExecutor.of(Path.of("target"));
root.addSource("sources", Bind.asSources(), Path.of("sources"));
root.addStep("generate", new GenerateSource(), "sources"); // writes sources/sample/Generated.java
root.addStep("classes", new Javac(ProcessHandler.Factory.of()), "sources", "generate");
root.addStep("artifacts", new Jar(ProcessHandler.Factory.of(), Jar.Sort.CLASSES), "classes");
root.execute(args);
BuildExecutor. is the root and writes everything under target/. addSource binds a directory so
changes to it invalidate downstream caches. addStep(name, step, predecessors…) chains a step whose
arguments come from the named predecessors. execute runs the graph (or a selector's subtree), reusing
cached outputs whose inputs are unchanged. The generate step above synthesises a Java source on the fly and
Javac, which reads the sources/ of every predecessor, compiles it next to the hand-written ones.
There is no phase lifecycle to fit into: a build is just steps wired to steps, and here you wire them
yourself.
Running a build inside another program
An entry point of your own is still a process. A program that already runs - an IDE, a test harness, a
server that builds what it serves - can run a build in its own JVM instead, through
java., the JDK's interface for a tool that runs in-process the way javac and jlink
do. build. publishes three, named after the commands they answer to:
| Tool | What it does | What follows the settings |
|---|---|---|
jenesis-make |
Builds the project | Selectors, as on a command line |
jenesis-exec |
Builds, then runs what it built | The program's own arguments |
jpx |
Runs a published program | The target and jpx's options |
StringWriter log = new StringWriter();
int code = ToolProvider.findFirst("jenesis-make").orElseThrow().run(
new PrintWriter(log), new PrintWriter(log),
"-Djenesis.project.version=1.0.0", "build");
The leading -Djenesis. arguments configure that run and nothing else: they are never written to the
JVM's properties, and the run never reads them from there, so two builds in one program can be configured
differently and neither leaves anything behind. Everything the build prints arrives on the writers you
passed, which the tool does not flush - they are yours, and an autoflushing PrintWriter already drains
itself. An @<file> argument works here as on a command line.
A setting that would replace the process a build runs in cannot be honoured in-process, and is refused by
name rather than ignored: jenesis. and jenesis. for all three, and
jenesis. for the program jenesis-exec runs. A -J option is refused as well, as the JDK's own
tools refuse one: a tool runs in its caller's JVM and is handed no options for one, so the -J of Execute.
and of jpx is the commands' alone. The refusal arrives on err with a non-zero code rather than as an
exception. jenesis-exec forks the program it runs, as its command does, so that
program writes to the JVM's own streams while the build's output goes to the writers.
The tools are found by name when build. is a resolved module or a jar on the class path. Source
mode registers no service, so a program there constructs new MakeTool(), new ExecuteTool() or
new JpxTool() itself; the contract is the same.