Extending Jenesis Repository

Jenesis Repository is assembled from Java modules, and every capability it has - a format, a storage backend, a security feed, a background pass, a console page - is a module that provides a service another module uses. Nothing registers a capability by name: a module on the module path is discovered through java.util.ServiceLoader, and taking it away takes the capability with it. An extension of your own works exactly the same way, so it can do anything a built-in module does, through the same seams.

How an extension plugs in

An extension is an ordinary Java module. Its module-info.java requires the module that declares the service and says which class provides it:

module com.example.repository.notice {
    requires build.jenesis.repository.store;      // PublicationObserver, ArtifactDescriptor, ArtifactStore
    requires build.jenesis.repository.settings;   // SettingsContributor, Setting

    provides build.jenesis.repository.store.PublicationObserver
            with com.example.repository.notice.NoticeObserver;
    provides build.jenesis.repository.settings.SettingsContributor
            with com.example.repository.notice.NoticeSettings;
}

Each release publishes the free modules to Maven Central under the group build.jenesis, every artifact named for its module, so a build resolves build.jenesis:build.jenesis.repository.store like any other dependency.

A few rules hold for every seam:

• Discovery happens in one place. Each service interface has a static that finds its providers - usually installed(), resolve(...) or discovered() on the interface itself - and that is the only code in the product that calls ServiceLoader for it. An extension never looks providers up itself; it provides one.
• Every seam states its contract. The javadoc of each service interface ends with a section titled Contract: which thread calls it, whether a call may be repeated, what a failure does, whether it may reach the network. Read it before implementing - it is the part a test of your extension should hold you to.
• A seam decides how many providers take part. Some take every provider (every format serves, every observer is told), some take at most one and name it by a setting (jenrepo.gc=mark-sweep), and a few always resolve exactly one, with a built-in default you can replace (the storage backend). The table below says which.
• An extension adds, it does not replace. The image's own modules are found before yours, so a module of yours with the same name as one the image carries is ignored. Replacing a built-in is done by selecting another provider through its setting, not by shadowing a module.

A few providers in code

React to a publish

A PublicationObserver is told after an artifact has been committed, removed, cached from an upstream or held. The descriptor names the artifact - ecosystem, coordinate, version, path, content hash - and the store is already scoped to the tenant and repository it went into:

public final class NoticeObserver implements PublicationObserver {

    @Override
    public void onPublished(ArtifactDescriptor artifact, ArtifactStore store) throws IOException {
        if (artifact.coordinate() == null || artifact.hash() == null) {
            return;   // a checksum, a signature or generated metadata: nothing to note
        }
        String note = artifact.ecosystem() + " " + artifact.coordinate() + " " + artifact.version();
        store.write("notices/" + artifact.hash(), new ByteArrayInputStream(note.getBytes(StandardCharsets.UTF_8)));
    }
}

The contract asks three things of it. It is called concurrently from every publishing request, so it keeps no state in fields. It may be called again for the same artifact, so what it writes must be an upsert. And it must not call out of the process inline: it leaves a durable note in the store it is handed, and a background pass delivers it. A thrown exception is logged and the publish stands - an observer never decides whether an artifact is accepted. That is what the screening seams are for.

Keep derived state right by walking

What an observer derives, a WalkConsumer can rebuild. The scheduled walk enumerates the store once for every consumer that listens on it, so a consumer is how an extension back-fills its state the first time it is installed and repairs it afterwards - never by walking the store on its own:

public final class NoticeRebuild implements WalkConsumer {

    @Override
    public String name() {
        return "notice-rebuild";   // its jenrepo.notice-rebuild=false switch, and its name on the walks screen
    }

    @Override
    public void onRetained(ArtifactDescriptor artifact, ArtifactStore store) throws IOException {
        new NoticeObserver().onPublished(artifact, store);   // the same upsert the live event performs
    }
}

The walk hands onRetained every artifact the store serves, at least once per pass. Which walk runs it, and how often, is the operator's choice on the walks screen, since a pass that reads the whole store is a cost.

Offer a setting

A SettingsContributor puts an extension's settings in the catalogue, so they appear on the settings screens, in GET /api/settings and in jenrepo settings for exactly as long as the module is installed:

public final class NoticeSettings implements SettingsContributor {

    @Override
    public List<Setting> settings() {
        return List.of(new Setting("notice-target", "Notices", "Notice target",
                "Where the notice drain delivers what the observer recorded. Empty delivers nothing.",
                Setting.Kind.URI, "", true).standard());
    }
}

The final true says the setting applies as soon as it is saved. A setting a module reads only as the server starts says false, and a write to it answers that it waits for a restart.

Add a console page

A ConsoleModuleProvider names a Spring @Configuration class that contributes the page's controllers, and the menu entries that lead to them. The page renders through the console's shared layout, so it looks like every other page:

public final class NoticeConsoleModule implements ConsoleModuleProvider {

    @Override
    public String name() {
        return "notices";
    }

    @Override
    public Class<?> configuration() {
        return NoticeConsoleConfig.class;   // a @Configuration declaring the page's @Controller
    }

    @Override
    public boolean enabledByDefault() {
        return true;
    }

    @Override
    public List<NavEntry> navEntries() {
        return List.of(new NavEntry("Notices", "/ui/notices", NavEntry.Access.ADMIN, NavEntry.Group.OPERATIONS));
    }
}

A page returns a view name and a model, never markup built as a string, and a page that needs to see the whole store starts that work off the request and shows its progress rather than waiting for it.

The same configuration can add panels to the console's dashboard: a DashboardContributor bean answers the panels a viewer sees - a title opening its page, a headline figure, a few lines - for the tenant they work in:

@Bean
DashboardContributor noticePanel(NoticeLog log) {
    return viewer -> List.of(new DashboardPanel("Notices", "/ui/notices",
            Integer.toString(log.pending(viewer.tenant())), "notices wait for delivery",
            DashboardPanel.Tone.NEUTRAL, List.of()));
}

A panel is drawn on every visit, so it costs a few point reads at most; a figure that needs every repository is counted in the background and read back with the time it was counted. A contributor that fails is shown as unreadable while the other panels render.

Store artifacts somewhere else

An ArtifactStoreProvider is a storage backend. It is chosen by name with jenrepo.store, and the filesystem store is the default when nothing is chosen:

public final class ExampleStoreProvider implements ArtifactStoreProvider {

    @Override
    public String name() {
        return "example";   // selected with JENREPO_STORE=example
    }

    @Override
    public ArtifactStore create(UnaryOperator<String> config) {
        return new ExampleStore(ArtifactStoreProvider.required(config, "jenrepo.example.bucket", name()));
    }
}

A backend is the one seam where the contract is long: compare-and-set writes, paged listings and a missing key answering as absent rather than as an error are what every other part of the product relies on. The store's contract kit is a test suite to run a new backend through before trusting it with artifacts.

Ship it in an image

The official image is built so that an image built FROM it can add modules without touching how it starts. Its module path ends with the folder /app/extensions/modulepath, and its class path with /app/extensions/classpath/*. The official image creates neither, and java skips a folder that does not exist, so they only matter once an image copies jars into them:

FROM jenesisbuild/jenesis-repository
COPY com.example.repository.notice.jar /app/extensions/modulepath/

docker build -t my-repository .
docker run -p 8080:8080 -v repository-data:/data my-repository

That is the whole image. A module that provides a service the server uses is resolved when the server starts and found by the seam's discovery, so no argument changes and the image needs no ENTRYPOINT of its own. Put jars in the two folders like this:

• A module goes in modulepath/, and so does any library it needs that the image does not already carry. A jar with no module-info of its own still works there, as an automatic module.
• A jar that cannot be a module goes in classpath/. It comes after the image's own class path, so it can add classes and META-INF/services entries but never replace a class the image already has.
• The image's own jars always win. Each is named on the module path ahead of the folder, so a jar of yours that carries a module the image already has is not the one loaded. Two jars claiming one package stop the server at start with an error naming both, rather than running with either.

JVM options - memory, system properties - go in JDK_JAVA_OPTIONS, which java reads before the image's own arguments:

ENV JDK_JAVA_OPTIONS="-XX:MaxRAMPercentage=75"

To confirm the extension was found, open Settings → Modules in the console and then Modules by contract, or run jenrepo spi: every seam is listed with the providers the running server discovered, yours among them.

The seams

Every seam below is a service interface in the free modules; the package is build.jenesis.repository followed by the part shown before the name. Every means each installed provider takes part, one, by setting means at most one is used and a setting names it when several are installed, and one means exactly one always resolves, with a built-in default.

Formats and storage

Screening what comes in

Background work

The server and access

The console and operations

The build cache