Observability

A repository you depend on needs to answer three questions: what is it doing, is it healthy, and is it configured safely. Jenesis Repository answers them over plain HTTP, so a console, a script or a monitoring system reads the same endpoints. This chapter covers the log tail, the health and metrics endpoints, the security-posture report, and the consistency check that keeps several nodes honest.

Who may read these endpoints

GET /api/logs, GET /api/consistency and the /actuator endpoints show deployment-wide state - every repository's log lines, every node, every metric. They are therefore gated to a key that holds a deployment-wide * grant; a key scoped to one repository is refused, and so is a keyless caller even when anonymous read rights are granted. On a deployment that runs with authentication off, anyone can read them. /actuator/health is always open, so probes need no credential.

Recent logs

The server keeps its most recent log entries in memory and serves them at GET /api/logs, which lets you read the tail without shell access to the host:

curl -H 'Jenesis-Repository-Key: jenk_…' \
  'http://localhost:8080/api/logs?level=WARN&q=proxy&limit=50'

{"cursor":1834,"count":2,"entries":[
  {"seq":1821,"timestamp":"2026-08-21T09:14:02.118Z","level":"WARN",
   "logger":"build.jenesis.observation","message":"jenreg.proxy.fetch [format=maven, outcome=miss] failed: …",
   "tenant":"default"}]}

The buffer holds the last 1 000 entries by default (jenreg.logs-buffer); older lines are gone from this view, but still reach your normal log output. Each entry carries a seq, so polling with since=<cursor> never repeats or skips a line.

The server's own operations log under the logger build.jenesis.observation: a completed operation at INFO, a failed one at WARN with its error. Today the one operation instrumented this way is the proxy fetch, jenreg.proxy.fetch, tagged with the format it served and the outcome - hit (served locally), miss (fetched from the upstream), negative (a remembered upstream 404), verified or withheld. Filter on q=proxy.fetch to watch your pull-through cache work. The same logger writes one line per HTTP request, http.server.requests, with its method, path and status - the server's access log; nothing else is logged per request.

Health and metrics

The server exposes three Spring Boot Actuator endpoints: /actuator/health, /actuator/info and /actuator/metrics. Health serves the liveness and readiness probes (/actuator/health/liveness, /actuator/health/readiness) a container platform expects, is never rate-limited, and is readable without a credential. Its detail is shown only to an authorised caller (management.endpoint.health.show-details is when-authorized); an anonymous probe sees up or down alone.

/actuator/metrics lists the JVM and HTTP request meters Spring Boot collects - request counts by URI and status, memory, threads. The proxy-fetch operation described above is also an observation, so a Micrometer registry on the module path receives it as a timer tagged with format and outcome. The server ships no registry of its own; the endpoint serves what the registry you install collects.

Security posture

GET /api/posture lists every setting that leaves the deployment less safe than it could be, each with the reason and the exact key and value that fix it. It never echoes a secret, so it is safe to surface on a dashboard:

curl -H 'Jenesis-Repository-Key: jenk_…' http://localhost:8080/api/posture

{"count":2,"critical":1,"warn":1,"info":0,"advisories":[
  {"id":"jenreg.auth.open","severity":"CRITICAL","scope":"DEPLOYMENT","tenant":null,
   "title":"Authorization is disabled - the instance is fully open",
   "why":"…","fix":"…","settingKey":"jenreg.auth","settingValue":"true","docs":"…"}]}

The advisories the server raises:

A clean deployment returns an empty list. The same report is shown in the console's Security posture panel, and the jenreg.auth.open advisory is also logged once at boot.

Running more than one node

Every node is stateless; the store is the only state, so several nodes behind a load balancer share one bucket or one mounted directory and serve the same content. Each node derives some state from the store - its view of the index, its configuration - and a node that falls behind or is configured differently would answer differently from its peers. The consistency check makes that visible.

Switch it on with jenreg.consistency.enabled=true on every node. Each node then publishes a small fingerprint of its derived state to the store on a heartbeat, and GET /api/consistency on any node compares all of them:

curl -H 'Jenesis-Repository-Key: jenk_…' http://localhost:8080/api/consistency

{"localNodeId":"repo-2","nodeCount":3,"liveCount":3,"converged":true,"singleNode":false,"truncated":false,
 "nodes":[{"nodeId":"repo-1","live":true,"stale":false,"heartbeatAgeMillis":4120,
           "indexCursor":128934,"snapshotVersion":"…","configGeneration":"3f2a…",
           "inventoryTotal":12894,"quotaUsed":73400320,"local":false}],
 "divergences":[]}

A node is live while its heartbeat is younger than jenreg.consistency.dead-after; only live nodes take part in the comparison, and a fleet of one live node is always converged. A node whose last heartbeat is older than staleness-window is flagged stale but stays in the comparison until dead-after. Three kinds of divergence are reported, and each also appears in the security-posture report under its own id:

The check only reports; it never blocks a request.

A node names itself with jenreg.consistency.node-id; unset, it uses the hostname, and falls back to a generated id (with a warning) only when no hostname is available. Give each node a stable id so a restart re-uses its fingerprint. Without the setting switched on, a node publishes nothing and writes no operational keys into the store.

A fingerprint whose node has been silent for longer than jenreg.consistency.forget-after (a day by default) is deleted by the next node that publishes, so a fleet that gives every restart a fresh hostname does not accumulate a fingerprint per host it ever ran on. A report reads at most 1 000 fingerprints; past that it answers with "truncated":true and compares the ones it read.

Settings

Every jenreg.* key is also an environment variable (JENREG_CONSISTENCY_ENABLED=true) or a -D system property.