Runtime 1: One Runtime, Many Callers

The problem

A SpiceGrinder model isn’t free to load — parsing the file, resolving <Import> nodes, and building the actual generator tree (with its own seeded RNG) is real work, and if the model carries a fixed seed, rebuilding it from scratch on every call means restarting that same RNG sequence every time, not continuing it forward. If a QA engineer’s script, a nightly CI job, and a teammate’s ad-hoc test all need draws from the same model, you don’t want each of them getting the same output back or paying that setup cost unnecessarily.

ModelServiceApp exists to let one running process serve all of these callers efficiently, so let’s look at exactly what it caches, and when.

Register isn’t build

Registering a model doesn’t build it:

POST /v1/models
<contents of claim-severity.xml>

{"modelId": "m-626f4de7-7138-4049-962d-fd43ba807a77", "name": null, "keepResident": false,
 "analysis": {"mode": "FULLY_COMPUTED", "nodeCount": 1, "minDepth": 1, "maxDepth": 1,
              "score": 6, "summary": "fully computed: nodes=1 minDepth=1 maxDepth=1 predictedAvgDepth=1.000 score=6"}}

That response comes back fast, and it tells you that the file was parsed and statically analyzed (node count, depth, a complexity score) and gives you an ID to reference the model by but nothing has been instantiated yet. No generator tree exist, no RNG has been seeded. The model is sitting in the service’s shared store as text plus metadata, available to any process that shares that store, not just the one you registered it against.

The data store is persistent — the model will remain until deleted, no matter how long it’s left in there

The first call builds it; the next one reuses it

The generator tree gets built the first time something actually asks for data:

POST /v1/models/m-626f4de7.../generate
{"count": 100}

{"modelId": "m-626f4de7...", "generationId": "g-ecddcebf...", "count": 100,
 "seed": "6936034287819168976", "seedSource": "random", "rngContinued": false, ...}

rngContinued: false indicates that there was no resident graph for this model yet, so this call built one from scratch, seeded it with a fresh random seed (since none was provided), and cached it before generating. Call /generate again with the identical request:

{"modelId": "m-626f4de7...", "generationId": "g-a2d59cb4...", "count": 100,
 "seed": "6936034287819168976", "seedSource": "random", "rngContinued": true, ...}

Same seed — because it’s the same graph reporting the same seed it was originally built with, not a new one — but now rngContinued: true. This call skipped the build step entirely and reused the graph still sitting in memory from the first call, picking up its RNG sequence where it left off rather than restarting it. That reuse is exactly what makes “one runtime, many callers” work: the second caller (and the third, and the tenth) isn’t paying the build cost the first caller already paid, and instead continues that same generator’s sequence rather than restarting it.

What “stays loaded” actually means

Reuse isn’t unconditional. Each process caches a bounded number of resident graphs (max.resident.graphs, default 20) — register and load more distinct models than that on one process and the least-recently-used (LRU) one gets evicted to make room, with the next call to it require rebuilding it from the shared store.

keepResident=true at registration time pins a model against that eviction, for a model important enough that you never want the rebuild cost, however rarely it’d actually come up. Of course, if you pin so many models the cache is still too full, one of them will have to be kicked out (using the same LRU logic), so try to pin models only when necessary to avoid this.

A resident graph that just sits idle long enough (idle.timeout.seconds, default 900) is dropped out of cache too, unless it’s pinned — “loaded” isn’t “loaded forever,” it’s “loaded for as long as it’s actually being used, or you’ve told the process not to forget it.”

Neither the cap nor the eviction touches the registered model. Deleting a resident graph from memory doesn’t delete the model from the shared store; it just means the next call rebuilds it, with a new random number engine. What gets discarded is the temporary in-memory structure, never the source of truth.

Optimization

The cap of 20 models loaded into memory should allow you to run the service smoothly on any hardware capable of running SpiceGrinder. If you have the extra resources, feel free to scale the limit higher if you want to be able to cache more models in memory at one time.

Running a model from memory instead of reloading it is more efficient, but don’t let that scare you, either. The process of loading a model is still relatively quick. We recommend never pinning a model just for speed concerns. Pinning is intended to make sure the models that need to be determinsitically reproducable don’t have to reset their random number generator.

Because SpiceGrinder is written in Java, the runtime optimizes as it goes — a long-running process like this winds up being more efficient than running the command line over and over. The overhead of switching in-memory models or even reloading from the shared store is generally less that the overall performance gain you get from a long-running process.