Models As Code

Models as code: why your synthetic data should live in version control

Infrastructure as code changed how teams manage servers, networks, and deployments. Instead of a wiki page describing what should be running, you get a file that describes what is running, reviewable in a pull request, diffable across changes, with a clear history of who changed what and why.

Test data almost never gets the same treatment. It’s usually something like a mystery CSV sitting in a shared drive, or a database snapshot someone took eighteen months ago and nobody’s touched since — no history, no review, no way to know what changed between “the tests passed yesterday” and “they don’t today.”

SpiceGrinder models are plain text — either XML or JSON, whichever you prefer — so they already get everything your existing version control is good at, for free. A model file simply lives in your repo, right next to the code it tests, if you set it up that way. Changes to a model show up in git diff. Reviewers can see exactly what shifted in a pull request, the same way they’d review a Terraform plan.

Here’s what that actually looks like. Say your actuarial team revises a loss-frequency assumption — claims are running hotter than the model expected:

- <Poisson name="Frequency" lambda="3.5"/>
+ <Poisson name="Frequency" lambda="4.2"/>

That’s the whole change. Anyone reviewing the pull request can see exactly what moved, why (your commit message says so), and when — the same review discipline you’d expect from a change to production code, applied to the data your tests actually run against.

Determinism is what makes this more than just a diff, though. A Terraform file only means something because terraform apply against it always produces the same infrastructure. A SpiceGrinder model only means something the same way because the same model plus the same seed always produces the exact same data — anywhere, any time, on anyone’s machine. That’s the difference between “a file that changed” and “a reproducable dataset” — a teammate can pull your branch, run the exact model you committed, and get byte-identical output to what you tested against, without you having to hand them a data export. You’ll be able to go back days, weeks, months… however far back your repository history goes to know exactly what tests you ran.

And when you do run it, you don’t have to take it on faith that it ran the way you think — turning the --audit flag on when generating data gives you a real record:

{"event.action":"run.summary","message":"model run summary",
 "spicegrinder.model.name":"SixStats",
 "spicegrinder.model.path":"samples/dnd-ability-scores.xml",
 "spicegrinder.seed":"6360338997739602050",
 "spicegrinder.observation_count":5,
 "spicegrinder.node_count":6,
 "spicegrinder.run.duration_ms":2,
 "spicegrinder.complexity_score":401}

The model name, the exact seed used, how long it took — the record is timestamped, so you’ll even know exactly when it ran — what you get is an audit trail of every generation run, in the same structured-logging format your infrastructure tooling already expects.

That is what “test data as code” actually means in practice. Your models can go through the exact same review, CI, and change-management process your infrastructure and your source code (and unit tests) already do.