Don’t let a silent prompt drift become a regulatory fine.
Golden datasets, trajectory assertions and CI regression gates for AI agents in healthcare, finance and law.
Instruments the agent stack you already run
LangChain
LlamaIndex
CrewAI
AutoGen
OpenAI Agents SDK
Anthropic
AWS Bedrock
GitHub Actions
GitLab CI
Generic benchmarks give you false confidence. Regulated production breaks in the edge cases.
Your agent passed every unit test on Monday. On Tuesday a prompt tweak meant to improve conciseness broke the JSON tool-call schema for insurance eligibility verification. In consumer chat that is a minor bug. In healthcare and finance it is a compliance incident with your name on it.
- of prompt-only pull requests changed at least one tool-call trajectory in a multi-turn suite while the final-answer score stayed flat.
- 41%
- of prompt-only pull requests changed at least one tool-call trajectory in a multi-turn suite while the final-answer score stayed flat.
- GONATURELLE regression study, 2026 (1,180 PRs across 14 production agents)
- Hallucination rate measured in leading AI legal research tools on a preregistered query set.
- 17% to 33%
- Hallucination rate measured in leading AI legal research tools on a preregistered query set.
- Magesh et al., Stanford RegLab and HAI, "Hallucination-Free?" (2024)
- Best agent's pass^8 on tau-bench retail: solving the same task correctly in all 8 tries.
- under 25%
- Best agent's pass^8 on tau-bench retail: solving the same task correctly in all 8 tries.
- Yao et al., tau-bench, arXiv:2406.12045 (2024)
- Maximum EU AI Act fine for breaching high-risk system obligations, of worldwide annual turnover.
- €15M or 3%
- Maximum EU AI Act fine for breaching high-risk system obligations, of worldwide annual turnover.
- Regulation (EU) 2024/1689 (EU AI Act), Article 99
From pack to blocked merge in four steps.
No rewrite of your LangGraph, LlamaIndex, CrewAI or in-house loop. GONATURELLE wraps the agent you have.
How the platform works- 01
Select a vertical pack
Choose expert-validated golden datasets for clinical triage, loan underwriting, contract analysis or claims processing, or upload your own cases.
healthcare/clinical-prior-auth-v2fintech/adverse-action-reg-blegal/citation-verification - 02
Instrument the trajectory
Three lines of Python or TypeScript wrap your agent loop and capture every tool call, intermediate step and API response.
@gonaturelle.trace(agent="prior-auth") - 03
Run calibrated judges
Score outputs against domain rubrics with judge ensembles calibrated to MD, JD and CPA consensus, with agreement published per judge.
κ 0.87 - 04
Gate the merge
Block pull requests in GitHub Actions when pass rates drop, latency spikes or a safety guardrail fails. Export the audit record.
gonaturelle/gate2 regressions · merge blocked
Evaluating the final answer is not enough. GONATURELLE evaluates the entire trajectory.
An agent can land on the right dose by accident after extracting an out-of-bounds parameter and skipping the allergy check. String-matching evals inspect one step: the last one. GONATURELLE asserts on every tool invocation, parameter boundary, state transition and reasoning step against deterministic constraints.
- Sequence constraints: verify identity before reading balances
- Parameter bounds from formularies, rate tables and policy limits
- Hallucinated arguments that no upstream step produced
- Cycle detection for runaway tool loops
- 01 · User intent"Dose amoxicillin for my 4-year-old, 16 kg, acute otitis media."
- 02 · Agent planlookup_guideline → compute_dose → check_allergies → respond
- 03 · Tool selectioncompute_dose(drug, weight_kg, mg_per_kg_day, divisions)
- 04 · Parameter extractionmg_per_kg_day = 900 (bound: 80 to 90, AAP high-dose regimen)
- 05 · Tool execution→ 7,200 mg/day, clamped by tool to max adult 4,000 mg/day
- 06 · State mutationallergy check skipped: plan step 3 never executed
- 07 · Response synthesis"Give 640 mg twice daily for 10 days." (matches reference by chance)
Six capabilities, one deployment gate.
assert_order("verify_identity",
"fetch_account_balance")Compliance ReportsEvery gated run becomes evidence: Annex IV sections, GMLP validation summaries, model-governance packs.Built for the five verticals where a wrong answer is a reportable event.
4 golden packs
Show allA note summarizer drops "patient on apixaban" from the medication list; downstream the triage agent clears a fall patient without a head CT flag.
- Critical Hallucination Rate
- Diagnostic Omission Index
- Contraindication Severity Score
Three lines into your agent loop. One YAML file into CI.
Load a versioned suite, evaluate with bounded concurrency, and assert like you would in pytest or Vitest. The GitHub Action posts results to the PR and fails the check on regression.
- Median fast-path gate
- 74 s
- Supported runtimes
- Python 3.10+
Node.js 20+
GONATURELLE platform telemetry, Q3 2026
from gonaturelle import EvalSuite, TrajectoryTester
suite = EvalSuite.load("healthcare/clinical-prior-auth-v2")
results = suite.evaluate(agent=my_clinical_agent, max_concurrency=10)
results.assert_zero_contraindications()
results.assert_tool_schema_valid()
# Sequence constraint: identity before benefits, every trajectory
TrajectoryTester(results).assert_order("verify_member", "fetch_benefits")“Trajectory assertions caught a broken SQL tool schema that our string evals had passed for three weeks. Manual prompt QA went from two days a sprint to a PR check.”
Security your CISO signs off on in one review.
- HIPAA BAA on every planSigned before your first PHI-bearing test case is uploaded, Developer tier included.
- SOC 2 Type IIAnnual audit of security, availability and confidentiality. Report and bridge letter in the Trust Center.
- Dedicated VPC or air-gapped KubernetesRun sandboxes and judges inside your AWS, GCP or Azure account with zero egress.
- Zero training on your dataZero-data-retention agreements with every model provider we call. Your prompts and cases never train a model.
- Signed, immutable run recordsSHA-256 manifests over datasets, prompts, model versions and scores for every gated run.
- SAML SSO, SCIM, RBACOkta and Microsoft Entra ID, per-suite roles, and a full audit log of who changed which threshold.
Frequently asked questions
How does GONATURELLE differ from LangSmith or Braintrust?
LangSmith and Braintrust are general-purpose tracing, experimentation and evaluation platforms, and many teams keep them for observability. GONATURELLE is narrower and deeper: expert-validated vertical golden datasets, deterministic trajectory assertions (sequence, schema, boundary), judges calibrated against domain experts with published agreement, and a CI gate whose output is an audit record. Several customers export traces from LangSmith into GONATURELLE suites.
How are your golden datasets created and verified?
Each case is drafted from de-identified real-world patterns or regulatory edge-traps, then independently labelled by at least two credentialed reviewers (physicians, attorneys, CPAs or former examiners). Disagreements go to adjudication, and we publish per-pack inter-annotator agreement. Packs are semantically versioned so a score on v2.3.0 always means the same thing.
Can we use our own proprietary test cases?
Yes. Bring-your-own-data uploads accept JSONL, CSV or traces from your observability tool. Cases are de-identified on ingest (configurable PHI and PII detectors), tagged, versioned and can be mixed with GONATURELLE packs in a single suite.
How fast do tests run in CI/CD?
The fast path runs 20 critical cases with three judges on every push; the median gate in GitHub Actions is 74 seconds (GONATURELLE platform telemetry, Q3 2026). The full suite of up to several thousand cases runs on merge requests or nightly, sharded across sandboxed runners.
Do you store our prompts or patient data?
Test inputs, outputs and traces are stored encrypted in your chosen region (US or India) for the retention period you set, or never leave your VPC on Enterprise. We operate under a BAA, use zero-data-retention terms with model providers, and never use customer data to train models.
What models can we evaluate?
Any agent you can call: hosted models from OpenAI, Anthropic, Google and AWS Bedrock, open-weight models on your own inference stack, and custom in-house agent loops. GONATURELLE evaluates the agent's behaviour through its tool calls and outputs, so it does not need model weights.
Stop testing regulated agents on generic prompts.
Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.