AgenticOpsfor productionAI agents and workflows.

A pre-release draft specification for describing runs, steps, model/tool calls, evaluations, and incidents. Product integrations are at different stages; stable conformance is not yet claimed.

What it defines

• Runs, steps & incidents schema
• Model & tool call vocabulary
• Evaluation & safety signals
• JSON Schema artifacts, v0.1–v0.4

• schema_valid
• semantic_valid
• aligned: true

Conformance check example: schema valid, semantic valid, aligned true.

A shared draft that supported integrations can validate against.

OpenTelemetry-native observability for production AI: ingest OTLP traces, track cost and latency per run, and surface evidence-backed findings for where to cut waste.

Observe · Make agents observable

• End-to-end observability
• Runtime evaluation
• Explain findings
• Evidence-backed recommendations
• Export reports
• Compare runs
• Audit evidence

Cost per run trending down across five recent runs, from $0.087 to $0.041, alongside six optimization findings.

Cost trending down over five runs after acting on its own findings.

Framework-agnostic scoring for any trace-shaped run — deterministic checks, rubric scorers, and LLM-as-judge evaluators, wired into a pass/fail gate you drop into CI.

Evaluate · Measure what matters

• Score agent outputs
• Benchmark runs
• LLM-as-a-Judge
• Trajectory evaluations
• Reusable evaluation suites
• Release gates
• Regression testing
• CI/CD quality gates

• trajectory-quality ≥ 0.850.91
• cost-per-run ≤ $0.05$0.031
• hallucination-rate ≤ 2%1.8%

Release gate: trajectory quality 0.91 passes an 0.85 threshold, cost per run $0.031 passes a $0.05 cap, hallucination rate 1.8% sits inside a 2% warning band.

Three thresholds, one CI gate — the merge blocks until every run clears them.

Fault injection and topology-aware drift testing across agent frameworks — provider failures, corrupted outputs, and tool errors, forced before production does it for you.

Break · Make your system more resilient

• Break agent workflows safely
• Validate resilience behavior
• Test stress scenarios
• Inject controlled faults
• Produce resilience evidence
• Measure recovery
• Improve robustness

• Recovered · 68%
• Degraded · 22%
• Failed · 10%

Outcome of 120 injected faults: 68% recovered, 22% degraded, 10% failed.

One in ten injected faults broke the agent outright — before a user hit it.

A real-time Decision Gate that watches an agent's next action against its declared intent — catching moves that are technically permitted but contextually wrong, not just disallowed content.

Supervise · Keep agents safe and aligned

• Supervise and govern agents
• Align actions with intent
• Judge / Critic modules
• Escalate for human approval
• Show live agent status
• Replan, pause, or approve
• Re-inject intent and context
• Apply agent guardrails

• Allow · 462
• Warn · 29
• Block · 9

Decision Gate outcomes across 500 actions: 462 allowed, 29 warned, 9 blocked as out of scope.

Escalating intervention, not a failure rate — a block is the Sidecar working.

A unified dashboard to monitor your AI agent fleet, review shared evidence, track alerts, and manage approved operations.

A unified dashboard to manage all your AI agents in one place

• SQLite-backed agent registry
• Capability and version inventory
• Fleet health and evidence posture
• Alerts, incidents, and approved operations

• Healthy · 41
• Degraded · 5
• Unhealthy · 2

Fleet status across 48 registered agents: 41 healthy, 5 degraded, 2 unhealthy.

One registry for every agent's runtime, framework, and capability versions.

A unified MCP interface over the shared model — exposing AgenticLens, Agentic Chaos, Agentic Sidecar, and Spec validation as one set of tools any MCP client can call.

Capabilities

• 14 callable tools
• Wraps Lens, Chaos, Sidecar & Spec
• 2 shared resources
• 3 prompt templates
• stdio transport, session state
• One identity for any MCP client

14 tools exposed, 1,284 calls routed today, 4 of 5 sibling adapters connected.

One MCP-native surface over every framework above, degrading honestly when one's missing.