Skip to main content

CLI reference

thaghr run

Run a fault-injection campaign against one agent.

thaghr run <example> [options]
FlagDefaultDescription
example (positional)Path to a directory containing agent.py
--trials10Number of trials to run
--max-cost1.0Hard cost cap in USD, the campaign stops and exits non-zero if exceeded
--fault-rate0.0Probability an HTTP 429 fires per call, 0 disables fault injection
--seed0RNG seed, for a reproducible fault sequence
--outputthaghr-results.csvWhere to write per-trial results
--k1k for the pass^k headline metric
--fail-underExit non-zero if the primary metric (pass^k, or GDS fallback) is below this threshold. Omit to run informationally without gating.
--prettyoffColorized report card via rich (pip install thaghr[pretty])

thaghr compare

Run a baseline (fault-free) and a faulted campaign back to back, report robustness and fault_tolerance.

thaghr compare <example> [options]
FlagDefaultDescription
example (positional)Path to a directory containing agent.py
--baseline-trials10Trials for the fault-free run
--fault-trials10Trials for the faulted run
--max-cost1.0Cap applied to each campaign separately, not combined
--fault-rate0.2Probability an HTTP 429 fires per call in the faulted campaign
--seed0RNG seed
--output-dir.Where to write baseline.csv and faulted.csv
--k1k for the pass^k headline metric
--prettyoffColorized report card via rich

thaghr proxy

Run thaghr as an HTTP proxy in front of an OpenAI-compatible endpoint. Point any OpenAI-compatible SDK's base_url at the proxy to get fault injection without changing your agent's code at all.

thaghr proxy --upstream https://api.openai.com [options]
FlagDefaultDescription
--upstreamRequired. Upstream OpenAI-compatible base URL
--fault-rate0.2Probability an HTTP 429 fires per call
--seed0RNG seed
--host127.0.0.1Host to bind
--port8135Port to bind

Point your SDK at it:

import openai

client = openai.OpenAI(base_url="http://127.0.0.1:8135/v1")

Next: GitHub Action to run any of this in CI.