GitHub Action

thaghr/thaghr-action wraps pip install thaghr and thaghr run so you don't write the shell yourself. It fails the build when --fail-under isn't met.

Usage

- uses: thaghr/thaghr-action@v1
  with:
    example: examples/01-hello-agent
    trials: 20
    fault-rate: 0.2
    fail-under: 0.8

Inputs

Input	Required	Default	Description
example	yes	—	Path to a directory containing agent.py
trials	no	10	Number of trials to run
fault-rate	no	0.2	Probability an HTTP 429 fires per call, 0 disables fault injection
seed	no	0	RNG seed
max-cost	no	1.0	Hard cost cap in USD
k	no	1	k for the pass^k headline metric
fail-under	no	—	Fail the build if the primary metric is below this threshold. Omit to run informationally without gating.
thaghr-version	no	latest	Pin to a specific thaghr release, e.g. 0.1.1

Outputs

Output	Description
results-csv	Path to the CSV file of per-trial results

Example: gate pull requests, not main

A common pattern: run thaghr on every PR so a reliability regression blocks the merge, without re-running (and re-spending API budget on) every push to main:

name: Agent reliability
on: [pull_request]
jobs:
  thaghr:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: thaghr/thaghr-action@v1
        with:
          example: examples/01-hello-agent
          trials: 30
          fail-under: 0.85

Pinning a version

By default the Action installs whatever's latest on PyPI. If you want reproducible CI runs that don't change behavior out from under you on a new thaghr release, pin it:

- uses: thaghr/thaghr-action@v1
  with:
    example: examples/01-hello-agent
    thaghr-version: "0.1.1"
    fail-under: 0.8