Skip to main content
The Axiom AI SDK CLI provides commands for running evaluations locally or in CI/CD pipelines.

​
Run evaluations

The simplest way to run evaluations is to execute all of them in your project:
You can also target specific evaluations by name, file path, or glob pattern:
To see which evaluations are available without running them:

​
Common options

For quick local testing without sending traces to Axiom, use debug mode:
To compare results against a previous evaluation, view both runs in the Axiom Console where you can analyze differences in scores, latency, and cost.

​
Run experiments with flags

Flags let you test different configurations without changing code. Override flag values directly in the command:
For complex experiments, load flag overrides from a JSON file:

​
Understanding output

When you run an evaluation, the CLI shows progress, scores, and a link to view detailed results in the Axiom Console:
Click the link to view results in the Console, compare runs, and analyze performance.

​
What’s next?

To learn how to view and analyze evaluation results, see Analyze results.