Skip to content

CI/CD

GitHub + Cloudmind

Run your Cloudmind eval suite as a required GitHub check and get the score delta commented on every pull request.

Start freeAll integrations

How it works

The GitHub Action runs your eval suite against a golden dataset and posts the result as a check. Score deltas appear as a PR comment showing which examples improved and which regressed, with a link to the full run.

Configure a threshold to fail the check when quality drops, which is what turns evals from a report into a gate.

Setup

- name: Run Cloudmind evals
  uses: cloudmind-ai/eval-action@v1
  with:
    api-key: ${{ secrets.CLOUDMIND_API_KEY }}
    suite: support-quality
    fail-under: 0.85

Supported

  • Required status checks

  • PR score delta comments

  • Per-example diffs

  • Configurable failure thresholds

  • Matrix runs across models

  • Artifact upload

Stop guessing whether your AI feature got better

Free for 50,000 traces a month. No credit card, no sales call, five minutes to your first trace.

Start free