> ## Documentation Index
> Fetch the complete documentation index at: https://hercules.app/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent test cases

> Collect the questions you want your Hercules Agent to handle. Add them from real runs, generate them with AI, or import a spreadsheet, then group and tag them.

<Note>
  Agents is in private beta, so things may change. If you hit a problem, tell us via [Support &
  feedback](/docs/company/contact).
</Note>

A test case is one question you want your Agent to handle well, and optionally the answer you expect back. Collect enough of them and you can measure whether a change made your Agent better or worse.

Go to **Agents** → open an Agent → **Evaluate** → **Test cases**.

<div className="screenshot candy">
  <img src="https://mintcdn.com/zeus-0f6dadbf/AUzCpLx0wjCzKFM0/images/agents/test-cases-main.png?fit=max&auto=format&n=AUzCpLx0wjCzKFM0&q=85&s=6253357933940872e0ce9936680baaef" alt="A test case dataset open as a grid, with Input, Expected output, and Tags columns" width="2228" height="1311" data-path="images/agents/test-cases-main.png" />
</div>

### What's a dataset?

A dataset is a group of test cases. The Test cases page opens on your datasets, one row each. Click one to see its cases in a spreadsheet-style grid.

Tags label a case *within* its dataset, so the two do different jobs. A dataset is the set you run, and a tag narrows it down. An [evaluation](/docs/agents/evaluation-runs) picks one or more datasets, then filters by tag if you want.

### How do I add test cases?

Four ways:

| Way                 | How                                                                                          |
| ------------------- | -------------------------------------------------------------------------------------------- |
| **From a real run** | Open a run in [Run history](/docs/agents/run-history) and add it to your test cases               |
| **By hand**         | **New test case**, then write the question and the answer you expect                         |
| **Generate**        | **...** → **Generate test cases**. Hercules writes them for you, and you can give it a topic |
| **Import**          | Upload a spreadsheet (CSV) or JSON file, up to 200 rows at a time                            |

Adding from real runs is usually best. Those are the questions your Agent actually gets asked.

### What's in a test case?

| Field               | What it is                                       |
| ------------------- | ------------------------------------------------ |
| **Input**           | What gets sent to the Agent                      |
| **Expected output** | The answer you want back, used by most scorers   |
| **Tags**            | Labels for grouping cases within the dataset     |
| **Metadata**        | Any extra details you want to keep with the case |

### How do I import a spreadsheet?

<Steps>
  <Step title="Open the import dialog">Open a dataset, then **...** → **Import**.</Step>
  <Step title="Upload the file">A CSV with a header row, or a JSON file.</Step>

  <Step title="Match up the columns">
    Say which column is the Input, the Expected output, Tags, or Metadata, and skip the ones you
    don't need. Hercules guesses from your column names first.
  </Step>
</Steps>

### Do I need to fill in the expected output?

Only if your scorer needs one. Some [scorers](/docs/agents/scorers) compare the Agent's reply against the answer you expected, and those skip any case that doesn't have one. Scorers that judge the reply on its own grade every case.

When you start an [evaluation](/docs/agents/evaluation-runs), Hercules tells you how many of your cases it can actually run.

### What are tags for?

Narrowing a dataset down. Tag cases by topic (`billing`, `refunds`) or by risk (`edge-case`), then run an evaluation against only those.

### Additional FAQ

<AccordionGroup>
  <Accordion title="Can I export my test cases?">
    Yes. Export everything, or only the rows you've ticked, as CSV or JSON. The file you get back can
    be imported again.
  </Accordion>

  <Accordion title="How many test cases should I have?">
    Enough to cover what you care about, including the things that have gone wrong before. A small set
    of real, hard questions beats a big set of made-up ones. One evaluation covers at most 100 cases,
    so split bigger sets across datasets.
  </Accordion>

  <Accordion title="Does generating test cases cost credits?">
    Yes, generating uses AI and costs credits. See [Costs and limits](/docs/agents/costs-and-limits).
  </Accordion>

  <Accordion title="Can I edit a case after importing it?">
    Yes. Edit a cell in the grid, or click a row to open it and change anything, including its tags.
  </Accordion>
</AccordionGroup>