# Compare classifiers

This guide walks through a fair bake-off on your own traces or on the bundled
paper dataset. Fair means every method sees the same prepared shots, the same
train validation and test cuts, the same shared filter bank unless a method
explicitly opts out, and the same metric definitions in the summary report.
If those ingredients differ between methods, the ranking is not comparable and
should not be treated as a leaderboard.

If you have never completed a zoo run, finish {doc}`../tutorials/first_zoo_run`
first so output directories and report filenames are familiar.

## Write a minimal configuration

Create a YAML file that points at your data, lists the methods to compare, and
enables the optional stages you care about. A typical starting point looks
like this:

```yaml
data:
  path: /path/to/your_traces.h5
  format: hdf5
  num_qubits: 5

classifiers:
  run: [threshold, lda, qda, svm, fnn, herqules]
  filter_types: [MF, RMF]

sweep:
  enabled: true

hardware:
  enabled: true

visualization:
  output_dir: output/my_bakeoff
  format: pdf
```

Fill in demodulation, preprocessing, and split fields so they match the
contract of your file. Those fields belong under ``data`` once for the whole
run. Do not change them between methods. Exact field names are documented in
{doc}`../reference/config`.

## Validate, then run

```bash
arcade validate path/to/my_config.yaml
arcade run path/to/my_config.yaml
```

Validation catches missing required fields and type errors before training
starts. The run command executes data preparation, classification for every
name in ``classifiers.run``, optional hardware estimation, and report
writing into ``visualization.output_dir``.

From Python the same entry point is available as:

```python
import arcade

arcade.run("path/to/my_config.yaml")
```

## Keep the recipe frozen

Changing demodulation, boxcar windowing, split ratios, or
``qubit_bit_order`` between methods invalidates the ranking. Put those knobs
under ``data`` once. List methods only under ``classifiers.run``. The
philosophy behind that rule is explained in
{doc}`../explanation/shared_recipe`.

## What fair means in practice

Every name in ``classifiers.run`` sees the same train, validation, and test
cuts. Filter-based methods share ``filter_types`` unless a method-specific
block overrides them. When the duration sweep is enabled, ARCADE truncates
traces at the same lengths for every method. The comparison table in the
summary report is the artifact you should share in a lab meeting or paper
supplement.

## Choose smoke, zoo, or paper fidelity

| Intent | Configuration pattern |
|--------|------------------------|
| Fast sanity check | ``run: [threshold, lda]`` with a small dataset or short training |
| Public multi-method bake-off | ``configs/paper_all_methods_benchmark.yaml`` |
| Paper-faithful reproduction | a dedicated ``configs/paper_*.yaml`` file that may differ slightly from the zoo recipe |

List registered names with ``arcade list``. Inspect one method with
``arcade describe`` followed by the registry name. Support levels and pitfalls
notes live in {doc}`../reference/classifiers`.

## After the run

Open the summary report under the configured output directory and follow
{doc}`../tutorials/read_the_report` so that joint accuracy, F5Q, and hardware
columns are interpreted correctly before you declare a winner.