Compare classifiers

This guide walks through a fair bake-off on your own traces or on the bundled paper dataset. Fair means every method sees the same prepared shots, the same train validation and test cuts, the same shared filter bank unless a method explicitly opts out, and the same metric definitions in the summary report. If those ingredients differ between methods, the ranking is not comparable and should not be treated as a leaderboard.

If you have never completed a zoo run, finish First zoo run first so output directories and report filenames are familiar.

Write a minimal configuration

Create a YAML file that points at your data, lists the methods to compare, and enables the optional stages you care about. A typical starting point looks like this:

data:
  path: /path/to/your_traces.h5
  format: hdf5
  num_qubits: 5

classifiers:
  run: [threshold, lda, qda, svm, fnn, herqules]
  filter_types: [MF, RMF]

sweep:
  enabled: true

hardware:
  enabled: true

visualization:
  output_dir: output/my_bakeoff
  format: pdf

Fill in demodulation, preprocessing, and split fields so they match the contract of your file. Those fields belong under data once for the whole run. Do not change them between methods. Exact field names are documented in Config schema.

Validate, then run

arcade validate path/to/my_config.yaml
arcade run path/to/my_config.yaml

Validation catches missing required fields and type errors before training starts. The run command executes data preparation, classification for every name in classifiers.run, optional hardware estimation, and report writing into visualization.output_dir.

From Python the same entry point is available as:

import arcade

arcade.run("path/to/my_config.yaml")

Keep the recipe frozen

Changing demodulation, boxcar windowing, split ratios, or qubit_bit_order between methods invalidates the ranking. Put those knobs under data once. List methods only under classifiers.run. The philosophy behind that rule is explained in Shared recipe.

What fair means in practice

Every name in classifiers.run sees the same train, validation, and test cuts. Filter-based methods share filter_types unless a method-specific block overrides them. When the duration sweep is enabled, ARCADE truncates traces at the same lengths for every method. The comparison table in the summary report is the artifact you should share in a lab meeting or paper supplement.

Choose smoke, zoo, or paper fidelity

Intent	Configuration pattern
Fast sanity check	run: [threshold, lda] with a small dataset or short training
Public multi-method bake-off	configs/paper_all_methods_benchmark.yaml
Paper-faithful reproduction	a dedicated configs/paper_*.yaml file that may differ slightly from the zoo recipe

List registered names with arcade list. Inspect one method with arcade describe followed by the registry name. Support levels and pitfalls notes live in Classifiers reference.

After the run

Open the summary report under the configured output directory and follow Read the report so that joint accuracy, F5Q, and hardware columns are interpreted correctly before you declare a winner.