Note

Run this notebook from the ARCADE repository root after pip install -e ".[torch,signature]" and, for paper configs, bash scripts/link_paper_data.sh.

Notebook: data stage in depth

This notebook walks Stage 1: Data in isolation. You will see how ARCADE loads traces, prepares labels, demodulates ADC when needed, splits shots, and writes caches. You will also learn which log lines to trust and how to debug shape, path, and bit-order failures before classification begins.

Prerequisites. Editable install, linked paper data when using bundled configs, and a short YAML that points at a real HDF5 file.

Pipeline position

[ Data ] → Features → Classify → Hardware → Report

Everything downstream assumes this stage froze the same traces and splits for every classifier. If demodulation or qubit_bit_order is wrong here, joint accuracy later will look mysteriously bad.

[ ]:

from pathlib import Path
import logging

from arcade._logging import configure_logging, get_logger
from arcade.config import load_config
from arcade.stages import DefaultDataStage

configure_logging(logging.INFO)
log = get_logger("tutorial.data")

# Prefer a small smoke config if you have one; otherwise the zoo YAML works
# and simply spends longer in later stages you will not run here.
CONFIG = Path("configs/example.yaml")
if not CONFIG.is_file():
    CONFIG = Path("configs/paper_all_methods_benchmark.yaml")

cfg = load_config(CONFIG)
print("data.path =", cfg.data.path)
print("num_qubits =", cfg.data.num_qubits)
print("qubit_bit_order =", cfg.data.qubit_bit_order)
print("demodulation.enabled =", cfg.data.demodulation.enabled)

Expected log pattern for a cold start

When the demod cache is missing, Stage 1 typically emits lines like:

[INFO] arcade.stages.data: === Stage 1: Data ===
[INFO] arcade.stages.data: Loading data from data/readout2019_adc.h5 (loader=hdf5)
[INFO] arcade.stages.data: Loaded: traces (...), labels (...)
[INFO] arcade.stages.data: After label preparation (format=...): unique labels=32
[INFO] arcade.stages.data: Split indices computed on raw labels: train=..., val=..., test=...
[INFO] arcade.stages.data: Demodulating raw ADC traces
[INFO] arcade.stages.data: Preprocessed traces: ...

When a demod cache hits, you instead see Loaded demod cache from ... and skips the ADC demodulation path. That is normal and faster on the second run.

[ ]:

# Run only the data stage. This is the primary debugging entry when loaders fail.
stage = DefaultDataStage()
data_out = stage.run(cfg)

# Inspect the prepared bundle keys your classify stage will see.
print(sorted(data_out.keys()) if isinstance(data_out, dict) else type(data_out))
if isinstance(data_out, dict):
    for key in ("traces", "labels", "splits", "label_set", "filter_set"):
        if key in data_out:
            val = data_out[key]
            shape = getattr(val, "shape", None)
            print(f"{key}: type={type(val).__name__} shape={shape}")

Debugging checklist for Stage 1

Symptom in logs / traceback	Likely cause	What to try
`FileNotFoundError` on `data.path`	Missing symlink or wrong path	Run `bash scripts/link_paper_data.sh` or fix YAML `data.path`
Loader / key errors from HDF5	Wrong `format`, `hdf5_keys`, or file contract	`arcade explore path --qubits N` then align YAML keys
Joint labels look like `2**N` classes but accuracy collapses later	Wrong `qubit_bit_order`	Compare `lsb0` vs `msb0` against the paper config
Demod runs every time and is slow	Cache disabled or path unset	Enable demod cache fields under `data.demodulation`
`unique labels` unexpected	`label_format` / `num_levels` mismatch	Inspect `LabelSet` and raw label histogram

Raise log detail

configure_logging(logging.DEBUG)

DEBUG is noisy. Use it when a single loader or demod helper fails without a clear INFO line.

[ ]:

# Validate config before blaming the loader.
from arcade.config import validate_and_explain

errors = validate_and_explain(CONFIG)
print("config valid:" if not errors else "config invalid:", not bool(errors))
for m in errors:
    print(" -", m)

Next

Continue with Classify stage using the same data_out object, or re-run data inside a full arcade.run once Stage 1 looks clean.