Method and evidence grades
Every number on this site comes from a published source, carries a grade that says how solid the evidence is, and is only compared with results measured the same way.
What we include
- Results published in a peer-reviewed paper, a preprint (for example on arXiv) or a company’s technical documentation.
- Announcements and press releases, but only with a grade C warning, and never in rankings.
- Results contributed through the journal, after the same checks as everything else.
We never take a number from a news story alone: we go to the original source. If we cannot see the number in the source ourselves, we leave it out.
Evidence grades
- A Independent and peer-reviewed
- Measured by someone other than the machine’s maker, and published in a peer-reviewed journal. A peer-reviewed paper written by the maker is not grade A.
- B Documented method
- The method is written up, in a preprint, a journal paper by the maker, or the maker’s technical documentation, but nobody independent has repeated it yet.
- C Announcement only
- A press release, or a number without enough detail to check. Shown with a warning and never used for rankings.
Separately from the grade, every result says who measured it, whether it was peer-reviewed, and whether anyone else has repeated it.
Comparing like with like
Two results are only compared if they were measured the same way: the same benchmark, the same version of its method and a comparable set-up. For example, a median gate fidelity is never ranked against a single best pair, and logical qubits from codes that only detect errors are never ranked against codes that correct them. Each result’s page shows the group it belongs to.
Conflicts of interest
Many benchmarks were designed by companies that also build quantum computers. Where a maker helped create a benchmark, or tested its own machine, we say so on the benchmark and result pages. Contributed articles state any link between the author and the machine they write about.
No company pays to be listed, graded or ranked.
Releases and corrections
We publish the data as a numbered monthly release. A published release never changes. If we find a mistake, we correct it in a new release and keep the old record, marked as corrected or replaced, with a note of what changed and when. Results the maker disputes are marked as disputed and are not ranked.
To report an error, contact us and quote the result number (for example r-000001).
Rankings
We don’t rank machines here. Our sister site QubitRank builds its rankings only from grade A and B results in our published releases, and links every rank back to its record on this site.