How we verify a figure
The steps every record goes through, from fetching a source to review, what the confidence and verifiedAt fields mean, and how corrections are recorded.
Tutorial · Updated
Records pass through five steps. Each step is there to catch a different kind of mistake. The first four apply to every record. The fifth, specialist review, has so far reached only part of the dataset; the methodology page gives the count.
1. Capture
We fetch the source page and store an exact copy of what came back. We also record a SHA-256 fingerprint: a short code computed from the stored bytes, which changes if even one character changes. Later checks are made against this stored copy, so an edit to the live page cannot quietly change what we reported.
A page can load successfully and still be empty. Some websites send a blank shell and fill it in with scripts afterwards. So the capture step also measures how much readable text a page contains, and marks a nearly empty page as thin. Thin pages are fetched again with a full web browser, which runs those scripts.
2. Extract
A program reads the stored copies and drafts records from them. Two rules apply here.
- Nothing is extracted as achieved. Every figure enters as claimed. Only a specialist review can promote it to achieved, and only with a date of measurement.
- What a source says is kept apart from what we added. Some facts in a draft come from outside the captured page: a classification, for example, or a founding year from general knowledge. These enter at the lowest confidence, with no verification date, and the record says so in its notes.
3. Validate
The whole dataset is checked against its schema, a formal description of which fields each record may have and what values they may take. Some errors cannot get through at all. For example, the schema has no field that could hold a gain figure without saying what it was divided by. Any error fails the automated check that runs on every change to the repository. Warnings are reported too. One marks a record whose content rests entirely on added facts, with nothing taken from a captured source.
4. Ledger
Every change is written to a ledger: a log that is only ever added to. Each entry includes a fingerprint of the entry before it, so altering or deleting an old entry would break the chain, and a routine check would detect it. Records are never deleted to settle a problem. Their status is lowered and a note explains why. The changelog is built from this ledger. It shows measurement status changes, retractions, records added, sources that went dead or changed, and candidates we declined to list. Other field-level edits are kept in the ledger but are not yet shown on the changelog.
5. Review
Specialist review passes then check records within one subject each: plasma physics, neutron physics, nuclear regulation, and funding. A review pass opens the source, compares it with the record, and corrects, downgrades or promotes the record. For a regulatory milestone, that means finding the step in the regulator’s own records, such as a Federal Register notice or a docket.
Confidence and verifiedAt
Two fields record how far a record has been checked.
Confidence is high, medium or low. It is our judgement of how well the evidence supports the record. Anything resting on facts added from outside a captured source enters as low.
verifiedAt is the date the source was last opened in a verification run and the record checked against it. If it is empty, that has not happened yet. It is only set when the source was actually opened during that run and supported the value. A related field, verifiedFields, lists which fields were checked, because a record can have its name and website confirmed while its classification has not been.
Each figure on a device page shows its confidence beside its sources. The methodology page states how many measurements and organisations have been verified so far, and the timeline marks verified entries.
Corrections
When a record is wrong, it is corrected in the open. The correction goes into the ledger. If it changes a measurement’s status or retracts a record, it also appears on the changelog and in the corrections feed at /feed.xml. Other field-level corrections are kept in the ledger but are not yet shown on either. Old values are not erased. A superseded or retracted figure stays in the record with its new status.
Reporting an error
Email info@hadamard2.com. Name the page and the figure, and if you can, a source that shows the correct value. FusionBenchmark is run by Hadamard LLC. Any correction is recorded in the ledger and appears on the changelog.
Sources
- Methodology, Changelog, and Terms of use on this site
More in Using the site
- Find companies by reactor type How to list the companies working on one kind of fusion machine or fission reactor, using the companies directory, the technology pages and the device table.
- Read a row on the benchmark and leaderboards What each column on the benchmark and leaderboard pages means, what achieved, claimed and planned tell you, and how to follow a figure back to where it was published.
- Compare approaches and devices How to use the comparison pages to set two approaches side by side, and the device table and leaderboards to compare individual machines.
- Check a source Where each figure's sources are shown, what the independence labels mean, how archived copies work, and how to tell when several sources are really one.