Corrections
Measurements go wrong and versions move on. Here is what happens when they do.
Three kinds of change
Not every edit is a correction, and the difference decides whether a note appears on the page.
- Correction
- A number, a conclusion or an instruction was wrong. The page gets a dated correction note at the top saying what it said before and what it says now. The note stays permanently.
- Re-verification
- The measurement was right when it was taken and has been checked again against current versions. The verification date updates, and any number that moved is called out in the article. The old measurement stays in the experiment record.
- Editorial fix
- Typos, broken links, clearer wording. These are fixed without a note, because nothing factual changed.
What is never done
- Silently editing a number after publication.
- Deleting an article that turned out to be wrong instead of correcting it.
- Updating a verification date without rerunning the measurement.
- Removing a stale record so the database looks fresher than it is.
Reporting something
The most useful report is a step that did not reproduce on your machine. To act on it, the following usually has to be known:
- The article, and which step or number is in question.
- Your GPU or SoC and how much VRAM or unified memory it has.
- Runtime version, driver version, and model build including quantization.
- The exact error text, copied rather than described.
- What you already tried.
Send that to contact@handsonailab.dev, or open an issue on the experiments repository, which is easier to follow up on because the workflow files are there.
What happens next
A report that points at a factual error gets checked against the experiment record first. If the record and the article disagree, the article is wrong and gets corrected. If they agree and your machine still behaves differently, that is a new data point, and the difference between the two configurations usually turns into its own row in the compatibility database.
Reports that cannot be reproduced here are not published as results. They can still be recorded in the database at the unverified report grade, clearly labelled as such.
Correction log
Corrections are published on the articles they belong to. Once the first one exists, it is listed here as well, so the record of what changed is readable in one place.
No corrections have been issued yet. The site has not published its first article.