How the score is calculated

This runs the actual scoring module, not a simplified copy of it. Set a design and an appraisal and watch the numbers move. Three things are worth trying: switch a criterion between no and not reported and watch the score refuse to drop; drag the confidence interval from a decisive null to a demonstrated benefit; and turn on a spin flag to watch a cap clamp the score rather than nudge it.

Rigor starts from a design prior drawn from a 47-family table and is held under a ceiling from the same row. The starting points are compressed and the ceilings wide, so execution decides where in the corridor a paper lands. A sham-controlled trial starts at 68.6 with no ceiling below 100; an adjusted registry cohort starts at 37.4 with a ceiling of 80 and therefore has 42.6 points of headroom; a retrospective case series starts at 8.8 under a ceiling of 42.

The ceiling is structural rather than a judgement about quality. A cross-sectional study cannot establish temporality however well it is conducted, and a cadaveric construct cannot speak to patient outcomes. Preclinical families are exempt from the spread that separates the clinical hierarchy, because their limitation belongs to impact, not to rigor: good bench science should read as good science answering a different question.

Each credited criterion is then summed and shrunk toward that prior by A / (A + 1.5), where A counts the criteria that could actually be assessed. A paper appraised on two criteria sits near its design prior; one appraised on twelve is scored on its own execution. Journal tier is the only proxy left in the model, worth at most 6 points, and it decays to exactly zero once 80% of the applicable criteria have been read.

Impact starts from a flat 45 for every paper, with no design prior at all. Design enters only at the bottom, as an absolute ceiling. The dominant term is not the author's conclusion but the confidence interval, normalised into units of the minimal clinically important difference and signed so that positive means benefit.

That choice is deliberate. Conclusion strength is exactly what spin inflates: 44.8% of orthopedic trials with a non-significant primary outcome contain spin, and spun papers are cited more, so scoring the rhetoric would rank the most spun papers highest by construction. The authors calling their own work practice-changing is worth two points, and it does not even count toward the shrink denominator.

Scores are computed continuously and displayed as one of five bands: exceptional from 85, strong from 68, sound from 48, limited from 30, weak below that. Composite critical-appraisal agreement runs around an intraclass correlation of 0.65 to 0.70, which supports four to six distinguishable strata and not a decimal place. Every card also carries an explicit uncertainty half-width of 6 + 14 × (1 − coverage): about ±6 when the whole paper could be appraised, and ±20 when there was no full text to read. A bare 78 out of 100 asserts a precision the underlying judgement cannot support.

Two things are deliberately absent. Citation counts, because at one or two weeks old they measure indexing speed rather than importance. And reader-relevance signals such as Canadian authorship or subspecialty, because they say nothing about how good a paper is; they inform editorial triage instead. Heat is a screening tool, not a verdict. Cold papers can still be great papers.