Half the light
A one pixel black and white checkerboard is exactly half black and half white, so any honest downscale has to average it. The right answer is sRGB 188, because the average is over light. Every resizer I measured returns about 128: sips, ffmpeg, the Rust image crate, fast_image_resize, Bun.Image, and the canvas in whatever browser is rendering this sentence. Their kernels agree with each other to seven decimal places. All of them average in the wrong space, which Eric Brasseur wrote up in 2007 and the defaults have outlived.
Your browser is doing it right now
This runs in your machine, on a checkerboard built a moment ago. The left square is what drawImage produces when it shrinks that checkerboard 16 times. The right square is the same reduction with the samples converted to light first.
The two grey squares should be visibly different, and the numbers under them say by how much. Nothing about the left square is a bug in your browser: it is what almost every image pipeline shipped this decade does.
Why 128 is the wrong grey
sRGB encodes perceived brightness rather than light. The code 128 sits near the middle of what your eye does with the range, and it carries about 21% of the light of code 255. So averaging the codes of black and white gives a grey with a fifth of the light of the pair it replaced, and the picture darkens everywhere fine detail used to be.
Textures are where this lands hardest, because texture is exactly where neighbouring pixels differ most. A photograph reduced this way loses contrast in foliage, fabric, hair and grain, and gains it nowhere.
The resampler, with every operational concern taken back out
Here is the whole idea at an integer reduction factor. It is the load-bearing version, and the shipping one is a few hundred lines around this shape.
function reduce(src, w, factor) {
const out = [], n = factor * factor;
for (let y = 0; y < w / factor; y++)
for (let x = 0; x < w / factor; x++) {
let sum = 0;
for (let dy = 0; dy < factor; dy++)
for (let dx = 0; dx < factor; dx++)
sum += toLinear(src[(y * factor + dy) * w + x * factor + dx]);
out.push(toSrgb(sum / n));
}
return out;
}Convert to light, average, convert back. Everything the real implementation adds is a separate concern wearing this function's clothes, and each one earns its place for a reason worth naming.
Arbitrary ratios need weights. An integer factor lets every source sample count once. A 3.86 times reduction does not, so each output sample carries a list of taps and weights instead of a rectangle.
The filter has to widen with the reduction. On a downscale the filter must low pass at the output Nyquist, so its support in source pixels grows by the reduction factor. A kernel evaluated at fixed width is sampling rather than filtering, and it aliases. This is why naive Lanczos3 downscales shimmer.
Weights are renormalised per output sample. At the frame edge the support window is clipped, and the weights that survive have to sum to one again or the border darkens into a vignette nobody can explain.
The two axes run separately. A 2D filter of this family factors into two 1D passes, which turns work per output pixel from support squared into support: about 13 taps instead of 44 at the reduction this site uses.
The forward transfer is a table. It takes an 8 bit value, so it has exactly 256 answers and does not need a powf 119 million times. That one change cut the resample core from 42.9 ms to 22.9 ms with byte identical output.
Six wrong answers, and the control that caught each one
The gamma defect is the easy part of this story. The hard part is that it survived years of measurement, and so did five other things, because in each case the instrument could not see the defect it was pointed at.
The pattern that connects them: a measurement that compares a thing against itself always agrees. Scoring an encode against the reference it came from cannot see a bad reference. Resizing to the size you already are cannot test a kernel that short circuits equal dimensions. Reading a metric you invented cannot tell you the metric is insensitive.
What worked every time was cheaper than it sounds. Build the control that would fail if the thing were broken, then break the thing on purpose and watch the control go red. The regression test for the rotation fix was written that way, and so was the probe column that finally caught sips leaving noise on a flat field.
What the correct version costs
Correct pixels compress worse, because the gamma incorrect average was quietly destroying high frequency energy that a correct one keeps. Regenerating this site's 660 thumbnail files moved them 15164 KB to 15392 KB, which is 1.5%.
Against a ground truth downscale, delivered quality on real photographs moved from 57.2 to 84.8 as a mean over 10 frames. On the Instagram export the same change let three Leica frames beat their old versions by about 4 ssimulacra2 points while landing 43% smaller, because the search could stop reaching for quality it was spending on aliasing.
Speed is a fair question and the answer is mixed. The scalar path here runs a 5952 by 3968 frame to 900 by 600 in about 70 ms single threaded, which beats fast_image_resize's own gamma correct path at 60.3 ms only after the channel specialisation above, and stays roughly twice the cost of its gamma wrong SIMD default at 28.6 ms. That gap is the price of the transfer function rather than of the kernel.
Offered upstream
Two of these are somebody else's to fix. image-rs/image#3101 proposes an opt in linear light resize, with measurements aimed at the specific objection that converting to a 16 bit linear space is too expensive to do transparently; their kernel is already correct, so the offer is the colour layer and the property tests rather than a replacement resampler. oven-sh/bun#40510 reports the same defect in Bun.Image with a self contained repro, where the argument is different: a runtime whose image API shipped this year has no back compatible golden images anchoring it to the wrong default.
What would change my mind
The byte cost is real and it is the honest objection. If a wider corpus showed the 1.5% growing into double digits at some other reduction ratio or subject matter, the right answer would be to spend the difference on the quality knob instead and take a smaller correctness win. The measurement that would settle it is a matched bytes comparison across a few hundred frames from several cameras, and this site has 165 frames from two.
The other open edge is the transfer curve itself. This pipeline assumes sRGB except where a file declares Gray Gamma 2.2, which covers its two cameras and would be wrong for anything shot in Display P3 or a log profile. Reading the curve from the ICC profile rather than from a filename pattern is the fix, and it is not written yet.
