[Bf-committers] using checksum instead of bundling images for regression tests
Sun, 23 May 2004 12:40:48 +0200
For as long the render code is being reorganized and improved, as the =20=
past 2 releases did, there's not much use in comparing pictures binary. =20=
Then there's the fact random jittering and sampling has been introduced =20=
for shadows and AO.
Even with an automatic check, it will still need a trained eye to =20
detect which changes are acceptable (improvements, or due to noise) and =20=
what changes are bugs or need further investigation. I prefer testing =20=
by intelligent humans who know what to look for, instead of devising =20
tests that can be reliably executed by 'monkeys'. :)
So if we make a simple set of testing files, with for each test =20
emphasis on a single rendering feature (or a certain combination), we =20=
should be able to find most of the problems just with a quick glance.
However, occasionally I enforce loading a 'background picture' in the =20=
spare render buffer (Jkey) to have a comparision, using Jkey to flip. =20=
It could be a useful feature to make that easier (like a 'copy actvive =20=
image in Image-Window to render buffer' command). If needed, even a =20
'diff' can be calculated then.
On Saturday, May 22, 2004, at 12:55 Europe/Amsterdam, Tom Musgrove =20
>> And strange suggestion anyway... do you think testers are not
>> smart enough to decide if the render is correct or not by just
>> comparing it? ;)
> As noted by Dan, it would allow the testing to be automated. Also, a =20=
> checksum would make certain that they were exactly the same, and thus =20=
> subtle differences that are easy to overlook could be caught. For =20
> instance, on my computer one of the librarys used for rendering pngs =20=
> is fairly bad, so doing a visual comparison could give false positives =
> (or negatives) for errors.
> Are the renders non repeatable due to usage of random functions? I'd =20=
> assume that we could use a fixed seed in a psuedo random function to =20=
> garuntee that the 'random number's selected could be the same.
> Not a major deal, but I thought it might be worth considering,
> Tom M.
> Is your PC infected? Get a FREE online computer virus scan from =20
> McAfee=AE Security. =20
> Bf-committers mailing list
Ton Roosendaal Blender Foundation firstname.lastname@example.org =20