[Bf-committers] using checksum instead of bundling images for regression tests

[Ton Roosendaal]

Hi,

For as long the render code is being reorganized and improved, as the =20=

past 2 releases did, there's not much use in comparing pictures binary. =20=

Then there's the fact random jittering and sampling has been introduced =20=

for shadows and AO.

Even with an automatic check, it will still need a trained eye to =20
detect which changes are acceptable (improvements, or due to noise) and =20=

what changes are bugs or need further investigation. I prefer testing =20=

by intelligent humans who know what to look for, instead of devising =20
tests that can be reliably executed by 'monkeys'. :)

So if we make a simple set of testing files, with for each test =20
emphasis on a single rendering feature (or a certain combination), we =20=

should be able to find most of the problems just with a quick glance.

However, occasionally I enforce loading a 'background picture' in the =20=

spare render buffer (Jkey) to have a comparision, using Jkey to flip. =20=

It could be a useful feature to make that easier (like a 'copy actvive =20=

image in Image-Window to render buffer' command). If needed, even a =20
'diff' can be calculated then.

-Ton-


On Saturday, May 22, 2004, at 12:55 Europe/Amsterdam, Tom Musgrove =20
wrote:

>
>>
>> And strange suggestion anyway... do you think testers are not
>> smart enough to decide if the render is correct or not by just
>> comparing it? ;)
>
> As noted by Dan, it would allow the testing to be automated.  Also, a =20=

> checksum would make certain that they were exactly the same, and thus =20=

> subtle differences that are easy to overlook could be caught.  For =20
> instance, on my computer one of the librarys used for rendering pngs =20=

> is fairly bad, so doing a visual comparison could give false positives =
=20
> (or negatives) for errors.
>
> Are the renders non repeatable due to usage of random functions?  I'd =20=

> assume that we could use a fixed seed in a psuedo random function to =20=

> garuntee that the 'random number's selected could be the same.
>
> Not a major deal, but I thought it might be worth considering,
>
> Tom M.
> LetterRip
>
> _________________________________________________________________
> Is your PC infected? Get a FREE online computer virus scan from =20
> McAfee=AE Security. =20
> http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3D3963
>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers@blender.org
> http://www.blender.org/mailman/listinfo/bf-committers
>
>
------------------------------------------------------------------------=20=

--
Ton Roosendaal  Blender Foundation ton@blender.org =20
http://www.blender.org