[Soc-2010-dev] Status Report (testing-week 6)

[Leif Andersen]

I'll try to make this short, as it's late here.

Even though I got sick this week (it turned out to be just a cold/flu), and
went out of town (and still am), I believe that this week has been one of
the most productive weeks so far.

This week started off with me implementing some of the GUI features I worked
with last week, which is to say I started making the image diff tests into
an operator.  (Previously they had been baked into the tests directly).
 They could be run from a Tests dropdown menu, although there was non
feedback outside of the console, leading the user to believe blender had
frozen while the tests were running.  Andrea pointed out that this was
pointless, and that it should just be a script.  As such, I started
implementing it in all in python 2.x.  The tests can be found in
tests/render, and I put together a slight howto on how to run them:
http://wiki.blender.org/index.php/User:LeifAndersen/GSoC2010/Howto

Render Tests

The current set of render tests requires PIL, and python 2.x. In order to
> run them, inside the blender source directory, look in: tests/render. There,
> you should find a collection of files. While the absolute path of those
> files doesn't matter (aka, you could move the tests/render folder anywhere,
> and it would do just fine), those files must be kept in the same
> relationship to each other. If blender is already in your system path, just
> go into the folder, and run python run.py, and the script will take it's
> course. If blender is not in your current path, open up run.py, and change
> the BLENDER_BIN variable to you blender binary, and than run python run.py.
> You can view the results in the console, otherwise you can see a list of all
> of the results, as well as diffs, by opening index.html in your web browser.


(my most recent commits have temporarily broken this as an operator, but I
will soon fix that).  There is a webpage that is outputted which allows you
to easily see the difference between the renders.  In addition, the
algorithm for analyzing the images is much improved.

I also moved my tests.hashcode module into an operator
(bpy.ops.tests.hashcode()), or Tests->Hashcode.  Although it still only
gives feedback in the terminal, and only takes some things into account when
making the hashcode.

As far as next week goes, I am going to spend a bit of time with the
webpage.  It's not quite flush with blender's webpage, but it's getting
closer.  But more importantly, I can see a few more options that would allow
the user to get more of the raw data behind the images (which I would assume
would make it debug).  Most of this I believe can be written in HTML, but
some javascript may be useful.  I also plan to get animations working with
these tests, and possibly bringing this back in to work with the blender
operator.  I also am going to try to get a lot more feedback this week, and
hope to have the tool used soon after.  I will also spend a bit of time
getting the hashcode operator to take more into account, as well as giving
more feedback to the user.  (As in feedback that's not only just console
output).  I would also work on aggregating the data in the blend file for
the user to test, but blender is already capable of that.  Finally, I plan
on improving the integration of Ctest and CDash, but that seems trivial to
do.



~Leif Andersen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/soc-2010-dev/attachments/20100702/888a92cb/attachment.htm