[Bf-committers] GSoC Proposal: Unit Testing

[Leif Andersen]

Hello, I am interested in participating in this years GSoC project.  In
particular, Unit testing.  An area that I though Unit tests would be good
was the API, but I'm very open to suggestions of any type.  I have attached
a prototype of my proposal, if anyone is willing to look at it, and give
constructive feedback, that would be greatly appreciated.  I am likely going
to change the bio at the end, as by the time Google will start accepting
applications, I will be done with the current projects I mention.

Again, thank you for your feedback.

*Development of Unit Tests for the Blender 2.5 API*


 *Name*

Leif Andersen


 *Email/IRC/WWW*

Emain – leif at leifandersen.net/leif.a.andersen at gmail.com/tbol3.14 at gmail.com

IRC – Leif/LeifAndersen

WWW – http://leifandersen.net


 *Synopsis*

* *Unit Testing is a very powerful tool used to determine the accuracy of
code components. While it is not generally used to test how various
desperate components of a project interact with each other directly, it can
be used to make sure that individual components are working properly, even
when components from different areas of the project are changed. For smaller
projects, and for components that are directly viewable to the user, this is
not a necessity. If there is a bug, it is directly viewable. However, for
larger projects, especially to portions of it that are hidden to the user,
this become very important. Unit testing helps developers of the project
determine which component the problems are coming from.

The goal of this project is to provide Unit Tests for all of the underlying
parts of the Blender API. Furthermore, if time permits, this project will
attempt to clean up the more stable portions of the Blender API, and expand
upon the current state of Blender development documentation.


 *Benefits to Blender*

According to the Blender code base,i <#127984db651a29d1_sdendnote1sym> there
currently is no main section for Unit Tests, leaving any tests in the code,
to be embedded within the files themselves. Furthermore, the main way to
determine how files are linked together, is to look at the makefiles
themselves, and to browse through the Blender code base. While this can be a
little daunting to new developers, the main problem is that this makes it
harder for regular contributers to test every component to make sure that
their slight changes to the code doesn't break anything.

This means that the current developers of Blender, especially the ones
working on the API, can spend more time making the code better, and less
time searching to find where their bugs are hidden. This is especially
useful for Blender with it's larger emphasis on the API. With lots of
development going on in that area, unit tests will be a valuable asset to
the Blender code base.


 *Deliverables*

By the end of this project, there will a simple set of tests that developers
can run to ensure that their modifications to the code did not change
anything major. Furthermore, these tests will give robust information as to
where the error occurs, to aid in the removal of any bugs. Finally, these
tests should be reliantly fast to run, and be built to be tested on both a
modular, and holistic level. That way the tests can be run often by the
developers, if they so choose to, but have the ability to perform all of the
tests with a simple button, when needed. This is useful so that the
developer need not completely recompile Blender, or the current module being
worked upon, while the work is done. Which should increase the speed of
development.

These tests will be placed in their own folder, or possibly, spread
throughout the code based delimited from the rest of the code.
Alternatively, the tests could be stored in a different section altogether
outside of the main blender svn folder. However, it would be useful to get
any changes made to the tests, whenever a developer gets the latest code
base.

These tests can be run either as the developer works on the project, or it
can happen as any new patches are committed into the code base.


 *Project Details*

CTestii <#127984db651a29d1_sdendnote2sym> is a powerful tool set which can
be used for generating test files. This is especially useful as blender is
already using CMake as an option to compile blender with. CDash can be used
as a straightforward front end, so the programmer can easily see the results
of the tests.

The first phase of the project is to create the Unit tests. They should test
the API for everything reasonably conceivably thrown at it, even if it is an
invalid parameter. This is important so developers, especially plugin
developers who are still new to the API, will get useful information back
when the put improper parameters into the function. The tests will not be
compiled together into a simple set of tests during this phase. Also, during
this phase, the tests will likely exist outside of the main trunk.

The second phase is to incorporate all of the tests into a concise package,
to make it easier for developers to use. CDash will be used to make
everything more readable. At this point, the tests will be usable, and
should compile successfully. However, they won't necessarily be stable, and
may change if better, more accurate, or more efficient tests are discovered.
These tests may be included in a separate blender/tests/ folder, or, as
suggested on the CMake website,iii <#127984db651a29d1_sdendnote3sym> the
tests could be set up to run whenever a new commit has been made.
Alternatively, if the Blender Community does not believe that they wish the
tests to be run on Blender's Servers, CDash hosts a free testing
environment, which can be temporarily for testing. However, using Blender's
Servers will allow for developers to get feedback, and as such, send
feedback for requested features, before the completion of the project.

The third phase of this project will be do document everything. While the
development workflow for blender will be nearly identical to that of what it
was before, with the option of testing your code, the current state of
documentation for new developers could be improved. While not entirely
pertinent to core developers, it is particularly useful for new plugin
developers, who aren't interested in learning every internal piece of
blender, but just want to develop a plugin that suites the piece of art that
they are working on at the moment. Possibly, several tutorials explaining
the components of the API could be generated, similar to Ira Krakow's
Blender 2.50 Python tutorials.iv <#127984db651a29d1_sdendnote4sym>

At this point, the main portion of the project should be completed. Any
remaining bugs will be dealt with, along with code cleanup, to make it
easier for the tests to be expanded upon as the API changes, and grows. If
not done so, the tests should be incorporated into the main blender code
base, or set up to run either nightly, or as a new commit is made.


 *Project Schedule*

* *Note that other then the start and end dates, these dates are predictions
as to how the Blender Foundation would like me to allocate my time. If
requested, I will be happy to change the dates for each of these phases, or
possibly even swap their positions, to spend more time on different segments
of the project.

I already understand portions of the blender code base and a fair chunk of
the Python API. Unless a catastrophe happens, I should be familiar with the
Blender code base by the May 24 start date. In fact, my last final is two
weeks before that, as such, I hope to have the proper CTest environment set
up, and to have started creating several preliminary tests by May 24.

>From May 24 to June 20 is phase one. I hope to take the preliminary tests
created before May 24, and turn them into rigorous test suites. By this
point, there should be no serious

>From June 20 to July 15 is phase two. I hope to take the tests suites, and
concatenate them into a fairly useful development tool. At this point, I
hope to have many developers try the tool, and submit further requests for
tests that went under the radar in phase one.

>From July 15 to August 16 is phase three. I hope to document the whole
process, along with improving the documentation of the API itself.
Furthermore, I will spend any extra time documenting other forms of
development that is lacking details, particularly relating to plugin
development, or new developers documentation.

I may spend a week during the first part of July on vacation. I will
continue to work on the code during this week. However, due to limited
internet connectivity, communication will have to be limited to daily email
that week.

In short, I will have no work or school over the summer. Per request, I do
not intend to take any summer classes, unless I'm explicit told that I can.
Even then though, it would only be one differential equations mathematics
class. The only work that I have done is freelance tutoring. However, as the
majority of schools in this area will be out, I will not be spending any
time on that.


 *Bio*

* *I am a 19 year old computer engineering and mathematics student studying
at the University of Utah. I first started programming in 2001 when my
parents bought a copy of Interplay's Learn to Program
Baisc,v<#127984db651a29d1_sdendnote5sym>a CD designed to get
adolescence interested in computer science by making
simple video games. Still interested in video games, I then went on to try
out Mark Overmar's Game Makervi <#127984db651a29d1_sdendnote6sym> (now
controlled by Yoyo Games). There, I created several platformers, and space
shooters. Also interested in multimedia and web production, I picked up a
copy of Macromedia Flash MX 2004, and made several small games and
animations with that. I even created a small website to show off some of my
work, originally programmed in basic HTML with simple Javascript, and then I
moved to Dreamweaver. In an attempt to break out of two space graphics, I
found Blender. Initially overwhelmed by all of the buttons, knobs, sliders,
menus, modes, and hotkeys, I eventually started creating several still
pictures. I then moved on to basic animations and character animations, and
a few very simple games. When I found the video sequencer, I used it to slap
together several films.vii <#127984db651a29d1_sdendnote7sym>

I started getting very serious with programming when I went to university.
There, the main classes that they have taught in was Java. As such, that is
the language the majority of my code has been in. Currently, my most
ambitious, although certainly not my most well written, completed project
was a file compressor, which turned out to do very little compression
whatsoever.viii <#127984db651a29d1_sdendnote8sym> I have a large chunk of my
code available for checkout on my
website.ix<#127984db651a29d1_sdendnote9sym>However, just because the
majority of my code is in Java, that does not mean
I don't know, and use other languages. Mainly to stretch my legs with C, but
also as something to do in my time, I've started working on a simple pong
game.x <#127984db651a29d1_sdendnote10sym>

I've started contributing to Blender by creating an OFF exporter, which was
in Blender 2.4x, but not 2.5x.xi <#127984db651a29d1_sdendnote11sym> I'm
currently working on an importer to go with it, and hope to have it done
within the next few days. Once done with that, I will further get to know
more about the actual core of Blender, by playing with it's window manager,
and fixing some simple bugs. Also, I am currently looking into the folder
blender/source/tools, to see, as reported on the file system page, if it is
really being used at all.i Currently, Scons will compile without it,
however, as it is a tools directory, it may be used for something besides
just tools to change the code for various platforms, which is what it
appears to be.

All in all, I know Java, C, Python, Matlab, and to a lesser extent C++,
Objective-C, Basic, Lisp, HTML, and a bit of PHP and Javascript.

i <#127984db651a29d1_sdendnote1anc>
http://wiki.blender.org/index.php/Dev:2.5/Doc/Blender_Source/Files_structure

ii <#127984db651a29d1_sdendnote2anc>
http://www.cmake.org/Wiki/CMake_Testing_With_CTest

iii <#127984db651a29d1_sdendnote3anc>http://www.cmake.org/

iv <#127984db651a29d1_sdendnote4anc>
http://www.youtube.com/watch?v=vmhU_whC6zw&feature=channel

v <#127984db651a29d1_sdendnote5anc>
http://www.amazon.com/Learn-Program-Basic-Windows-Macintosh/dp/B000N3W2L4

vi <#127984db651a29d1_sdendnote6anc>http://gamemaker.nl/

vii <#127984db651a29d1_sdendnote7anc>Http://leifandersen.blip.tv

viii <#127984db651a29d1_sdendnote8anc>
http://sourceforge.net/projects/lcompress/

ix <#127984db651a29d1_sdendnote9anc>svn://leifandersen.net

x <#127984db651a29d1_sdendnote10anc>http://sourceforge.net/projects/pongrpg/

xi <#127984db651a29d1_sdendnote11anc>
https://projects.blender.org/tracker/index.php?func=detail&aid=21717&group_id=153&atid=467