[Bf-committers] GSoC Proposal: Unit Testing

[Keir Mierle]

Congratulations Leif for having your proposal accepted!

I thought I would drop in to plug the Google Test unittesting framework.
Yes, it's C++, but it has some features like automatic test registration and
concise syntax (among others) that make it superior to the pure C
frameworks. Furthermore, I am a committer on the project and offer my
support. I will be available for the duration of GSoC. I can also help with
build integration, as I've integrated Google Test with several projects.

http://code.google.com/p/googletest/

I'm happy answer any questions.

Keir


On Thu, Mar 25, 2010 at 7:29 PM, Leif Andersen <leif.a.andersen at gmail.com>wrote:

> Hello, I am interested in participating in this years GSoC project.  In
> particular, Unit testing.  An area that I though Unit tests would be good
> was the API, but I'm very open to suggestions of any type.  I have attached
> a prototype of my proposal, if anyone is willing to look at it, and give
> constructive feedback, that would be greatly appreciated.  I am likely
> going
> to change the bio at the end, as by the time Google will start accepting
> applications, I will be done with the current projects I mention.
>
> Again, thank you for your feedback.
>
> *Development of Unit Tests for the Blender 2.5 API*
>
>
>  *Name*
>
> Leif Andersen
>
>
>  *Email/IRC/WWW*
>
> Emain – leif at leifandersen.net/leif.a.andersen at gmail.com/tbol3.14 at gmail.com
>
> IRC – Leif/LeifAndersen
>
> WWW – http://leifandersen.net
>
>
>  *Synopsis*
>
> * *Unit Testing is a very powerful tool used to determine the accuracy of
> code components. While it is not generally used to test how various
> desperate components of a project interact with each other directly, it can
> be used to make sure that individual components are working properly, even
> when components from different areas of the project are changed. For
> smaller
> projects, and for components that are directly viewable to the user, this
> is
> not a necessity. If there is a bug, it is directly viewable. However, for
> larger projects, especially to portions of it that are hidden to the user,
> this become very important. Unit testing helps developers of the project
> determine which component the problems are coming from.
>
> The goal of this project is to provide Unit Tests for all of the underlying
> parts of the Blender API. Furthermore, if time permits, this project will
> attempt to clean up the more stable portions of the Blender API, and expand
> upon the current state of Blender development documentation.
>
>
>  *Benefits to Blender*
>
> According to the Blender code base,i <#127984db651a29d1_sdendnote1sym>
> there
> currently is no main section for Unit Tests, leaving any tests in the code,
> to be embedded within the files themselves. Furthermore, the main way to
> determine how files are linked together, is to look at the makefiles
> themselves, and to browse through the Blender code base. While this can be
> a
> little daunting to new developers, the main problem is that this makes it
> harder for regular contributers to test every component to make sure that
> their slight changes to the code doesn't break anything.
>
> This means that the current developers of Blender, especially the ones
> working on the API, can spend more time making the code better, and less
> time searching to find where their bugs are hidden. This is especially
> useful for Blender with it's larger emphasis on the API. With lots of
> development going on in that area, unit tests will be a valuable asset to
> the Blender code base.
>
>
>  *Deliverables*
>
> By the end of this project, there will a simple set of tests that
> developers
> can run to ensure that their modifications to the code did not change
> anything major. Furthermore, these tests will give robust information as to
> where the error occurs, to aid in the removal of any bugs. Finally, these
> tests should be reliantly fast to run, and be built to be tested on both a
> modular, and holistic level. That way the tests can be run often by the
> developers, if they so choose to, but have the ability to perform all of
> the
> tests with a simple button, when needed. This is useful so that the
> developer need not completely recompile Blender, or the current module
> being
> worked upon, while the work is done. Which should increase the speed of
> development.
>
> These tests will be placed in their own folder, or possibly, spread
> throughout the code based delimited from the rest of the code.
> Alternatively, the tests could be stored in a different section altogether
> outside of the main blender svn folder. However, it would be useful to get
> any changes made to the tests, whenever a developer gets the latest code
> base.
>
> These tests can be run either as the developer works on the project, or it
> can happen as any new patches are committed into the code base.
>
>
>  *Project Details*
>
> CTestii <#127984db651a29d1_sdendnote2sym> is a powerful tool set which can
> be used for generating test files. This is especially useful as blender is
> already using CMake as an option to compile blender with. CDash can be used
> as a straightforward front end, so the programmer can easily see the
> results
> of the tests.
>
> The first phase of the project is to create the Unit tests. They should
> test
> the API for everything reasonably conceivably thrown at it, even if it is
> an
> invalid parameter. This is important so developers, especially plugin
> developers who are still new to the API, will get useful information back
> when the put improper parameters into the function. The tests will not be
> compiled together into a simple set of tests during this phase. Also,
> during
> this phase, the tests will likely exist outside of the main trunk.
>
> The second phase is to incorporate all of the tests into a concise package,
> to make it easier for developers to use. CDash will be used to make
> everything more readable. At this point, the tests will be usable, and
> should compile successfully. However, they won't necessarily be stable, and
> may change if better, more accurate, or more efficient tests are
> discovered.
> These tests may be included in a separate blender/tests/ folder, or, as
> suggested on the CMake website,iii <#127984db651a29d1_sdendnote3sym> the
> tests could be set up to run whenever a new commit has been made.
> Alternatively, if the Blender Community does not believe that they wish the
> tests to be run on Blender's Servers, CDash hosts a free testing
> environment, which can be temporarily for testing. However, using Blender's
> Servers will allow for developers to get feedback, and as such, send
> feedback for requested features, before the completion of the project.
>
> The third phase of this project will be do document everything. While the
> development workflow for blender will be nearly identical to that of what
> it
> was before, with the option of testing your code, the current state of
> documentation for new developers could be improved. While not entirely
> pertinent to core developers, it is particularly useful for new plugin
> developers, who aren't interested in learning every internal piece of
> blender, but just want to develop a plugin that suites the piece of art
> that
> they are working on at the moment. Possibly, several tutorials explaining
> the components of the API could be generated, similar to Ira Krakow's
> Blender 2.50 Python tutorials.iv <#127984db651a29d1_sdendnote4sym>
>
> At this point, the main portion of the project should be completed. Any
> remaining bugs will be dealt with, along with code cleanup, to make it
> easier for the tests to be expanded upon as the API changes, and grows. If
> not done so, the tests should be incorporated into the main blender code
> base, or set up to run either nightly, or as a new commit is made.
>
>
>  *Project Schedule*
>
> * *Note that other then the start and end dates, these dates are
> predictions
> as to how the Blender Foundation would like me to allocate my time. If
> requested, I will be happy to change the dates for each of these phases, or
> possibly even swap their positions, to spend more time on different
> segments
> of the project.
>
> I already understand portions of the blender code base and a fair chunk of
> the Python API. Unless a catastrophe happens, I should be familiar with the
> Blender code base by the May 24 start date. In fact, my last final is two
> weeks before that, as such, I hope to have the proper CTest environment set
> up, and to have started creating several preliminary tests by May 24.
>
> >From May 24 to June 20 is phase one. I hope to take the preliminary tests
> created before May 24, and turn them into rigorous test suites. By this
> point, there should be no serious
>
> >From June 20 to July 15 is phase two. I hope to take the tests suites, and
> concatenate them into a fairly useful development tool. At this point, I
> hope to have many developers try the tool, and submit further requests for
> tests that went under the radar in phase one.
>
> >From July 15 to August 16 is phase three. I hope to document the whole
> process, along with improving the documentation of the API itself.
> Furthermore, I will spend any extra time documenting other forms of
> development that is lacking details, particularly relating to plugin
> development, or new developers documentation.
>
> I may spend a week during the first part of July on vacation. I will
> continue to work on the code during this week. However, due to limited
> internet connectivity, communication will have to be limited to daily email
> that week.
>
> In short, I will have no work or school over the summer. Per request, I do
> not intend to take any summer classes, unless I'm explicit told that I can.
> Even then though, it would only be one differential equations mathematics
> class. The only work that I have done is freelance tutoring. However, as
> the
> majority of schools in this area will be out, I will not be spending any
> time on that.
>
>
>  *Bio*
>
> * *I am a 19 year old computer engineering and mathematics student studying
> at the University of Utah. I first started programming in 2001 when my
> parents bought a copy of Interplay's Learn to Program
> Baisc,v<#127984db651a29d1_sdendnote5sym>a CD designed to get
> adolescence interested in computer science by making
> simple video games. Still interested in video games, I then went on to try
> out Mark Overmar's Game Makervi <#127984db651a29d1_sdendnote6sym> (now
> controlled by Yoyo Games). There, I created several platformers, and space
> shooters. Also interested in multimedia and web production, I picked up a
> copy of Macromedia Flash MX 2004, and made several small games and
> animations with that. I even created a small website to show off some of my
> work, originally programmed in basic HTML with simple Javascript, and then
> I
> moved to Dreamweaver. In an attempt to break out of two space graphics, I
> found Blender. Initially overwhelmed by all of the buttons, knobs, sliders,
> menus, modes, and hotkeys, I eventually started creating several still
> pictures. I then moved on to basic animations and character animations, and
> a few very simple games. When I found the video sequencer, I used it to
> slap
> together several films.vii <#127984db651a29d1_sdendnote7sym>
>
> I started getting very serious with programming when I went to university.
> There, the main classes that they have taught in was Java. As such, that is
> the language the majority of my code has been in. Currently, my most
> ambitious, although certainly not my most well written, completed project
> was a file compressor, which turned out to do very little compression
> whatsoever.viii <#127984db651a29d1_sdendnote8sym> I have a large chunk of
> my
> code available for checkout on my
> website.ix<#127984db651a29d1_sdendnote9sym>However, just because the
> majority of my code is in Java, that does not mean
> I don't know, and use other languages. Mainly to stretch my legs with C,
> but
> also as something to do in my time, I've started working on a simple pong
> game.x <#127984db651a29d1_sdendnote10sym>
>
> I've started contributing to Blender by creating an OFF exporter, which was
> in Blender 2.4x, but not 2.5x.xi <#127984db651a29d1_sdendnote11sym> I'm
> currently working on an importer to go with it, and hope to have it done
> within the next few days. Once done with that, I will further get to know
> more about the actual core of Blender, by playing with it's window manager,
> and fixing some simple bugs. Also, I am currently looking into the folder
> blender/source/tools, to see, as reported on the file system page, if it is
> really being used at all.i Currently, Scons will compile without it,
> however, as it is a tools directory, it may be used for something besides
> just tools to change the code for various platforms, which is what it
> appears to be.
>
> All in all, I know Java, C, Python, Matlab, and to a lesser extent C++,
> Objective-C, Basic, Lisp, HTML, and a bit of PHP and Javascript.
>
> i <#127984db651a29d1_sdendnote1anc>
>
> http://wiki.blender.org/index.php/Dev:2.5/Doc/Blender_Source/Files_structure
>
> ii <#127984db651a29d1_sdendnote2anc>
> http://www.cmake.org/Wiki/CMake_Testing_With_CTest
>
> iii <#127984db651a29d1_sdendnote3anc>http://www.cmake.org/
>
> iv <#127984db651a29d1_sdendnote4anc>
> http://www.youtube.com/watch?v=vmhU_whC6zw&feature=channel
>
> v <#127984db651a29d1_sdendnote5anc>
> http://www.amazon.com/Learn-Program-Basic-Windows-Macintosh/dp/B000N3W2L4
>
> vi <#127984db651a29d1_sdendnote6anc>http://gamemaker.nl/
>
> vii <#127984db651a29d1_sdendnote7anc>Http://leifandersen.blip.tv
>
> viii <#127984db651a29d1_sdendnote8anc>
> http://sourceforge.net/projects/lcompress/
>
> ix <#127984db651a29d1_sdendnote9anc>svn://leifandersen.net
>
> x <#127984db651a29d1_sdendnote10anc>
> http://sourceforge.net/projects/pongrpg/
>
> xi <#127984db651a29d1_sdendnote11anc>
>
> https://projects.blender.org/tracker/index.php?func=detail&aid=21717&group_id=153&atid=467
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>