[Bf-committers] Proposal for 2.36 release

[Matthew H Plough (mplough at Princeton.EDU)]

After watching two releases go not as well as we had
hoped, I'm beginning to think that the problem lies
just where it always does with code I write for
classes -- insufficient testing.  A CVS freeze is a
possible solution, but I don't think it's the whole
solution.  We need to come up with a way to test
code more completely; I'll try to outline this.

A couple bugs slipped through that we should have
caught immediately -- stuff like tapers crashing,
but the vast majority of the code performs very,
very well in most cases.  Curve tapering is a new
feature that, if I recall correctly, is not included
in the regression suite.  Thus, the first part of
better testing is to make sure that our test tools
keep up with new features.  Every feature needs to
be tested as if someone were trying to use it.  

However, that's not the whole deal.  I feel like I'm
spouting debugging mantras from my algorithms and
data structures class, but we need to thoroughly
test boundary conditions.  No vertices, lots of
vertices, odd scales, etc. all need to be tested.  

Now that I've said that, I'll try to make something
of it.  I've been doing horrible things to Blender
lately, so I'll do my best to flesh them out into
.blends that could go in the regression suite.

I don't know what to do about people not running the
regression suite, though -- any ideas about how to
get more people to perform the tests?  I'll gladly
perform them, but our current testers + 1 really
doesn't help a whole lot.

Matt

----- Original Message -----
From: Jean-Luc Peuriere <jlp at nerim.net>
Date: Monday, November 22, 2004 1:23 pm
Subject: Re: [Bf-committers] Proposal for 2.36 release

> 
> Le 22 nov. 04, à 10:22, Ted Schundler a écrit :
> 
> > While shorter releases isn't a bad idea, I don't
know that it's a
> > solution to the issue at hand.
> >
> Shorter release cycle means more frequents CVS
freezes of at least 
> 15 
> days.
> 4-6 releases/year seems good for me. Many
commercial apps release a 
> major
> version only every other year, with sometimes an
interim version in 
> between
> 
> Also for major improvements, you are reducing the
launch window, 
> which 
> is
> not good.
> 
> Or we need to spawn 2 CVS, stable and devel, which
is possible with 
> tools
> like subversion, but a pain with CVS to keep in sync.
> 
> Now the quality level of blender is still very
good. What does not 
> work,is the beta phase. Posts in builds forum at
b.o leads to users 
> playingwith features and builds at various level
of patching, not 
> testing the
> quality of the official builds which are, btw,
loosely identified.
> 
> > I think Bart's on the right track with official
release candidate
> > versions - publicized and posted on b3d.o,
Elysiun, etc., not buried
> > in the b.o forums. For a normal user, trying out
a beta or test
> > version is sort of shaky business (if they even
find it). But a
> > Release Candidate means it's effectively ready
to go. And if it's
> > listed on an official download page, it looks a
lot safer to try 
> out.
> Separate officials builds from users and
experimental builds is good,
> I agree. Now advertising on many forums will only
lead to more users
> playing with the app, not testing it.
> 
> from real life experience, to get good tests
results, you need a 
> support,
> which can take the form of a completed regression
suite,
> plus a form with questions about day to day use
but the most important
> is real testers, not casual users.. In my view,
the test
> builds should be available to registered users
which agree to run the
> regression suite and complete the form. To be not
a pain, most of the
> suite should be an automated process.
> >
> If such a place is created, I as platform manager
can engage myself to
> provide builds regularly (on a weekly or bi-weekly
basis), not only
> in B-con4 condition.
> 
> > To be safe with the official release, rather
than release when it
> > seems good, wait for X there being no new
significant bug reports 
> for> X days or something. Then that RC becomes the
release.
> 
> I agree on that, but that's what we have done
here, and get no bad 
> report
> until RC became Release. either X was way too
short, or users were 
> onlyplaying. I think later is the true answer,
that's why you need 
> committed testers.
> 
> 
> -- 
> Jean-Luc (lukep)
> insane Mac user since 89
> 
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at projects.blender.org
>
http://projects.blender.org/mailman/listinfo/bf-committers
>