[Bf-committers] Avoiding regressions, changes to the release cycle.

[Thomas Dinges]

Hi Campbell,
thanks for bringing this topic up here on the mailing list, answers inline.

Am 29.07.2013 13:43, schrieb Campbell Barton:
> Last meeting we discussed changes to release because of bad
> regressions in recent releases.
> It was suggested that we move to 3 month release cycles.
>
> I believe this wont help, since its not really solving the problem,
> consider that we had 1 week extra to fix bugs for 2.68 (and we _did_
> fix a lot of bugs), but even then bad regressions still got in.
It is correct, that a longer release cycles isn't a guarantee that we 
won't have regressions and an "a" release.
Nevertheless, I think we could evaluate this again. I am happy to try 
other solutions first (see your points beneath), but in general I just 
don't see why we really have to do 6 releases per year.
But let's keep the 2 months for now, and check other things firt.
> To be clear there were 3 regressions that I'd consider show stoppers.
> - Crash deleting a sequence strip r58374
> - Removing vertex colors crashes r58436
> - Painting Undo Enable Face paint Crash r58509
>
> Notice all 3 were caused by `fixes` made after 2.68 RC (made r57908)
> (There were other bad bugs but not regressions AFAIK)
>
>
> I think there are 2 problems with how we are doing releases currently.
>
> - We're making risky changes/fixes too close to the release date.
>
> - RC's are not really 'release-candidates', we are asking users to
> test a build, then making many changes afterwards that aren't properly
> tested.
Agreed, a RC really should be a "final" build, but we abuse it as Beta 
release.
> So I'd propose rather then change the time between releases, we should
> be more disciplined _not_ to make fixes which have hard-to-predict
> side effects just before release.
>
> After the last release candidate is a good time to only focus on show
> stopper bugs. We could also use BCON4, as long as it doesn't last too
> long, (IMHO more then 2 weeks would become inefficient).
>
> Some guidelines...
> - Simple fixes such as NULL checks, array index range checks, obvious
> memory leaks... etc, are fine.
> - If the fix is more involved but not a show-stopper or regression,
> then postpone until next release.
> - If there is some exception (for whatever reason is really important
> to resolve), then review throughly (have 2 other devs review for eg).
>
> Just listing guidelines to make it more clear the kind of change I'm suggesting.
> This doesn't have to be any change in policy or new rules, we could
> just agree not to make risky changes after an RC (or right before the
> release) I think this would be enough.
This sounds good, +1. We should be more strict here, and every change 
*after* the RC should be a fix for a regression/showstopper. But I'd not 
make this "review changes more thoroughly" an optional thing, ideally we
should apply this method for all commits after the RC.  When we really 
stick to "fix regressions only", then reviewing the handful of commits 
is not a problem.
> Other things to look into (which I'm not covering)...
>
> - We should really investigate automated testing in areas shown to
> have caused problems in previous releases (big topic).
>
> - Since we can't completely avoid bug-fix releases, we could try to
> make the process less onerous, minor releases shouldn't be difficult,
> if they are, we could look into automating this more too, or sharing
> tasks with more devs.
>
Good points, but indeed a bigger topic.

Best regards,
Thomas

-- 
Thomas Dinges
Blender Developer, Artist and Musician

www.dingto.org