[Bf-committers] Beta Testing for pre-release versions?

[Juan J. Pena Mena]

El s=C3=A1b, 15 de 05 de 2004 a las 07:12, Ton Roosendaal escribi=C3=B3:

>=20
> There are 18 fixes mentioned in the 2.33a release notes. Only three of =20
> these were bugs related to features that worked in 2.32 and before:
>=20
> 1. the Windows installer

> 2. The audio sequencer crash
> 3. The error in UV unwrapping menu

You forgot about the antialiased fonts on Linux, and the alpha mapping
on the game engine (although the alpha thing is easy to understand
because the hole come-back situation).  By the way, the AA isn't totally
fixed on 2.33a (see bug tracker).

> -> stricter release/test scheduling
> Just comes back each time, and I don't even expect we'll ever fully =20
> solve it. Deadlines just are the only means to get adrenaline pumped =20
> into coders veins, and get them to work late nights fixing the last =20
> details. :)

I'm not propposing to remove deadlines, I'm speaking about a mandatory
release hold AFTER the deadline is over.

> I don't know really how an 'official' betatester group could be =20
> organized... but personally I'd welcome people who want to spend time =20
> on creating and maintaining a good regression suite, and be standby on =20
> forums and on this mailing list in the week(s) before release.
>=20
> -Ton-

It isn't enough to have the group of volunteers, it also takes that each
platform manager to post (the so far final) binary builds done by
himself, and those are the builds that the beta group would test.=20

It makes not sense that every tester downloaded a CVS checkout and
compiled the code himself. Many errors would come from compile-time
choices made by the tester.. and new errors could slip inside just at
final compile time on the platform manager machine.  That is why the
testers should evaluate =C2=A8the official=C2=A8 binary.

Finally, regression suits aren't enough. They are good for detecting
output errors, but they have almost no use for detecting workflow errors
(like a misplaced hotkey, or a missing menu). Only by using =C2=A8the real
thing=C2=A8 on =C2=A8real work=C2=A8 projects can those bugs be detected, a=
nd for that
time is needed, more time than it takes to run a regression test.

Apollux.