[Bf-committers] OpenGL compliance test

[Gregor Mückl]

Hi,

Sounds like someone should step forward and solve this one for real. Since 
other projects occationally stumble across OpenGL problems, too, it seems to 
be worth the effort of putting together a test suite that gathers data on the 
various clients and sends the results back to a central database on the web.

Such a program should at least gather the following information about the 
system:

- Video hardware
- Video driver version
- OS kernel version
- Video subsystem version (e.g. XFree version)
- CPU model and speed (a different CPU model could trigger bugs in GL)
- All queried IDs as unmodified plain text

The actual compliance testing will be the real pain. Basically this will 
consist of cycling OpenGL states and ask the user if things are displayed 
correctly, unless you can automate the tests in some way or another (which is 
difficult because different graphics hardware may produce slightly different 
output while still conforming to the OpenGL specifications).

The biggest problem I see with this approach is that the test results that get 
sent back to the server might be tampered with.

Regards,
Gregor

Am Mittwoch, 19. November 2003 14:08 schrieb Ton Roosendaal:
> Hi,
>
> With more and more people using Blender, we get into troubles with
> OpenGL again. It resembles the situation in NaN, where we had to invest
> in all types of gfx cards and build a testing farm. Daniel Dunbar did
> some nice work then in testing several PC configurations (Linux,
> Windows) and drivers settings.
>
> For an open source organization we have to do this differently... and I
> don't know yet how to exactly solve it. We could seek cooperation with
> other OpenGL projects, but most likely the gamers side of OpenGL world
> won't have the troubles we encounter. I could also ask opengl.org for
> help with it...
>
> One idea I had was developing a simple Ghost based test program,
> opening 2 windows and doing all kinds of checks, including asking the
> user to close windows, hide them, switch desktop or focus, etc.
> The most common errors are in 2d commands, like pixel reading and
> drawing, frontbuffer drawing, and this using glViewport and glScissor
> switching a lot.
>
> I'm curious to find out if there's a good cross platform other OpenGL
> application existing we could use as verification test. People are less
> likely to blame us when they find out similar programs all suffer from
> the same errors. (And no, the famous OpenGL Gears is a plain dumb
> example!).
>
> Then there's the problem of actively managing a database with reports
> from users... like the thread Matt posted a long time ago on the 'news
> and chat' forum. We need someone to digest all this info, and build a
> good spreadsheet out of it. Maybe this is something to write in
> php/mysql, where people can easily submit their own system stats.
>
> So:
> - other opengl projects we can cooperate with?
> - other opengl progs we can use as verification?
> - someone interested in building a database?
>
> -Ton-
>
>
> ------------------------------------------------------------------------
> --
> Ton Roosendaal  Blender Foundation ton@blender.org
> http://www.blender.org
>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers@blender.org
> http://www.blender.org/mailman/listinfo/bf-committers