[Bf-committers] Graphics Cards and Blenchmarks

[Matt Ebb]

Hi,

I've recently been in contact with Mark D. Butala, who emailed me offering
to help collate the information in the Graphics Cards Compatibility thread
on blender.org (
http://www.blender.org/modules.php?op=modload&name=phpBB2&file=viewtopic&t=1419 )
and update the Graphics Cards page on blender3d.org (
http://www.blender3d.org/Support/?sub=GraphicsCards ). He's also proposing
to work on creating a new 'blenchmark' since the blacksmith one is out of
date, and not very useful for benchmarking today's computers.

Personally I'd love to see a brand new blenchmark and really try and push
Blender as being a benchmarking tool to test performance under real-world 3D
graphics creation. Currently, most reviewers use 3D Studio MAX test scenes.
Firstly I think we can do better than this (like in the blacksmith file) by
offering a scene that's from, or simulates real-world production and the
sorts of things that indicate how the graphics card will perform under real
working conditions, unlike the MAX files which contain things like testing
how fast it can display a plane with primitives and teapots on it. Of course
there's the other advantage of Blender being free, so it may make it easier
to persuade reviewers to include it in their testing. Getting more reviewers
to use Blender as a '3D graphics creation' component of a graphics card
review is not only great publicity for Blender, but will also create a good
impression of Blender being a useful, professional 3D tool, especially if we
can use a test scene that reflects real life work. Anyway, maybe that's a
bit of a pipe dream of mine, but I think it's possible and I'd love to see
it happen.

We have a few questions for people who are familiar with the internals of
Blender and who may be able to shed some light on this idea.

* Is the CTRL-ALT-T method a reliable and valid method of benchmarking? What
exactly is it testing and is there anything in the code that may make it
favour some systems to others, or perhaps be reliant on other things like
CPU or RAM? Is it tweaked to be best for values that may have been best at
the time when it was made, but perhaps irrelevant now?

* Is there a way that we could get an FPS score over the course of an
animation in the viewport rather than the arbitrary value from the
CTRL-ALT-T blenchmark? This would be much more meaningful to people as when
they are shopping, they could look at a graphics card's blenchmark and think
"hmm 10 FPS.. this card will be a bit sluggish" or "wow, 100 FPS. This card
will be nice and fast in Blender". If this would take a lot of hacking,
would something like this be possible in Python?

* What bottlenecks exist and how can we stress them? What sort of things in
a scene stress what components and how can we isolate and work with these
(for example using scenes that test certain bits of the hardware, certain
specifics like textures, polygon counts, other features like lighting,
etc.).

This may be a bigger job than we imagine, but it would be good to at least
get some feedback. It could be a great way to get Blender out in the minds
of the public a bit more, and also make it easier for Blender users to
choose a graphics card that works and gives good performance.

So, any ideas or thoughts?

Matt