[Bf-blender-cvs] [d49b850399a] master: Tests: run geometry nodes benchmarks multiple times to reduce noise

[Jacques Lucke]

Commit: d49b850399aa12ab1e62b40e38bec7c5d0096190
Author: Jacques Lucke
Date:   Fri Jan 6 13:01:22 2023 +0100
Branches: master
https://developer.blender.org/rBd49b850399aa12ab1e62b40e38bec7c5d0096190

Tests: run geometry nodes benchmarks multiple times to reduce noise

Ideally, we would also get variance information out of the test, but that
seems a bit more complex to implement. For now just run the test a couple
of times and average the timings.

The test now runs between 5 and 100 times, depending on how long it
to run the test once.

===================================================================

M	tests/performance/tests/geometry_nodes.py

===================================================================

diff --git a/tests/performance/tests/geometry_nodes.py b/tests/performance/tests/geometry_nodes.py
index 3505f5278bd..591c78045ae 100644
--- a/tests/performance/tests/geometry_nodes.py
+++ b/tests/performance/tests/geometry_nodes.py
@@ -11,18 +11,33 @@ def _run(args):
     # Evaluate objects once first, to avoid any possible lazy evaluation later.
     bpy.context.view_layer.update()
 
-    # Tag all objects with geometry nodes modifiers to be recalculated.
-    for ob in bpy.context.view_layer.objects:
-        for modifier in ob.modifiers:
-            if modifier.type == 'NODES':
-                ob.update_tag()
-                break
-
-    start_time = time.time()
-    bpy.context.view_layer.update()
-    elapsed_time = time.time() - start_time
-
-    result = {'time': elapsed_time}
+    test_time_start = time.time()
+    measured_times = []
+
+    min_measurements = 5
+    max_measurements = 100
+    timeout = 5
+
+    while True:
+        # Tag all objects with geometry nodes modifiers to be recalculated.
+        for ob in bpy.context.view_layer.objects:
+            for modifier in ob.modifiers:
+                if modifier.type == 'NODES':
+                    ob.update_tag()
+                    break
+
+        start_time = time.time()
+        bpy.context.view_layer.update()
+        elapsed_time = time.time() - start_time
+        measured_times.append(elapsed_time)
+
+        if len(measured_times) >= min_measurements and test_time_start + timeout < time.time():
+            break
+        if len(measured_times) >= max_measurements:
+            break
+
+    average_time = sum(measured_times) / len(measured_times)
+    result = {'time': average_time}
     return result