[Bf-python] Image pixels (very)fast access

[Kerim Borchaev]

Hi all!

I'm Kerim Borchaev, participating in development of a renderer addon
for Blender.
One of the big bottlenecks for the addon is image data access. And
bitmap data is the biggest part of many(most?) scenes.
Today it takes about 1 second to retrieve all pixels from a 2k blender
image. This is at least an order of magnitude slower than it is
theoretically possible. This is expected, of course. Python tuple(or
iterator) access is inherently slow - pixel data needs to be kept in a
contiguous buffer.

I've made a simple patch(below) to demonstrate this, comparing
creation of a numpy array for 2k texture pixels using existing api for
prop_array(pixels[:]) and from a returned PyBytes object:

iterations: 10, size: 64.0 mb
make_from_pixels() 9.168 s
    throughput: 69 mb/s
make_from_bytes() 0.364 s
    throughput: 1759 mb/s

I'm new to blender codebase but I can see a couple of ways this can be
accomplished:
- additional property/method of bpy.types.Image. Returning python
'bytes' object - 'bpy.data.images[0].pixels_bytes'.
- bytes api for pyrna_prop_array, e.g.
bpu.data.images[0].pixels.as_bytes(). Same as I did in in the patch.
But I guess making it 'right'(like supporting all array types) is
different from a simple hack like this.
- something else(e.g. replacing tuple for slicing array_prop with
numpy array, etc)?

Is this idea is something that might be accepted by the Blender Team?
I'd love to implemented it in that case.

Kerim

---
 source/blender/python/intern/bpy_rna.c | 20 ++++++++++++++++++++
 tests/python/bl_image_pixels_test.py   | 29 +++++++++++++++++++++++++++++
 2 files changed, 49 insertions(+)
 create mode 100644 tests/python/bl_image_pixels_test.py

diff --git a/source/blender/python/intern/bpy_rna.c
b/source/blender/python/intern/bpy_rna.c
index dc247f28539..169ae9261f5 100644
--- a/source/blender/python/intern/bpy_rna.c
+++ b/source/blender/python/intern/bpy_rna.c
@@ -3082,6 +3082,25 @@ static int
pyrna_prop_array_ass_subscript(BPy_PropertyArrayRNA *self, PyObject *
    return ret;
 }

+static PyObject *pyrna_prop_array_as_bytes(BPy_PropertyRNA *self)
+{
+   Py_ssize_t len = (Py_ssize_t)pyrna_prop_array_length(self);
+
+   int itemsize = 4;
+   int length = itemsize * len;
+
+   PyObject *pybytes = PyBytes_FromStringAndSize(NULL, length);
+   RNA_property_float_get_array(&self->ptr, self->prop,
PyBytes_AsString(pybytes));
+   return pybytes;
+}
+
+PyDoc_STRVAR(pyrna_prop_array_as_bytes_doc,
+".. method:: as_bytes()\n"
+"\n"
+"   Return the raw data of the array\n"
+);
+
+
 /* for slice only */
 static PyMappingMethods pyrna_prop_array_as_mapping = {
    (lenfunc) pyrna_prop_array_length,               /* mp_length */
@@ -5021,6 +5040,7 @@ static struct PyMethodDef pyrna_prop_methods[] = {
 };

 static struct PyMethodDef pyrna_prop_array_methods[] = {
+    {"as_bytes", (PyCFunction)pyrna_prop_array_as_bytes, METH_NOARGS,
pyrna_prop_array_as_bytes_doc},
    {NULL, NULL, 0, NULL}
 };

diff --git a/tests/python/bl_image_pixels_test.py
b/tests/python/bl_image_pixels_test.py
new file mode 100644
index 00000000000..c18d5e7b359
--- /dev/null
+++ b/tests/python/bl_image_pixels_test.py
@@ -0,0 +1,29 @@
+import timeit
+import numpy as np
+
+import bpy
+
+bpy.ops.image.new(name="Untitled", width=2048, height=2048,
alpha=True, generated_type='COLOR_GRID', float=True)
+
+
+def make_from_bytes():
+    return np.frombuffer(bpy.data.images[0].pixels.as_bytes(),
dtype=np.float32)
+
+
+def make_from_pixels():
+    return np.array(bpy.data.images[0].pixels[:], dtype=np.float32)
+
+
+assert list(make_from_bytes()) == list(make_from_pixels())
+
+image_size_bytes = make_from_bytes().nbytes
+
+iterations = 10
+
+print("iterations: %s, size: %s mb" % (iterations, image_size_bytes/2**20))
+
+for code in ['make_from_pixels()', 'make_from_bytes()']:
+    t = timeit.timeit(code, number=iterations, globals=globals())
+    print(code, "%.3f s" % t)
+    print('   ', 'throughput: %d mb/s' %
((image_size_bytes*iterations)/2**20/t))
+
--