[Bf-blender-cvs] [a5b7cf9b5f1] master: Fix T53058: Crash when rendering to Quicktime RLE codec

[Sybren A. Stüvel]

Commit: a5b7cf9b5f134febd726dc7a92e7602dc60862ad
Author: Sybren A. Stüvel
Date:   Tue Jul 2 18:18:39 2019 +0200
Branches: master
https://developer.blender.org/rBa5b7cf9b5f134febd726dc7a92e7602dc60862ad

Fix T53058: Crash when rendering to Quicktime RLE codec

The root cause seems to be an assumption in
[generate_video_frame()](https://developer.blender.org/diffusion/B/browse/master/source/blender/blenkernel/intern/writeffmpeg.c)
that we're always using 4 bytes per pixel. This is not true when using
QTRLE in RGB mode, because that uses the RGB24 pixel format (so 3 bytes
per pixel). Just updating the `linesize` property doesn't fix it though,
but just creates a crash somewhere else.

This at least fixes the crash by always forcing RGBA to be written, even
when the user selects RGB.

===================================================================

M	source/blender/blenkernel/intern/writeffmpeg.c

===================================================================

diff --git a/source/blender/blenkernel/intern/writeffmpeg.c b/source/blender/blenkernel/intern/writeffmpeg.c
index 681c03e25b6..f3336adda30 100644
--- a/source/blender/blenkernel/intern/writeffmpeg.c
+++ b/source/blender/blenkernel/intern/writeffmpeg.c
@@ -696,9 +696,9 @@ static AVStream *alloc_video_stream(FFMpegContext *context,
   }
 
   if (codec_id == AV_CODEC_ID_QTRLE) {
-    if (rd->im_format.planes == R_IMF_PLANES_RGBA) {
-      c->pix_fmt = AV_PIX_FMT_ARGB;
-    }
+    /* Always write to ARGB. The default pixel format of QTRLE is RGB24, which uses 3 bytes per
+     * pixels, which breaks the export. */
+    c->pix_fmt = AV_PIX_FMT_ARGB;
   }
 
   if (codec_id == AV_CODEC_ID_PNG) {