[Bf-vfx] On Format Woes

[Peter Schlaile]

Hi Ton,

> I'm personally a bit concerned about the image quality itself  - the amount of noise is disturbingly high. I would expect a nicer signal-noise ratio.
> 
> I also like to figure out how sony stores a frame in 10 Mbyte, and exports that to 50 MByte files. We probbly have to do more testing. :) And talk to sony i guess!
> (There seems to be a F65 SDK too, but I didn't check on that yet)

you should actually be worried about that (at least a little, see
below) :)

My observations so far:

* I've taken a look using a hexeditor and mxflib on the MXF-files you
provided.

The audio tracks are standard encoding and happily decoded by mxflib, so
at least no problem at this front.

* video raw data is a different story. Taking a look using a hex editor,
they seem to store the bayer planes compressed into small junks in
varying size. One can see (using a hexeditor, search for 0x40 0x00 0x00
0x22) that there is a block structure and we have 2 blocks of roughly
(read varying!) size X and one block of roughly four times X. 

That fits nicely with the specification of the camera, which states,
that they used a modified bayer matrix, which has twice as much pixels
for green than usual. (they use: 4 pixels of green together with 1 pixel
of red and 1 pixel of blue, where as RED (and any other camera) uses
only 2 pixels of green.)

Take a look here, for the gory details of RED color encoding:
http://peter.schlaile.de/redcode (they actually do use JPEG2000 to
encode their bayer planes, newer versions use a slightly modified /
obfuscated version).

Take a look here, for the details of the Sony F65 sensor:
http://image-sensors-world.blogspot.de/2011/04/sony-clearvid-lands-in-digital-cinema.html )

Regarding compression ratio, we can do a little math on file size:
mxfdump tells us, that you recorded a file with 135 frames. The video
frames have a stored resolution of 4160x2204 pixels (Display resolution
is 4096x2160).

File size per video frame (roughly, subtracting audio frames, ignoring
mux overhead): (1390741040-12960000)/135 = 10205785 bytes.

Raw sensor data should accumulate to: 
4160 x 2204 x (1 Green + 1/4 Blue + 1/4 Red) x 16bit = 27505920 bytes.

Roughly a compression ratio of 1:3. There is still hope, that this can
be done losslessly or at least with very mild compression.

For comparison: a RED encoded 4k RAW frame has the following specs:
(Taking the 4k-ColorDots-T8 from bealecorner.org)

File size: 1292268 bytes

Raw sensor data:

4096 x 2304 x (1/2 Green + 1/4 Blue + 1/4 Red) x 12bit = 14155776 bytes

This is a completely different story, namely a factor of round about
1:11. (That said: recent RED cameras can be tuned to 6:1 in 4k
resolution).

Looking at ARRI Alexa frames BTW reveals, that they store indeed simple
uncompressed data 1:1 no strings attached. It's a pity, that this camera
only delivers 2k - at least, it does it totally uncompressed.

So I see three ways to go:

a) ask Sony, if they will clarify this issue and/or provide us with a
SDK.

b) ask ARRI, if they have a 4k Alexa in the pipeline (despite the
statement within their FAQ which tells us otherwise...) :)

c) ask RED, if they ... wait, is there any point in doing so? 

Seriously: even the Wavelet encoded RED files look very nice. They do
pretty clever difference encoding for the R and B-channel (they are
compressed relative to the G-channel).

On the minus side is there deliberate codec obfuscation on current
firmware releases, which is a pity, when used within an Open Movie
Project.

The Sony F65 should nevertheless deliver the better picture quality
regarding the used codec compression ratio (keep in mind, could be
lossless, we don't really know.), the higher bit depth and the better
resolution within the green channel (which is very important since Green
has the strongest contribution to the luminance channel).

At least in theory, since there is a lot more to picture quality than
the codec in use.

<troll mode>
Have we already talked about the type of lenses, you intend to use for
Mango :) ?

Did you find a focus puller :) ?
</troll mode>

Cheers,
Peter

P.S.: If I find out, how the bayer plane compression within the F65
works, I'll let you know.