Ilias Giarimis

 

Here's my research... Resolve is taking the reduced-resolution image metadata for the Magic Lantern DNGs.

 

Cinema DNG has different EXIF data to stills standard DNG.

For a start no reduced res image EXIF.

 

Resolve seems only designed to read the Cinema DNG EXIF data.

 

Since DNG and Cinema DNG are practically the same format aside from metadata, if they make the metadata Cinema DNG standard in Raw2DNG I think that might fix the problem.

For examining the influence of the in DNG embedded jpeg preview you can try converting the ML DNGs with Adobe DNG converter (disabling "skip source image if destination already exists)  to new DNGs with a) full preview b) midsize preview c)no preview and load in Resolve.

Added bonus will be the DNG lossless compression applied which shrinks the file size from 3.5MB to around 2.3MB on average.

One more option is to use DNG lossy compression which shrinks 3.5MB to about 1.0MB exporting a file with 8bit log encoding. It will be interesting to check the robustness of this format in gradation.

Interesting. How would it impact dynamic range though? If you look at the dreadful DR in the H.264, I think a lot of that is due to 8bit compression.

It is not because of 8 bit, proved by the conversions of our RAWs to jpeg 8bit. An 8bit video with a rec.709 gamma can support 10 stops DR while retaining acceptable tonality without posterizations and with robustness  enough for (up to) middle gradation.

It's due to silly contrasty tone curves used by Canon/Panny etc which clip highlight/shadows.

14bit to 12bit makes a lot of sense to me. I hope they find a way to do it.

 

14bit to 10bit I think would be trickier - wouldn't you have to remap the dynamic range to avoid clipping?

Yes with 10bits linear there could be a problem, we have to use log (or just rec.709 gamma) encoding. This way It's pretty robust for gradation, proved by the use of 10bit log as a standard for Digital Intermediates.

Not totally insignificant too is that the BMCC is also writing 12bit log from a 16bit linear capture.  

Indeed, this 12bit log raw is a nice implementation gives better elasticity than 14bit linear and it became a necessity in the absense of analog amplification. To withstand 5-6 stops digital amplification a file must have excessive bit depth .. it's the same situation as with Medium Format raws with no analog ISO but 16bit linear data. 

Just wondering, was the lighting exactly the same for both images?

 

If you look at that strong highlight on the round base of the desklamp, you'll see that the BMCC can completely recover it, while the 5D3 cannot.  This with the "Whites" and "Highlights" slider pulled all the way down to recover as much as possible. 

 

I'd rather shoot 5D3 RAW because of the sensor size, but if the lighting was the same for both images, it really shows the tremendous dynamic range that the BMCC is capable of.

This happens because BMCC's RAW is just at base ISO with no analog or digital amplification and with just an exif tag (look for "baseline exposure" in exif) giving the info to the a raw converter which makes use of this tag (ACR/LR) that a 3.4 stop "exposure" increase is needed to render the shot at correct lightness.

If we open the BMCC raw with a converter like Rawtherapee which does not make use of this tag the photo shows as 3.4 stop underexposed.

Meanwhile Canon's DNG is about 0.5 stops over exposed, with average raw green levels at 2.4 stops lower than saturation instead of the usual 3.0 .. this gave an advantage regarding noise at the dark areas at the expense of burned highlights.

So the BMCC has about 4 stops greater headroom for highlights

I'm curious, how do you mesure DR using black raw frames? whats the process.

Open the black frame DNG with RawDigger http://www.rawdigger.com/ , go to "preferences" and disable "subtract black" and then read the values under sigma (σ). The displayed values are the stdev of the black frame which stdev is the definition of noise in engineering.

This way you have the "read noise" or else the "noise floor".

Then find in exif data (exif button in RawDigger) the "White Level" value and subtract from this the exif "Black Level" or better subtract the avg value (rawdigger display) of the inspected channel. This way you have the "Max Level"

Dynamic range is Max Level/Noise floor. This is the so called "engineering DR" for a single pixel.. it's what DxO measures as "Display DR" and what BlackMagic reports as DR.

To make a fair comparison we have to compare at the same output size. So if the DR of 5dIII is for 1920X1080 (2073600 pixels) output and BM is 2432X1366 (3322112 pixels) we have to normalize the later for 1920X1080 output ..

DR(BM1080) = DR(BM) + log2(sqrt(3322112/2073600)) = DR(BM) + 0.34 stops ..

http://www.dxomark.com/index.php/Publications/DxOMark-Insights/Detailed-computation-of-DxOMark-Sensor-normalization

Correct me if I am wrong, but I thought each photosite has the same dynamic range for what it can measure in terms of signal. And if you are imputing DR by using multiple photosites, more photosites would mean more DR, not less, as your statement implies. 

You are correct about the implication of more pixels as shown a few lines before in this message, ... but wrong at interpreting my message.

I mean that if there is no skipping the above normalisation for final size is OK and by resampling/binning 8Mp to 2Mp gives +1 stop DR. But if there exists any line or pixel skipping we have to account for the real number of used pixels.

If for example, due to skipping, only half of the pixels contribute to the final image we have to normalize 4Mp to 2Mp and the differense will be 0.5 stops.

So we agree isn't it ?? More pixels with given DR result in more DR when we refer to the same final size ..

If you looks at the actual raw files the DR advantage for the BMCC is clear.

If we have raw black frames it's easy to mesure the DR of BM vs 5DIII.

BTW if 5DIII uses all the pixels of the 16/9 sensor crop (no skipping) in video mode then the expected DR at 1920X1080 is about 1 stop more than DxO's measured DR of 11.74 stops for Canon at 8Mp. 

Andrew wrote..

"Colour cast
I notice that noise in the shadows and blacks in general have a magenta cast if you have a warmer white balance and the magenta tint slider to the right but if you put the magenta tint slider back towards green, the warmer yellows go way too green for my liking. This could be a light issue rather than a camera one. Energy saving practical lights do give a green cast. The solution in post was to avoid anything higher than 3200k white balance – this was more keeping with the LED practical light I had for fill lighting in the scene any way."

What's up with the magenta tint to all this RAW footage? The H264 versions seems to filter it out?

A sensor issue. The image processing engine filters it out and counter acts it normally.

Usually when developing RAW files, the magenta tint at the dark areas is due to RAW Black Point set at a lower than the correct value. Inverselly if the BP is higher than the correct we take green tint.

This behavior is because of the WB multiplicators which are almost always bigger for the R and B channels than the Green. So the result is excessive R and B components which combined gives magenta tint plus less contrast and the combination is magenta fog ...

Canon 5DIII's RAW Black Point (photo mode) is at 2047-2048. Same was the BP with the first gen DNGs by ML, those was not cropped and included the side "optically black" area where one can measure the Black Point value with RawDigger (and I did so ..it was 2047).

But ML tags the BP (DNG metadata) a bit lower at 2038. I believe this is an error by ML team and the BP should be 2047-2048.

There is a possibility that they chose lower BP to have the beneffit of lower noise (negative and positive deviations counteract) and better gradation at the darks ... and correct the resulting inbalance and lowish contrast with a LUT at a later stage (something that looks to me a very difficult task).

At the moment a user can change this tag in DNG exif (with exitool) or develop the raw with Rawtherapee and set there (in RT) the Raw.Black levels at +10.0.

BlackMagick went the other way .. set the BP higher (to clip what is mostly noise) .. thats why there was the problem with greenish tint developing the early DNG samples.