Attila Bakos

I analyzed the raw data in the DNG converted from ProRes RAW and now I know exactly how it's messed up.
I adjusted my tool and I think I have a working fix now. It fixes the raw data itself, so the result will be perfect in every software that reads DNG.
Before (see the broken line in the middle):

After:

 
I'm still testing my tool, it can read 12/16bit DNG, uncompressed, or lossless compressed, the lossy formats are not supported. It uses multiple threads, on my recent i7 laptop it converts with 25-35fps.
It's a pain to convert twice but at least I got it working, and learned a bit of C++ on the way.
If only I could fix the framerate issue as well, but no chance for that.

I analyzed the raw data in the DNG converted from ProRes RAW and now I know exactly how it's messed up.
I adjusted my tool and I think I have a working fix now. It fixes the raw data itself, so the result will be perfect in every software that reads DNG.
Before (see the broken line in the middle):

After:

 
I'm still testing my tool, it can read 12/16bit DNG, uncompressed, or lossless compressed, the lossy formats are not supported. It uses multiple threads, on my recent i7 laptop it converts with 25-35fps.
It's a pain to convert twice but at least I got it working, and learned a bit of C++ on the way.
If only I could fix the framerate issue as well, but no chance for that.

I analyzed the raw data in the DNG converted from ProRes RAW and now I know exactly how it's messed up.
I adjusted my tool and I think I have a working fix now. It fixes the raw data itself, so the result will be perfect in every software that reads DNG.
Before (see the broken line in the middle):

After:

 
I'm still testing my tool, it can read 12/16bit DNG, uncompressed, or lossless compressed, the lossy formats are not supported. It uses multiple threads, on my recent i7 laptop it converts with 25-35fps.
It's a pain to convert twice but at least I got it working, and learned a bit of C++ on the way.
If only I could fix the framerate issue as well, but no chance for that.

I analyzed the raw data in the DNG converted from ProRes RAW and now I know exactly how it's messed up.
I adjusted my tool and I think I have a working fix now. It fixes the raw data itself, so the result will be perfect in every software that reads DNG.
Before (see the broken line in the middle):

After:

 
I'm still testing my tool, it can read 12/16bit DNG, uncompressed, or lossless compressed, the lossy formats are not supported. It uses multiple threads, on my recent i7 laptop it converts with 25-35fps.
It's a pain to convert twice but at least I got it working, and learned a bit of C++ on the way.
If only I could fix the framerate issue as well, but no chance for that.

I analyzed the raw data in the DNG converted from ProRes RAW and now I know exactly how it's messed up.
I adjusted my tool and I think I have a working fix now. It fixes the raw data itself, so the result will be perfect in every software that reads DNG.
Before (see the broken line in the middle):

After:

 
I'm still testing my tool, it can read 12/16bit DNG, uncompressed, or lossless compressed, the lossy formats are not supported. It uses multiple threads, on my recent i7 laptop it converts with 25-35fps.
It's a pain to convert twice but at least I got it working, and learned a bit of C++ on the way.
If only I could fix the framerate issue as well, but no chance for that.

Remember we discussed earlier that X-H2s 6.2K ProRes RAW has a hot pixel at the exact same location for everyone?
I spent days to create an executable in C++ that removes that hot pixel from the DNG raw file created from ProRes RAW.
I was just about to post it here but then I noticed that this is not a hot pixel issue at all! All pixels in the same row right from the hot pixel are shifted towards the right.
See here:

It starts with the hot pixel, and it goes all the way to the right edge.
Or here (this is the right edge of an image, you don't see the hot pixel here, it's somewhere in the middle):

 
Atomos says it's Fuji's fault, and as usual, Fuji couldn't care less.
Atomos is also silent about the framerate bug I discovered. A new firmware was released just now and it's not fixed, it's not even listed in the known bugs.
Stuff like this makes me really disillusioned.
 

Remember we discussed earlier that X-H2s 6.2K ProRes RAW has a hot pixel at the exact same location for everyone?
I spent days to create an executable in C++ that removes that hot pixel from the DNG raw file created from ProRes RAW.
I was just about to post it here but then I noticed that this is not a hot pixel issue at all! All pixels in the same row right from the hot pixel are shifted towards the right.
See here:

It starts with the hot pixel, and it goes all the way to the right edge.
Or here (this is the right edge of an image, you don't see the hot pixel here, it's somewhere in the middle):

 
Atomos says it's Fuji's fault, and as usual, Fuji couldn't care less.
Atomos is also silent about the framerate bug I discovered. A new firmware was released just now and it's not fixed, it's not even listed in the known bugs.
Stuff like this makes me really disillusioned.
 

Remember we discussed earlier that X-H2s 6.2K ProRes RAW has a hot pixel at the exact same location for everyone?
I spent days to create an executable in C++ that removes that hot pixel from the DNG raw file created from ProRes RAW.
I was just about to post it here but then I noticed that this is not a hot pixel issue at all! All pixels in the same row right from the hot pixel are shifted towards the right.
See here:

It starts with the hot pixel, and it goes all the way to the right edge.
Or here (this is the right edge of an image, you don't see the hot pixel here, it's somewhere in the middle):

 
Atomos says it's Fuji's fault, and as usual, Fuji couldn't care less.
Atomos is also silent about the framerate bug I discovered. A new firmware was released just now and it's not fixed, it's not even listed in the known bugs.
Stuff like this makes me really disillusioned.
 

Remember we discussed earlier that X-H2s 6.2K ProRes RAW has a hot pixel at the exact same location for everyone?
I spent days to create an executable in C++ that removes that hot pixel from the DNG raw file created from ProRes RAW.
I was just about to post it here but then I noticed that this is not a hot pixel issue at all! All pixels in the same row right from the hot pixel are shifted towards the right.
See here:

It starts with the hot pixel, and it goes all the way to the right edge.
Or here (this is the right edge of an image, you don't see the hot pixel here, it's somewhere in the middle):

 
Atomos says it's Fuji's fault, and as usual, Fuji couldn't care less.
Atomos is also silent about the framerate bug I discovered. A new firmware was released just now and it's not fixed, it's not even listed in the known bugs.
Stuff like this makes me really disillusioned.
 

Remember we discussed earlier that X-H2s 6.2K ProRes RAW has a hot pixel at the exact same location for everyone?
I spent days to create an executable in C++ that removes that hot pixel from the DNG raw file created from ProRes RAW.
I was just about to post it here but then I noticed that this is not a hot pixel issue at all! All pixels in the same row right from the hot pixel are shifted towards the right.
See here:

It starts with the hot pixel, and it goes all the way to the right edge.
Or here (this is the right edge of an image, you don't see the hot pixel here, it's somewhere in the middle):

 
Atomos says it's Fuji's fault, and as usual, Fuji couldn't care less.
Atomos is also silent about the framerate bug I discovered. A new firmware was released just now and it's not fixed, it's not even listed in the known bugs.
Stuff like this makes me really disillusioned.
 

Remember we discussed earlier that X-H2s 6.2K ProRes RAW has a hot pixel at the exact same location for everyone?
I spent days to create an executable in C++ that removes that hot pixel from the DNG raw file created from ProRes RAW.
I was just about to post it here but then I noticed that this is not a hot pixel issue at all! All pixels in the same row right from the hot pixel are shifted towards the right.
See here:

It starts with the hot pixel, and it goes all the way to the right edge.
Or here (this is the right edge of an image, you don't see the hot pixel here, it's somewhere in the middle):

 
Atomos says it's Fuji's fault, and as usual, Fuji couldn't care less.
Atomos is also silent about the framerate bug I discovered. A new firmware was released just now and it's not fixed, it's not even listed in the known bugs.
Stuff like this makes me really disillusioned.
 

The maximum I could get out from my 5D3 is 3584 x 1730 in 14bit lossless. That's not binned, it uses a 1.6x crop.
We are now at around 150MB/s maximum with card spanning, sd card overclocking and several hacks to boost performance. And some talented people are still tweaking this thing.

Well, just got reply from Atomos about the bad pixel issue in 6.2K raw:
Please note that our engineering has gone over your case and replicated and tested your setup.
Unfortunately, the issue lies with the camera as it is outputting the bad pixel in 6.2 k RAW.
We would love to help you fix the issue but as it is on the camera end our hands are tied.
Please contact the camera manufacturer to get this issue resolved.
Fuji doesn't give a shit about my emails, so I wouldn't expect a fix anytime soon. This, and the framerate issue, and also the fact that Fuji's ProRes RAW has noticeably less dynamic range than F-Log, not to mention F-Log2, makes me loose faith in this company, they do a lot of things right but they always have to fuck up little things like this.
 

Yes for 4K the bitrate are very similar (I am around 780Mbps for ProRes HQ in 4KDCI) But for 6,2K I have a bitrate of around 2300 Mbps with ProRes HQ. This is really different from the 720 Mbps of H265
(Angelbirds SE 512GB cards are very good at a reasonable price, I got some around 180 euros)
https://www.angelbird.com/prod/av-pro-cfexpress-se-type-b-2919/?category=238

I needed similar in-camera and external resolutions for a fair comparison. 6.2K was the only way because in ProRes RAW you only have full-sensor 6.2K or cropped 4.8K. I don't have a CF card so I can only record ProRes HQ to the Ninja, and 6.2K is not available there, it's only available in ProRes RAW mode. This is the reason for internal H.265 instead of ProRes HQ.
And if you record in UHD 24p, internal H.265 has actually more bitrate than ProRes HQ (720Mbps vs 707Mbps). I tested it, H.265 actually looks better due to more efficient compression.
To you other question, I did not use any LUTs during my testing, I usually work with color space transform or aces nodes.

Summary of my findings so far:
 

Summary of my findings so far:
 

Well, just got reply from Atomos about the bad pixel issue in 6.2K raw:
Please note that our engineering has gone over your case and replicated and tested your setup.
Unfortunately, the issue lies with the camera as it is outputting the bad pixel in 6.2 k RAW.
We would love to help you fix the issue but as it is on the camera end our hands are tied.
Please contact the camera manufacturer to get this issue resolved.
Fuji doesn't give a shit about my emails, so I wouldn't expect a fix anytime soon. This, and the framerate issue, and also the fact that Fuji's ProRes RAW has noticeably less dynamic range than F-Log, not to mention F-Log2, makes me loose faith in this company, they do a lot of things right but they always have to fuck up little things like this.
 

I heard back from Atomos support as well last week. They said the exact same-thing to me.
I called Fuji to report the feedback from Atomos and the guy told me he tried the exact combo himself and doesn't see it but his tone was shit… He seemed very uninterested and dismissive. He actually said its not a problem for the camera, blamed Atomos, and said just because 3 or 4 cameras have this issue doesn't mean its a problem for the camera. I’m thinking they have known about the issue and sitting and quietly scaling customer feedback. I emailed them last Wednesday and haven't received response to that email. I guess it's a shoot internally for now wait and see game. I really like the camera as its fun to use but for my first time dealing with Fuji support I am very disappointed.
As you mention the dynamic range internally is better. I’m just gonna use my SmallHd indie 5 monitor and stick with F-Log2

Well, just got reply from Atomos about the bad pixel issue in 6.2K raw:
Please note that our engineering has gone over your case and replicated and tested your setup.
Unfortunately, the issue lies with the camera as it is outputting the bad pixel in 6.2 k RAW.
We would love to help you fix the issue but as it is on the camera end our hands are tied.
Please contact the camera manufacturer to get this issue resolved.
Fuji doesn't give a shit about my emails, so I wouldn't expect a fix anytime soon. This, and the framerate issue, and also the fact that Fuji's ProRes RAW has noticeably less dynamic range than F-Log, not to mention F-Log2, makes me loose faith in this company, they do a lot of things right but they always have to fuck up little things like this.
 

A small update on the X-H2s + Ninja raw recording. We tried another body and another recorder as well, and this is a general issue, not related to my unit. When you put the body to 6.2K raw mode sometimes you will get like 12 fps. It's easy to reproduce it when you switch between 23.98 and 24fps 10-20 times, at some point you will enter this low fps mode, and sometimes it stays in this mode even if you hit record. Really annoying.
I also checked this another X-H2s body for that specific "bad pixel", and it's there, only in 6.2K raw mode, so this is a general issue as well. However, it was not visible when we used the Ninja V, it's related to the Ninja V Plus.
I'm discussing these with Atomos already, will keep you posted. However, Fuji doesn't reply to any of my emails, so if it's their fault, I'm not sure what I can do.

To anyone who is interested in recording ProRes RAW with the X-H2s, I just got confirmation from Fuji that they convert X-Trans to Bayer in the camera, as I suspected. Because the two patterns are different, you have to calculate what's not there. So let's say at a given pixel X-Trans stores red but for Bayer we need blue. I asked Fuji if they interpolate this blue using the surrounding blue values in the X-Trans pattern or they just copy the value of a blue neighboring pixel. The latter is faster but more prone to artifacts. Unfortunately I received an one-liner that this is proprietary info.
I'll know more when I test this out myself, I'll receive a Ninja V+ in the following days.

To anyone who is interested in recording ProRes RAW with the X-H2s, I just got confirmation from Fuji that they convert X-Trans to Bayer in the camera, as I suspected. Because the two patterns are different, you have to calculate what's not there. So let's say at a given pixel X-Trans stores red but for Bayer we need blue. I asked Fuji if they interpolate this blue using the surrounding blue values in the X-Trans pattern or they just copy the value of a blue neighboring pixel. The latter is faster but more prone to artifacts. Unfortunately I received an one-liner that this is proprietary info.
I'll know more when I test this out myself, I'll receive a Ninja V+ in the following days.

To anyone who is interested in recording ProRes RAW with the X-H2s, I just got confirmation from Fuji that they convert X-Trans to Bayer in the camera, as I suspected. Because the two patterns are different, you have to calculate what's not there. So let's say at a given pixel X-Trans stores red but for Bayer we need blue. I asked Fuji if they interpolate this blue using the surrounding blue values in the X-Trans pattern or they just copy the value of a blue neighboring pixel. The latter is faster but more prone to artifacts. Unfortunately I received an one-liner that this is proprietary info.
I'll know more when I test this out myself, I'll receive a Ninja V+ in the following days.

To anyone who is interested in recording ProRes RAW with the X-H2s, I just got confirmation from Fuji that they convert X-Trans to Bayer in the camera, as I suspected. Because the two patterns are different, you have to calculate what's not there. So let's say at a given pixel X-Trans stores red but for Bayer we need blue. I asked Fuji if they interpolate this blue using the surrounding blue values in the X-Trans pattern or they just copy the value of a blue neighboring pixel. The latter is faster but more prone to artifacts. Unfortunately I received an one-liner that this is proprietary info.
I'll know more when I test this out myself, I'll receive a Ninja V+ in the following days.