Attila Bakos

15 hours ago, androidlad said:

It's really important to expose HLG correctly because it maps 18% grey at a low 22IRE.

How did you find out that 22%? For a moment let's forget about Lutcalc data. I just shot a grey card at 46% in F-Log, using ISO640 and 1/40s. Then I switched the camera to HLG and shot the same card at ISO1000 and 1/60s. Looking by the numbers it's the same exposure (aperture did not change) but if there's no added gain in HLG then it's practically 2/3 stops lower than F-Log. In this case the grey level is about 32-33IRE.

1 hour ago, androidlad said:

I meant altering the tags to all BT.709 as well as removing the full range tag (or deliberately changing it to limited), because I heard Premiere Pro handles full range tagged files as yuvj420p format and decode using the wrong matrix regardless of resolution. Could you experiment with that if you have time? Thanks.

I tried removing the full range flag, no change. Also tried removing the full range flag & altering all color related tags, no change. Premiere and Resolve seem to completely ignore these tags.

5 minutes ago, androidlad said:

Interesting. In Premiere Pro, that LUT gives a perfect visual match.

Do you think altering the tags on H.264/H.265 files could be achieved without transcoding? And in batch?

Yes in most cases that LUT will give good results, but I had some shots with pink colors where it was ever so slightly off.

Yes you can alter the tags without transcoding, but I don't see why you would do that. Fuji's tags are actually correct, the problem is that the mainstream editors ignore them. If you modify the tags the files will look off even in editors that actually care about the metadata.

@androidlad Upon further inspection I found out that the Rec709toRec601.cube LUT is not 100% accurate, sometimes it will be slightly off. This problem might be something that can't be fixed with a single LUT, or matrix. For a really accurate conversion transcoding seems to be the only way:

ffmpeg -i INPUT.MOV -vf colorspace=all=bt709,scale=in_range=full:out_range=full -c:v prores_ks -profile:v 2 -c:a copy OUTPUT.MOV

Just remember to switch ProRes to full range once it's imported.

2 hours ago, thebrothersthre3 said:

For anyone interested the method I ended up using was adding the Fast Color Corrector tool in Video Effects under Obsolete. I changed the Output Black Level to 16 and the Output White Level to 235. Both transcoded prores and original H265 files look the same now (no more blown highlights and crushed shadows). 

Yeah that's the old way of doing it, but the method androidlad mentioned is faster, I'd recommend that one. The presets are under Effects->Lumetri presets->Technical.

36 minutes ago, androidlad said:

Do you have X-T3 with Ninja V? Would be interesting to see a comparison between internal H.265 HLG and external ProRes HLG. HLG metadata seems to be precise and complete.

Yes I'm interested in that as well, but unfortunately no external recorder here

If you check the HLG files in Assimilate Scratch and in Resolve, they look different. But if you go to the media section in Scratch and change data format from the recognized Rec2020 to Lin/sRGB and change HLG to SDR, they will look the same.

I checked a some footage with the correction matrix, and the funny thing is that I actually prefer the wrong interpretation in many shots

FYI I contacted Blackmagic support about the issue, it has been forwarded to the development team, but that's all they told me.

35 minutes ago, androidlad said:

Decoding a BT.709 YCbCr signal using BT.601 and BT.709 matrix should produce two different results, so, if Scratch was truly respecting the tags, internal and external should look different. Of course this is all based on the assumption that the camera originated YCbCr data is the same between internal and external.

That's why I believe that assumption is wrong and the data itself is different.

EDIT: Forgot to tell you that I experimented with extracting the raw stream and inserting it into another container without any flags. In this case even Scratch shows the difference, so my theory seems to be correct.

EDIT2: Or just use this script: ffmpeg -i INPUT.MOV -c copy -bsf:v hevc_metadata=matrix_coefficients=1 OUTPUT.MOV
It only changes the matrix coefficients tag to BT.709, and immediately you can see a difference in Scratch. So it's respecting the tags.

I think the underlying YCbCr data is different in the internal version compared to the ProRes version. Both share the same BT.709 primaries but the internal version requires the BT.601 matrix coefficients to convert to the correct RGB values. Resolve and Premiere seems to ignore this flag and they use the BT.709 matrix. This is perfect for the external recording, but not so much for the internal as we have seen. The LUT and the matrix that's included in that package is basicly a multiplication of two matrices, the first one converts from RGB to YCbCr using the 701 matrix, the second one converts YCbCr to RGB with the 601 matrix. So it basicly undoes the wrong YCbCr->RGB conversion and redoes it correctly. Scratch does one thing that Premiere & Resolve does not, it actually gives a shit about the flags

UPDATE: I believe we were looking at this the wrong way. There is only one place for these tags and it's at the stream level, however Premiere and Resolve doesn't even read these flags. Assimilate Scratch does, and if you modify the flags with FFMpeg the footage will look different in Scratch. The difference is also visible with MPC-HC, or FFPlay. At this point I don't think we can do much, maybe tell BM Support that they should care about these flags.

This also leads me to believe that the Fuji flags are correct, and the bad interpretation is not a result of a confusion.

16 minutes ago, androidlad said:

were you able to change metadata tags for both stream and container ? What does ffprobe say for the altered file?

I can change the stream metadata tags with the commands I posted earlier.
If I list the streams with ffprobe (ffprobe -v error -show_streams INPUT.MOV), then I get this in the original HEVC stream:
color_space=smpte170m
color_transfer=smpte170m
color_primaries=bt709

After converting all to bt709 I get this:
color_space=bt709
color_transfer=bt709
color_primaries=bt709

What I don't know is how I can change things on the container level.

EDIT:

ffprobe -show_format actually shows the container tags, and this is what I get for the original file:

[FORMAT]
filename=DSCF7556.MOV
nb_streams=3
nb_programs=0
format_name=mov,mp4,m4a,3gp,3g2,mj2
format_long_name=QuickTime / MOV
start_time=0.000000
duration=25.000000
size=635467264
bit_rate=203349524
probe_score=100
TAG:major_brand=qt
TAG:minor_version=0
TAG:compatible_brands=qt
TAG:creation_time=2019-01-14T16:13:13.000000Z
TAG:original_format=Digital Camera
TAG:original_format-eng=Digital Camera
TAG:comment=FUJIFILM DIGITAL CAMERA X-T3
TAG:comment-eng=FUJIFILM DIGITAL CAMERA X-T3
[/FORMAT]

So there is nothing here about colors. Then it's interesing why changing metadata on the stream level does nothing in Resolve/Premiere.

14 minutes ago, androidlad said:

BBC already published a tool to change colour metadata and I tried it, it didn't work because all it does (including recent ffmpeg builds) is changing the metadata tags at container level (QuickTime), while the actual metadata tags are also present at stream level (H.264/H.265), to change metadata tags at stream level you might need to go extremely sophisticated with hex editor.

I see. So it's theoretically possible but we don't have the tools. Doing this by hand is beyond my skills I'm afraid.

Btw ffmpeg's manual says, that it modifies headers in the stream: https://ffmpeg.org/ffmpeg-bitstream-filters.html#hevc_005fmetadata

With recent ffmpeg builds you can change HEVC/H.264 flags, and it doesn't solve the issue. There must be something else too.

If you want to try it, use this command to change all the color related flags to BT.709:

ffmpeg -i INPUT.MOV -c copy -bsf:v hevc_metadata=colour_primaries=1:transfer_characteristics=1:matrix_coefficients=1 OUTPUT.MOV

And this to change all to BT.601:

ffmpeg -i INPUT.MOV -c copy -bsf:v hevc_metadata=colour_primaries=6:transfer_characteristics=6:matrix_coefficients=6 OUTPUT.MOV

If you're a Resolve Studio user, you can try this DCTL, it does the same thing as the Rec709toRec601.cube mentioned earlier, but it won't clip values, and it's a bit more precise: https://www.dropbox.com/s/pm3nbyco6wg25kx/FujifilmColorFix.dctl?dl=1

2 hours ago, androidlad said:

It happens with all fuji camera internal H.264 and H.265 files, with the exception of HLG.

We had a discussion in another thread but I forgot what the consensus was. I still have those files with the man in red jacket coming from the X-T3 and from the Ninja V. In Assimilate Scratch they look the same, but in Premiere and Resolve they look different. We already have the LUTs to make them look identical, but I don't know which one is correctly displayed.

Edit: nevermind, just found the thread where you said that the Ninja is showing the correct colours. I think I'll do a DCTL for the Fuji internal files for more precision.

9 hours ago, thebrothersthre3 said:

 

Would you use the vingette in a video for that purpose or just still images? 

It's just a window around the person with a very soft edge, it can be tracked as well. I use this many times to help focusing on the subject. If you don't overdo it can be quite helpful sometimes.

Here's a quickie. A BT.2020-Rec.709 conversion, a curve to highlight the person, some vignette to darken everything else, Kodak 2383 D55 on 40%, slightly boosted saturation and desaturated shadows.

1 hour ago, ErisC said:

Thanks for the reply, I don't use Premiere Pro, I use Davinci Resolve and  the h.265(HEVC) is much darker, there is a solution on Mac?

Try setting both clips to full range in Resolve. Do they show a difference?

10 hours ago, androidlad said:

Portkeys/IKAN has one, brighter at 1500nit, similar feature set and quite a bit cheaper at $279

https://suikui.myshopify.com/products/lh5-hdr-5-1500nit-4k-signal-support-touch-screen-1920-x-1080-hdmi-on-camera-monitor-with-hdr

I wish it had waveform/parade. 

1 hour ago, BrunoCH said:

Yes the waweform has three options : Luma, RGB, parade.

 

swit 55 parade.mov

Great, thanks!

Does this monitor have RGB parade?

7 minutes ago, androidlad said:

It's starting to look like it's intentional, to manage processing power.

That's what I thought when the issue appeared. I don't think it will be fixed.

2 hours ago, thephoenix said:

what would be so specific to xt3 ??

The name

25 minutes ago, BrunoCH said:

@Attila Bakos

It's a little bit off topic but not too much because the XT3 can be powered by Anker batteries. Here is the solution I found to hold my battery: 

https://www.amazon.fr/gp/product/B00X752FNA/ref=ppx_yo_dt_b_asin_title_o00__o00_s00?ie=UTF8&psc=1 + cable tie

After just attach it to any monitor or LED holder mount.

Curious to know if someone has found another solution?

Looks good, I use this phone holder until I get something better: https://m.aliexpress.com/item/32797019174.html?trace=wwwdetail2mobilesitedetail&spider=y&productId=32797019174&productSubject=Ulanzi-IRON-MAN-II-Aluminum-Metal-Smartphone-Tripod-Mount-with-Cold-Shoe-Mount-Cell-Phone-Tripod