Dolby Vision: Parse RPU in HEVC and AV1

[cjee21]

Parse RPU in HEVC which is needed for resolving #1482, #2118 and #2468

Also parse EL in BL+EL+RPU type HEVC streams.

Also parse RPU in AV1 metadata OBU.

[cjee21]

To resolve #2118, refer to 6.2 Annex II (page 23) of https://professionalsupport.dolby.com/s/article/What-is-Dolby-Vision-Profile?language=en_US

[cjee21]

Since it is complex to parse the RPU, I do not know if there are still bugs in the parsing.

I only parse the RPU and not fill or check integrity. I'll leave that to @JeromeMartinez since I do not know what info to extract and how to indicate enhancement layer type. The temporary section is to be removed when that information is filled and shown in output. I did not add any MEDIAINFO_TRACE guards either since that parsing is likely needed to resolve the two GitHub issues mentioned.

[cjee21]

Some interesting Dolby Vision IQ / Dolby Vision 2 stuff found in iPhone sample:

01189AC       ext_metadata_block (5 bytes)
01189AC        ext_block_length:                4 (0x4)
01189AC        ext_block_level:                 11 (0x0B) - (8 bits)
01189AD        ext_block_payload (4 bytes)
01189AD         content_type:                   4 (0x04) - (8 bits) - User Generated Content
01189AE         white_point:                    0 (0x0) - (4 bits) - D65
01189AF         reference_mode_flag:            No
01189AF         reserved:                       0 (0x0) - (3 bits)
01189AF         sharpness:                      0 (0x0) - (2 bits) - Default
01189AF         noise_reduction:                0 (0x0) - (2 bits) - Default
01189B0         mpeg_noise_reduction:           0 (0x0) - (2 bits) - Default
01189B0         frame_rate_conversion:          0 (0x0) - (2 bits) - Default
01189B0         brightness:                     0 (0x0) - (2 bits) - Default
01189B0         color:                          0 (0x0) - (2 bits) - Default
01189B1         reserved1:                      0 (0x0) - (2 bits)
01189B1         reserved2:                      0 (0x0) - (2 bits)

[JeromeMartinez]

Since it is complex to parse the RPU, I do not know if there are still bugs in the parsing.

Thank you, it helps a lot.

I'll leave that to @JeromeMartinez since I do not know what info to extract and how to indicate enhancement layer type.

I'll check (but only in October :( ), but do you have any hing about FEL vs MEL?

[cjee21]

do you have any hing about FEL vs MEL?

Hint? Find this part of the code added in this PR:

    // TEMPORARY ==========================================================================
    if (isMEL)
        Param2("EL Type", "Minimum Enhancement Layer (MEL)");
    else
        Param2("EL Type", "Full Enhancement Layer (FEL / non-MEL)");
    // end TEMPORARY ======================================================================

It is based on what's mentioned above in #2392 (comment).

[cjee21]

Hmm... Dolby sometimes calls RPU Reference Processing Unit and sometimes Reference Picture Unit...

Also...

Different Dolby Vision profiles use different mechanisms for carriage of the Dolby Vision reference processing unit. These include:
• Unspecified NAL units (NALu)
• T.35 versatile supplementary enhancement information (VSEI) using Dolby terminal provider code.
• T.35 Metadata open bitstream unit (OBU) using Dolby terminal provider code and wrapping the Dolby Vision reference processing unit in an EMDF

For now only parsed the Unspecified NALu type.

[JeromeMartinez]

T.35 versatile supplementary enhancement information (VSEI) using Dolby terminal provider code.

I never saw such content but if we find it, it would be easy to map the code to this one.

[cjee21]

T.35 Metadata open bitstream unit (OBU) using Dolby terminal provider code and wrapping the Dolby Vision reference processing unit in an EMDF

This one Dolby Vision profile 10 in AV1.

[JeromeMartinez]

This one Dolby Vision profile 10 in AV1.

I have missed something, link?

[cjee21]

This one Dolby Vision profile 10 in AV1.

I have missed something, link?

I have not looked into this but I saw it at the following places:
https://professionalsupport.dolby.com/s/article/What-is-Dolby-Vision-Profile?language=en_US
https://kodi.wiki/view/Samples
https://github.com/quietvoid/dovi_tool/blob/main/dolby_vision/src/av1/emdf.rs

[cjee21]

The L11 metadata part has mismatch between Dolby docs and Dolby patent so may have to tweak once there are more samples.

[JeromeMartinez]

The L11 metadata part has mismatch between Dolby docs and Dolby patent so may have to tweak once there are more samples.

dolby docs are usually at a later time and have fixes the patent doesn't have, so priority on Dolby docs.

[JeromeMartinez]

@cjee21 please split the PR, the 2 commits are 2 different things and I may merge the small one quickly.

[cjee21]

dolby docs are usually at a later time and have fixes the patent doesn't have, so priority on Dolby docs.

For the part that are in docs and sample files I follow that (docs did not say how many bits but it is parsed in a way that correct data is obtained from actual files) but some parts that I guess are used by Dolby Vision 2 are only seen in patent for now.

[ValeZAA]

Just to mention again, only those parameters are MEL, anything else and it is FEL, so called trivial NLQ parametrs, since any other NLQ parameters require FEL, THAT IS THEY require second HEVC stream

#2118 (comment)

haasn/libplacebo#286

And yes, this is indeed 6.2 Annex II (page 23) of https://professionalsupport.dolby.com/s/article/What-is-Dolby-Vision-Profile?language=en_US

[cjee21]

Anyone has DV FEL and MEL samples so that I can test that the detection works properly?

[ValeZAA]

Anyone has DV FEL and MEL samples so that I can test that the detection works properly?

There are a lot of different samples I have.
Hundreds.

https://drive.google.com/drive/folders/1V-onFW4PwgKAFdVlwXeuKOnUvdZXqsY6

Including samples where the RPU is one frame off the video. https://drive.google.com/drive/folders/1nMz95KqgkO95EDGrYhrk-Ifx6vRYDEfs see folder Out of sync RPU

[cjee21]

I added a temp commit for easier testing.

---

@ValeZAA which video contains MEL? Can't seem to find one.

Also, what does white point value 1 mean in L11 Dolby Vision P8 IQ TEST White Point (works only if DV IQ is enable).mp4?
https://professionalsupport.dolby.com/s/article/Dolby-Vision-IQ-Content-Type-Metadata-L11?language=en_US only lists two white points.

[cjee21]

Added CMv...

Temp test filling:

Color range                              : Full
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : IPT-PQ-C2
Dolby_Vision                             : RPU Present
 RPU_Profile                             : 0
 Base_Layer                              : 10 bits
 Enhancement_Layer                       : 10 bits
 VDR                                     : 12 bits
 Base_Layer_Video_Range                  : Full
 Content_Mapping_version                 : 4.0
 Content_Type_Metadata                   : Present
  Content_Type                           : Movies
  White_Point                            : D65
  Sharpness                              : Default
  Noise_Reduction                        : Default
  MPEG_Noise_Reduction                   : Default
  Frame_Rate_Conversion                  : Default
  Brightness                             : Default
  Color                                  : Default
 Deduced_Profile                         : 5
Codec configuration box                  : hvcC+dvcC

[quietvoid]

Not sure if it was added but another useful thing for profile 7 would be MEL vs. FEL.

See in https://professionalsupport.dolby.com/s/article/What-is-Dolby-Vision-Profile
6.2 Annex II: Differentiating MEL and non-MEL bitstreams

[cjee21]

Not sure if it was added but another useful thing for profile 7 would be MEL vs. FEL.

There is

Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2
Dolby_Vision                             : RPU Present
 RPU_Profile                             : 1
 Base_Layer                              : 10 bits
 Enhancement_Layer                       : 10 bits
 VDR                                     : 12 bits
 Base_Layer_Video_Range                  : Limited
 Enhancement_Layer_Type                  : Full (FEL)
 Content_Mapping_version                 : 4.0
 Deduced_Profile                         : 7
Codec configuration box                  : hvcC+dvcC

[cjee21]

Dolby Vision also contains aspect ratio metadata that may be used to indicate cropping or active display area. But it is dynamic and may vary throughout the video so I am not sure if it should be shown in MediaInfo.

https://github.com/xbmc/xbmc/issues/27033#issuecomment-3497128236

[JeromeMartinez]

Dolby Vision also contains aspect ratio metadata that may be used to indicate cropping or active display area. But it is dynamic and may vary throughout the video so I am not sure if it should be shown in MediaInfo.

Please show this information, it is classic to have dynamic metadata (also for e.g. AVC aspect ratio), and we have a long term plan for better showing that when it changes, better to have the information now so it will be ready for metadata change display when we implement it.

[cjee21]

Please show this information

Rebased and done.

Using the faulty (double-cropped) sample from the xbmc/Kodi issue link above:

Width                                    : 3 840 pixels
Height                                   : 1 612 pixels
Display aspect ratio                     : 2.39:1
Active width                             : 3 840 pixels / 3 840 pixels / 3 840 pixels / 3 840 pixels / 3 840 pixels
Active height                            : 1 059 pixels / 1 059 pixels / 1 059 pixels / 1 059 pixels / 1 059 pixels
Active display aspect ratio              : 3.630