Support for VideoEncParams #779
Description
Overview
I'm doing some quick and simple bitstream analysis in python using pyav and I'd like to get quantization params. The old way of getting these through side data/through the API was deprecated and the "new" way of doing it that works with some of the modern codecs: AVVideoEncParams was never added.
I can currently do it with something like this:
import argparse
from dataclasses import dataclass
from typing import Dict, List
import av
from bitstring import ConstBitStream
parser = argparse.ArgumentParser()
parser.add_argument("input")
parser.add_argument("--stream", type=int, default=0, required=False)
args = parser.parse_args()
container = av.open(args.input)
stream = container.streams.video[0]
stream.codec_context.options["export_side_data"] = "venc_params"
@dataclass
class VideoBlockParams:
src_x: int = 0
src_y: int = 0
w: int = 0
h: int = 0
block_delta: int = 0
@dataclass
class VideoEncParams:
delta_qp: List[Dict[str, int]]
blocks: List[VideoBlockParams]
nb_blocks: int = 0
blocks_offset: int = 0
block_size: int = 0
qp_type: int = 0
qp: int = 0
def parse_venc_params(sd: av.sidedata.sidedata.SideData) -> VideoEncParams:
params = VideoEncParams(delta_qp=[], blocks=[])
stream = ConstBitStream(sd.to_bytes())
params.nb_blocks = stream.read("uintle:32")
stream.read(32) # 4 bytes padding, probably compiler dependent
params.blocks_offset = stream.read("uintle:64")
params.block_size = stream.read("uintle:64")
params.qp_type = stream.read("uintle:32")
params.qp = stream.read("intle:32")
params.delta_qp = []
for _ in range(4):
params.delta_qp.append({"dc": stream.read("intle:32"), "ac": stream.read("intle:32")})
stream.pos = params.blocks_offset * 8
for _ in range(params.nb_blocks):
bp = VideoBlockParams()
bp.src_x = stream.read("intle:32")
bp.src_y = stream.read("intle:32")
bp.w = stream.read("intle:32")
bp.h = stream.read("intle:32")
bp.block_delta = stream.read("intle:32")
params.blocks.append(bp)
return params
for frame in container.decode(video=args.stream):
for sd in frame.side_data:
if int(sd.type) == 21:
vencparams = parse_venc_params(sd)
print(f"Got QPs for {len(vencparams.blocks)} blocks")
but I expect that wont work all the time since I'm parsing a C structure and there could be padding added to it. It also wasn't a ton of fun to write.
Existing FFmpeg API
https://www.ffmpeg.org/doxygen/trunk/structAVVideoEncParams.html
Expected PyAV API
It could be as simple as accessing motion vectors, something like
Example:
if sd.type == av.sidedata.sidedata.Type.VIDEO_ENC_PARAMS:
print(sd.qp)
for b in range(sd.blocks):
print(sd.qp + b.delta_qp)
mimicking the structure of the FFMPEG object, see the dataclasses in my initial example for something of a full interface
Investigation
There's no problem other than this API doesn't exist yet, I investigated the docs and found it missing I suppose?
Reproduction
Not sure how to reproduce an API that doesn't exist. Try to call the nonexistent functions and observe an exception I guess?
Versions
- OS: Up to date arch linux
- PyAV runtime:
PyAV v8.0.3
git origin: git@github.com:PyAV-Org/PyAV
git commit: v8.0.3
library configuration: --disable-doc --disable-static --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-shared --enable-version3 --enable-zlib --prefix=/tmp/vendor
library license: GPL version 3 or later
libavcodec 58. 91.100
libavdevice 58. 10.100
libavfilter 7. 85.100
libavformat 58. 45.100
libavutil 56. 51.100
libswresample 3. 7.100
libswscale 5. 7.100
- PyAV build: Who's setup.py am I supposed to be running and how is this relevant?
- FFmpeg: pretty sure all this info is in the pyav version command but here you go
ffmpeg version n4.3.2 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
Additional context
I think this template should be edited there seem to be some irrelevant fields