Allow automatically rotating video based on file metadata

[zt2]

First of all, I would like to thank you for this fantastic library for manipulating videos in Python:)

Overview

Here is my situation: I have developed a REST-ful API to wrap a trained video model, and I need to make sure that all uploaded videos are not rotated before I extract frames from it.

FFmpeg executable support automatically re-rotating video based on file metadata, it was enabled by default, but I don't know how to enable this feature in PyAV.

You can find this feature from FFmpeg document:

-noautorotate
    Disable automatically rotating video based on file metadata.

I'm not sure this feature implemented in FFmpeg executable or library, I also tried to search in PyAV documents and issues, but nobody mentions this.

Existing FFmpeg API

In https://ffmpeg.org/doxygen/trunk/ffplay_8c_source.html#l00355:

static int autorotate = 1;

Expected PyAV API

Allow user set option like autorotate when calling av.open, next time container.decode(video=0) called the video frame would be auto-rotated by video metadata.

Example:

container = av.open('VIDEO_PATH', autorotate=True)

Investigation

Search document in PyAV and FFmpeg, dig into FFmpeg code.

Reproduction

Versions

• OS: Ubuntu 16.04.5 LTS
• PyAV runtime:

# python -m av --version
PyAV v6.2.0
git origin: git@github.com:mikeboers/PyAV
git commit: unknown-commit
library configuration: --disable-static --enable-shared --disable-doc
library license: LGPL version 2.1 or later
libavcodec     57. 64.100
libavdevice    57.  1.100
libavfilter     6. 65.100
libavformat    57. 56.100
libavutil      55. 34.100
libswresample   2.  3.100
libswscale      4.  2.100

• PyAV build:

{{ Complete output of `python setup.py config --verbose`. }}

• FFmpeg:

# ffmpeg -version
ffmpeg version 3.2 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --disable-static --enable-shared --disable-doc
libavutil      55. 34.100 / 55. 34.100
libavcodec     57. 64.100 / 57. 64.100
libavformat    57. 56.100 / 57. 56.100
libavdevice    57.  1.100 / 57.  1.100
libavfilter     6. 65.100 /  6. 65.100
libswscale      4.  2.100 /  4.  2.100
libswresample   2.  3.100 /  2.  3.100

[mikeboers]

Unfortunately, this is implemented in the executables, not the library.

Doubly unfortunate, it is implemented via an input filter graph. PyAV does not have any such thing. While I feel like perhaps that is something PyAV ought to have, I'm not sure how I would like to build that.

This will take some thought and time.

You could, if you so desired, build this in user space with a filter graph, just like the executables do. Assuming you know how much you want to rotate it, of course.

There is a side data branch that provides the display matrix for figuring out the angle, but you'd either need to do the matrix math yourself or we expose this.

[zt2]

Thanks for your suggestion. I will try to implement this in userspace.

[wizyoung]

It seems there indeed is a way to do it? In decord lib it works like:
https://github.com/dmlc/decord/blob/e0b7f7cabd8c7c224c2265453ee807ed19eb4d8a/src/video/video_reader.cc#L202