av sync drifts (rtmp streaming)

[darrellenns]

I'm not sure that this is really an issue with libbmd, but hopefully someone could at least point me in the right direction. I am using the bmd branch of libav to send an rtmp stream to akamai. I've found that the audio is leading the video by about 1 second after 24 hours of streaming.

I'm using a DeckLink SDI 4K card, and the latest git versions of libbmd, libav, and rtmpdump. The script I'm using is shown below (URLs, passwords, etc changed). The CPU usage is low (around 13%), so no issues there.

Is anyone using this for long form streaming/recording? Are there some avconv options I should be using (maybe some combination of -vsync and the asyncts filter)? I would not expect there to be any need to do any special syncing, as all the data is clocked to the same SDI source. Or am I missing something?

A huge thanks to lu-zero for making the glue to put all these pieces together!

!/bin/sh

HIGH_WIDTH=1280
HIGH_HEIGHT=720
HIGH_RATE=2500k

MEDIUM_WIDTH=854
MEDIUM_HEIGHT=480
MEDIUM_RATE=1000k

LOW_WIDTH=640
LOW_HEIGHT=360
LOW_RATE=750k

ENTRY_POINT="rtmp://myentrypoint.akamaientrypoint.net/EntryPoint"
USERNAME=myakamaiuser
PASSWORD="myakamaipass"
STREAM_ID1="akamai_id1"
STREAM_ID2="akamai_id2"
STREAM_ID3="akamai_id3"

GOP_LENGTH=150

##########################

#limit to 8GB of ram in case of memory leak
ulimit -v 8000000

nice -n -10 avconv \
-stats -loglevel verbose \
-analyzeduration 0 -f bmd -video_mode 8 -audio_connection 2 -video_connection 4 -video_format 1 -i default \
 \
-c:v libx264 -pre:v medium -profile:v baseline -level 40 -b:v ${HIGH_RATE} -minrate ${HIGH_RATE} -maxrate ${HIGH_RATE} -bufsize 2048k \
-tune fastdecode -g ${GOP_LENGTH} -refs 0 -threads 12 \
-c:a libfdk_aac -ac 2 -ar 44100 -b:a 128k \
-filter:v "scale=w=${HIGH_WIDTH}:h=${HIGH_HEIGHT}" \
-f flv "${ENTRY_POINT} live=true pubUser=${USERNAME} pubPasswd=${PASSWORD} playpath=${STREAM_ID1} flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)" \
 \
-c:v libx264 -pre:v medium -profile:v baseline -level 40 -b:v ${MEDIUM_RATE} -minrate ${MEDIUM_RATE} -maxrate ${MEDIUM_RATE} -bufsize 1024k \
-tune fastdecode -g ${GOP_LENGTH} -refs 0 -threads 12 \
-c:a libfdk_aac -ac 2 -ar 44100 -b:a 128k \
-filter:v "scale=w=${MEDIUM_WIDTH}:h=${MEDIUM_HEIGHT}" \
-f flv "${ENTRY_POINT} live=true pubUser=${USERNAME} pubPasswd=${PASSWORD} playpath=${STREAM_ID2} flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)" \
 \
-c:v libx264 -pre:v medium -profile:v baseline -level 40 -b:v ${LOW_RATE} -minrate ${LOW_RATE} -maxrate ${LOW_RATE} -bufsize 192k \
-tune fastdecode -g ${GOP_LENGTH} -refs 0 -threads 12 \
-c:a libfdk_aac -ac 2 -ar 44100 -b:a 128k \
-filter:v "scale=w=${LOW_WIDTH}:h=${LOW_HEIGHT}" \
-f flv "${ENTRY_POINT} live=true pubUser=${USERNAME} pubPasswd=${PASSWORD} playpath=${STREAM_ID3} flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/9272392-av-sync-drifts-rtmp-streaming?utm_campaign=plugin&utm_content=tracker%2F256780&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F256780&utm_medium=issues&utm_source=github).

[lu-zero]

4k is known to have issues with certain drivers. Which version are you
using?
On 6 Mar 2015 18:39, "dack" notifications@github.com wrote:

I'm not sure that this is really an issue with libbmd, but hopefully
someone could at least point me in the right direction. I am using the bmd
branch of libav to send an rtmp stream to akamai. I've found that the audio
is leading the video by about 1 second after 24 hours of streaming.

I'm using a DeckLink SDI 4K card, and the latest git versions of libbmd,
libav, and rtmpdump. The script I'm using is shown below (URLs, passwords,
etc changed). The CPU usage is low (around 13%), so no issues there.

Is anyone using this for long form streaming/recording? Are there some
avconv options I should be using (maybe some combination of -vsync and the
asyncts filter)? I would not expect there to be any need to do any special
syncing, as all the data is clocked to the same SDI source. Or am I missing
something?

A huge thanks to lu-zero for making the glue to put all these pieces
together!

!/bin/sh

HIGH_WIDTH=1280
HIGH_HEIGHT=720
HIGH_RATE=2500k

MEDIUM_WIDTH=854
MEDIUM_HEIGHT=480
MEDIUM_RATE=1000k

LOW_WIDTH=640
LOW_HEIGHT=360
LOW_RATE=750k

ENTRY_POINT="rtmp://myentrypoint.akamaientrypoint.net/EntryPoint"
USERNAME=myakamaiuser
PASSWORD="myakamaipass"
STREAM_ID1="akamai_id1"
STREAM_ID2="akamai_id2"
STREAM_ID3="akamai_id3"

GOP_LENGTH=150

##########################

#limit to 8GB of ram in case of memory leak
ulimit -v 8000000

nice -n -10 avconv
-stats -loglevel verbose
-analyzeduration 0 -f bmd -video_mode 8 -audio_connection 2 -video_connection 4 -video_format 1 -i default

-c:v libx264 -pre:v medium -profile:v baseline -level 40 -b:v ${HIGH_RATE} -minrate ${HIGH_RATE} -maxrate ${HIGH_RATE} -bufsize 2048k
-tune fastdecode -g ${GOP_LENGTH} -refs 0 -threads 12
-c:a libfdk_aac -ac 2 -ar 44100 -b:a 128k
-filter:v "scale=w=${HIGH_WIDTH}:h=${HIGH_HEIGHT}"
-f flv "${ENTRY_POINT} live=true pubUser=${USERNAME} pubPasswd=${PASSWORD} playpath=${STREAM_ID1} flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

-c:v libx264 -pre:v medium -profile:v baseline -level 40 -b:v ${MEDIUM_RATE} -minrate ${MEDIUM_RATE} -maxrate ${MEDIUM_RATE} -bufsize 1024k
-tune fastdecode -g ${GOP_LENGTH} -refs 0 -threads 12
-c:a libfdk_aac -ac 2 -ar 44100 -b:a 128k
-filter:v "scale=w=${MEDIUM_WIDTH}:h=${MEDIUM_HEIGHT}"
-f flv "${ENTRY_POINT} live=true pubUser=${USERNAME} pubPasswd=${PASSWORD} playpath=${STREAM_ID2} flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

-c:v libx264 -pre:v medium -profile:v baseline -level 40 -b:v ${LOW_RATE} -minrate ${LOW_RATE} -maxrate ${LOW_RATE} -bufsize 192k
-tune fastdecode -g ${GOP_LENGTH} -refs 0 -threads 12
-c:a libfdk_aac -ac 2 -ar 44100 -b:a 128k
-filter:v "scale=w=${LOW_WIDTH}:h=${LOW_HEIGHT}"
-f flv "${ENTRY_POINT} live=true pubUser=${USERNAME} pubPasswd=${PASSWORD} playpath=${STREAM_ID3} flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

—
Reply to this email directly or view it on GitHub
#11.

[darrellenns]

I'm on driver 10.3.7 (the latest). Is there are certain version that is known to work better?

[darrellenns]

Tried 10.3.1 driver - same issue. I'm going to try some older versions and see if I can get them to work. What's the oldest SDK version supported by libbmd?

[lu-zero]

Pretty much all are supported beside the oldest, but those won't support
your device anyway
On 9 Mar 2015 18:26, "dack" notifications@github.com wrote:

Tried 10.3.1 driver - same issue. I'm going to try some older versions and
see if I can get them to work. What's the oldest SDK version supported by
libbmd?

—
Reply to this email directly or view it on GitHub
#11 (comment).

[darrellenns]

No luck with different driver versions. The oldest one I could get to work is 10.1.1, and I tried a bunch of others as well. After running for 20 hours, the audio is leading the video by about 10 frames. Do you have any links to info about the 4k driver issues you mentioned?

[lu-zero]

I hope to have time to buy either a firewire enclosure or a fast desktop to
test directly, I have a good laptop but the desktop I use to try gets quite
killed when I try 4k...

Try adding -af asyncts:compensate=1 and tell me if it works any better. I
have a rewrite of libbmd in progress, let see if I can make it more
efficient.

On Thu, Mar 12, 2015 at 4:54 PM, dack notifications@github.com wrote:

No luck with different driver versions. The oldest one I could get to work
is 10.1.1, and I tried a bunch of others as well. After running for 20
hours, the audio is leading the video by about 10 frames. Do you have any
links to info about the 4k driver issues you mentioned?

—
Reply to this email directly or view it on GitHub
#11 (comment).

[darrellenns]

I've been trying a bunch of combinations of avsyncts, -vsync, -copyts, etc. So far no luck. Because the video always lags behind the audio, it seems that something must be applying the wrong presentation timestamps, or buffering up video frames before the timestamp is applied. The unknown factor is whether this is happening in the driver, libbmd, the libav bmd patch, or elsewhere in libav. I've also discovered that I can induce the video lag by connecting a video switcher to the input and switching back and forth a few times to create some discontinuity (though it does drift even with a solid continuous input, this makes it a bit easier to test without waiting a day for it to drift).

[darrellenns]

No combinations of asyncts filter, vsync, and copyts seem to make any difference. My current best guess is that buffering of the video frames is occurring somewhere before the frames are tagged with the libav PTS (resulting in an incorrect PTS being applied). It could be in libbmd, the bmd patch for libav, or the drivers themselves.

[lu-zero]

I'm checking today, and I found already a couple of items I should fix ASAP
but unrelated to your problem.

On Fri, Mar 13, 2015 at 5:18 PM, dack notifications@github.com wrote:

No combinations of asyncts filter, vsync, and copyts seem to make any
difference. My current best guess is that buffering of the video frames is
occurring somewhere before the frames are tagged with the libav PTS
(resulting in an incorrect PTS being applied). It could be in libbmd, the
bmd patch for libav, or the drivers themselves.

—
Reply to this email directly or view it on GitHub
#11 (comment).

[darrellenns]

Thanks, much appreciated. If there's anything else I can do or other data I can provide, just let me know.

[lu-zero]

More spare time =)

On Fri, Mar 13, 2015 at 6:19 PM, dack notifications@github.com wrote:

Thanks, much appreciated. If there's anything else I can do or other data
I can provide, just let me know.

—
Reply to this email directly or view it on GitHub
#11 (comment).