AVsync is broken at high playback speeds in some streams

[Brad Martin <mythtv@…>]

At playback speeds of 1.5x or greater, I have found that the video can often surge way ahead of the audio, by tens of seconds or more. This often happens during a commercial break, where I was in sync prior to the commercial, but completely lose sync upon the transition. It may eventually sync again when the show comes back on. FWIW I have a frame-doubling interlacer configured, which may be implicated as well.

I have tracked this down to a couple of causes:

• The VideoSync class is not being kept up to date with the frame interval and frame-doubling state at any given time as the stream and playback speed change.
• The VideoSync is attempting to clamp the maximum sync adjustment to a small multiple of the wrong frame interval due to the above.
• The avsync_predictor is continually dropping frames, because it is failing to account for extra time due to the repeat_pict setting on some video frames, as well as any dynamically-applied sync delay.

Attached is a patch against the fixes/0.28 branch that improves these issues for me.

[Peter Bennett]

Can you supply a sample video which is subject to the video surging ahead of the audio? I just watched part of a 1080i recording which has AC3 sound at 2x speed including a commercial and I did not see any occasion where video and audio were out of sync. Perhaps your commercial break is at a different resolution from the program. I would like to compare playback before and after this fix.

[Brad Martin <mythtv@…>]

I don't think it's the resolution, I think it's the framerate. The show I debugged with (NBC Nightly News) had a repeat_pict of zero during the actual program, but positive values during different commercials.

I'll crop and upload a verified sample when I get a chance.

[Brad Martin <mythtv@…>]

At this link you can find a snippet of a transport stream from an NBC Nightly News recording, which shows the problem as it goes into commercial. I also included a cellphone video of how this clip plays on my machine without the patch.

​https://drive.google.com/open?id=1BE4NP_liMhVYlsq4E-EBVAFv2nVT1Tvl

[Brad Martin <mythtv@…>]

I just added another clip at that same location from SNL (not during commercial).

[Peter Bennett]

I tested this patch with VAAPI, VDPAU and Raspberry Pi. It works well on the test videos, and I could not find any negative impact.

The first test video (NBC Nightly News) audio gets jerky at 2.0 stretch. That problem is fixed if you increase the Audio Read Ahead (available in master, see #13172).

I did not find any video of my own that exhibits the avsync problem when speeded up. I have a recording of that same Nightly News broadcast and it is fine speeded up without the patch, even when going into commercials. The problem must be dependent on the individual broadcast station.

[Mark Spieth]

Can you list the formats res + interlacing, deinterlacers used by this video. This would allow a better decision to be made. What it could mean also is CPU decoding power limited if the stream is HD or 4k 60fps progressive. The video stream after decoding is bursty. I havent analysed this case yet though.

Maybe for testing it might be good to have both modes available with a settings change.

This might be too difficult though from a practical pov.

[mythtv@…]

The problem is not jerkiness, rather the video is surging well ahead of the audio, as you can see in the mkv recording. I have been fighting this issue regularly for six months or longer, so it is neither a new problem nor is it exclusive to highly specific videos. The problem happens on two different machines for me, both running the same myth version. Both of my machines seem to be using "usleep+busy wait" as the sync method, however a brief test using RTC timing showed no difference. Both machines are running Ubuntu 16.04 with NVIDIA graphics. The first machine is using the VDPAU High Quality playback profile. The primary deinterlacer is "Advanced (2x HW)", and the fallback is "Advanced (1x HW)". The second machine (the one the mkv was captured on) is using the OpenGL High Quality profile, with "Greedy HighMotion (2x)" and "Greedy HighMotion" deinterlacers.

I don't believe a CPU limitation could explain the video surging *ahead* of the audio in this manner.

[Peter Bennett]

Maybe it is not clear from my comment - I think the patch is good and worthwhile installing.

[Mark Spieth]

Hmmm.

Increased audio buffer works though. Then your patch must be making the playback smoother or adding its own latency to compensate for the A/V timestamp separation.

It would be interesting to see how hardware sync from dri which I believe causes less issues with sync especially tearing without requiring double/triple buffering by the video card.

Is the stream 1080p60?

[Mark Spieth]

Replying to pbennett:

Maybe it is not clear from my comment - I think the patch is good and worthwhile installing.

OK commit it. Im comfortable with the level of testing you have done. I will install and trial ASAP.

[Peter Bennett]

Replying to markspieth:

OK commit it. Im comfortable with the level of testing you have done. I will install and trial ASAP.

I will wait a couple of days in case anybody else want to test. The code looks reasonable. I may think of other test cases in the meantime and I will test it with a few other videos to make sure I see no adverse impact. Let me know if you see any problems.

The test videos are 1280x720 60fps MPEG2, Audio is AC-3, 6 channels.

[Peter Bennett]

Patch fixed for applying to master branch

[Peter Bennett]

The original patch has a conflict with master. This is the same patch but fixed for master.

[Stuart Auchterlonie]

:shipit:

[Brad Martin <mythtv@…>]

In 67a11c1599f2d4e33d4e0859dfa30ed537f8563d/mythtv:

Error: Processor CommitTicketReference failed

GIT backend not available