2

I am trying to stream video and audio through two separate RTP channels and then retrieve and play them on a client through webRTC and play them in sync. I am able to retrieve the packets on a webRTC client, but the audio is very choppy. My System is as follows:

  • Ubuntu 20.04
  • Ffmpeg 4.2.2
  • 16 GB RAM
  • 6 core Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz

The following is the command I ran.

ffmpeg \
-f x11grab -video_size 1280x720 -i :20.0 \
-f pulse -ac 2 -i default \

-c:v libx264 -copyts -an -r 30 -maxrate 2000k -minrate 1000k -b:v 1500k \
-bufsize 500k -tune zerolatency -preset veryfast -pix_fmt yuv420p \
-payload_type 101 -ssrc 22222222 -pkt_size 1000 -f rtp rtp://127.0.0.1:5004 \

-c:a libopus -vn -ar 48000 -b:a 48k \
-payload_type 102 -ssrc 11111111 -pkt_size 1000 -f rtp rtp://127.0.0.1:5008

Further investigation has lead me to realise that RTP streamed audio by itself was stuttery. The command being the following:

ffmpeg \
-f pulse -ac 2 -i default \
-c:a libopus -ar 48000 -b:a 48k -ac 2 -payload_type 102 \
-ssrc 11111111 -pkt_size 1000 -f rtp rtp://127.0.0.1:5008

What I've found

In the following cases, a YouTube video was played, audio recorded and sent to a browser on another computer (through webRTC).

  1. The audio on first play sounds extremely distorted, and takes 2 to 3 seconds, before it gets back to normal.
  2. Pausing and playing the video again, does not result in any sound distortion.
  3. Resuming a video after approx 20+ seconds results in distorted audio for 2-3 seconds before returning back to normal.
  4. Changing YouTube videos always results in audio distortion (for 2-3 seconds).

Here's a recording of me jumping around to three different YouTube videos. You can hear the issues that I described, particularly the audio stuttering, distortion, and the delay when starting: https://streamable.com/xz0uq8

Ffmpeg Logs

When a YouTube video is paused for approx 20+ seconds, I see a log that looks like this. I see a large stream of these Non-montonous DTS messages before returning back to normal. When a video is playing continuously, I do not experience any of these stutters or see any of these messages.

    Last message repeated 1 times
[rtp @ 0x55ba5d0219c0] Non-monotonous DTS in output stream 1:0; previous: 76391534506976, current: 76391534506064; changing to 76391534506976. This may result in incorrect timestamps in the output file.

[rtp @ 0x55ba5d0219c0] Non-monotonous DTS in output stream 1:0; previous: 76391534506976, current: 76391534506652; changing to 76391534506976. This may result in incorrect timestamps in the output file.

[libopus @ 0x55ba5d044640] Queue input is backward in time

[rtp @ 0x55ba5d0219c0] Non-monotonous DTS in output stream 1:0; previous: 76391534513260, current: 76391534512685; changing to 76391534513260. This may result in incorrect timestamps in the output file.

[libopus @ 0x55ba5d044640] Queue input is backward in time

frame=78361 fps= 30 q=0.0 size=   78742kB time=442080:38:22.83 bitrate=   0.0kbits/s dup=334 drop=333 speed=6.1e+05x    
[libopus @ 0x55ba5d044640] Queue input is backward in time

    Last message repeated 1 times
frame=78377 fps= 30 q=0.0 size=   78745kB time=442080:38:23.30 bitrate=   0.0kbits/s dup=334 drop=333 speed=6.1e+05x    
frame=78392 fps= 30 q=0.0 size=   78746kB time=442080:38:23.80 bitrate=   0.0kbits/s dup=334 drop=333 speed=6.1e+05x    
frame=78406 fps= 30 q=0.0 size=   78748kB time=442080:38:24.30 bitrate=   0.0kbits/s dup=334 drop=333 speed=6.09e+05x

I need to achieve smooth audio playback without any of the issues I described above. Does anybody have any idea what might be going on? Thanks.

jason
  • 21
  • 3

0 Answers0