2011/03/21

Audio Video Sync Problem

This is how AV sync is done:
Audio time is get and set to video processing module.
video block compare the current video time and audio time and decide the should play video faster or slower.
How video can play video faster or slower?
Video blocks decodes a frame and then display it on the screen, by extending or shortening the display time, video can be played faster or slower.
Please be noticed that decoding time can never be reduced and no frame is dropped.
AV mis-sync example:
1.
We consume too much time in video decoding process, so that even we display the decoded video frame at shortest time, video still can not catch up with audio.
This is how we spent too much time in video decoding. We align every byte of video ring-buffer before we decode one frame, and the video is 30fps, which will consume a lot of computation time.
2.
A/V mis-sync happened during encoding. This problem happens because start of audio encoding is too much late than start of video encoding, And all the first audio/video encoded frame starts with the timestamp 0. So when we play the encoded video file, we can see the video is delayed.
3.
A/V mis-sync happened during VT. The network only has max 48kbps bandwidth to transfer encoded video frames. But at the start of the video encoding, the bitrate at the first second burst into 56kbps( then drop to 48kbps as we set). This will the delay the transfer of the video data for a about 0.5( or 2?) secs. As in the coming time, all the 48kbps bandwidth is still fully occupied, the delayed part can never find a chance to find time to do a make up. So the delay happend at the beginning will continue stably… turn no worse or better.

No comments:

Post a Comment

Post Code on Blogger

Simplest way to post code to blogger for me: <pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black;overflow-x:...