Audio & Video Sync Issue

[nirmala-ncompass]

Hi @pedroSG94 My Custom Implementation is:
I am recording audio and video in file, mean while I am extending and modified AndroidMuxerRecordController class to get audio and video buffers. In my custom class I am receiving it in one method (onBuffervailable) Here based on my video frame count i am splitting chunks and writing in different file. Here I am resetting Buffer PTS value for each file. Kindly check below code. Audio and video is good in local video. But i my case when i am stetching those videos in mediaLive. The video is good but audio is playing bit delay. Its not in sync. Could you please help me How to set new PTS value for the buffers in one file with audio and video in sync?

`

   private static final int VIDEO_TIME_BASE = 1000 * 1000; 
   private static final int AUDIO_TIME_BASE = 1000 * 1000;
  private static final int VIDEO_FRAME_RATE = 30;
  private static final long VIDEO_FRAME_DURATION = VIDEO_TIME_BASE / VIDEO_FRAME_RATE;
  private static final int AUDIO_SAMPLE_RATE = 32000; // 32 kHz
  private static final int AUDIO_FRAME_SIZE = 1024; // AAC frame size
 private static final long AUDIO_FRAME_DURATION = (long) AUDIO_FRAME_SIZE * AUDIO_TIME_BASE / 
 AUDIO_SAMPLE_RATE;
 private long videoPtsCounter = 0;
 private long audioPtsCounter = 0;

     public void onBufferAvailable(ByteBuffer byteBuffer, MediaCodec.BufferInfo bufferInfo, boolean isVideo) {
   
      synchronized (this) {
        
      // Copy the ByteBuffer to prevent overwriting
        ByteBuffer bufferCopy = ByteBuffer.allocate(byteBuffer.remaining());
        bufferCopy.put(byteBuffer);
        bufferCopy.flip();

        long calculatedPts;

        if (isVideo) {
            // Calculate video PTS
            calculatedPts = videoPtsCounter * VIDEO_FRAME_DURATION;
            videoPtsCounter++;
        }
        else {
            // Calculate audio PTS
            calculatedPts = audioPtsCounter * AUDIO_FRAME_DURATION;
            audioPtsCounter++;
        }

        // Set the final adjusted PTS
        bufferInfo.presentationTimeUs = calculatedPts;
        
        // Create buffer data object
        BufferData bufferData = new BufferData(bufferCopy, new MediaCodec.BufferInfo(), isVideo);
        bufferData.bufferInfo.set(bufferInfo.offset, bufferInfo.size, bufferInfo.presentationTimeUs, bufferInfo.flags);

        // Track video frame count
        if (isVideo) {
            videoFrameCounter++;
        } else audioFrameCounter++;
        // Add buffer data to the buffer list
        if (useFirstArray) {
            bufferList1.add(bufferData);
        } else {
            bufferList2.add(bufferData);
        }
        // Process buffer if we hit the frame limit
        if (videoFrameCounter >= MAX_VIDEO_FRAMES) {
            processBufferInBackground(useFirstArray ? bufferList1 : bufferList2, false);
            if (useFirstArray) {
                bufferList1.clear();
            } else {
                bufferList2.clear();
            }
            audioPtsCounter = 0;
            videoPtsCounter = 0;
            audioFrameCounter = 0;
            videoFrameCounter = 0; // Reset frame counter
            useFirstArray = !useFirstArray; // Switch buffer list
        }
    }
}

`

[pedroSG94]

Hello,

Did you tried using the clock to create the pts instead of calculate it based in the config?
You can try it in both (video/audio) or only in one to test:

bufferInfo.presentationTimeUs = System.nanoTime() / 1000 - presentTimeUs;

Where presentTimeUs is (when you start encoder or receive the first frame):

System.nanoTime() / 1000