🐛 After pauseRecording(), stopRecording() takes 4 seconds longer than if not paused

[Mikewa33]

What's happening?

I logged every appendBuffer call at all if cases that are commented. While the video is recording we get a ton of

2024-04-09 16:05:17.089435-0400 Hetal[11959:1948327] [native] VisionCamera.appendBuffer(_:clock:type:): 3: Last Written Timestamp Buffer Optional(__C.CMTime(value: 204422346222041, timescale: 1000000000, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0))

But our design calls for a pause function before saving. So we pause the video before calling stopRecording. When paused the buffer logs don't write cause the camera is paused. We have a button for the user to save and that calls stopRecording(). When that is called then these logs happen

2024-04-09 16:05:21.210329-0400 Hetal[11959:1948456] [native] VisionCamera.stop(clock:): Requesting stop at 204426.518807333 seconds for AssetWriter with status "writing"...
2024-04-09 16:05:25.410054-0400 Hetal[11959:1948327] [native] VisionCamera.stop(clock:): Waited 4.0 seconds but no late Frames came in, aborting capture...

On the react side the camera init looks like this

<Camera
       style={StyleSheet.absoluteFill}
       ref={cameraRef}
       device={device}
       isActive={true}
       format={format}
       video={true}
       fps={format.maxFps}
       enableZoomGesture={true}
     />

When the user pauses the recording we call
cameraRef.current.pauseRecording();

We then have buttons to save or retake the video. On save we call
cameraRef.current.stopRecording();

I think because it is in a pause state when we call stop no buffers are being called and that prevent finish() from happening as it normally would

Reproduceable Code

cameraRef.current.pauseRecording();
cameraRef.current.stopRecording();

Relevant log output

2024-04-09 16:05:17.089435-0400 Hetal[11959:1948327] [native] VisionCamera.appendBuffer(_:clock:type:): 3: Last Written Timestamp Buffer Optional(__C.CMTime(value: 204422346222041, timescale: 1000000000, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0))
2024-04-09 16:05:21.210329-0400 Hetal[11959:1948456] [native] VisionCamera.stop(clock:): Requesting stop at 204426.518807333 seconds for AssetWriter with status "writing"...
2024-04-09 16:05:25.410054-0400 Hetal[11959:1948327] [native] VisionCamera.stop(clock:): Waited 4.0 seconds but no late Frames came in, aborting capture...

Camera Device

{
  "hardwareLevel": "full",
  "hasTorch": true,
  "formats": [],
  "isMultiCam": false,
  "sensorOrientation": "landscape-right",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "minZoom": 1,
  "minExposure": -8,
  "hasFlash": true,
  "name": "Back Camera",
  "maxZoom": 123.75,
  "minFocusDistance": 10,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "neutralZoom": 1,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "position": "back"
}

Device

iPhone SE

VisionCamera Version

3.9.1

Can you reproduce this issue in the VisionCamera Example app?

No, I cannot reproduce the issue in the Example app

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[mrousavy]

yea, this is a bug.

[thanhtungkhtn]

Any update on this? facing similar issue

"react": "18.2.0",
"react-native": "0.74.1",
"react-native-vision-camera": "^4.0.3"

[mrousavy]

No update yet, if you want me to fix this for ya, contact me thru my agency or sponsor this issue.

[mrousavy]

Hey all!

I just spent a few days on thinking about a battleproof timestamp synchronization solution, and I came up with a great idea.
I built a TrackTimeline helper class which represents a video or audio track - it can be started & stopped, paused & resumed, and even supports nesting pauses without issues.

• The total duration of the video is summed up from the difference between the first and the last actually written timestamps, minus the total duration of all pauses between a video. No more incorrect video.duration! 🥳
• Whereas before I just had a 4 second timeout if no frames arrive, I now just wait twice the frame latency (a few milliseconds) to ensure no frames are left out at maximum! 🎉
• A video can be stopped while it is paused without any issues, as a pause call is taken into consideration before stopping 💪
• A video file's session now exactly starts at the start() timestamp, and ends at the exact timestamp of the last video frame - this ensures there can never be any blank frames in the video, even if the audio track is longer 🤩

This was really complex to built as I had to synchronize timestamps between capture sessions, and the entire thing is a producer model - a video buffer can come like a second or so later than the audio buffer, but I need to make sure the video track starts before the audio track starts, and ends after the audio track ends - that's a huge brainf*ck! 🤯😅

There's also no helper APIs for this on iOS, and it looks like no other Camera framework (not even native Swift/ObjC iOS Camera libraries) support this - they all break when timestamps have a delay (e.g. video stabilization enabled) (or dont even support delays at all) ; so I had to build the thing myself.

Check out this PR and try if it fixes the issue for you; #2948

Thanks! ❤️