🐛 V3 runAsync doesn't work

[manolo-battista]

What's happening?

On docs is described:

But I think it needs update, cause request frame as first parameter and func (callback) as second parameter.

The callback function is never called. I saw that code reach the func on runOnAsyncContext function, but I think enter always on finally cause I can't see my log ok 2.

Reproduceable Code

No response

Relevant log output

No response

Camera Device

No response

Device

iPhone 14 Pro

VisionCamera Version

3.0.0

Can you reproduce this issue in the VisionCamera Example app?

• I can reproduce the issue in the VisionCamera Example app.

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[abdelrahman-muntaser]

@manolo-battista did you find any solution for this ?

[manolo-battista]

@manolo-battista did you find any solution for this ?

no, unfortunately I still didn't find any solution. For my projects at the moment I'm using runAtTargetFps to run lower FPS rate.

const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log("I'm running synchronously at 60 FPS!")
  runAtTargetFps(2, () => {
    'worklet'
    console.log("I'm running synchronously at 2 FPS!")
  })
}, [])

[mrousavy]

Hi. Would've been great if you could've sent a PR for that to fix it in docs.

But in my tests, runAsync works - can you put a console.log in there to double check that it does not get called?

Can you reproduce it in the example app?

[rodgomesc]

hey @mrousavy i think we should keep this issue open, i'm facing the same issue since this function was released, can you provide a minimal example that's working for you?

[rodgomesc]

some weirdness scope things happening

UseCase1:

logcat throws JSI rethrowing JS exception: Regular javascript function cannot be shared. Try decorating the function with the 'worklet' keyword to allow the javascript function to be used as a worklet."

  const frameProcessor = useFrameProcessor(
    (frame) => {
      "worklet";
      runAsync(frame, () => {
        "worklet";
        console.log("my awesome log");
      });
    },
    []
  );
  

UseCase2:

do not throws anything onlogcat but frameContext is always undefined, the callback is not being triggered for some reason

  const myFn = (frameContext) => {
    "worklet";
    console.log("Here", frameContext);
  };

  const frameProcessor = useFrameProcessor(
    (frame) => {
      "worklet";
      runAsync(frame, myFn);
    },
    [myFn]
  );
  

[mrousavy]

Oh! Hm that's weird, I remember this worked when I built it.

@chrfalch do you maybe have any insights?

[rodgomesc]

oh wait

it seems that by design the frame it's not being passed to myFn as a callback prop, this forces us to uses arrow functions to hook the frame scope, also it seems that's no guarantees that internal.decrementRefCount() is being called after my long running function finishes the execution?, which causes a java.lang.IllegalStateException: Image is already closed,

react-native-vision-camera/package/src/FrameProcessorPlugins.ts

Lines 50 to 55 in 764897d

	runOnAsyncContext = Worklets.createRunInContextFn((frame: Frame, func: () => void) => {
	'worklet'
	try {
	// Call long-running function
	func()
	} finally {

[mrousavy]

Wait what? not sure if I follow -

it seems that by design the frame it's not being passed to myFn as a callback prop

no you're right, you can just use the Frame inside your lambda directly, the lambda has no parameters.

there's no guarantees that internal.decrementRefCount() is being called after my long running function finishes the execution

Why not? It runs in finally, no?

[rodgomesc]

@mrousavy yep you are totally right, i was very drunk that day sorry

found a pattern that doesn't make sense for me, example app works my app doesn't work, tried different combination of libs and now i'm using same version that's on example but it always end up with

[rodgomesc]

i have a serie of frame processors at this moment running in sequence

something like this

const frameProcessor = (frame) => {
    'worklet';

     // processors
     const faces = faceDetectorProcessor(frame)
     const luminanceResults = luminanceDetectionProcessor(frame, faces)
     const spoofResults = antiSpoofingProcessor(frame);


    // callbacks
    handleSpoofingDetection(spoofResults)
    handleLuminanceDetection(luminanceResults)

},[handleSpoofingDetection, handleLuminanceDetection])

the problem is that at this point the frame are being deallocated before i finish my operations,
so i'm thinking to implement something like this, to drop some frames while my processors still running

const frameProcessor = (frame) => {
    'worklet';

    // just drop the frame
    if(!jsiUtil.finishedOperations) return;  

    const frameCopy = jsiUtil.copyFrame(frame);

     // processors      
     const faces = faceDetectorProcessor(frameCopy)
     const luminanceResults = luminanceDetectionProcessor(frameCopy, faces)
     const spoofResults = antiSpoofingProcessor(frameCopy);

    // cleanup frame copy
    jsiUtil.ReleaseFrameCopy()    

    // callbacks
    handleSpoofingDetection(spoofResults)
    handleLuminanceDetection(luminanceResults)

},[handleSpoofingDetection, handleLuminanceDetection])

would be nice to hear your opinion on this temporary solution to make sure i'm not missing anything

some final notes:

i'm linking my lib with visionCamera.so and unwraping the frame, i've tried increment refcount before my operations and decrement on the last one, but it seems to be ignored need further investigation

[mrousavy]

Woah, that's a weird error. Can you add console.log statements to your code (begin and end of sync calls, and begin and end of async calls) and then also to the native Frame's release method?

This is weird and you shouldn't need such workarounds - it should just work magically with ref counting.

[rodgomesc]

i still couldn't figure out what's happening, i'll try to upload something reproducible