When run Super Resolution CoreML Model, App will Freeze

[kr1s0404]

Environment: Real device

When I performe Super Resolution, the app will freeze and UI won't update.
The loading spineer (in commit b97bac7) won't show up either

Then Xcode gives out the following information in consol.

[coreml] espresso_network_query_blob_shape returned rank != 4 for image output. Unknown how to handle an image with rank: 3, proceeding without output backing.

After few second, the app become normal and the image has be process successfully.

I'm not sure what cause this, might have to look into data raise or something else.

[kr1s0404]

ChatGPT 4 gives out the following answer:

The error message "returned rank != 4 for image output. Unknown how to handle an image with rank: 3, proceeding without output backing." indicates that the Core ML model is expecting an output of rank 4, but it is receiving an output of rank 3 instead. In the context of Core ML and neural networks, the "rank" refers to the number of dimensions in the tensor (or multi-dimensional array) that represents the image.

Here's what the ranks typically represent:

Rank 4: Usually, this represents [Batch size, Height, Width, Channels] or [Batch size, Channels, Height, Width].
Rank 3: This often represents [Height, Width, Channels], but it lacks the batch size dimension.
To resolve this issue, you need to ensure that the output from your model aligns with the expected rank. Since the error mentions a rank 3 output while expecting rank 4, it seems that the batch size dimension is missing.

Possible Solutions:
Check Model's Output Configuration: Review your Core ML model's configuration and output specifications. Ensure that your implementation matches the expected output format of the model.

Modify Output Handling: If your model is designed to output rank 3 tensors, you may need to modify how you handle the output in your code to add the batch dimension or adapt it accordingly.

Review Image Conversion: Ensure that the way you convert images to buffers and back is compatible with the model's expectations. Sometimes, discrepancies in image processing can lead to mismatches in tensor ranks.

Debug and Inspect: Utilize debugging tools to inspect the shapes of the tensors at various points in your code. This can help you identify where the mismatch occurs.

Consult Model Documentation: If you have access to the documentation or the person who trained the model, it might provide insights into the expected input and output formats.

Model Modification: If you have the capability and it's necessary, you might need to modify the model itself to ensure it produces the expected output rank.

Without more detailed information about the specific model and its expected input/output formats, these are general guidelines to investigate and resolve the issue.

But still can't find any solution.

[kr1s0404]

Just solved this.

Perform the super-resolution processing in a background thread and then update the UI on the main thread once the processing is complete. This can be achieved by using Swift's Grand Central Dispatch (GCD) to dispatch the heavy task to a background queue.

see detail: 363f0fd