JPEG Encoder problem on Jetson Orin NX

[bwsw]

Version

0.6.0.32

Describe the bug.

Hello, dear developers.

Thank you for a solution that is very valuable for the Nvidia ecosystem.

I try to save a CuPy-allocated image with the Encoder. The same source code works just fine on dGPU.

Minimum reproducible example

from nvidia import nvimgcodec
import cupy as cp

codec = nvimgcodec.Encoder(backends=[nvimgcodec.Backend(nvimgcodec.BackendKind.HW_GPU_ONLY)])

encode_params = nvimgcodec.EncodeParams(
            quality_type=nvimgcodec.QualityType.QUALITY, quality_value=95
        )

cupy_image = cp.zeros((1280, 720, 3), dtype=cp.uint8)

codec.write(
   f'screenshot.jpg',
   nvimgcodec.as_image(cupy_image),
   'jpeg',
   params=encode_params
)

Environment details

JetPack 6.1
DeepStream 7.0 is working well with NVJPEG

Relevant log output

[INFO] [nvimgcodec] Registering extension nvimgcodec_builtin_parsers version:0.6.0
[INFO] [nvimgcodec] Framework is registering parser (id:bmp_parser codec:bmp)
[INFO] [nvimgcodec] Codec bmp not yet registered, registering for first time
[INFO] [nvimgcodec] Framework is registering parser (id:jpeg_parser codec:jpeg)
[INFO] [nvimgcodec] Codec jpeg not yet registered, registering for first time
[INFO] [nvimgcodec] Framework is registering parser (id:jpeg2k_parser codec:jpeg2k)
[INFO] [nvimgcodec] Codec jpeg2k not yet registered, registering for first time
[INFO] [nvimgcodec] Framework is registering parser (id:png_parser codec:png)
[INFO] [nvimgcodec] Codec png not yet registered, registering for first time
[INFO] [nvimgcodec] Framework is registering parser (id:pnm_parser codec:pnm)
[INFO] [nvimgcodec] Codec pnm not yet registered, registering for first time
[INFO] [nvimgcodec] Framework is registering parser (id:tiff_parser codec:tiff)
[INFO] [nvimgcodec] Codec tiff not yet registered, registering for first time
[INFO] [nvimgcodec] Framework is registering parser (id:webp_parser codec:webp)
[INFO] [nvimgcodec] Codec webp not yet registered, registering for first time
[INFO] [nvimgcodec] Loading extension module: /home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvtiff_ext.so.0.6.0
[INFO] [nvimgcodec] Registering extension nvtiff_extension version:0.6.0
[INFO] [nvimgcodec] Framework is registering decoder (id:nvtiff_cuda_decoder codec:tiff)
[INFO] [nvimgcodec] Framework is registering encoder (id:nvtiff_cuda_encoder codec:tiff)
[INFO] [nvimgcodec] Loading extension module: /home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libjpeg_turbo_ext.so.0.6.0
[INFO] [nvimgcodec] Registering extension libjpeg_turbo_extension version:0.6.0
[INFO] [nvimgcodec] Framework is registering decoder (id:libjpeg_turbo_decoder codec:jpeg)
[INFO] [nvimgcodec] Loading extension module: /home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvjpeg_ext.so.0.6.0
[INFO] [nvimgcodec] Registering extension nvjpeg_extension version:0.6.0
[INFO] [nvimgcodec] Framework is registering encoder (id:nvjpeg_cuda_encoder codec:jpeg)
[INFO] [nvjpeg-ext] HW decoder not supported by this platform. Skip.
[INFO] [nvimgcodec] Framework is registering decoder (id:nvjpeg_cuda_decoder codec:jpeg)
[INFO] [nvimgcodec] Framework is registering decoder (id:nvjpeg_lossless_decoder codec:jpeg)
[INFO] [nvimgcodec] Loading extension module: /home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvpnm_ext.so.0.6.0
[INFO] [nvimgcodec] Registering extension nvpnm_extension version:0.6.0
[INFO] [nvimgcodec] Framework is registering encoder (id:nvpnm_encoder codec:pnm)
[INFO] [nvimgcodec] Loading extension module: /home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libtiff_ext.so.0.6.0
[INFO] [nvimgcodec] Registering extension libtiff_extension version:0.6.0
[INFO] [nvimgcodec] Framework is registering decoder (id:libtiff_decoder codec:tiff)
[INFO] [nvimgcodec] Loading extension module: /home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libopencv_ext.so.0.6.0
[INFO] [nvimgcodec] Registering extension opencv_extension version:0.6.0
[INFO] [nvimgcodec] Framework is registering decoder (id:opencv_jpeg_decoder codec:jpeg)
[INFO] [nvimgcodec] Framework is registering decoder (id:opencv_jpeg2k_decoder codec:jpeg2k)
[INFO] [nvimgcodec] Framework is registering decoder (id:opencv_png_decoder codec:png)
[INFO] [nvimgcodec] Framework is registering decoder (id:opencv_bmp_decoder codec:bmp)
[INFO] [nvimgcodec] Framework is registering decoder (id:opencv_pnm_decoder codec:pnm)
[INFO] [nvimgcodec] Framework is registering decoder (id:opencv_tiff_decoder codec:tiff)
[INFO] [nvimgcodec] Framework is registering decoder (id:opencv_webp_decoder codec:webp)
[INFO] [nvimgcodec] Framework is registering encoder (id:opencv_jpeg_encoder codec:jpeg)
[INFO] [nvimgcodec] Framework is registering encoder (id:opencv_jpeg2k_encoder codec:jpeg2k)
[INFO] [nvimgcodec] Framework is registering encoder (id:opencv_png_encoder codec:png)
[INFO] [nvimgcodec] Framework is registering encoder (id:opencv_bmp_encoder codec:bmp)
[INFO] [nvimgcodec] Framework is registering encoder (id:opencv_pnm_encoder codec:pnm)
[INFO] [nvimgcodec] Framework is registering encoder (id:opencv_tiff_encoder codec:tiff)
[INFO] [nvimgcodec] Framework is registering encoder (id:opencv_webp_encoder codec:webp)
[INFO] [nvimgcodec] Loading extension module: /home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvbmp_ext.so.0.6.0
[INFO] [nvimgcodec] Registering extension nvbmp_extension version:0.6.0
[INFO] [nvimgcodec] Framework is registering encoder (id:nvbmp_encoder codec:bmp)
[INFO] [nvimgcodec] Framework is registering decoder (id:nvbmp_decoder codec:bmp)
[INFO] [nvimgcodec] Loading extension module: /home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvjpeg2k_ext.so.0.6.0
[INFO] [nvimgcodec] Registering extension nvjpeg2k_extension version:0.6.0
[INFO] [nvimgcodec] Framework is registering encoder (id:nvjpeg2k_encoder codec:jpeg2k)
[INFO] [nvimgcodec] Framework is registering decoder (id:nvjpeg2k_decoder codec:jpeg2k)
[INFO] [nvimgcodec] Requested num_threads=0, cpu_cores_count=8, selected num_threads=7
[INFO] [nvimgcodec] encode num_samples=1
[INFO] [nvimgcodec] Sample #0 can't be processed
[WARNING] [pynvimgcodec] Something went wrong during encoding image #0 there will be None on corresponding output position
[INFO] [nvimgcodec] Unregistering extension libjpeg_turbo_extension version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering decoder (id:libjpeg_turbo_decoder codec:jpeg)
[INFO] [nvimgcodec] Unloading extension module:/home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libjpeg_turbo_ext.so.0.6.0
[INFO] [nvimgcodec] Unregistering extension libtiff_extension version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering decoder (id:libtiff_decoder codec:tiff)
[INFO] [nvimgcodec] Unloading extension module:/home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libtiff_ext.so.0.6.0
[INFO] [nvimgcodec] Unregistering extension nvbmp_extension version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering encoder (id:nvbmp_encoder codec:bmp)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:nvbmp_decoder codec:bmp)
[INFO] [nvimgcodec] Unloading extension module:/home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvbmp_ext.so.0.6.0
[INFO] [nvimgcodec] Unregistering extension nvimgcodec_builtin_parsers version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering parser (id:bmp_parser codec:bmp)
[INFO] [nvimgcodec] Framework is unregistering parser (id:jpeg_parser codec:jpeg)
[INFO] [nvimgcodec] Framework is unregistering parser (id:jpeg2k_parser codec:jpeg2k)
[INFO] [nvimgcodec] Framework is unregistering parser (id:png_parser codec:png)
[INFO] [nvimgcodec] Framework is unregistering parser (id:pnm_parser codec:pnm)
[INFO] [nvimgcodec] Framework is unregistering parser (id:tiff_parser codec:tiff)
[INFO] [nvimgcodec] Framework is unregistering parser (id:webp_parser codec:webp)
[INFO] [nvimgcodec] Unregistering extension nvjpeg2k_extension version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering encoder (id:nvjpeg2k_encoder codec:jpeg2k)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:nvjpeg2k_decoder codec:jpeg2k)
[INFO] [nvimgcodec] Unloading extension module:/home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvjpeg2k_ext.so.0.6.0
[INFO] [nvimgcodec] Unregistering extension nvjpeg_extension version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering encoder (id:nvjpeg_cuda_encoder codec:jpeg)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:nvjpeg_cuda_decoder codec:jpeg)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:nvjpeg_lossless_decoder codec:jpeg)
[INFO] [nvimgcodec] Unloading extension module:/home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvjpeg_ext.so.0.6.0
[INFO] [nvimgcodec] Unregistering extension nvpnm_extension version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering encoder (id:nvpnm_encoder codec:pnm)
[INFO] [nvimgcodec] Unloading extension module:/home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvpnm_ext.so.0.6.0
[INFO] [nvimgcodec] Unregistering extension nvtiff_extension version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering decoder (id:nvtiff_cuda_decoder codec:tiff)
[INFO] [nvimgcodec] Framework is unregistering encoder (id:nvtiff_cuda_encoder codec:tiff)
[INFO] [nvimgcodec] Unloading extension module:/home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libnvtiff_ext.so.0.6.0
[INFO] [nvimgcodec] Unregistering extension opencv_extension version:0.6.0
[INFO] [nvimgcodec] Framework is unregistering decoder (id:opencv_jpeg_decoder codec:jpeg)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:opencv_jpeg2k_decoder codec:jpeg2k)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:opencv_png_decoder codec:png)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:opencv_bmp_decoder codec:bmp)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:opencv_pnm_decoder codec:pnm)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:opencv_tiff_decoder codec:tiff)
[INFO] [nvimgcodec] Framework is unregistering decoder (id:opencv_webp_decoder codec:webp)
[INFO] [nvimgcodec] Framework is unregistering encoder (id:opencv_jpeg_encoder codec:jpeg)
[INFO] [nvimgcodec] Framework is unregistering encoder (id:opencv_jpeg2k_encoder codec:jpeg2k)
[INFO] [nvimgcodec] Framework is unregistering encoder (id:opencv_png_encoder codec:png)
[INFO] [nvimgcodec] Framework is unregistering encoder (id:opencv_bmp_encoder codec:bmp)
[INFO] [nvimgcodec] Framework is unregistering encoder (id:opencv_pnm_encoder codec:pnm)
[INFO] [nvimgcodec] Framework is unregistering encoder (id:opencv_tiff_encoder codec:tiff)
[INFO] [nvimgcodec] Framework is unregistering encoder (id:opencv_webp_encoder codec:webp)
[INFO] [nvimgcodec] Unloading extension module:/home/administrator/.local/lib/python3.10/site-packages/nvidia/nvimgcodec/extensions/libopencv_ext.so.0.6.0

Other/Misc.

without specifying backend:

[ERROR] [nvjpeg_cuda_encoder] Could not create nvjpeg encoder - nvjpeg error #9 (implementation not supported) when running nvjpegCreateEx(NVJPEG_BACKEND_DEFAULT, nullptr, nullptr, 0, &handle_) at /home/jenkins/agent/workspace/nvimagecodec/helpers/release_v0.6.0/Release_12/build/extensions/nvjpeg/cuda_encoder.cpp:279

Check for duplicates

• I have searched the open bugs/issues and have found no duplicates for this bug report

[smatysik-nv]

Hi @bwsw,

I think the issue is that only nvimgcodec.BackendKind.HW_GPU_ONLY is selected, which is why the logs show [INFO] [nvjpeg-ext] HW decoder not supported by this platform. Skip. Unfortunately, even though the ORIN platform has a hardware JPEG decoder, it’s not accessible through the standard interface supported by NVJPEG. As a result, nvImageCodec currently cannot use it. (DeepStream, on the other hand, uses a Jetson-specific interface for the hardware JPEG decoder engine so it works) If you leave the default backends (just not define them), it should fall back to using the CUDA JPEG decoder."

[bwsw]

@smatysik-nv hi the problem is with the encoder, not the decoder. Please look at the code sample.

[smatysik-nv]

@bwsw I am sorry - I've been too focused on decoding lately, but with encoder is similar problem. Does it acctually work on a dGPU with nvimgcodec.BackendKind.HW_GPU_ONLY? - we do not have yet HW jpeg encoder in nvImageCodec so I am wondering if, due to a potential bug, it might be silently falling back to the CUDA encoder anyway.

[bwsw]

@smatysik-nv thank you for your reply.

With codec = nvimgcodec.Encoder(backends=[nvimgcodec.Backend(nvimgcodec.BackendKind.GPU_ONLY)]) it fails on Jetson as well (with the same log message):

[WARNING] [pynvimgcodec] Something went wrong during encoding image #0 there will be None on corresponding output position

On dGPU, I run like:

https://github.com/insight-platform/Savant/pull/1043/files#diff-f6d25d003715b6e08cd4a3b714e47d5b950a7f3023f72d7164fc603d6116a27bR20-R22

And it works just fine.

I wanted to clarify the differences between the two backends. So, on Jetson, we have NVJPEG ASIC, which we definitely want to use because it offloads CUDA cores, which are always busy with neural networks and other efficient computations.

I thought that HW_GPU_ONLY means that the library works on NVJPEG ASIC, not GPU (basically, it is what we as Jetson users desire). We have a custom C++ implementation that does the job, so we know it works (and DeepStream uses it as well), but we wanted to use a unified and less custom solution.

Thus, the question is whether the library currently supports the NVJPEG ASIC. According to my understanding, the default good priority sequence for 'jpeg' should be:

• NVJPEG ASIC
• CUDA
• CPU

[smatysik-nv]

@bwsw Thank you for sharing the code and the clarification.

In the code you provided, for dGPU (assuming the platform is not aarch64), it runs with the default backends, allowing nvImageCodec to select any supported backend. However, on aarch64, it explicitly forces the unsupported HW_GPU_ONLY backend:

        if not is_aarch64():
            self.codec = nvimgcodec.Encoder(device_id=0)
        else:
            self.codec = nvimgcodec.Encoder(
                backend_kinds=[nvimgcodec.BackendKind.HW_GPU_ONLY]
            )

To resolve this, you can either remove the if statement and let nvImageCodec automatically choose the appropriate backend, or explicitly set the backend to HYBRID_CPU_GPU (if you want to avoid falling back to a CPU-only encoder). The current CUDA implementation still relies partially on the CPU, and nvimgcodec.BackendKind.GPU_ONLY is not supported - it's reserved for future implementations that are fully (or nearly fully) GPU-based.

we definitely want to use because it offloads CUDA cores, which are always busy with neural networks and other efficient computations.
I thought that HW_GPU_ONLY means that the library works on NVJPEG ASIC,

You're absolutely right - that’s the intention. However, unfortunately, the nvjpeg library (which is part of CUDA) does not support the hardware JPEG engine on Orin. This can be confusing, as the hardware engine on Orin is sometimes also referred to as "nvjpeg," but it's not actually supported by the nvjpeg library.

On Orin, you can either use DeepStream, or implement a custom nvImageCodec plugin using the Orin-specific interface to access the hardware JPEG encoder. This is documented here:
Jetson Multimedia API - JPEG Encode

As far as I know, this approach is used by GStreamer as well:
DeepStream GStreamer Plugin - nvjpegdec

Apologies for the inconvenience — in the next release of nvImageCodec, this should be improved and unified for Thor.

[bwsw]

@smatysik-nv hi. Could you clarify if hardware encoding VIC/CUDA is implemented for Orin in the recent release or not? I see a note for Thor, but not Orin.

[hoang-dt]

Hi @bwsw, unfortunately nvImageCodec 0.7.0 does not have support for hardware JPEG decoding or encoding on Orin.