Errors while trying to run nvblox with a Gemini335L

[nhathout]

So, i have been trying to get this example working with my Gemini 33L camera, and had to make quite a bit of edits to get everything looking correct in rqt node graph.

I remapped inputs to the nvblox and vslam nodes, as well as made sure my odometry_flattener package was being built. Everything looks fine now, but I noticed an issue when remapping the topic '/camera/depth/camera_info'. Whenever I correctly remap this topic, it breaks my program, ending in the error:

[component_container_mt-1] F0816 13:58:18.676508 11700 camera_cache.cpp:41] Check failed: camerasEqual(itr->second, camera)
[component_container_mt-1] *** Check failure stack trace: ***
[ERROR] [component_container_mt-1]: process has died [pid 11670, exit code -6, cmd '/opt/ros/humble/lib/rclcpp_components/component_container_mt --ros-args -r __node:=shared_nvblox_container'].

If I get rid of this remapping (which would render that topic an empty topic, or not published by the camera), the program runs but doesn't work as intended. It throws a bunch of messages in the terminal regarding to dropped frames and exceeding of frame thresholds.

Could I get some assistance with this? I will also insert my current rqt node graph (including the correct remapping for the above mentioned topic)

[jian-dong]

Hello, thank you for your feedback on this issue. I ran the code I pushed to GitHub yesterday, and I didn't notice that the topic mapping was incorrect. It should indeed map to /camera/depth/camera_info. I have already fixed it.

Please note that this is a test version, and our internal testing is also ongoing in parallel.

[nhathout]

Thanks for the reply.

I have a follow up issue though, I have connected all the topics to nodes correctly, and this now works for a colleague using a Gemini 335, but when I run the code I get a black screen for the color image in Rviz, as well as outputs in the terminal that look like the image pasted to this comment. Would you know any fix for this?

[jian-dong]

@nhathout Please try editing the file /sys/module/usbcore/parameters/usbfs_memory_mb using sudo vim and increase the value to 128. I believe this will resolve the issue.

[jian-dong]

Can you take a picture your Rviz2 UI

[nhathout]

Ok, let me try editing that file first and I will send the Rviz2 UI

[nhathout]

@jian-dong The issue is still not resolved. However, I did notice the vslam portion does work, meaning the camera knows when I move it, however I believe this issue arises due to the lack of color image.

when I run the orbbec camera alone using the package orbbec_camera and the launch file gemini_330_series.launch.py, the color image is able to show using rviz2, but when I run the whole orbbec/nvblox example, it doesnt seem to...

[jian-dong]

@nhathout I could run it on my computer, but the RGB image is black on your side. After carefully comparing the differences, the only discrepancy I noticed was the firmware version. Yours is 1.2.20, while my camera's firmware version is 1.3.25. Could you please upgrade your firmware and try again? Thank you!

[nhathout]

Yes, @jian-dong I fixed that last evening, which did fix the RGB image from showing, but I still get the same outputs in my terminal where the messages are dropped and delta between frames exceeds the threshold.. I will keep debugging.

[jian-dong]

Hi @nhathout

I don't know if the data lag or frame drops are caused by the large amount of data being transmitted by ROS2. Could you try adjusting the DDS settings according to this document ? I set these Fast DDS parameters on my computer and didn't experience any frame drops.

[nhathout]

Hi @jian-dong, that helped a lot thank you. Now I am not experiencing any frame drops.

By the way, I tried running this example completely ignoring the negotiated package and it worked perfectly fine, so I am pretty sure all of those steps are not necessary. I simply made sure all the required repositories are properly cloned, then made sure the relevant dependencies are all installed.

An important note is that I still have to comment out line 41 of the file /src/isaac_ros_nvblox/nvblox_ros/src/lib/camera_cache.cpp or else the program crashes at that line. Not sure why this happens but the example will crash at that line if not commented out.

After doing this I simply rebuilt all ros2 packages by running colcon build, then ran source install/setup.bash and then ros2 launch isaac_orbbec_launch orbbec_dynamics_example.launch.py and everything worked well without frame drops.

[jian-dong]

Hi @nhathout , I'm glad to hear everything is running now! Since I'm not familiar with NVBlox's internal logic, I need more time to figure out why that specific line is causing the crash. I appreciate your patience!

[nhathout]

No worries at all :)

[jian-dong]

Hi @nhathout, my colleague and I discussed it, and we believe that the line of code is checking whether the camera info of the depth and color cameras are the same. Could you please confirm if the depth_registration is set to true? Also, could you run the following commands to check if the camera info for the color and depth cameras are the same?

1. ros2 topic echo /camera/color/camera_info
2. ros2 topic echo /camera/depth/camera_info

Thanks!