Kinect1 in Docker not working

[siehlema]

My configuration is not working like expected. I want to use one Kinect1 and one Kinect2 in a multi_sensor Docker setting. Kinect2 runs like expected and performs object detection, person detection and pose estimation. But Kinect1 does not provide any data. The tracking output stays the same no matter if the kinect1 detection is running or not. So I think all the data is provided from the Kinect2.

I also followed the steps given in the Intrinsic Calibration and tried to run the roslaunch tracking detection_and_tracking.launch. RViz and the detection view open up, but the Detections View has a black screen, which doesn't change. In RViz I can see the RGB image (/camera/rgb/image_color) and the depth image (/camera/depth_registered/image). The topic /camera/depth/image doesn't provide any data.

I tried freenect and openni.

So I guess smth is wrong with my Kinect1 configuration. Or is that a Problem on the OpenPTrack side? I would really appreciate if anybody had a similar problem and knows how to solve it. It would also be very helpful to know, if this just doesn't work at the moment in the Docker container.

---

Here is the output of detection_and_tracking:
started roslaunch server http://127.0.0.1:41383/

SUMMARY

PARAMETERS

• /HaarDispAdaNode/HaarDispAdaPrior: 7.5
• /HaarDispAdaNode/Kinect_Disparity_Fix: False
• /HaarDispAdaNode/Q_Size: 20
• /HaarDispAdaNode/UseMissingDataMask: False
• /HaarDispAdaNode/classifier_file: /root/workspace/r...
• /HaarDispAdaNode/haar_disp_ada_min_confidence: 2.0
• /HaarDispAdaNode/mode: load
• /HaarDispAdaNode/num_Training_Samples: 350
• /HaarDispAdaNode/use_disparity: True
• /HaarDispAdaViewer/label: 1
• /HaarDispAdaViewer/mode: roi_display
• /HaarDispAdaViewer/show_confidence: True
• /camera/camera_nodelet_manager/num_worker_threads: 4
• /camera/depth_rectify_depth/interpolation: 0
• /camera/depth_registered_rectify_depth/interpolation: 0
• /camera/disparity_depth/max_range: 4.0
• /camera/disparity_depth/min_range: 0.5
• /camera/disparity_registered_hw/max_range: 4.0
• /camera/disparity_registered_hw/min_range: 0.5
• /camera/disparity_registered_sw/max_range: 4.0
• /camera/disparity_registered_sw/min_range: 0.5
• /camera/driver/depth_camera_info_url:
• /camera/driver/depth_frame_id: /camera_depth_opt...
• /camera/driver/depth_registration: True
• /camera/driver/device_id: Pose recognition has trouble with black clothes #1
• /camera/driver/rgb_camera_info_url:
• /camera/driver/rgb_frame_id: /camera_rgb_optic...
• /ground_based_people_detector/apply_denoising: False
• /ground_based_people_detector/background_resolution: 0.3
• /ground_based_people_detector/background_seconds: 3.0
• /ground_based_people_detector/background_subtraction: False
• /ground_based_people_detector/classifier_file: /root/workspace/r...
• /ground_based_people_detector/ground_based_people_detection_min_confidence: -1.75
• /ground_based_people_detector/ground_estimation_mode: 0
• /ground_based_people_detector/heads_minimum_distance: 0.3
• /ground_based_people_detector/lock_ground: False
• /ground_based_people_detector/max_distance: 5.0
• /ground_based_people_detector/maximum_person_height: 2.3
• /ground_based_people_detector/mean_k_denoising: 5
• /ground_based_people_detector/minimum_luminance: 120
• /ground_based_people_detector/minimum_person_height: 1.0
• /ground_based_people_detector/pointcloud_topic: /camera/depth_reg...
• /ground_based_people_detector/rate: 60.0
• /ground_based_people_detector/read_ground_from_file: False
• /ground_based_people_detector/sampling_factor: 4
• /ground_based_people_detector/sensor_tilt_compensation: True
• /ground_based_people_detector/std_dev_denoising: 0.3
• /ground_based_people_detector/use_rgb: True
• /ground_based_people_detector/valid_points_threshold: 0.3
• /ground_based_people_detector/voxel_size: 0.06
• /moving_average_filter_node/heartbeat_time: 1.0
• /moving_average_filter_node/max_history_size: 1000
• /moving_average_filter_node/position_lifetime: 0.6
• /moving_average_filter_node/publish_empty: True
• /moving_average_filter_node/rate: 30.0
• /moving_average_filter_node/track_lifetime_with_no_detections: 0.6
• /moving_average_filter_node/window_size: 5
• /ros2udp_converter/json/indent_size: 0
• /ros2udp_converter/json/newline: False
• /ros2udp_converter/json/spacing: False
• /ros2udp_converter/json/use_tabs: False
• /ros2udp_converter/udp/buffer_length: 2048
• /ros2udp_converter/udp/heartbeat_interval: 0.25
• /ros2udp_converter/udp/hostip: localhost
• /ros2udp_converter/udp/port: 21234
• /ros2udp_converter_object/json/indent_size: 0
• /ros2udp_converter_object/json/newline: False
• /ros2udp_converter_object/json/spacing: False
• /ros2udp_converter_object/json/use_tabs: False
• /ros2udp_converter_object/udp/buffer_length: 2048
• /ros2udp_converter_object/udp/heartbeat_interval: 0.25
• /ros2udp_converter_object/udp/hostip: localhost
• /ros2udp_converter_object/udp/port: 21234
• /ros2udp_converter_pose/json/indent_size: 0
• /ros2udp_converter_pose/json/newline: False
• /ros2udp_converter_pose/json/spacing: False
• /ros2udp_converter_pose/json/use_tabs: False
• /ros2udp_converter_pose/udp/buffer_length: 2048
• /ros2udp_converter_pose/udp/heartbeat_interval: 0.25
• /ros2udp_converter_pose/udp/hostip: localhost
• /ros2udp_converter_pose/udp/port: 21234
• /rosdistro: kinetic
• /rosversion: 1.12.14
• /tracker_node/HaarDispAdaPrior: 7.5
• /tracker_node/Kinect_Disparity_Fix: False
• /tracker_node/Q_Size: 20
• /tracker_node/UseMissingDataMask: False
• /tracker_node/acceleration_variance: 100
• /tracker_node/debug_active: True
• /tracker_node/detection_debug: True
• /tracker_node/detection_history_size: 500
• /tracker_node/detections_to_validate: 3
• /tracker_node/detector_likelihood: True
• /tracker_node/detector_weight: -0.25
• /tracker_node/extrinsic_calibration: False
• /tracker_node/gate_distance_probability: 0.999
• /tracker_node/haar_disp_ada_min_confidence: 2.0
• /tracker_node/history_pointcloud: True
• /tracker_node/history_size: 500
• /tracker_node/markers: True
• /tracker_node/min_confidence_initialization: 0.0
• /tracker_node/mode: load
• /tracker_node/motion_weight: 0.25
• /tracker_node/num_Training_Samples: 350
• /tracker_node/orientation/vertical: False
• /tracker_node/position_variance_weight: 30
• /tracker_node/rate: 30
• /tracker_node/sec_before_fake: 2.4
• /tracker_node/sec_before_old: 8
• /tracker_node/sec_remain_new: 1.2
• /tracker_node/tracking_results: True
• /tracker_node/use_disparity: True
• /tracker_node/velocity_in_motion_term: True
• /tracker_node/voxel_size: 0.06
• /tracker_node/world_frame_id: /world

NODES
/camera/
camera_nodelet_manager (nodelet/nodelet)
depth_metric (nodelet/nodelet)
depth_metric_rect (nodelet/nodelet)
depth_points (nodelet/nodelet)
depth_rectify_depth (nodelet/nodelet)
depth_registered_hw_metric_rect (nodelet/nodelet)
depth_registered_metric (nodelet/nodelet)
depth_registered_rectify_depth (nodelet/nodelet)
depth_registered_sw_metric_rect (nodelet/nodelet)
disparity_depth (nodelet/nodelet)
disparity_registered_hw (nodelet/nodelet)
disparity_registered_sw (nodelet/nodelet)
driver (nodelet/nodelet)
ir_rectify_ir (nodelet/nodelet)
points_xyzrgb_hw_registered (nodelet/nodelet)
points_xyzrgb_sw_registered (nodelet/nodelet)
register_depth_rgb (nodelet/nodelet)
rgb_debayer (nodelet/nodelet)
rgb_rectify_color (nodelet/nodelet)
rgb_rectify_mono (nodelet/nodelet)
/
HaarDispAdaNode (detection/HaarDispAda174)
HaarDispAdaViewer (opt_utils/roi_viewer)
camera_base_link (tf2_ros/static_transform_publisher)
camera_base_link1 (tf2_ros/static_transform_publisher)
camera_base_link2 (tf2_ros/static_transform_publisher)
camera_base_link3 (tf2_ros/static_transform_publisher)
ground_based_people_detector (detection/ground_based_people_detector)
moving_average_filter_node (tracking/moving_average_filter)
ros2udp_converter (opt_utils/ros2udp_converter)
ros2udp_converter_object (opt_utils/ros2udp_converter_objects)
ros2udp_converter_pose (opt_utils/ros2udp_converter_pose)
rviz (rviz/rviz)
tracker_node (tracking/tracker)

ROS_MASTER_URI=http://127.0.0.1:11311/

process[camera/camera_nodelet_manager-1]: started with pid [19505]
[ INFO] [1559726263.783153692]: Initializing nodelet with 4 worker threads.
process[camera/driver-2]: started with pid [19506]
[ INFO] [1559726264.324119672]: Number devices connected: 1
[ INFO] [1559726264.324169507]: 1. device on bus 001:14 is a SensorV2 (2ae) from PrimeSense (45e) with serial id 'A00363806896051A'
[ INFO] [1559726264.324747970]: Searching for device with index = 1
[ INFO] [1559726264.343852644]: Opened 'SensorV2' on bus 1:14 with serial number 'A00363806896051A'
[ INFO] [1559726264.391893779]: rgb_frame_id = '/camera_rgb_optical_frame'
[ INFO] [1559726264.391911843]: depth_frame_id = '/camera_depth_optical_frame'
[ WARN] [1559726264.394960969]: Camera calibration file /root/.ros/camera_info/depth_A00363806896051A.yaml not found.
[ WARN] [1559726264.394977511]: Using default parameters for IR camera calibration.
process[camera/rgb_debayer-3]: started with pid [19527]
process[camera/rgb_rectify_mono-4]: started with pid [19722]
process[camera/rgb_rectify_color-5]: started with pid [19892]
process[camera/ir_rectify_ir-6]: started with pid [19958]
process[camera/depth_rectify_depth-7]: started with pid [20024]
process[camera/depth_metric_rect-8]: started with pid [20090]
process[camera/depth_metric-9]: started with pid [20156]
process[camera/depth_points-10]: started with pid [20215]
process[camera/register_depth_rgb-11]: started with pid [20274]
process[camera/points_xyzrgb_sw_registered-12]: started with pid [20291]
process[camera/depth_registered_sw_metric_rect-13]: started with pid [20365]
process[camera/depth_registered_rectify_depth-14]: started with pid [20383]
process[camera/points_xyzrgb_hw_registered-15]: started with pid [20442]
process[camera/depth_registered_hw_metric_rect-16]: started with pid [20508]
process[camera/depth_registered_metric-17]: started with pid [20525]
process[camera/disparity_depth-18]: started with pid [20589]
process[camera/disparity_registered_sw-19]: started with pid [20643]
process[camera/disparity_registered_hw-20]: started with pid [20665]
process[camera_base_link-21]: started with pid [20687]
process[camera_base_link1-22]: started with pid [20708]
process[camera_base_link2-23]: started with pid [20719]
process[camera_base_link3-24]: started with pid [20730]
process[ground_based_people_detector-25]: started with pid [20741]
process[HaarDispAdaNode-26]: started with pid [20752]
process[tracker_node-27]: started with pid [20904]
process[moving_average_filter_node-28]: started with pid [20972]
process[ros2udp_converter_pose-29]: started with pid [21112]
process[ros2udp_converter-30]: started with pid [21166]
process[ros2udp_converter_object-31]: started with pid [21209]
process[HaarDispAdaViewer-32]: started with pid [21258]
process[rviz-33]: started with pid [21295]
Valid frame found!

Ground plane initialization starting...
Manual mode for ground plane estimation.
-0.431174 1.00289 2.733
0.314887 1.00413 2.669
-0.0814322 0.0881172 4.296
done.
Ground plane coefficients: -0.0412644, -0.866315, -0.497791, 2.21149.
Ground plane saved to /root/workspace/ros/src/open_ptrack/detection/conf/ground_camera_rgb_optical_frame.txt

[ERROR] [1559726283.388490039]: transform exception: "world" passed to lookupTransform argument target_frame does not exist. If you are seeing just one error like this do not worry, I am probably working!

---

If there is anything else needed to solve this issue, I'll provide you the information. Thanks in advance!

---

Supplement:
I found out, that the detection array (published on detector/detections from HaarDispAdaNode) is always empty. So it does not detect persons. The /ground_based_people_detector_kinect/detections and the other topics, which are used by the HaarDispAdaNode, on the other hand provide information. So could the problem lie in HaarDispAdaNode?

[siehlema]

Supplement2:
I found a solution by taking a closer look at how the kinect2 single_camera_tracking_node works. I created new launch files for the kinect1, where needed. Then I created a ROS workflow analogue to the one given for the kinect2. Key step here was to use rtpose_wrapper with the correct kinect1 topics (/kinect/rgb/image_color, /kinect/depth_registered/image_raw and /kinect/rgb/camera_info). Finally I was able to perform skeleton tracking with kinect1.

Next step is to add this functionality to the multi sensor solution to combine kinect1 & kinect2 skeleton tracking.

I can provide my solution if anyone faces a similar issue. Maybe it would make sense to add this functionality to the repo as well?

[jburkeucla]

Martin, Thanks! We’re going to have a few people working on imager support later in the summer and can roll this in. Please do a pull request or just post the approach here. From: Martin Siehler <notifications@github.com> Reply-To: OpenPTrack/open_ptrack_v2 <reply@reply.github.com> Date: Wednesday, June 12, 2019 at 4:44 AM To: OpenPTrack/open_ptrack_v2 <open_ptrack_v2@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: Re: [OpenPTrack/open_ptrack_v2] Kinect1 in Docker not working (#64) Supplement2: I found a solution by taking a closer look at how the kinect2 single_camera_tracking_node works. I created new launch files for the kinect1, where needed. Then I created a ROS workflow analogue to the one given for the kinect2. Key step here was to use rtpose_wrapper with the correct kinect1 topics (/kinect/rgb/image_color, /kinect/depth_registered/image_raw and /kinect/rgb/camera_info). Finally I was able to perform skeleton tracking with kinect1. Next step is to add this functionality to the multi sensor solution to combine kinect1 & kinect2 skeleton tracking. I can provide my solution if anyone faces a similar issue. Maybe it would make sense to add this functionality to the repo as well? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#64?email_source=notifications&email_token=AADFGR4WRNJIQE75W2XZFU3P2DOQXA5CNFSM4HTUUL22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXQEK7A#issuecomment-501237116>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AADFGRZRLGRM3FCGDWGCUALP2DOQXANCNFSM4HTUUL2Q>.

[jburkeucla]

Yes, we would be happy to include these in the repo. Please provide a pull request against the 1804 branch.

[siehlema]

The current 1804 branch doesn't support KinectV1 anymore. Should I provide a pull request against it anyway?