How to decode optical flow GTs?

[lynnezixuan]

I tried to load optical flow GTs with the commands below:

with h5py.File("./Shift/discrete/images/train/left_stereo/flow.hdf5", "r") as hdf5: # load the HDF5 file im_bytes = bytearray(hdf5['00aa-d36e/00000000_flow_left_stereo.npz']) # select the file we want flow = np.load(BytesIO(im_bytes)) # same as opening an ordinary png file from IO stream.

The output of np.asarray(flow['flow'], dtype=np.float32).max() is 0.01171875. The value range is not correct. (I tried to wrap frame 1 to frame 0, the results are not correct obviously)

Could you share the correct way to decode this value? Thanks!

[suniique]

Hey, @lynnezixuan. Thanks for the question!

First, I would like to mention the definition of optical flow in Carla see here, where the optical flow values need to be multiplied by the image_size (1280 in SHIFT) to get the flow in the pixel unit.

Second, the flow data are captured with the videos (10fps), not images (1fps). So, you are recommended either

• to use the flow with video frames (i.e., discrete/videos/train/left_stereo/{img, flow}.hdf5), or
• further multiply the value by 10 if you want to wrap between the 0th and the 10th frames. Notice that this is only an approximation of the accumulated flow.

If you have any further questions, please don't hesitate to discuss!

[MiaoJieF]

Hey, @lynnezixuan. Thanks for the question!

First, I would like to mention the definition of optical flow in Carla see here, where the optical flow values need to be multiplied by the image_size (1280 in SHIFT) to get the flow in the pixel unit.

Second, the flow data are captured with the videos (10fps), not images (1fps). So, you are recommended either

• to use the flow with video frames (i.e., discrete/videos/train/left_stereo/{img, flow}.hdf5), or
• further multiply the value by 10 if you want to wrap between the 0th and the 10th frames. Notice that this is only an approximation of the accumulated flow.

If you have any further questions, please don't hesitate to discuss!

So the image here doesn't correspond to the optical flow?