error reading in data from these two halo lidar files

[isabels]

Hi,

In testing out xradar on a variety of halo lidar files, I ran across two files that xradar cannot successfully read in. We aren't actively trying to use xradar to process these files in particular, so I would say this is a low priority, but I wanted to let you know in case you hadn't seen this error before.

Xradar version: 0.10.0
Python version: 3.13.5
OS version: RedHat 9.6

I've included the two files that produce errors, User1_20_20231003_140127.hpl and User1_257_20231008_140441.hpl, in a google drive folder here.

To replicate the error:

import xradar as xd
datatree = xd.io.open_hpl_datatree("User1_20_20231003_140127.hpl") # or the other file

The error produced:

Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    datatree = xd.io.open_hpl_datatree("User1_20_20231003_140127.hpl")
  File "/h/eol/isabels/.conda/envs/halo/lib/python3.13/site-packages/xradar/io/backends/hpl.py", line 655, in open_hpl_datatree
    "/": _get_required_root_dataset(ls_ds, optional=optional).rename(
         ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/h/eol/isabels/.conda/envs/halo/lib/python3.13/site-packages/xradar/io/backends/common.py", line 259, in _get_required_root_dataset
    root = _assign_root(ls)
  File "/h/eol/isabels/.conda/envs/halo/lib/python3.13/site-packages/xradar/io/backends/common.py", line 95, in _assign_root
    [[ts.time.values.min(), ts.time.values.max()] for ts in sweeps[1:]]
      ~~~~~~~~~~~~~~~~~~^^
  File "/h/eol/isabels/.conda/envs/halo/lib/python3.13/site-packages/numpy/_core/_methods.py", line 47, in _amin
    return umr_minimum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation minimum which has no identity

If I specify a sweep number to extract with the sweep keyword, I can get some data out of both these files. For file User1_20_20231003_140127.hpl, I can successfully create a datatree from sweep 0, but it fails for sweep 1 (with the same error as above). For file User1_257_20231008_140441.hpl, I can successfully create a datatree from any of the even numbered sweeps, but get the same error on all the odd sweeps.

If this is due to some kind of misconfiguration in the halo lidar/in the file format, feel free to ignore.

Thanks!
Isabel

[kmuehlbauer]

Thanks @isabels for filing this issue. From a first look at the headers of the provided files, they have additional or shifted information. Our reader currently does hard-coded reading, so that's most likely the issue here.

We should take this as an argument to check, if we can have a more clever and robust header reader. Deducing the different parts from the header information itself should be possible.

One question for you @isabels. I'm wondering if this LIDAR data would also be available in a binary or any other compressed format (like NetCDF4/HDF5)? Back in the days I was working with data in formats LAS and XYZ, which where binary.

[isabels]

Hi Kai,

I've checked with my colleague, and we do not have this lidar data available in another format. Sorry!