Skip to content

HDF5 Dask DataFrame issue: File ID#960

Closed
ax3l wants to merge 2 commits intoopenPMD:devfrom
ax3l:fix-hdf5DaskDataFrame
Closed

HDF5 Dask DataFrame issue: File ID#960
ax3l wants to merge 2 commits intoopenPMD:devfrom
ax3l:fix-hdf5DaskDataFrame

Conversation

@ax3l
Copy link
Member

@ax3l ax3l commented Apr 8, 2021

When removing those lines - and only with HDF5 - I get the following error on electrons.to_dask():

openpmd_api/DaskDataFrame.py in particles_to_daskdataframe(particle_species)
     65         for k_rc, rc in r.items():
     66             if not rc.constant:
---> 67                 chunks = rc.available_chunks()
     68                 break
     69         if chunks:

RuntimeError: [HDF5] File ID not found with file name

Seen with the 3D openPMD HDF5 example data set, at iteration 400.

Printing the records and components in the loop shows:

  • charge Scalar (skipped because constant)
  • mass Scalar (skipped because constant)
  • momentum x (calls available_chunks)

The patch from #959 is applied.

When removing those lines - and only with HDF5 - I get the following
error on `electrons.to_dask()`:
```
openpmd_api/DaskDataFrame.py in particles_to_daskdataframe(particle_species)
     65         for k_rc, rc in r.items():
     66             if not rc.constant:
---> 67                 chunks = rc.available_chunks()
     68                 break
     69         if chunks:

RuntimeError: [HDF5] File ID not found with file name
```

Seen with the 3D openPMD HDF5 example data set, at iteration 400.
Printing the records and components in the loop shows:
- charge Scalar (skipped because constant)
- mass Scalar (skipped because constant)
- momentum x (calls available_chunks)
@ax3l ax3l force-pushed the fix-hdf5DaskDataFrame branch from f2dea43 to 7fc3676 Compare April 8, 2021 03:07
@lgtm-com
Copy link
Contributor

lgtm-com bot commented Apr 8, 2021

This pull request introduces 2 alerts when merging 7fc3676 into 6fd3936 - view on LGTM.com

new alerts:

  • 2 for Unused import

@lgtm-com
Copy link
Contributor

lgtm-com bot commented Apr 8, 2021

This pull request introduces 2 alerts when merging 37e08fb into 6fd3936 - view on LGTM.com

new alerts:

  • 2 for Unused import

@ax3l
Copy link
Member Author

ax3l commented Apr 8, 2021

demonstrator moved to #961

@ax3l ax3l closed this Apr 8, 2021
@ax3l ax3l deleted the fix-hdf5DaskDataFrame branch April 8, 2021 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant