IDEX l2b epoch job failure

[lacoak21]

Change Summary

Overview

I noticed that IDEX l2b job failed due to epoch not increasing monotonically. This was due to the input files not being sorted by date and also using np.unique() to determine the unique list of DOYs. Np.unique also sorts the array which I did not know.

File changes

I made sure to pass in the housekeeping and science datasets sorted by date as well as keeping those lists preserved throughout the code. I could have just sorted the arrays at the end by epoch but this helps with debugging to have everything sorted nicely.

[laspsandoval]

Looks good.

[Copilot]

Pull request overview

Fixes IDEX L2B processing failures caused by non-monotonic epoch ordering by ensuring inputs are processed in chronological order and by preserving day-of-year (DOY) ordering across year boundaries.

Changes:

• Preserve chronological DOY order (without np.unique re-sorting) when computing daily L2B/L2C products.
• Sort IDEX L2B science and housekeeping inputs by epoch in the CLI before calling idex_l2b.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
imap_processing/idex/idex_l2b.py	Replaces np.unique with order-preserving DOY de-duplication to avoid year-boundary mis-ordering.
imap_processing/cli.py	Sorts loaded IDEX science/housekeeping datasets by epoch before passing to idex_l2b.

Comments suppressed due to low confidence (1)

imap_processing/idex/idex_l2b.py:131

• epoch_doy is computed here, but compute_counts_by_charge_and_mass() recalculates epoch_to_doy(l2a_dataset["epoch"].data) inside the per-day loop, which repeatedly performs SPICE conversions and can be a significant performance cost. Consider passing the already-computed epoch_doy into compute_counts_by_charge_and_mass() (or computing it once inside that function) and using it to build current_day_indices.

    epoch_doy = epoch_to_doy(l2a_dataset["epoch"].data)
    # Use dict.fromkeys to preserve order while getting unique DOYs. We want to make
    # sure the order of DOYs stays the same in case we are dealing with data that
    # spans over the new year. E.g., we want 365 to come before 1 if we have data from
    # Dec and Jan.
    epoch_doy_unique = np.array(list(dict.fromkeys(epoch_doy)))
    (
        counts_by_charge,
        counts_by_mass,
        counts_by_charge_map,
        counts_by_mass_map,
        daily_epoch,
    ) = compute_counts_by_charge_and_mass(l2a_dataset, epoch_doy_unique)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[subagonsouth]

LGTM.