2750 update create goodtimes dataset to use l1b de as input dataset

[subagonsouth]

Change Summary

Overview

Following on from changes to the Hi L1B DE. I added some variables to make the Hi Goodtimes dependencies more straightforward.

File changes

Changes to hi_goodtimes.py:

1. create_goodtimes_dataset:
   - Parameter: l1a_de → l1b_de
   - MET extraction: meta_seconds + meta_subseconds/1000 → esa_step_met
   - ESA step: esa_step → esa_energy_step
   - Updated docstrings
2. mark_incomplete_spin_sets:
   - Parameter: l1a_de → l1b_de
   - MET extraction: meta_seconds + meta_subseconds/1000 → esa_step_met
   - Spin invalid check: spin_invalids != 0 → (ccsds_qf & SPIN_INVALID_QF_MASK) != 0
   - Added constant: SPIN_INVALID_QF_MASK = 0x02 (bit 1)

Changes to test_hi_goodtimes.py:

1. New helper function: _create_l1b_de_dataset() - reduces duplication across test fixtures
2. Updated fixtures:
   - mock_l1b_de: Uses esa_step_met and esa_energy_step
   - All l1a_de_* fixtures renamed to l1b_de_* with L1B fields
   - ccsds_qf replaces spin_invalids (bit 1 = 0x02 for spin invalid)
3. Renamed test class: TestGoodtimesFromL1aDe → TestGoodtimesFromL1bDe

Closes: #2750

[Copilot]

Pull request overview

This pull request updates the Hi Goodtimes processing to use L1B Direct Event (DE) data instead of L1A DE data as input. The change is motivated by new variables added to L1B DE that simplify dependencies. The PR introduces two new variables to L1B DE (esa_step_met and ccsds_qf), updates the goodtimes creation and validation functions to use these new L1B fields, and comprehensively updates the test suite.

Changes:

• Added de_esa_step_met() and de_ccsds_qf() functions to compute new L1B DE variables
• Updated create_goodtimes_dataset() and mark_incomplete_spin_sets() to accept L1B DE instead of L1A DE
• Replaced direct MET computation with esa_step_met variable access
• Replaced spin_invalids checks with ccsds_qf bitmask checks
• Added comprehensive test coverage for new functions

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
imap_processing/hi/hi_l1b.py	Added de_esa_step_met() and de_ccsds_qf() functions to compute new L1B variables; integrated them into annotate_direct_events() pipeline
imap_processing/hi/hi_goodtimes.py	Updated create_goodtimes_dataset() and mark_incomplete_spin_sets() to use L1B DE with new variable names and quality flag checks
imap_processing/quality_flags.py	Added ImapHiL1bDeFlags class defining PACKET_FULL and SPIN_INVALID flags
imap_processing/tests/hi/test_hi_l1b.py	Added comprehensive tests for de_esa_step_met() and de_ccsds_qf() functions; updated expected data variable counts
imap_processing/tests/hi/test_hi_goodtimes.py	Added _create_l1b_de_dataset() helper; updated all fixtures and tests to use L1B DE structure with new variable names
imap_processing/cdf/config/imap_hi_variable_attrs.yaml	Added CDF attributes for esa_step_met and ccsds_qf variables

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[laspsandoval]

Nice documentation. The only question that I have is why was it marked l1a to begin with?

[subagonsouth]

The only question that I have is why was it marked l1a to begin with?

When this code was first implemented, the L1B DE did not have the esa_step_met variable carried through from L1A. Those times are needed to generate the Goodtimes so I was planning to just use L1A and L1B. But then some other things shifted so I decided to clean up the the upstream dependencies by carrying that variable through to L1B.