Skip to content

Prevent fetch from downloading unnecessary data and skipping data download for certain instruments#175

Merged
j-atkins merged 2 commits intomainfrom
fetch-fix-skipping
May 2, 2025
Merged

Prevent fetch from downloading unnecessary data and skipping data download for certain instruments#175
j-atkins merged 2 commits intomainfrom
fetch-fix-skipping

Conversation

@j-atkins
Copy link
Collaborator

@j-atkins j-atkins commented May 1, 2025

Closes #174

@j-atkins j-atkins requested a review from VeckoTheGecko May 1, 2025 13:58
Copy link
Collaborator

@VeckoTheGecko VeckoTheGecko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small change so that we aren't comparing against a repr :)


if "DRIFTER" in instruments_in_schedule:
print("Drifter data will be downloaded")
if "DRIFTER" in str(instruments_in_schedule):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering what the motivation for this is? Was it not working before?

Since instruments_in_schedule is a call to schedule.get_instruments() giving a set[InstrumentType], I think the right way to do this would be

Suggested change
if "DRIFTER" in str(instruments_in_schedule):
if InstrumentType.DRIFTER in instruments_in_schedule:

(also making sure InstrumentType is imported at the top)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, much more sensible. I've made these changes. I've added the InstrumentType import within the _fetch function to avoid circular import errors. Does that seem sensible?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that sounds good. I'm not sure why we're getting circular imports, might be worth me looking at the package structure.

But yes, a lot of code sometimes puts the import in the function body rather than the top of the file

@j-atkins j-atkins requested a review from VeckoTheGecko May 2, 2025 09:06
Copy link
Collaborator

@VeckoTheGecko VeckoTheGecko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@j-atkins j-atkins merged commit d910ff0 into main May 2, 2025
10 checks passed
@j-atkins j-atkins deleted the fetch-fix-skipping branch May 2, 2025 11:28
j-atkins added a commit that referenced this pull request May 9, 2025
…nload for certain instruments (#175)

* fixes for over-extensive data downloads and skipping certain instruments

* fixes based on PR feedback #175
j-atkins added a commit that referenced this pull request May 15, 2025
* update init command to use CTD_BGC as new instrument when making schedule

* update example ship_config file to also include CTD_BGC instrument config

* Prevent fetch from downloading unnecessary data and skipping data download for certain instruments (#175)

* fixes for over-extensive data downloads and skipping certain instruments

* fixes based on PR feedback #175

* Managing conflicts on branch. Move InstrumentType to ship_config.py

* Move Waypoint to schedule.py

* update init command to use CTD_BGC as new instrument when making schedule

* add configuration for CTD_BGC

* add bgc data download option for CTD_BGC instrument

* add CTD_BGC to instruments which prompts ship data download

* Review feedback on complete_download

---------

Co-authored-by: Vecko <36369090+VeckoTheGecko@users.noreply.github.com>
j-atkins added a commit that referenced this pull request May 15, 2025
* update init command to use CTD_BGC as new instrument when making schedule

* update example ship_config file to also include CTD_BGC instrument config

* Prevent fetch from downloading unnecessary data and skipping data download for certain instruments (#175)

* fixes for over-extensive data downloads and skipping certain instruments

* fixes based on PR feedback #175

* Managing conflicts on branch. Move InstrumentType to ship_config.py

* Move Waypoint to schedule.py

* update init command to use CTD_BGC as new instrument when making schedule

* add configuration for CTD_BGC

* add bgc data download option for CTD_BGC instrument

* add CTD_BGC to instruments which prompts ship data download

* Review feedback on complete_download

* bgc input data .nc files for testing

* add CTD_BGC to static/example schedule (also necessary for ship_config test)

* update test suite for new bgc sampling using ctd_bgc

* new ctd_bgc instrument

* add ctd_bgc sampling capability to virtualship run execution

---------

Co-authored-by: Vecko <36369090+VeckoTheGecko@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fetch skips downloading data for some instruments

2 participants