Skip to content

Conversation

@lewisjared
Copy link
Contributor

@lewisjared lewisjared commented Jul 16, 2025

Description

Use a self-hosted runner to locally cache downloaded data

Checklist

Please confirm that this pull request has done the following:

  • Data registry up to date (regenerate if necessary with a comment on this PR of /regenerate)
  • Documentation added (where applicable)
  • Changelog item added to changelog/

@lewisjared
Copy link
Contributor Author

/regenerate

@lewisjared
Copy link
Contributor Author

/regenerate

@lewisjared
Copy link
Contributor Author

@nocollier @bouweandela @lee1043 This is a pretty big QoL improvement. I've moved the pipeline to a self-hosted node which caches the fetched data. The test time is down to a few minutes!

I've also fixed the pipeline so it can now process the obs4ref data. The fix was to ignore decimating files smaller than a given threshold (10 MB).

I'm merging this as is as I need to play with the regeneration workflow on main

@lewisjared lewisjared merged commit 7de5f94 into main Jul 17, 2025
0 of 3 checks passed
@bouweandela bouweandela deleted the use-self-hosted branch September 11, 2025 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants