fix(core): runtime inputs shouldn't be cached at task_hasher layer and filesets should be in the hash_plans layer#34971
Conversation
✅ Deploy Preview for nx-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for nx-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
View your CI Pipeline Execution ↗ for commit 73bed1f
☁️ Nx Cloud last updated this comment at |
|
Failed to publish a PR release of this pull request, triggered by @AgentEnder. |
…d filesets should be in the hash_plans layer
6b6a693 to
531c09c
Compare
…d filesets should be in the hash_plans layer [Self-Healing CI Rerun]
🐳 We have a release for that!This PR has a release associated with it. You can try it out using this command: npx create-nx-workspace@22.7.0-pr.34971.531c09c my-workspaceOr just copy this version and use it in your own command: 22.7.0-pr.34971.531c09c
To request a new release for this pull request, mention someone from the Nx team or the |
There was a problem hiding this comment.
Important
At least one additional CI pipeline execution has run since the conclusion below was written and it may no longer be applicable.
Nx Cloud has identified a possible root cause for your failed CI:
We investigated the failing tasks and determined they are unrelated to the PR's changes. The e2e-gradle tests fail due to the foojay.io Java toolchain provisioning API being unavailable (HTTP 400 / cache restore errors), which is an external service outage. The e2e-nx:workspace-legacy task has a known non-zero flakiness rate and its project is not affected by this PR.
No code changes were suggested for this issue.
Trigger a rerun:
🔔 Heads up, your workspace has pending recommendations ↗ to auto-apply fixes for similar failures.
🎓 Learn more about Self-Healing CI on nx.dev
🐳 We have a release for that!This PR has a release associated with it. You can try it out using this command: npx create-nx-workspace@22.7.0-pr.34971.73bed1f my-workspaceOr just copy this version and use it in your own command: 22.7.0-pr.34971.73bed1f
To request a new release for this pull request, mention someone from the Nx team or the |
…d filesets should be in the hash_plans layer (#34971) This pull request refactors the caching strategy in the task hashing logic to improve cache isolation and correctness. The main change is to eliminate shared, long-lived caches in favor of creating fresh caches for each invocation, preventing stale data from persisting across CLI commands. Additionally, new caches for project and workspace file set hashes are introduced, and cache handling is streamlined for better maintainability. **Cache management improvements:** * Removed the `runtime_cache` field from the `TaskHasher` struct and now create fresh `DashMap` caches for runtime, project file set, and workspace file set hashes within each invocation of the main hash function. This ensures no stale cache data persists across CLI commands. [[1]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bL153) [[2]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bL184) [[3]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bL196-R208) * Updated the `hash_runtime` function and its usage to accept a reference to a `DashMap` instead of an `Arc<DashMap>`, simplifying ownership and concurrency concerns. [[1]](diffhunk://#diff-77cbfff0572545027ac8ad107859d5135e6f78ee7c32babc56b2c9bbf89cfaccL5-R11) [[2]](diffhunk://#diff-77cbfff0572545027ac8ad107859d5135e6f78ee7c32babc56b2c9bbf89cfaccL54-R61) **File set hash caching:** * Introduced the `CachedFileSetHash` struct to store both the hash value and the list of matched files for project and workspace file set hashing, enabling more efficient input collection and cache lookups. * Added per-invocation caches for project and workspace file set hashes, with logic to check the cache before computing a new hash and to insert results after computation. [[1]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bL196-R208) [[2]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR353-R385) [[3]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR412-R438) [[4]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR554-R556) **Function signature and argument updates:** * Updated function signatures and argument passing to propagate the new cache references throughout the hashing logic, including `HashInstructionArgs` and related functions. [[1]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR264-R266) [[2]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR341-R343) [[3]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR554-R556) Fixes #30170 --------- Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com> (cherry picked from commit c96de0b)
|
This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request. |
This pull request refactors the caching strategy in the task hashing logic to improve cache isolation and correctness. The main change is to eliminate shared, long-lived caches in favor of creating fresh caches for each invocation, preventing stale data from persisting across CLI commands. Additionally, new caches for project and workspace file set hashes are introduced, and cache handling is streamlined for better maintainability.
Cache management improvements:
runtime_cachefield from theTaskHasherstruct and now create freshDashMapcaches for runtime, project file set, and workspace file set hashes within each invocation of the main hash function. This ensures no stale cache data persists across CLI commands. [1] [2] [3]hash_runtimefunction and its usage to accept a reference to aDashMapinstead of anArc<DashMap>, simplifying ownership and concurrency concerns. [1] [2]File set hash caching:
CachedFileSetHashstruct to store both the hash value and the list of matched files for project and workspace file set hashing, enabling more efficient input collection and cache lookups.Function signature and argument updates:
HashInstructionArgsand related functions. [1] [2] [3]Fixes #30170