Skip to content

fix(core): runtime inputs shouldn't be cached at task_hasher layer and filesets should be in the hash_plans layer#34971

Merged
FrozenPandaz merged 3 commits into
masterfrom
fix/tasks-hasher-cache
Mar 24, 2026
Merged

fix(core): runtime inputs shouldn't be cached at task_hasher layer and filesets should be in the hash_plans layer#34971
FrozenPandaz merged 3 commits into
masterfrom
fix/tasks-hasher-cache

Conversation

@AgentEnder
Copy link
Copy Markdown
Member

This pull request refactors the caching strategy in the task hashing logic to improve cache isolation and correctness. The main change is to eliminate shared, long-lived caches in favor of creating fresh caches for each invocation, preventing stale data from persisting across CLI commands. Additionally, new caches for project and workspace file set hashes are introduced, and cache handling is streamlined for better maintainability.

Cache management improvements:

  • Removed the runtime_cache field from the TaskHasher struct and now create fresh DashMap caches for runtime, project file set, and workspace file set hashes within each invocation of the main hash function. This ensures no stale cache data persists across CLI commands. [1] [2] [3]
  • Updated the hash_runtime function and its usage to accept a reference to a DashMap instead of an Arc<DashMap>, simplifying ownership and concurrency concerns. [1] [2]

File set hash caching:

  • Introduced the CachedFileSetHash struct to store both the hash value and the list of matched files for project and workspace file set hashing, enabling more efficient input collection and cache lookups.
  • Added per-invocation caches for project and workspace file set hashes, with logic to check the cache before computing a new hash and to insert results after computation. [1] [2] [3] [4]

Function signature and argument updates:

  • Updated function signatures and argument passing to propagate the new cache references throughout the hashing logic, including HashInstructionArgs and related functions. [1] [2] [3]

Fixes #30170

@AgentEnder AgentEnder requested a review from a team as a code owner March 23, 2026 22:01
@AgentEnder AgentEnder requested a review from leosvelperez March 23, 2026 22:01
@netlify
Copy link
Copy Markdown

netlify Bot commented Mar 23, 2026

Deploy Preview for nx-docs ready!

Name Link
🔨 Latest commit 73bed1f
🔍 Latest deploy log https://app.netlify.com/projects/nx-docs/deploys/69c2934f6aa2a900082693eb
😎 Deploy Preview https://deploy-preview-34971--nx-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify
Copy link
Copy Markdown

netlify Bot commented Mar 23, 2026

Deploy Preview for nx-dev ready!

Name Link
🔨 Latest commit 73bed1f
🔍 Latest deploy log https://app.netlify.com/projects/nx-dev/deploys/69c2934c6ad1e000088c6866
😎 Deploy Preview https://deploy-preview-34971--nx-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@nx-cloud
Copy link
Copy Markdown
Contributor

nx-cloud Bot commented Mar 23, 2026

View your CI Pipeline Execution ↗ for commit 73bed1f

Command Status Duration Result
nx affected --targets=lint,test,build,e2e,e2e-c... ✅ Succeeded 56m 47s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 8s View ↗
nx-cloud record -- pnpm nx conformance:check ✅ Succeeded 7s View ↗
nx build workspace-plugin ✅ Succeeded 3m 43s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 1s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded 1s View ↗

☁️ Nx Cloud last updated this comment at 2026-03-24 14:41:25 UTC

@github-actions
Copy link
Copy Markdown
Contributor

Failed to publish a PR release of this pull request, triggered by @AgentEnder.
See the failed workflow run at: https://github.com/nrwl/nx/actions/runs/23462352826

…d filesets should be in the hash_plans layer
@AgentEnder AgentEnder force-pushed the fix/tasks-hasher-cache branch from 6b6a693 to 531c09c Compare March 23, 2026 22:09
nx-cloud[bot]

This comment was marked as outdated.

…d filesets should be in the hash_plans layer [Self-Healing CI Rerun]
@github-actions
Copy link
Copy Markdown
Contributor

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx create-nx-workspace@22.7.0-pr.34971.531c09c my-workspace

Or just copy this version and use it in your own command:

22.7.0-pr.34971.531c09c
Release details 📑
Published version 22.7.0-pr.34971.531c09c
Triggered by @AgentEnder
Branch fix/tasks-hasher-cache
Commit 531c09c
Workflow run 23462352826

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

Copy link
Copy Markdown
Contributor

@nx-cloud nx-cloud Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

At least one additional CI pipeline execution has run since the conclusion below was written and it may no longer be applicable.

Nx Cloud has identified a possible root cause for your failed CI:

We investigated the failing tasks and determined they are unrelated to the PR's changes. The e2e-gradle tests fail due to the foojay.io Java toolchain provisioning API being unavailable (HTTP 400 / cache restore errors), which is an external service outage. The e2e-nx:workspace-legacy task has a known non-zero flakiness rate and its project is not affected by this PR.

No code changes were suggested for this issue.

Trigger a rerun:

Rerun CI

Nx Cloud View detailed reasoning on Nx Cloud ↗

🔔 Heads up, your workspace has pending recommendations ↗ to auto-apply fixes for similar failures.


🎓 Learn more about Self-Healing CI on nx.dev

@github-actions
Copy link
Copy Markdown
Contributor

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx create-nx-workspace@22.7.0-pr.34971.73bed1f my-workspace

Or just copy this version and use it in your own command:

22.7.0-pr.34971.73bed1f
Release details 📑
Published version 22.7.0-pr.34971.73bed1f
Triggered by @AgentEnder
Branch fix/tasks-hasher-cache
Commit 73bed1f
Workflow run 23462352826

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

@FrozenPandaz FrozenPandaz merged commit c96de0b into master Mar 24, 2026
23 checks passed
@FrozenPandaz FrozenPandaz deleted the fix/tasks-hasher-cache branch March 24, 2026 17:18
FrozenPandaz pushed a commit that referenced this pull request Mar 26, 2026
…d filesets should be in the hash_plans layer (#34971)

This pull request refactors the caching strategy in the task hashing
logic to improve cache isolation and correctness. The main change is to
eliminate shared, long-lived caches in favor of creating fresh caches
for each invocation, preventing stale data from persisting across CLI
commands. Additionally, new caches for project and workspace file set
hashes are introduced, and cache handling is streamlined for better
maintainability.

**Cache management improvements:**

* Removed the `runtime_cache` field from the `TaskHasher` struct and now
create fresh `DashMap` caches for runtime, project file set, and
workspace file set hashes within each invocation of the main hash
function. This ensures no stale cache data persists across CLI commands.
[[1]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bL153)
[[2]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bL184)
[[3]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bL196-R208)
* Updated the `hash_runtime` function and its usage to accept a
reference to a `DashMap` instead of an `Arc<DashMap>`, simplifying
ownership and concurrency concerns.
[[1]](diffhunk://#diff-77cbfff0572545027ac8ad107859d5135e6f78ee7c32babc56b2c9bbf89cfaccL5-R11)
[[2]](diffhunk://#diff-77cbfff0572545027ac8ad107859d5135e6f78ee7c32babc56b2c9bbf89cfaccL54-R61)

**File set hash caching:**

* Introduced the `CachedFileSetHash` struct to store both the hash value
and the list of matched files for project and workspace file set
hashing, enabling more efficient input collection and cache lookups.
* Added per-invocation caches for project and workspace file set hashes,
with logic to check the cache before computing a new hash and to insert
results after computation.
[[1]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bL196-R208)
[[2]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR353-R385)
[[3]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR412-R438)
[[4]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR554-R556)

**Function signature and argument updates:**

* Updated function signatures and argument passing to propagate the new
cache references throughout the hashing logic, including
`HashInstructionArgs` and related functions.
[[1]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR264-R266)
[[2]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR341-R343)
[[3]](diffhunk://#diff-d81ca7875513ef544822c24671104ce52cd09409a41a796768aba507d87b3c0bR554-R556)

Fixes #30170

---------

Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com>
(cherry picked from commit c96de0b)
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Mar 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runtime cache input simply does not work

2 participants