acl: fixes a bug where client RPC's fail for node pool "all"

[mismithhisler]

Description

These changes add ACL code to handle client RPC's where the requested entitity is in node_pool = "all". Without this, multiple client RPC endpoints will fail whenever a job is created in "all" nodepools.

In a followup PR, we should be using the already defined node.IsInPool, but it's going to take a small/medium refactor to get working nicely. The included E2E test should catch any future regressions.

Testing & Reproduction steps

Links

Fixes: #27961

Contributor Checklist

• Changelog Entry If this PR changes user-facing behavior, please generate and add a
  changelog entry using the make cl command.
• Testing Please add tests to cover any new functionality or to demonstrate bug fixes and
  ensure regressions will be caught.
• Documentation If the change impacts user-facing functionality such as the CLI, API, UI,
  and job configuration, please update the Nomad product documentation, which is stored in the
  web-unified-docs repo. Refer to the web-unified-docs contributor guide for docs guidelines.
  Please also consider whether the change requires notes within the upgrade
  guide. If you would like help with the docs, tag the nomad-docs team in this PR.

Reviewer Checklist

• Backport Labels Please add the correct backport labels as described by the internal
  backporting document.
• Commit Type Ensure the correct merge method is selected which should be "squash and merge"
  in the majority of situations. The main exceptions are long-lived feature branches or merges where
  history should be preserved.
• Enterprise PRs If this is an enterprise only PR, please add any required changelog entry
  within the public repository.

• If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

[SavelevArtemD]

Builded test binary from this branch, and deployed on test cluster.
MY scenario:

• Client and Server running on 2.0.1. Job with node_pool: all stacked in pending state, client spams with RPC errors for rpc=Alloc.GetAllocs.
• Server updated to 2.0.2-dev - binary I builded from this branch.
• Node_pool_all job started execution, RPC errors disappeared.
• Node_pool_all job stopped fully.
• Started new Node_pool_all job, and it started succesfully. But in client nomad.service logs I found client.rpc: error performing RPC to server: error="rpc error: Permission denied" rpc=Job.Register server=x.x.x.x. Looks like it doesn't affect currently running job, but not 100% sure.

Job I'm using for test:

job "test-node-pool-all-long" {
  region      = "global"
  datacenters = ["nomad"]
  node_pool   = "all"
  type        = "service"

  group "test" {
    count = 1

    task "hello" {
      driver = "docker"

      config {
        image   = "alpine:3.20"
        command = "sh"
        args = [
          "-c",
          "i=0; while true; do echo \"hello-from-node-pool-all-long iteration=$i host=$(hostname) time=$(date -Iseconds)\"; i=$((i+1)); sleep 10; done"
        ]
      }

      resources {
        cpu    = 100
        memory = 64
      }
    }
  }
}

[mismithhisler]

@SavelevArtemD I am not able to reproduce this log in my test cluster. Do you have acl's enabled? The job registration RPC shouldn't be affected by this bug as we don't call the client ACL check method during job registration.

Are there any other error or debug logs to indicate what is happening here? If you register a job with an invalid token and get Permission Denied, the job should under no circumstances deploy, so I am definitely interested in figuring this out.

[tgross]

LGTM!

[tgross]

LGTM!

[pkazmierczak]

👍