kbatch service account permissions

[iameskild]

After applying the helm chart for the kbatch-proxy, I ran into an issue where the kbatch service account was Forbidden from making the desired Kubernetes API calls.

Version info:

• kbatch - 0.3.1
• kbatch-proxy - 0.3.1 helm-chart

Logs from the kbatch-proxy pod after kbatch job list -o table:

HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"serviceaccounts is forbidden: User \"system:serviceaccount:dev:kbatch-kbatch-proxy\" cannot list resource \"serviceaccounts\" in API group \"\" in the namespace \"eskild\"","reason":"Forbidden","details":{"kind":"serviceaccounts"},"code":403}

I was able to get around this by creating a ClusterRole and ClusterRoleBinding for the aforementioned service account.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kbatch-clusterrole
rules:
- apiGroups: ["batch", ""]
  resources: ["*"]
  verbs: ["get", "watch", "list", "patch", "create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kbatch-clusterrolebinding
  namespace: dev
subjects:
- kind: ServiceAccount
  name: kbatch-kbatch-proxy
  namespace: dev
  apiGroup: ""
roleRef:
  kind: ClusterRole
  name: kbatch-clusterrole
  apiGroup: rbac.authorization.k8s.io

These permissions will likely need to be tightened or modified but that said, with these permissions I was able to submit and list kbatch jobs.

I guess I have two remaining questions:

• Is it possible that I missed something in the kbatch/kbatch-proxy setup that might have led me to this issue?
• If we believe this is indeed an issue, I'd be happy to submit a PR to the helm-chart repo to add the ClusterRole and ClusterRoleBinding. Is this something you are amenable to?

Thanks a lot for all your great work :)

[TomAugspurger]

FWIW, here are the roles I've discovered in microsoft/PlanetaryComputer#93 (comment) (so far):

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kbatch-role
rules:
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["create", "get", "list", "delete"]

- apiGroups: [""]
  resources: ["pods", "pods/log", "configmaps"]
  verbs: ["get", "watch", "list", "create", "delete", "patch"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kbatch
subjects:
- kind: User
  name: system:serviceaccount:prod:dhub-prod-kbatch-proxy
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: kbatch-role
  apiGroup: rbac.authorization.k8s.io

I'm applying these with kubectl apply -f kbatch-roles.yaml.

I haven't really used Kubernetes RBAC before, so I'm not sure how it's typically handled, but I'm guessing this is something the helm chart should do for you. The zero-to-jupyterhub-kubernetes helm chart probably does things properly. https://zero-to-jupyterhub.readthedocs.io/en/latest/resources/reference.html#rbac-create and https://zero-to-jupyterhub.readthedocs.io/en/latest/resources/reference.html#hub-serviceaccount seem relevant.

[yuvipanda]

Just ran into this too. I think this should go into the helm chart