Problem
When using Kubernetes Authentication in Vault, the agent-init container throws permission denied error during authentication, and upon investigating the Vault Operational Log, the following error is logged:
cannot create resource \"tokenreviews\" in API group \"authentication.k8s.io\
Prerequisites
- Using Vault Kubernetes Authentication
- The error cannot create resource "tokenreviews" in API group appears in Vault Operational Log.
Cause
There are multiple causes which are discussed below. But the main cause is:
- Service Accounts used in this auth method will need to have access to the TokenReview API. If Kubernetes is configured to use RBAC roles, the Service Account should be granted permissions to access this API.
Alternatively, it could also be due to the following:
- The service account is configured in kubernetes initial namespaces such as kube-system. If that's the case, please create a new namespace or use the default namespace.
Solutions:
- Vault does provide an example ClusterRoleBinding that can be referenced.
-
Review the ClusterRoleBinding and double check to make sure the name and namespace matches with the service account being used. The commands kubectl get sa and kubectl get clusterrolebinding can help to examine the serviceaccounts and clusterrolebindings.
- You can get the jwt token from the following command (reference from Official Kube Docs). Please replace <REPLACE_ME> with the appropriate service account that's used on your clusterrolebinding.
kubectl get secret $(kubectl get serviceaccount <REPLACE_ME> -o jsonpath='{.secrets[0].name}') -o jsonpath='{ .data.token }' | base64 --decode
- The Kubernetes RBAC official documentation is a good article to understand more about how clusterrolebinding works.