Introduction
For the use-case to leverage Vault Azure Auth method for application running inside kubernetes cluster, the vault-agent-init
container might give some login failure error.
This article is around the possible error you may lend into, and how we could overcome the same error to let the authentication succeed.
Problem
One reported GitHub Issue, where workload running inside AKS uses Azure Auth Method, which is configured on vault server (hosted on Azure VM), then workload container "vault-agent-init" throws following error.
Error
oidc: id token issued by a different provider, expected "https://sts.windows.net/TenantID/" got "https://login.microsoftonline.com/TenantId/v2.0"
Prerequisites (if applicable)
-
One vault cluster on Azure VM. I used 1.19.0+ent version.
-
Created one Azure AKS cluster with workload identity by using
--enable-oidc-issuer
and the--enable-workload-identity
parameters. -
Install helm chart on the AKS cluster by passing reference of external Vault server VM IP.
helm install vault hashicorp/vault \
--set "global.externalVaultAddr=http://<external-vault-ip>:8200"
-
Create and configure Managed Identity for workload on AKS. Ref.
Deploy and configure an AKS cluster with workload identity - Azure Kubernetes Service
-
Created Azure Auth method on the vault server, using official guide. Ref.
Azure - Auth Methods | Vault | HashiCorp Developer
Cause
With above setup in place, once we create a sample application by passing following annotations, then vault-agent-init
container throws following error for login failure.
Annotations
...
annotations:
vault.hashicorp.com/agent-inject: 'true'
vault.hashicorp.com/role: 'dev-role'
vault.hashicorp.com/agent-inject-secret-credentials.txt: 'secret/data/devwebapp/config'
vault.hashicorp.com/auth-path: 'auth/azure/'
...
Error
2025-03-26T09:01:12.295Z [ERROR] agent.auth.handler: error authenticating:
error=
| Error making API request.
|
| URL: PUT http://172.190.143.193:8200/v1/auth/azure/login
| Code: 500. Errors:
|
| * oidc: id token issued by a different provider, expected "https://sts.windows.net/ad651998-3955-403c-afec-ee5b5e063194/" got "https://eastus.oic.prod-aks.azure.com/ad651998-3955-403c-afec-ee5b5e063194/5924206b-13f3-4678-99fd-5bb7e4d1bc8e/"
backoff=4m26.77s
Overview of possible solutions (if applicable)
Solutions:
The error occured because vault-agent-init
container by default uses auth-type
as kubernetes
.
We need to explicitly define the azure auth method using annotation vault.hashicorp.com/auth-type: 'azure'
.
Also, we need to pass resource field which is mandatory, hereby we need to pass another annotation vault.hashicorp.com/auth-config-resource: "<https://management.azure.com/>"
.
Ref. https://developer.hashicorp.com/vault/docs/platform/k8s/injector/annotations#vault-hashicorp-com-auth-config
https://developer.hashicorp.com/vault/docs/agent-and-proxy/autoauth/methods/azure#resource
Annotations to use:-
...
annotations:
vault.hashicorp.com/auth-type: 'azure'
vault.hashicorp.com/auth-config-resource: "<https://management.azure.com/>"
...
Outcome
With above solution, vault-agent-init
container would be able to successfully login using azure auth method.
Additional Information
https://developer.hashicorp.com/vault/api-docs/auth/azure#login
https://learn.microsoft.com/en-us/azure/aks/use-managed-identity