Obtain HCP Vault Token from Gitlab CI/CD and Terraform Deployment

Scope

While performing Terraform deployments via GitLab-CI and HCP Vault, there are two situations. First, where secrets are needed; and second, where we need token.

In this article, we will learn more on how to "fetch" the vault token that is "in action" when using the Gitlab CI/CD pipeline to pass to the Terraform deployment. The primary use case is the Terraform Vault Provider uses a TOKEN to authenticate. This article describes how to fetch a vault token via GitLab CI/CD and Terraform using JWT (JSON Web Token) authentication method. 

Pre-Requisites

1. Ensure you have a HCP Cloud Vault Instance set up and accessible. Export the Vault Namespace in your script (refer script provided below).
2. Generate a JWT Token
3. GitLab CI/CD Configuration : Set up GitLab CI/CD pipeline to fetch the necessary credentials securely.
4. Terraform Configuration : Set up HCP Vault provider in your Terraform configuration.
   
   

Steps for Authentication of a GitLab CI/Cd job with Vault

Snippet for obtaining vault token :

image:
  name: hashicorp/terraform:light

variables:
  TF_VAR_vault_role: <Role-Name>
  VAULT_NAMESPACE: <Namespace-Name>
  VAULT_SERVER_URL: <URL of Vault Server>

vault_auth:
  stage: test
  script:
    - export VAULT_NAMESPACE=$VAULT_NAMESPACE
    - export VAULT_ADDR=$VAULT_SERVER_URL
    - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=$TF_VAR_vault_role jwt=$CI_JOB_JWT)"
    - vault token lookup

deploy:
  stage: deploy
  script:
    - terraform init
    - terraform plan -var "VAULT_TOKEN=${VAULT_TOKEN}" -out=tfplan

For vault_auth job, two script level commands are executed here : 

• The first allows authentication to HCP Vault via JWT method.
   - `role=$TF_VAR_vault_role` : This parameter specifies the role within Vault's JWT authentication method that the JWT token is being used for. 
   -  `jwt=$CI_JOB_JWT` : This parameter specifies the JWT token itself, obtained from the GitLab CI/CD environment variables (CI_JOB_JWT).
   - Finally, the HCP Vault token is retrieved and set to the VAULT_TOKEN environment variable. This will be used again by Terraform for the rest of its actions.
• The second line vault token lookup allows to check the duration of the HCP Vault token, the attached policies, etc.

After executing this script, the resulting token obtained from Vault's authentication is exported as the `VAULT_TOKEN` environment variable, which can then be used by subsequent commands or scripts to interact with HCP Vault on behalf of the authenticated users. This approach ensures secure authentication and authorization when interacting with HCP Vault from within the GitLab CI/CD pipeline.

`terraform plan -var "VAULT_TOKEN=${VAULT_TOKEN}" -out=tfplan` : In this command, terraform will use the `VAULT_TOKEN` environment variable as a variable in its execution plan, ensuring that it can authenticate with HCP Vault and access any secrets or resources required during the planning process.

Workflow (for reference)

To verify the authenticity of the JWT token information, HCP Vault instance will rely on the issuer's JSON Web Key Sets (JWKS) containing the sets of public keys used to sign the JWT. Once the token has been proven authentic, Vault instance will check the JWT's information and compare it with the expected information such as project ID, issuer, target branch, etc. Once the token information matches the Vault's expectations, Vault delivers its token with the appropriate policy to the CI, allowing the recovery of the secrets. This gives the following workflow : 

Reference Documents 

1. Securing terraform deployment via GitLab-CI and Vault
2. https://gitlab.com/-/snippets/2169718