Setting up Consul API GW to get JWT token with Azure AD as IdP

Avatar
Himanshu Sharma
  • Updated

Introduction

In modern micro-services architectures, secure communication and authentication are critical aspects to ensure that only authorized users and services can interact with each other.

When dealing with authentication, JWT (JSON Web Tokens) are commonly used to facilitate stateless and scalable authentication, enabling systems to verify the identity of a user or service by validating a token rather than making frequent calls to a centralized authentication service. Azure Active Directory (Azure AD) is a widely used identity provider (IdP) that supports OAuth2, OpenID Connect (OIDC), and other authentication protocols to issue JWTs.

API Gateways (e.g., Kong, Traefik, NGINX) sit between clients and micro-services, acting as a reverse proxy to route requests while enforcing security policies like authentication, rate limiting, and routing logic. In this setup, Consul API Gateway is used to integrate with identity providers like Azure AD to authenticate and authorize incoming requests via JWTProvider CRD.

Expected Outcome

Azure AD is integrated as the JWT provider for Consul and an API Gateway, allowing Consul to validate incoming JWT tokens issued by Azure AD and enable secure communication within the cluster.

The architecture involves:

  1. Azure AD issuing JWT tokens for authenticated users or services.

  2. The API Gateway validating incoming JWT tokens from clients before forwarding requests to backend services.

  3. Consul validating and using these JWT tokens for secure service-to-service communication, ensuring that only authorized services can interact.

To achieve this, we need to set up a JWT Provider Custom Resource Definition (CRD) in Kubernetes, configure Consul to trust Azure AD as an identity provider, and configure the API Gateway to authenticate requests using JWT tokens from Azure AD.

Prerequisites

  1. Basic Knowledge and Familiarity

  1. Ensure that Consul is installed and running in your environment with following minimum version constraint . You can either run it on-premises, in a cloud VM, or within a Kubernetes cluster.

  • The helm chart version: 1.5.3

  • server image version 1.19.2

  • Dataplane image version: 1.5.3

  • k8s-control-plane image version: 1.5.3

  1. Follow this guide to setup Consul API GW on K8s cluster and to control access to services inside the mesh.

Use Case

Some common and impactful use cases for setting up this integration:

  1. Centralized Authentication and Authorization for Microservice Architecture - In a microservice architecture, different services need to authenticate and authorize each other to ensure that only legitimate requests are processed. Using Azure AD as the identity provider allows you to centralize authentication, reducing the overhead of managing credentials within individual services.

  2. Federated Authentication Across Multiple Systems - When multiple systems or organizations need to collaborate, federated authentication allows users to authenticate using their existing identity provider (e.g., Azure AD) without requiring them to create new credentials for each system.

  3. Secure API Gateway Authentication with External Identity Providers - Many organizations prefer to integrate their API Gateway with an external identity provider (IdP) like Azure AD for authentication. This removes the need for the API Gateway to manage authentication and offloads security concerns to a trusted IdP.

  4. Multi-Tenant Architecture with Role-Based Access Control (RBAC) - In multi-tenant applications, each tenant requires access to its own set of resources, but with the same underlying microservices. Role-based access control (RBAC) ensures that users can only access resources that belong to their tenant or role.

Procedure

Setting up a Consul Cluster JWT provider with an API Gateway on Azure AD involves configuring both Consul and Azure AD to trust and communicate via JWT tokens for authentication and authorization. Here’s a step-by-step guide for this integration:

1. Configure Azure Active Directory (Azure AD)

First set up Azure AD as the Identity Provider (IdP) to issue JWT tokens that Consul can validate.

Step 1.1: Register an Application in Azure AD

  1. Go to Azure Portal and navigate to Azure Active Directory.

  2. Under App registrations, click on New registration.

  3. Provide a name for your app (e.g., JWT DEMO).

  4. Set the Supported account types based on your needs (e.g., "Accounts in this organizational directory only").

  5. Set the Redirect URI (optional at this stage) if needed, typically used for OAuth flow.

  6. After registration, note the Application (client) ID and Directory (tenant) ID.

 

image-20241216-071123.png

Screenshot 2024-12-16 at 12.36.48 PM.png

Step 1.2: Configure API Permissions

  1. Navigate to your registered app in Azure AD > API permissions.

  2. Click Add a permission, then select Microsoft Graph or Other API depending on your use case.

  3. Add the necessary permissions (e.g., openid, profile, email for OIDC).

  4. Grant Admin Consent for the added permissions.

 

image-20241216-071629.png

Screenshot 2024-12-16 at 12.45.07 PM.png

Step 1.3: Obtain the OpenID Connect (OIDC) metadata

  1. In Azure AD, you can find the OIDC metadata endpoint by navigating to the App Registration > Endpoints.

    • Copy the OpenID Connect metadata document URL (e.g., https://login.microsoftonline.com/{tenant-id}/v2.0/.well-known/openid-configuration).

This URL will be used by Consul to verify the JWT tokens issued by Azure AD.

Step 1.4: Create a Client Secret (if using OAuth2 flow)

If you're using OAuth2 flow to authenticate the API Gateway (i.e., for token issuance), you’ll need a client secret:

  1. In the Certificates & secrets section of your app in Azure AD, click New client secret.

  2. Note the value of the client secret, which you’ll need to configure in your API Gateway or Consul service.

Screenshot 2024-12-16 at 12.43.16 PM.png

image-20241216-071643.png

 

Postman Configuration

In Postman, we configure an OAuth 2.0 as the Authorization so we can get the Access Token.

  1. Go to the Authorization tab

  2. In the Type column select “OAuth 2.0”

  3. Set “Add authorization data” to “Request Headers”

  4. Set “Grant Type” to “Client Credentials”

  5. Set the Header Prefix value to “Bearer”

  6. Set the Access Token URL value to the “OAuth 2.0 token endpoint (v2)” URL value from Azure AD Application Endpoints.

  7. Set the Client ID value to the “Application (client) ID” value from the Azure AD App.

  8. Set the Client Secret value to the Client secret value of the Azure AD App.

  9. Set the Scope to the Application ID URI you copied from Azure but append /.default at the end (Example: “api://<READCT>/.default”

  10. Set Client Authentication to “Send client credentials in body”

image-20241216-073736.png

Screenshot 2024-12-16 at 1.05.40 PM.png

11) Click “Get New Access Token” and then “Proceed”
12) Copy the access token value and click “Use token”

Screenshot 2024-12-16 at 1.09.07 PM.png

image-20241216-073924.png

Decoding the Token: The user will need to decode the JWT token to get the Identity Provider URL. It will be done using Consul API GW. This guide demonstrates token decoding using JWT.IO

  1. Go to JWT.IO in your web browser and paste your token into the “Encoded” text box there.

  2. On the Decode side, look for "iss", and copy the URL to a notepad.

image-20241216-074242.png

Screenshot 2024-12-16 at 1.10.26 PM.png

2. Configure Consul for JWT Authentication

Now that Azure AD is set up to issue JWT tokens, the user needs to configure Consul to use these tokens for authenticating incoming requests.

Step 2.1: Enable the JWT Authentication Method

In the Consul cluster, the user needs to enable JWT authentication and configure it to trust Azure AD as the Identity Provider.

The JWTProvider CRD will configure Consul to trust JWT tokens issued by Azure AD. This is how Consul will validate incoming JWT tokens for authentication.

Then the user needs to define access control policies in Consul that will govern which users or services can access particular services based on the JWT claims (e.g., roles, user ID). The GatewayPolicy CRD allows you to specify rules for API Gateway behavior, such as which JWT claims should be used for authorization and what roles are required for access.

Ref.JWT provider configuration reference | Consul | HashiCorp Developer

GatewayPolicy configuration reference | Consul | HashiCorp Developer

--- apiVersion: consul.hashicorp.com/v1alpha1 kind: JWTProvider metadata: name: api-jwt-provider namespace: consul spec: issuer: "https://sts.windows.net/{tenant-id}/" jsonWebKeySet: remote: uri: "https://login.microsoftonline.com/{tenant-id}/discovery/v2.0/keys" jwksCluster: discoveryType: STRICT_DNS cacheDuration: 5m forwarding: headerName: user-token --- apiVersion: consul.hashicorp.com/v1alpha1 kind: GatewayPolicy metadata: name: api-gateway-policy namespace: consul spec: targetRef: name: api-gateway sectionName: http group: gateway.networking.k8s.io/v1beta1 namespace: consul kind: Gateway override: jwt: providers: - name: "api-jwt-provider" default: jwt: providers: - name: "api-jwt-provider" verifyClaims: - path: - appid value: <APP_ID> ---
% kubectl get jwtprovider -A NAMESPACE NAME AGE consul api-jwt-provider 8s % kubectl get gatewaypolicy -A NAMESPACE NAME SYNCED LAST SYNCED AGE consul api-gateway-policy 37s % kubectl get po -n consul NAME READY STATUS RESTARTS AGE api-gateway-64fb9f5d5b-248k8 1/1 Running 0 1d consul-connect-injector-d47445d8c-tp4t6 1/1 Running 0 1d consul-server-0 1/1 Running 2 (55d ago) 1d consul-terminating-gateway-74db79698-5d79c 1/1 Running 0 1d consul-webhook-cert-manager-687f69b9c5-49b2q 1/1 Running 0 1d % kubectl get gateways.gateway.networking.k8s.io -A NAMESPACE NAME CLASS ADDRESS PROGRAMMED AGE consul api-gateway consul a64fef46f6f9c4e1cb4dcf81d2bfea1b-846729335.eu-west-1.elb.amazonaws.com True 1d

 

With above setup, when the user tries to check consul-api-gateway pod logs it will show following messages with different endpoint IPs for the Azure AD.

2024-12-16T07:55:53.332Z+00:00 [debug] envoy.http(23) [Tags: "ConnectionId":"1896444","StreamId":"10323561514510472677"] encoding headers via codec (end_stream=false): ':status', '200' 'content-type', 'text/plain; charset=UTF-8' 'cache-control', 'no-cache, max-age=0' 'x-content-type-options', 'nosniff' 'date', 'Mon, 16 Dec 2024 07:55:53 GMT' 'server', 'envoy' 'x-envoy-upstream-service-time', '1' 'connection', 'close' 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.client(23) [Tags: "ConnectionId":"527138"] response complete 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.http(23) [Tags: "ConnectionId":"1896444","StreamId":"10323561514510472677"] Codec completed encoding stream. 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896444"] closing data_to_write=277 type=0 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896444"] setting delayed close timer with timeout 1000 ms 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.pool(23) [Tags: "ConnectionId":"527138"] response complete 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.pool(23) [Tags: "ConnectionId":"527138"] destroying stream: 0 remaining 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896444"] write flush complete 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896444"] closing socket: 1 2024-12-16T07:55:53.332Z+00:00 [debug] envoy.conn_handler(23) [Tags: "ConnectionId":"1896444"] adding to cleanup list 2024-12-16T07:55:55.715Z+00:00 [debug] envoy.dns(14) dns resolution for login.microsoftonline.com started 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.dns(14) dns resolution for login.microsoftonline.com completed with status 0 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.11:443 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.17:443 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.18:443 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.16:443 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 20.190.181.6:443 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.19:443 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 20.190.181.2:443 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.7:443 2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) DNS refresh rate reset for login.microsoftonline.com, refresh rate 5000 ms 2024-12-16T07:55:56.282Z+00:00 [debug] envoy.conn_handler(23) [Tags: "ConnectionId":"1896445"] new connection from 192.168.63.75:53898 2024-12-16T07:55:56.282Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896445"] remote close

 

Additional Information

https://support.hashicorp.com/hc/en-us/articles/14834216621971-Configuring-Azure-Active-Directory-AAD-with-Consul-using-OpenID-Connect-OIDC

https://support.hashicorp.com/hc/en-us/articles/23181468381843-Configuring-Azure-Active-Directory-AAD-with-Nomad-using-OpenID-Connect-OIDC

https://github.com/hashicorp/consul/blob/main/CHANGELOG.md#1192-august-26-2024
https://github.com/hashicorp/consul/pull/21604

https://github.com/hashicorp/consul-k8s/blob/v1.5.3/CHANGELOG.md#153-august-30-2024

 

Articles in this section

See more

Related articles