Introduction
When accessing Vault via a load balancer, a common issue that customer's encounter is the IP of the load balancer being logged as the remote address in audit logs as opposed to the actual client IP. This article will discuss how to instead have the client IP logged in audit logs. While this demo makes use of an AWS Application load balancer, the same steps can be broadly applied to any Layer 7 load balancer in any environment.
If you are instead using a Layer 4 load balancer (such as an AWS Network Load Balancer) please review our article on Proxy Protocol Support.
Expected Outcome
Client IP is associated with requests in audit logs instead of the load balancer IP.
Prerequisites (if applicable)
- Running and unsealed Vault cluster
- Audit device enabled
- Load balancer pointed at the Vault cluster
Procedure
-
First, verify the IP address of the client that is using to access Vault.
~ curl -s https://api.ipify.org
99.999.99.99
-
Then, retrieve the private IP address(es) of the load balancer (assuming the load balancer is communicating with Vault over its private network interface)
~ aws ec2 describe-network-interfaces | jq -r '.[] | .[] | select(.Association.IpOwnerId == "amazon-elb") | .PrivateIpAddress'
01.23.45.67
02.34.56.78
-
Next, lets verify the default behavior when making a request against Vault via a load balancer. We'll make a simple request to list available Vault auth methods and view the remote address that gets logged in the audit logs
# Make this request from the client
~ vault auth list
# View the request in audit logs (we use a file audit device for this example)
tail -1 audit.log | jq -r '.request .remote_address'
01.23.45.67
-
In the previous step, we see that the IP address of the load balancer is logged as the remote_address. We want this to instead be the client IP address found in step 1. To do this, lets update the listener stanza in Vault's server configuration file to include the following parameter:
listener "tcp" {
...
x_forwarded_for_authorized_addrs = ["01.23.45.67", "02.34.56.78"]
}
Here we've added the parameter x_forwarded_for_authorized_addrs to include the load balancer's IP addresses.
- Now that we've updated the configuration file, let's restart Vault to update the TCP listener configuration.
sudo systemctl restart vault
- Finally, after restarting and unsealing Vault, we can make the same request from the client and verify that the client's IP is logged as the remote_address
# Make this request from the client
~ vault auth list
# View the request in audit logs (we use a file audit device for this example)
tail -1 audit.log | jq -r '.request .remote_address'
99.999.99.99