Prior to Consul version 1.9.0, when Consul's HTTP connections rate-limiting threshold is breached, they get silently reset without any kind of error message present in the API calls, nor a message in the Consul logs. This could lead to a potential misdirection of troubleshooting into investigating networking and security solutions on the path of the connections. This can present in the vault logs with messages similar to:
error="Get http://127.0.0.1:8500/v1/.../?keys=&separator=%2F: EOF"
Prerequisites (if applicable)
- A Consul cluster version 1.8.19 or older
- An amount of HTTP API calls to the cluster higher than the configured limit; or higher than the default limit of 200
Raise the connection limit's value (default 200). Below - JSON configuration example:
Consul documentation for HTTP rate limiting