Problem
During a system patch update on a Terraform Enterprise server, Docker was upgraded to version 20.10.8. After the update, Terraform Enterprise cannot pull Docker images and the application fails to start.
The Terraform Enterprise UI dashboard displays the following error message:
"sequential image pull failed: 1 error occurred: * public/docker:latest; API error (500) Get "https:registry-1.docker.io/v2": net/http: request canceled while waiting for connection (Client Timeout exceeded while awaiting headers)"
Cause
This issue occurs because requests to docker.io are blocked by a firewall. Docker version 20.10.8 introduced a change requiring an additional $HTTPS_PROXY entry in the proxy settings.
The Docker 20.10.8 release notes state:
Due to net/http changes in Go 1.16, HTTP proxies configured through the $HTTP_PROXY environment variable are no longer used for TLS (https://) connections. Make sure you also set an $HTTPS_PROXY environment variable for handling requests to https:// URLs.
Solution
To resolve this issue, you must add the required HTTPS_PROXY setting to the Docker proxy configuration file.
Procedure
- Edit the
http-proxy.conffile. The default location is/etc/systemd/system/docker.service.d/http-proxy.conf. - Add the
HTTPS_PROXYenvironment variable to the file and save your changes. -
Restart the Docker service.
$ sudo systemctl restart docker
-
Restart the Replicated services. You can do this from the TFE Admin Console at
https://<tfe-url>:8800/dashboardor from the command line.To restart from the command line, first stop the service.
$ replicatedctl stop
Wait one to two minutes, then start the service.
$ replicatedctl start
- Monitor the startup progress on the Terraform Enterprise UI dashboard to confirm the application starts successfully.
Outcome
After you update the proxy configuration, Terraform Enterprise should successfully pull the required Docker images and complete the startup process. The application will be fully functional once startup is complete.
Additional Information
- For more details on this change, refer to the Docker Engine 20.10.8 release notes.