Problem
When a Terraform Enterprise instance is low on disk space, you may encounter one or more of the following symptoms:
-
Terraform runs fail with a
no space left on deviceerror message.Error: write /...: no space left on device
-
The Replicated management dashboard displays a low disk space warning.
Low disk space! There may not be enough disk space on the following nodes: <xxx>. Please increase drive space to at least 1 GB.
- Attempts to generate a support bundle fail or produce an empty file.
Cause
These issues occur because the available disk space on the virtual machine hosting Terraform Enterprise is exhausted. This can be caused by an accumulation of Docker data, old run artifacts, or large log files.
Solutions
This guide provides several methods to reclaim disk space on the instance.
Solution 1: Prune Unused Docker Data
The docker system prune command removes all unused containers, networks, dangling images, and optionally, volumes. This can often reclaim a significant amount of space.
Before running the command, review all stopped containers and unused images, volumes, build caches, and networks to ensure you do not remove anything important.
Execute the following command to perform the cleanup.
$ docker system prune --volumes
Solution 2: Delete Unused Workspaces
Deleting unused workspaces removes all associated run data, including plans, states, and configuration versions. Note that garbage collection may not occur immediately, so it may take some time for the disk space to be fully reclaimed.
You can delete workspaces manually in the Terraform Enterprise UI through Workspace Settings or automate the process using the Delete a Workspace API endpoint.
Preventive Measures
To prevent disk space issues from recurring, consider implementing the following long-term strategies:
-
Optimize Configuration Uploads: Review your Terraform configuration directories and add unnecessary paths to a
.terraformignorefile to reduce the size of uploaded content. - Review Repository Structure: Moderate the use of large mono repositories, as they can lead to larger configuration versions.
-
Configure Log Rotation: Implement log rotation for both Docker and
journaldto manage the size of system and container logs.