Problem
Generating a support bundle results in a tarball that only contains a single README.md
file
Cause
There are a few reasons why support bundle generation may fail:
-
The Replicated version in use has a known bug that results in empty support bundles.
-
The support bundle process times out while gathering log files.
-
The system does not have enough disk space to complete bundle generation.
Solutions
Replicated Version
Some previous versions of Replicated have known issues with support bundle generation which result in empty support bundles. Use the following steps to verify the current Replicated version and upgrade as needed.
-
Connect to the instance using SSH.
-
Run the following command to gather the current version of Replicated.
$ replicated -version
-
Check the Replicated release notes to verify whether the current version of Replicated installed on the system was affected by a bug preventing support bundle generation.
-
If the current version was affected by a bundle generation bug, or is outdated, refer to the Upgrading Replicated guide for steps on how to upgrade the version of Replicated installed on the system.
-
Attempt to generate a new support bundle.
Process Times Out
Generally, the cause of the process timing out can be linked to a large journal file. The following steps detail the process for checking the size of the journal file and reducing the size if needed.
-
Connect to the instance using SSH.
-
The journal size can be checked using the following command:
$ journalctl --disk-usage
-
The file can be reduced to the the last 500MB with the following command:
$ journalctl --vacuum-size=500M
-
Run the
--disk-usage
command again to confirm that the vacuum command worked as expected. -
Attempt to generate a new support bundle.
Note: For a long term solution, examine how logrotate
is configured on the server and keep the size of the journal below 1GB.
System Storage
Because support bundle generation involves copying the log files for each of the Docker containers that make up the Terraform Enterprise application, the system must have at least enough available storage for twice the current size of these logs. In the event that there is currently not enough storage to complete this task, the following steps may be used to help reclaim storage space and allow for bundle generation to complete.
-
Connect to the instance using SSH.
-
Run the following command to determine how much disk space Docker is currently using.
$ docker system df
-
Run the following command to verify how much disk space is available. The Docker filesystem is generally found at
/var/lib/docker
, so the relevant line of the output is whichever one contains this path.$ df -h
-
If the amount of disk space available from step two is less than twice the amount shown in step one, the following steps may be used to clean up the Docker filesystem and free up additional storage space. It is important to verify that both Terraform Enterprise and Replicated are running prior to taking steps to clean up the Docker filesystem to prevent data loss.
-
Verify that the Terraform Enterprise application is currently running by using the following command.
$ replicatedctl app status
-
Verify that all three parts of the Replicated application are currently running by using the following command.
$ systemctl status replicated replicated-ui replicated-operator
-
After verifying the applications are running, the following command may be used to clean up the Docker filesystem.
$ docker system prune
-
-
Repeat steps one and two to verify whether there is now twice the amount of storage used by the Docker filesystem available. If this process did not free up enough space, it may be necessary to extend the amount of storage available on the system.
-
Attempt to generate a new support bundle.
Monitoring Support Bundle Generation Logs
If support bundle generation continues to fail after following the previous steps, additional logging from the process may be inspected using the following steps. These logs may also be helpful in the event that additional support from HashiCorp is necessary.
-
Connect to the instance using SSH.
-
Start the process of generating a support bundle using the following command.
$ replicatedctl support-bundle
-
While the support bundle generation process is running, use the following command to gather the ID of the container running the process. The ID will be the first column of the output and is a string of 12 alphanumeric characters.
$ docker ps | grep support-bundle
-
Using the ID gathered in the previous step, use the following command to monitor the logs.
$ docker logs -f $ID
-
If assistance is required in determining the cause of any errors in the logs, please reach out to HashiCorp support.
Additional Information
Further Steps
If the steps outlined in this document do not result in successful bundle generation, please reach out to HashiCorp support for additional assistance.