Problem
When you upgrade Terraform Enterprise with Replicated Mounted Disk to version v202406-1 (build 776) or higher, the process may appear to hang. The logs may show the following message without further updates:
Running as builtin tfe user, ensuring ownership of scratch directories...
The Replicated dashboard may also display an error message:
Ready state command canceled: context deadline exceeded
Prerequisites
- You are upgrading Terraform Enterprise to version
v202406-1(build 776) or higher. - Your installation uses the Replicated Mounted Disk operational mode.
- The mounted disk contains a large amount of data (e.g., over 1TB).
Cause
This behavior is an expected part of the upgrade process and not an error. To accommodate new security improvements, the upgrade procedure must update file permissions across all directories on the mounted disk. For large data directories, this permission update is a time-consuming operation that does not provide progress indicators in the logs.
Solution
The process can take a significant amount of time to complete, depending on the total amount of data and the performance of the underlying disk storage. You should budget up to one hour for this task to complete during each application restart when upgrading to version v202406-1 or higher.
Allow the process to run uninterrupted. The upgrade will continue automatically once the file permission updates are complete.
Additional Information
- For more details on storage requirements, refer to the Mounted Disk Storage Requirements documentation.