Upgrade Warning
The Replicated deployment option is limited to customers who purchased Terraform Enterprise before January 2024. Terraform Enterprise now supports new deployment options and will end support for the Replicated Native Scheduler option. The final Replicated release of Terraform Enterprise will be in November 2024. HashiCorp will support this release until April 1, 2026.
Scenario
When upgrading a Terraform Enterprise installation with multiple nodes, it’s crucial to attempt the upgrade with only one instance running before scaling back up to the usual number of instances. This is because the upgrade process updates the application’s database through a process called “migrations.” This process also updates a sequence number indicating the version of the database schema.
Here’s an example of a sequence number in a log file:
2024-01-10T02:11:48.233107447Z Current version: 20230725220632.
The database updates occur during the application's first launch during the upgrade process. These updates are typically logged in the stdout log for a process named migrations. If the database version and installation version don't match, the application won't start, and the database will require a rollback before a new upgrade attempt.
Recommendation
When upgrading Terraform Enterprise, it's important to stop the application after waiting for all in-flight runs to finish before starting the upgrade.
The upgrade process differs depending on your chosen deployment option:
- Flexible Deployment Options: The upgrade guide for Flexible Deployments can be found here. For a multi-node setup, scale down to a single node before stopping the application and installing updates.
- Terraform Enterprise Replicated Deployment: The upgrade process for a multi-node Active/Active deployment can be found here. Upgrading a Terraform Enterprise deployment with multiple Active/Active nodes requires a specific approach. Please be sure to review the automated install guide, our best practices for deployment, and the Admin CLI guide before beginning. The primary requirements include an auto scaling group (ASG) or equivalent, with a single instance running Terraform Enterprise. This ASG should be behind a load balancer. We recommend the installation be completely automated to allow scaling to zero and back to one without intervention. Currently, the safest way to perform an upgrade is to shut down all node instances, rebuild one node to validate a successful upgrade, and then scale to additional nodes (currently max 5).
Additional Information
Additional documentation for for the changes made during the "migrations", is here:
- The rails, vault, registry, and task_worker PostgreSQL schemas must be created on the database. These schemas will be automatically created if they do not already exist.
- A PostgreSQL user must be created with the following permissions on the database:
- The ability to create, modify, and read all tables and indices on all schemas within the database. Usually this is granted if the user is an owner of the database.
- The ability to create extensions.
NOTE: Introduced in v202309-1, Terraform Enterprise supports flexible deployment options, such as Docker Engine and Kubernetes. If you are using Replicated today and want to migrate your Terraform Enterprise installation to an alternate deployment option, refer to our migration guide.