Migrate to Terraform Enterprise or HCP Terraform VCS from any backend

Introduction

There are many considerations when planning to migrate to a Terraform Enterprise (TFE) or HCP Terraform workspace, such as:

• Should the workspace be created and configured before migrating the state?
• When should the VCS repository with the Terraform code be connected to the workspace?
• What will happen to the resources after the state file is migrated?

Prerequisites

Before you begin, ensure you have completed the following steps:

1. Have an active instance of Terraform Enterprise or an HCP Terraform account.
2. Confirm you have access to your current state file (e.g., locally, Amazon S3, Azure Storage).
3. Confirm you have access to the VCS repository containing the Terraform configuration (e.g., GitHub, GitLab, Bitbucket).
4. Verify you have permissions to connect your VCS provider to an external application. For more information, refer to the Configure Workspace VCS Connections documentation.
5. If module sources are hosted in a private VCS repository, you must configure authentication with SSH keys.
6. Create a local clone of the repository containing your Terraform configuration.
7. In your local repository, run terraform init to connect to your current backend.
8. Confirm your configuration and infrastructure are synchronized by running terraform plan and verifying the output shows No changes.
9. Verify that all expected resources are present in the state by running the following command.

   $ terraform state list

10. Create a backup of your current state file.

    $ terraform state pull > migration-backup.tfstate

Procedure

This section outlines three methods for migrating to a Terraform Enterprise or HCP Terraform workspace.

Method 1: Use terraform init to Create the Workspace (Recommended)

This method uses the terraform init command to create the workspace and migrate the state in a single process. The workspace does not need to be created in Terraform Enterprise or HCP Terraform ahead of time.

1. Verify the version of Terraform you are using locally, as this version will be set on the new workspace.

   $ terraform -v

2. In your root module, replace any existing backend configuration with the cloud block for the HCP Terraform cloud integration.

   terraform {
     cloud {
       hostname     = "dev.my-tfe-instance.com"
       organization = "my-org"
   
       workspaces {
         name = "infra-mgmt"
       }
     }
   }

3. Log in to your Terraform Enterprise or HCP Terraform instance from the CLI. Create an API token when prompted and paste it into the terminal.

   $ terraform login dev.my-tfe-instance.com

4. Initialize the new backend. Terraform will detect the existing state and prompt you to migrate it.

   $ terraform init

   The command will display the following output.

   Initializing HCP Terraform...
   
   Do you wish to proceed?
   As part of migrating to Terraform Enterprise, Terraform can optionally copy
   your current workspace state to the configured Terraform Enterprise workspace.
   
   Answer "yes" to copy the latest state snapshot to the configured
   Terraform Enterprise workspace.
   
   Answer "no" to ignore the existing state and just activate the configured
   Terraform Enterprise workspace with its existing state, if any.
   
   Should Terraform migrate your existing state?
     Enter a value:

5. Type yes and press Enter. A successful migration will show the message: HCP Terraform has been successfully initialized!

6. Verify the resources are still tracked correctly.

   $ terraform state list

7. Navigate to your Terraform Enterprise or HCP Terraform organization. A new workspace named infra-mgmt will be present. Select this workspace and go to the States tab to confirm the migrated state is visible.

   Note: Do not queue a run yet. The workspace is not yet connected to your VCS repository.

8. Connect the workspace to your VCS repository.

   1. Go to Settings > Version Control.
   2. Click Connect to version control and choose the Version Control Workflow.
   3. Follow the prompts to select your repository and configure all required settings (e.g., Terraform Working Directory).

9. Configure your workspace Variables, including any Terraform variables (e.g., var.server_name) or environment variables (e.g., AWS_REGION). Ensure any credentials have the necessary permissions to manage all resources in your state file.

10. Trigger a run by merging a change to your VCS branch or by starting a run manually in the UI. The plan should result in No changes.

Backout Plan

If a backout is necessary, follow these steps:

1. Ensure the state backup taken earlier is still available (e.g., migration-backup.tfstate).
2. If a terraform.tfstate file is present, delete it.
3. Roll back the changes made to the backend block by editing the configuration or checking out an original copy (e.g., git checkout main.tf).
4. Delete the .terraform folder and .terraform.lock.hcl file.
5. Execute terraform init.
6. Execute terraform plan. The output No changes. Your infrastructure matches the configuration. should be displayed.
7. To delete the new workspace, navigate to Settings > Destruction and Deletion. For more details, refer to the documentation on deleting a workspace.

Method 2: Create Workspace Before Migration (Not Recommended)

This method is not recommended due to a higher potential for misconfiguration, such as a mismatch between the local Terraform version and the version set in the workspace.

1. Verify your local Terraform version.

   $ terraform -v

2. In your Terraform Enterprise or HCP Terraform organization, manually create and configure a new workspace, including all variables and VCS settings.
3. In the workspace's General settings, set the Terraform Version to match the version you are using locally.
4. In your local Terraform configuration, add the cloud block as shown in Method 1.
5. Log in to your instance.

   $ terraform login [hostname]

6. Initialize the backend and type yes when prompted to migrate the state.
7. Verify in the workspace UI that the state was migrated successfully.

Method 3: Bulk Migration Using tf-migrate

For migrating a large number of state files to many workspaces, using a tool like tf-migrate is recommended. This approach requires significant preparation but is more efficient at scale. For the procedure, refer to the tutorial: Migrate to HCP Terraform in bulk.

Additional Information

• To manage workspaces and other Terraform Enterprise or HCP Terraform resources as code, you can use the TFE Provider.
• Workspaces, variables, and other components can also be managed programmatically using the API.