Migrate to Terraform Enterprise or HCP Terraform VCS from any backend

Introduction

There are many considerations when planning to migrate to a Terraform Enterprise (TFE) or HCP Terraform (HCPTF) workspace such as:

• Should the workspace be created and configured before migrating the state? 
• When should the VCS repository with the Terraform code be connected to the workspace?
• What will happen to the resources after the state file is migrated?

Prerequisites

Before you begin, ensure you have the following:

1. An active instance of Terraform Enterprise or an HCP Terraform account.
2. Access to your current state file (e.g., locally, Amazon S3, Azure Storage).
3. Access to the VCS repository containing the Terraform configuration (e.g., GitHub, GitLab, Bitbucket).
4. Permissions to connect your VCS provider to an external application. For more information, refer to the Configure Workspace VCS Connections documentation.
5. If module sources are hosted in a private VCS repository, you must configure authentication with SSH keys.
6. A local clone of the repository containing your Terraform configuration.
7. Within your local repository, run $ terraform init to connect to your current backend.
8. Confirm your configuration and infrastructure is synchronized by running $ terraform plan and verifying the output shows No changes.
9. Verify that all expected resources are present in the state by running $ terraform state list.
10. Create a backup of your current state file $ terraform state pull > migrationbackup.tfstate.

Migration Procedures

This section outlines 3 methods for migrating to a TFE or HCPTF workspace.

Method 1: Use terraform init to Create the Workspace - Recommended 

This method uses the $ terraform init command to create the workspace and migrate the state in a single process. The workspace does not need to be created in TFE or HCP Terraform ahead of time.

Procedure

1. Verify the version of Terraform you are using locally, as this version will be set on the new workspace $ terraform -v.
2. In your root module, replace any existing backend configuration with the cloud block.

terraform {
  cloud {
    hostname = "dev.my-tfe-instance.com"
    organization = "my-org"

    workspaces {
    name = "infra-mgmt"
    }
  }
}

3. Log in to your TFE or HCP Terraform instance from the CLI. Create an API token when prompted and paste it into the terminal $ terraform login dev.my-tfe-instance.com
4. Initialize the new backend $ terraform init. Terraform will detect the existing state and prompt you to migrate it. The command will display the following output:

Initializing HCP Terraform...

Do you wish to proceed?
As part of migrating to Terraform Enterprise, Terraform can optionally copy
your current workspace state to the configured Terraform Enterprise workspace.

Answer "yes" to copy the latest state snapshot to the configured
Terraform Enterprise workspace.

Answer "no" to ignore the existing state and just activate the configured
Terraform Enterprise workspace with its existing state, if any.

Should Terraform migrate your existing state?

Enter a value:

5. Type "yes" and press Enter. A successful migration will show the message: HCP Terraform has been successfully initialized!
6. Verify the resources are still tracked correctly $ terraform state list.
7. Navigate to your TFE or HCP Terraform organization. A new workspace named "infra-mgmt" will be present. Select this workspace and go to the States tab to confirm the migrated state is visible.

**Note:** Do not queue a run yet. The workspace is not yet connected to your VCS repository.

8. Connect the workspace to your VCS repository.
     - Go to Settings > Version Control.
     - Click Connect to version control and choose the Version Control Workflow.
     - Follow the prompts to select your repository and configure all required settings (e.g., Terraform Working Directory).
9. Configure your workspace Variables, including any Terraform variables (e.g., var.server_name) or environment variables (e.g., AWS_REGION). Ensure any credentials have the necessary permissions to manage all resources in your state file.
10. Trigger a run by merging a change to your VCS branch or by starting a run manually in the UI. The plan should result in No changes.

Backout Plan

If you run into issues and need to backout, please follow these steps:

1. Ensure the backup of state taken earlier is still available (e.g., migrationbackup.tfstate).
2. If there is a terraform.tfstate file is present, delete it.
3. Roll back the changes made to the backend by editing the configuration or checking out an original copy (e.g., git checkout main.tf).
4. Delete the .terraform folder and .terraform.lock.hcl.
5. Execute terraform init. 
6. Execute terraform plan and  No changes. Your infrastructure matches the configuration should be displayed.  
7. To delete the problematic workspace, navigate to "Settings" -> "Destruction and deletion" (documentation). 

Method 2: Create Workspace Before Migration - Not Recommended

This method is not recommended due to a higher potential for misconfiguration, such as a mismatch between the local Terraform version and the version set in the workspace.

Procedure

1. Verify your local Terraform version $ terraform -v.
2. In your TFE or HCPTF organization, manually create and configure a new workspace, including all variables and VCS settings.
3. In the workspace's General settings, set the Terraform Version to match the version you are using locally.
4. In your local Terraform configuration, add the cloud backend block as shown in Method 1.
5. Log in to your instance $ terraform login [hostname].
6. Initialize the backend and type "yes" when prompted to migrate the state.
7. Verify in the workspace UI that the state was migrated successfully.

Method 3: Bulk Migration with tf-migrate

For migrating a large number of state files to many workspaces, using a tool like `tf-migrate` is recommended. This approach requires significant preparation but is more efficient at scale.

Procedure: Migrate to HCP Terraform in bulk

Additional Information

• To manage workspaces and other TFE/HCPTF resources as code, you can use the TFE Provider.
• Workspaces, variables, and other components can also be managed programmatically using the API.