Migrate to Terraform Enterprise or HCP Terraform VCS from any backend

Avatar
Julie Erlemeier
  • Updated

Introduction

There are many considerations when planning to migrate to a Terraform Enterprise (TFE) or HCP Terraform (HCPTF) workspace such as:

  • Should the workspace be created and configured before migrating the state? 
  • When should the VCS repository with the Terraform code be connected to the workspace?
  • What will happen to the resources after the state file is migrated?

 

Prerequisites

Please confirm you have the following:

  • Active instance of TFE or HCP TF account.
  • Terraform plan showing "No changes" in the output.
  • Access to your state file (e.g., locally, Amazon S3, Azure Storage). 
  • Access to the VCS repository containing the Terraform configuration, also known as the root module (e.g., Github, Gitlab, Bitbucket). If module sources are hosted in a private VCS repository, configure authentication with SSH keys.
  • Permissions to connect your VCS to an external application. For more info, please read Configure workspace VCS connections.
  • Determined if workspaces will be managed using the TFE Provider in the long term. If yes, review "Solution D" below.

Once completed, perform the following:

  • Clone the repository containing the Terraform configuration locally.
  • Run terraform init to connect to the backend. 
  • Execute terraform state list to ensure the expected resources are shown.
  • Backup this state terraform state pull > migrationbackup.tfstate

 

Solutions

Here are 5 solutions for migrating to Terraform Enterprise or HCP Terraform workspaces from another backend.

 

Solution A: Use terraform init to create the workspace and migrate the state

  • The workspace does not need to be created ahead of time. 
  • Terraform init will set the Terraform version in the workspace to the version of Terraform used during migration.
  • After state migration, navigate to the new workspace to connect your VCS. 

Steps

  • Access your source code. In this example, it creates 10 random_pet resources. These are present in the existing state in the current backend.
resource "random_pet" "r" {
  count = 10
}
  • Execute terraform -v and make note of your version (i.e., v1.10.4).
  • Add your cloud integration to your root module code. If there is a previous backend configuration, replace it with cloud.
terraform {
  cloud { 
    hostname = "dev.my-tfe-instance.com" 
    organization = "my-org" 

    workspaces { 
      name = "foobar-root" 
    } 
  } 
}

module "test" {
  source = "github.com/my_org_name/foobar-source.git"
}
  • Execute terraform login [hostname] (source). Create a token and paste it into the terminal.
  • Execute terraform init, which should display the following output:
Initializing HCP Terraform...
Do you wish to proceed?
  As part of migrating to Terraform Enterprise, Terraform can optionally copy
  your current workspace state to the configured Terraform Enterprise workspace.
  
  Answer "yes" to copy the latest state snapshot to the configured
  Terraform Enterprise workspace.
  
  Answer "no" to ignore the existing state and just activate the configured
  Terraform Enterprise workspace with its existing state, if any.
  
  Should Terraform migrate your existing state?

  Enter a value:
  • Type "yes" and press enter. If the migration is successful, the following output will be shown:  "HCP Terraform has been successfully initialized!"
  • Executing terraform state list should still show the expected resources. Navigate to your TFE organization. There will be a workspace automatically created named "foobar-root". Select this workspace and go to "States" to see that the migrated state is present.
  • Do NOT queue a run. This workspace is not yet configured to connect to the VCS
  • Perform the following to connect to VCS:
    • Go to "Settings", "Version Control", "Connect to version control", choose "Version Control Workflow". Follow the prompts and configure all settings (e.g., Terraform Working Directory). 
    • Configure your workspace "Variables". These are either Terraform variables (e.g., var.server_name) or environment variables (e.g., AWS_REGION). If configuring credentials, make sure those credentials give Terraform Enterprise the ability to perform all necessary operations on ALL the resources in your state file.
  • Now you can trigger a run using your VCS connection (e.g., merging code to main) or manually in the UI. 
  • The run should display "No changes Your infrastructure matches the configuration".

Backout Plan

If you run into issues and need to backout, please follow these steps:

  • Ensure the backup of state taken earlier is still available (e.g., migrationbackup.tfstate).
  • If there is a terraform.tfstate file is present, delete it.
  • Roll back the changes made to the backend by editing the configuration or checking out an original copy (e.g., git checkout main.tf).
  • Delete the .terraform folder and .terraform.lock.hcl.
  • Execute terraform init. 
  • Execute terraform plan and  No changes. Your infrastructure matches the configuration should be displayed.  
  • To delete the problematic workspace, navigate to "Settings" -> "Destruction and deletion" (documentation). 

 

Solution B: Create Workspace Before Migration - Not Recommended

  • There are many points of failure. 
  • You must ensure your the Terraform version used for the migration matches the version you set on your workspace.

Steps

  • All runs must be complete.
  • Execute terraform plan and make sure you see No changes. Your infrastructure matches the configuration. 
  • Execute terraform -v and make note of your version (i.e., v1.10.4).
  • Make your workspace manually and configure VCS:
    • In your organization, create your workspace. Configure variables and all necessary settings.
    • Choose "Version Control" and select your provider. Follow the prompts and then choose your repository.
    • In "Workspace Settings", in "General", choose the "Terraform version" (i.e., v1.10.4). Save your setting. This must match the version used to perform the migration.
  • In the Terraform block, add the cloud backend. See "Solution A" above for details.
  • Execute terraform login [hostname] (source). Create the token and paste it into the terminal.
  • Execute terraform init. The output will be: Should Terraform migrate your existing state? type "yes".
  • In the workspace, you will see the state was migrated.

Backout plan

See "Solution A" above.

 

Solution C: tf-migrate

  • Recommended for bulk migration of many state files to many workspaces.
  • There is significant prep-work. This may be overkill for a small number of workspaces. 
  • Procedure: Migrate to HCP Terraform in bulk

 

Solution D: TFE Provider

In this case, the first step is to create one workspace. Using infrastructure-as-code, this workspace will manage all of your TFE/HCPTF infrastructure(e.g., TFE token with necessary permissions to do the work, workspaces, VCS connections, ect.)

If you need to make future changes, you have a clear, established pattern. Once workspaces are created, you can migrate state to them from your local machine. We will not cover this topic in this article. If you would like more information, please open a ticket with HashiCorp Support. 

 

Solution E: API Workflow

Use rest APIs to manage workspaces, variables, ect. We will not cover this topic in this article. If you would like more information, please open a ticket with HashiCorp Support. 

 

Articles in this section

See more