Migrating and Pushing Terraform State from One Backend to Another

Overview

This guide provides detailed steps for migrating a Terraform state file from one backend (e.g., GitLab HTTP backend) to another. It also covers how to manually edit the state file if required before the final push.

This is particularly useful when you need to change your backend provider or split consolidated states into individual ones without losing historical data.

Use Case

You want to:

Pull the current state file from your existing backend
(Optionally) modify the state file manually
Push it to a new backend (e.g., from GitLab HTTP to AWS S3 or Terraform Cloud).

Step-by-Step Instructions

1. Pull the Existing State File

Run the following command from the environment currently configured with the old backend:

terraform state pull > current.tfstate

This will export your current state file in JSON format.

2. Edit the State File (Optional)

Before migrating, you may want to:

Remove unnecessary resources
Change resource names
Update module paths

Use jq or a text editor like VS Code or Sublime Text to make modifications.

Be cautious with edits. Always validate structure and ensure the serial number is correct.

Example:

To update a resource name:

"resources": [
  {
    "type": "aws_instance",
    "name": "new_name",
    ...
  }
]

To increment the serial number:

"serial": 123  ➜  "serial": 124

3. Reinitialize Terraform with the New Backend

In a clean working directory:

For AWS S3:

Set up your new backend configuration, for example:

terraform {
  backend "s3" {
    bucket = "my-tf-states"
    key    = "envs/prod/terraform.tfstate"
    region = "us-west-2"
  }
}

Initialize Terraform with -migrate-state=false to avoid triggering automatic migration:

terraform init -migrate-state=false

4. Push the Edited State to New Backend

Now use the following command:

terraform state push -force current.tfstate

-force overrides backend state with your local copy. Use this with caution.

Common Errors and Fixes

State Lock Error

Error: Error acquiring the state lock

Use:

terraform force-unlock <LOCK_ID>

404 or Backend URL Errors

Failed to persist state: HTTP error: 404

Make sure:

Backend URL is correct
The address and related settings in your backend match the new backend format

Recommendations

Always back up your state before making manual edits or pushing.
Test changes locally with terraform plan before applying.
Validate the final state JSON structure.