Using the Controlled Remote State Access Migration Tool

Introduction

To help ease adoption of the new controlled remote state access feature across a large numbers of workspaces, we’ve included a command line tool with Terraform Enterprise v202104-1 which examines usage of the terraform_remote_state data source in each existing workspace within Terraform Enterprise and makes recommendations about how to configure the remote state consumers list so as to avoid interrupting existing workflows.

Procedure

Running the migration tool

The migration tool is a wrapper around a rake task within the ptfe_atlas container that runs as a part of a Terraform Enterprise installation. As such an administrator with privileges to connect to an instance running Terraform Enterprise with superuser access is required. The procedure for running the migration tool is outlined below.

In the event that there are any errors, timeouts, or other issues, it is safe to re-run the migration tool multiple times.

1. Connect to the instance (or one of the nodes if running in clustered mode) using SSH.

2. Connect to the ptfe_atlas container (for Terraform Enterprise v202205-1 or later, this container is called tfe-atlas) using the following command.

    $ sudo docker exec -it ptfe_atlas /bin/bash

   For Terraform Enterprise v202205-1 or later:

    $ sudo docker exec -it tfe-atlas /bin/bash

3. (Optional) Once connected to the *atlas container, the configuration options for the migration script may be viewed by running the following command.

    $ ./scripts/populate_state_sharing_settings.sh --help

4. The migration script may then be be run using the following command.

   $ ./scripts/populate_state_sharing_settings.sh

5. The output from running the migration tool will be similar to the following.

   ===
   Organization: hashicorp
   ===
   
   output-workspace-1's state is used by:
     -> consumer-workspace
   
   output-workspace-2's state is used by:
     -> consumer-workspace
   
   Establish relationships between these workspaces? (y/n)
   
   Choosing y means that state from the producing workspace can
   only be accessed by the consuming workspace(s), instead of being
   available to any workspace in the organization. You can always
   adjust these settings via the workspace settings page.

6. Once all workspaces that are accessing state from other workspaces have been assessed, the migration tool will offer the option to switch all remaining workspaces to restricted sharing.

   Require explicit remote state access for remaining workspaces in this organization? (y/n)
   
   There are 27 workspace(s) left in this organization which are 
   configured to allow remote state access from any other workspace 
   in the organization. Choosing y means that going forward, these 
   workspace(s) must explicitly choose to allow state access. This 
   setting can always be adjusted via the workspace settings page.

Troubleshooting

Terraform state parser

The tool relies on data populated by the version of the Terraform state parser service that shipped in Terraform Enterprise v202104-1. This data is automatically populated during Terraform runs, but if there are workspaces that haven’t performed a run since the upgrade, the following message will be displayed.

Some workspaces in organization hashicorp are missing
remote state access data. Populating missing data...
...........................Done!

If this step times out, it’s likely that the Terraform state parser service is experiencing difficulties. To get around this, it is possible to lower the --batch-size option from its default of 10 to keep from overwhelming the service. To do so, re-run the migration tool as follows.

$ ./scripts/populate_state_sharing_settings.sh --batch-size 5

Renamed workspaces and organizations

The terraform_remote_state data source relies on the workspace and organization names in order to discover which workspace to pull state from. As a result, the migration tool may skip over workspaces that have been renamed since the last Terraform run.

To correct this issue, queue a run in any workspace that has been renamed since the last time a run has been queued within it, as well as any workspaces that use the terraform_remote_state data source to access state from the recently renamed workspace.

Additional Information

If you experience issues using the methods described in this article, please contact HashiCorp Support to request for further assistance.