How to verify TFE version in PostgreSQL data

Problem

The Terraform Enterprise (TFE) application version must be compatible with the schema version of its PostgreSQL database. If these versions are mismatched, TFE may fail to start, presenting various errors.

Common scenarios include:

1. TFE version is lower than the database schema version: TFE will fail to start.
2. TFE version is higher than the database schema version: TFE will attempt to upgrade the database schema. This may succeed, or it may fail if a mandatory intermediate release was skipped during an upgrade.

Example error messages in the application status may include Container ptfe_registry_migrations failed, context deadline exceeded, or Timeout waiting for event nginx started.

## Example error 1
$ replicatedctl app status
[ { "AppID": "15d424fba1e64bf47618f01b764e765d", "State": "stopped", "Error": "Container ptfe_registry_migrations failed: Container ... exited with non-zero exit status 1: "... }]

## Example error 2
$ replicatedctl app status
[ { "AppID": "ff26b23cbe3042f852c5ec5ab0ee27d2", "State": "started", "Error": "Ready state command canceled: context deadline exceeded", ... }]

## Example error 3
$ replicatedctl app status
[ { "AppID": "837ee5c406334c744b60eb3086f09573", "State": "stopped", "Error": "Container tfe-nginx failed: Timeout waiting for event nginx started-80", ... }]

To troubleshoot these issues, you must first verify the current schema version stored in the PostgreSQL database.

Solutions

This guide provides several methods to check the PostgreSQL schema version for both Replicated and Flexible Deployment Options (FDO) installations.

Solution 1: Verify Schema on Replicated Deployments

You can use one of the following methods depending on your environment's status and your access level.

Method A: Use the tfe-atlas container

Execute the db:version Rake task inside the tfe-atlas container.

$ docker exec -it tfe-atlas /usr/bin/init.sh /app/scripts/wait-for-token -- \
  bash -i -c 'cd /app && ./bin/rake db:version'

The command returns the current schema version.

Current version: 20230206193105

Method B: Query PostgreSQL directly

1. Log in to the TFE PostgreSQL environment.

   $ sudo docker exec -ti tfe-atlas /bin/bash
   # psql -h <fqdn> -p 5432 -U <username>

2. Connect to your TFE database. In this example, the database is named tfe.

   postgres=> \c tfe

3. Run the following query to retrieve the latest schema version.

   tfe=> SELECT * FROM rails.schema_migrations ORDER BY version DESC LIMIT 1;

   The command returns the version.

   version      
   ----------------
    20230206193105

Method C: Check logs in a support bundle

After generating a support bundle, inspect the ptfe_migrations.stdout or tfe-migrations.stdout log file for an entry indicating the schema version.

## File: primary/app/logs/ptfe_migrations.stdout

Current version: 20230206193105

Solution 2: Verify Schema on Flexible Deployment Options (FDO)

1. Log in to the Rails console using tfectl.

   $ tfectl support console

2. From the console, run the following commands to get the database, application schema, and private registry schema versions.

   • Database Version

     ActiveRecord::Base.connection.execute("SELECT version();").first["version"]

   • TFE Application Schema Version

     ActiveRecord::Base.connection.execute("SELECT * FROM rails.schema_migrations ORDER BY version DESC LIMIT 1").first["version"]

   • TFE Private Registry Schema Version

     result = ActiveRecord::Base.connection.execute("SELECT * FROM registry.schema_migrations;").first
     version = result["version"]
     dirty = result["dirty"]
     puts "Version: #{version}, Dirty: #{dirty}"

TFE Version to Schema Version Mapping

Use the following table to correlate your TFE release version with its corresponding PostgreSQL schema versions.

Additional Information

For a complete list of versions and required upgrade paths, refer to the Terraform Enterprise release notes.