Problem
Terraform Enterprise fails to start after upgrading to releases v202207-1 and v202207-2
Prerequisites
- Terraform Enterprise mounted disk operational mode
- Upgrade to releases v202207-1 and v202207-2
Cause
- The upgrade from Postgres 12 to Postgres 14 fails to complete due to not finding
pg_hba.confinitdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
Is the server running locally and accepting connections on that socket?
LOG: starting PostgreSQL 14.4 (Debian 14.4-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
LOG: listening on IPv4 address "0.0.0.0", port 5432
LOG: listening on IPv6 address "::", port 5432
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
LOG: database system was shut down at 2023-03-31 14:08:10 UTC
LOG: database system is ready to accept connections
LOG: received smart shutdown request
LOG: background worker "logical replication launcher" (PID 258) exited with exit code 1
LOG: shutting down
LOG: database system is shut down
cp: cannot stat '/backup/12/pg_hba.conf': No such file or directory
rm: cannot remove '/var/lib/postgresql/data': Device or resource busy
Solution:
- SSH into the Terraform Enterprise host and edit the
pg_hba.conffile
# Retrieve the disk_path. Output might differ on your system.
replicatedctl app-config export --template '{{.disk_path.Value}}'
/opt/hashicorp/data
# Edit the pg_hba.conf with your favorite editor
sudo vim /opt/hashicorp/data/postgres/pgdata/pg_hba.conf
# Add the following line at the bottom of the file
host all all all md5
# Save & exit - Stop and Start the Terraform Enterprise application
# Stop TFE
replicatedctl app stop -f
# Start TFE
replicatedctl app start
Outcome
During startup check the logs of the tfe-postgres-setup container and look for this line
PostgreSQL data upgrade completed successfully! Exiting.