During upgrades, the Replicated dashboard shows that the application is transitioning to "starting up" status. When this eventually fails with the error Container [ptfe_base_startup/tfe_base_startup] failed: Timeout waiting for event startup sequencing - phase 1, follow these diagnostic steps:
Symptom Diagnosis
- Confirm the existing installation operates in mounted disk mode
- SSH into the Terraform Enterprise host machine
- Execute
sudo docker logs [ptfe_base_startup/tfe_base_startup]and verify the output matches:
2020/11/13 13:23:38 version: 91a42b2 2020/11/13 13:23:38 setting default postgres connect_timeout to 30 2020/11/13 13:23:39 amqp://rabbitmq.[ptfe/tfe]:5672: not ready yet: Exception (501) Reason: "read tcp 172.17.0.18:53612->172.17.0.1:5672: read: connection reset by peer" 2020/11/13 13:23:39 redis://redis:6379: ready 2020/11/13 13:23:39 postgres://postgres: not ready yet: pq: password authentication failed for user "hashicorp" [repeated authentication failures continue] 2020/11/13 13:23:40 amqp://rabbitmq.[ptfe/tfe]:5672: ready
-
For Terraform Enterprise v202205-1 or later only: Execute
sudo docker logs db_pw_migrateand verify output matches:
2020-11-13T13:23:48.505522000Z [+] executing password migration logic... 2020-11-13T13:23:48.565785000Z psql: FATAL: password authentication failed for user "hashicorp" 2020-11-13T13:23:48.566170000Z [+] updating postgres password... 2020-11-13T13:23:48.617737000Z psql: FATAL: password authentication failed for user "hashicorp"
Solution
If these indicators are all matched, the problem comes from incorrect internal PostgreSQL password being used, the How To Recover PostgreSQL Password on Mounted Disk guide is provided in order to recover the password.
Additional information
If the issue continues to occur after completing the steps in this guide, please contact HashiCorp Support to request further assistance.