Problem

The Terraform Enterprise Web UI occasionally shows error message “error fetching data internal server error” when trying to access workspaces. However,  Terraform runs are not affected.

Cause

The issue is commonly attributed to connectivity disruptions with the external PostgreSQL database. The atlas.log may show errors related to database connection issues, such as:

error=ActiveRecord::ConnectionNotEstablished id=xxxxx message=connection to server at "10.x.x.x", port 5432 failed: FATAL: sorry, too many clients already

Followed by multiple failed GET requests for workspaces:

"method":"GET","path":"/api/v2/workspaces/ws-xxxxxx","format":"jsonapi","status":500

The error indicates the PostgreSQL instance managing the Terraform Enterprise DB has exceeded its max_connections limit, causing requests to retrieve workspace information to fail.

Solutions

To resolve it, please review the PostgreSQL instance performance and take the following actions:

1. Check the current connection usage (pg_stat_activity) to identify if too many active or idle connections are consuming resources.
2. Increase the max_connections limit if the instance allows it, or optimize existing connections.
3. Each connection in PostgreSQL consumes memory, ensure your server has the necessary capacity to support the requirements and allocate additional memory as needed.

Outcome

The web UI should operate seamlessly without any error message. 

References:

• PostgreSQL - max_connections
• PostgreSQL - pg_stat_activity