Problem
When using a GitLab.com VCS connection with Terraform Enterprise, you may encounter the following error, which invalidates the VCS connection.
The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client. Server responded with code 401, message: Token was revoked. You have to re-authorize from the user.
Cause
A race condition can occur if GitLab sends a high volume of webhooks simultaneously. Each webhook may trigger an OAuth token refresh, which revokes the previously active token. This can lead to subsequent webhook processing failures and invalidate the VCS connection in Terraform Enterprise.
You can verify this by reviewing the GitLab production logs and searching for /oauth/token POST requests. In the example below, a successful token refresh request took 60 seconds to complete. During that time, another refresh request was initiated, which failed with a 401 error because the original token had already been invalidated by the first request.
{
"method": "POST",
"path": "/oauth/token",
"format": "*/*",
"controller": "Doorkeeper::TokensController",
"action": "create",
"status": 200,
"duration": 60.17,
"view": 0.0,
"params": [
{
"key": "client_id",
"value": ""
},
{
"key": "client_secret",
"value": "[FILTERED]"
},
{
"key": "grant_type",
"value": "refresh_token"
},
{
"key": "refresh_token",
"value": "[FILTERED]"
}
],
"correlation_id": "qyYcaGBHM94"
}
{
"method": "POST",
"path": "/oauth/token",
"format": "*/*",
"controller": "Doorkeeper::TokensController",
"action": "create",
"status": 401,
"duration": 3.43,
"view": 0.0,
"params": [
{
"key": "client_id",
"value": ""
},
{
"key": "client_secret",
"value": "[FILTERED]"
},
{
"key": "grant_type",
"value": "refresh_token"
},
{
"key": "refresh_token",
"value": "[FILTERED]"
}
],
"correlation_id": "dKsHi6Ej3t2"
}Solutions
There are two approaches to resolve this issue. The first solution restores access immediately but may not prevent the problem from recurring. The second solution is the recommended long-term fix.
Solution 1: Restore the VCS Connection Manually
To restore VCS access quickly, you must revoke and re-establish the VCS connection from the Terraform Enterprise UI. This action will require you to reconnect all workspaces that were using the old connection.
Note that this is a temporary fix. If the underlying race condition in GitLab occurs again, the connection may become invalid.
Solution 2: Recreate the VCS Connection using a Personal Access Token (Recommended)
To prevent this race condition, recreate the VCS connection using a GitLab Personal Access Token (PAT) instead of the standard OAuth flow. A PAT is generated by a user and remains valid until it is manually revoked, so it is not subject to the token refresh race condition.
You can create the VCS connection programmatically using one of the following methods:
- Terraform Enterprise API: Use the Create an OAuth Client API endpoint to configure the connection.
-
TFE Provider: Use the
tfe_oauth_clientresource in your Terraform configuration.
Additional Information
This behavior is related to a known issue in GitLab. For more details, refer to the public GitLab issue: GitLab Issue #243844.