Introduction
The SIC-001 error is a generic failure to process a Terraform slug. SIC stands for Source Ingress Controller, and it manages VCS intake. To get more specific errors and the root cause, a support bundle will be needed. Typically, SIC-001s are resolved either via recreating the VCS connections on the affected workspaces.
Problem
SIC-001 comes in many forms:
- The Oauth token expires
- Permissions on the VCS side are changed or revoked
- Network issues keeping TFE from reaching the VCS or its internal store of slugs
- TFE is misconfigured
- Both the workspace and TFE are attempting to manage the VCS auth, causing a race condition.
- Use of symlinks that link outside of the workspace
- Extremely large repository sizes
- Incorrect or non-existent Terraform Working Directory
If Gitlab is being used as a VCS provider, there is some strange behavior that causes the OAuth token to expire and not refresh correctly. We’ve informed GItlab about this. Our temporary workaround is to use a Personal Access Token for authentication against Gitlab.
Cause
Workspace runs will error with something like:
Configuration errored
Your configuration has been uploaded but contains errors (Internal error: SIC-001).
Failed to ingress slug: Failed to reach repo: Failed on ls-remote: exit status 128
Output: remote: HTTP Basic: Access denied
fatal: Authentication failed for 'https://gitlab.hhla-ad.local/cloud-infrastructure/tos-test/tos-environments.git/'
This is typically caused by the following:
Using symlinks that link outside of the repository
Extremely large repository sizes
Incorrect or non-existent Terraform Working Directory
Failed authenticationThe ingress container might have log lines such as:
2020-12-09T06:59:44.414180818Z {"@level":"error","@message":"Failed on ls-remote","@module":"slug-ingress","@timestamp":"2020-12-09T06:59:44.414004Z","error":"exit status 128","output":"remote: HTTP Basic: Access denied\nfatal: Authentication failed for 'https://gitlab.hhla-ad.local/cloud-infrastructure/tos-test/tos-environments.git/'\n"}or
Internal error: SIC-001
Failed to ingress slug: Failed to upload slug: failed uploading: PUT http://archivist.ptfe:7675/v1/object/dmF1bHQ6djE6OFB4alY0RHRHdXhMelRuSkd0ZDk2dHRhU0hXTEc3SEFOY0hmeSt1RndFVnJ6QlpoQmswSHhmcXIwdDdHWUZDZHprUW5uYlNzak1QQ1N2NnNvUWZKaEREOE1aNysrcWNaTGQyaWYvNUdEQkdwbkwyN1pQZFUvdjc5ZWFnMEVBaVVZWVZPQVpoRW12WlRuN1ljWW1aSFlsUThOeTRuMDhZTDIybVgwM1dyb1BvMEltRmM1SUozc1RwS0xYcUhRVktlcDIxeUlTb1N1YWM5Sy9xUEUvMklVTGplVUZzK2loWUJINllUNWhQaHNjd05ReERIQWFjR0hXclFFcUl0bnVGcTJ2SUFwMFJzcFJpMVREbVRBSFRwWEVvckhYZHh1S1E9 giving up after 5 attemptsFor Terraform Enterprise v202205-1 or later:
Internal error: SIC-001
Failed to ingress slug: Failed to upload slug: failed uploading: PUT http://archivist.tfe:7675/v1/object/dmF1bHQ6djE6OFB4alY0RHRHdXhMelRuSkd0ZDk2dHRhU0hXTEc3SEFOY0hmeSt1RndFVnJ6QlpoQmswSHhmcXIwdDdHWUZDZHprUW5uYlNzak1QQ1N2NnNvUWZKaEREOE1aNysrcWNaTGQyaWYvNUdEQkdwbkwyN1pQZFUvdjc5ZWFnMEVBaVVZWVZPQVpoRW12WlRuN1ljWW1aSFlsUThOeTRuMDhZTDIybVgwM1dyb1BvMEltRmM1SUozc1RwS0xYcUhRVktlcDIxeUlTb1N1YWM5Sy9xUEUvMklVTGplVUZzK2loWUJINllUNWhQaHNjd05ReERIQWFjR0hXclFFcUl0bnVGcTJ2SUFwMFJzcFJpMVREbVRBSFRwWEVvckhYZHh1S1E9 giving up after 5 attempts
Solution
As a workaround for Gitlab, some customers have recreated the VCS connection through the TFE provider or the API. When creating a VCS connection through the API or the provider, it generates a personal access token and Terraform Enterprise can use this token in place of the Oauth token and the personal access token should not need to be refreshed since this token is generated by the user. It should remain valid until you invalidate it.
Creating new oauth clients for VCS can be automated using either the API that TFE provides, or via the TFE Terraform provider with the oauth_client resource.