Problem
After a new installation or an update of Terraform Enterprise on a recent Amazon Linux 2 AMI (from January 2024 or later), the Replicated service fails to start.
When you inspect the replicated service logs, they show pthread_create failed: Operation not permitted errors.
To view the logs, run the following command:
# journalctl -u replicated
The log output shows the following errors:
runtime/cgo: pthread_create failed: Operation not permitted SIGABRT: abort
Prerequisites
- Terraform Enterprise installed on Amazon Linux 2 (AMI dated on or after January 1, 2024).
- Root or
sudoaccess to the instance.
Cause
Recent security updates in Amazon Linux 2 have introduced a more restrictive default seccomp profile for Docker. The libseccomp library limits the system calls (syscalls) a process can make to enhance security.
The hardened profile sets certain required syscalls to SCMP_ACT_ERRNO, which causes the kernel to deny the request with a Permission Denied error. This prevents some Terraform Enterprise services from starting correctly.
Solution
To resolve this issue, you must modify the Docker seccomp profile to allow the necessary syscalls and then restart the services.
- Follow the initial steps in the Docker Engine using a modified libseccomp profile documentation to create the custom seccomp profile file. This involves creating and populating the
/etc/docker/seccomp.jsonfile. - Open the
/etc/docker/seccomp.jsonfile in a text editor and locate thedefaultActionkey. -
Change the value from
"SCMP_ACT_ERRNO"to"SCMP_ACT_ALLOW". This change explicitly permits the required syscalls.Before:
"defaultAction": "SCMP_ACT_ERRNO",
After:
"defaultAction": "SCMP_ACT_ALLOW",
- Save the changes to the
/etc/docker/seccomp.jsonfile. -
Stop the Replicated services.
# systemctl stop replicated replicated-operator replicated-ui
-
Restart the Docker service to apply the new
seccompprofile.# systemctl restart docker
-
Start the Replicated services again.
# systemctl start replicated replicated-ui replicated-operator
Outcome
After you apply the modified seccomp profile and restart the services, Terraform Enterprise should start successfully.