Vault: Error initializing core: Failed to lock memory: cannot allocate memory – HashiCorp Help Center

Introduction

Problem

The Terraform Enterprise application fails to start with(container ID will differ):

Container tfe-base-startup failed: Container 5275bd7d6892a2dd400dab2e47f9529f4fe75d95a3ded95e16bb06d87d1b5e38 exited with non-zero exit status 1:

Inspecting the aforementioned container logs will display the following lines:

INFO: Vault token retrieval timeout not yet reached
ERROR: Operation timed out waiting for vault token

Checking the logs for the vault container with docker logs tfe-vault displays the following error:

Error initializing core: Failed to lock memory: cannot allocate memory

This usually means that the mlock syscall is not available.
Vault uses mlock to prevent memory from being swapped to
disk. This requires root privileges as well as a machine
that supports mlock. Please enable mlock on your system or
disable Vault from using it. To disable Vault from using it,
set the `disable_mlock` configuration option in your configuration
file.

The vault container will appear to restart several times until exiting completely.

Prerequisites

Terraform Enterprise v202205-1 or later

Cause

The following option is enabled in /etc/docker/daemon.json
```
{
    "no-new-privileges": true
}
```

This option is used to increase Docker Security and prevents container processes from gaining additional privileges, hence commands like su or sudo will not work causing the vault-server process to fail during startup as is now associated with the systemd Daemon (In previous versions the process ran as the root user)

systemd+ 564409 564352 0 13:50 ? 00:00:02 vault server -config /etc/vault

Solution:

First stop the Terraform Enterprise application via the Replicated Console or CLI
```
replicatedctl app stop -f
```

Remove the option from /etc/docker/daemon.json and reload the daemon configuration

sudo systemctl daemon-reload

Restart the Docker service
```
sudo systemctl restart docker
```

Confirm the Replicated service is ready and start the Terraform Enterprise application.

replicatedctl system status
{
   "Replicated": "ready",
   "Retraced": "ready"
}

replicatedctl app start

Outcome

The Terraform Enterprise application will now start successfully. Check the container logs for confirmation:

docker logs tfe-base-startup
..
INFO: Vault token retrieval timeout not yet reached
INFO: Decrypting token

docker logs tfe-vault
..
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 1.10.2
Storage Type postgresql
Cluster Name vault-cluster-3e9a1e9d
Cluster ID e37de6c2-6ec2-8d37-c8fb-f8b4de7227db
HA Enabled false
+ Creating vault token for use by services
+ Successfully created vault token
+ renewing vault token

Additional Information