Introduction

When upgrading from an older version of Vault to a newer one, an error may be encountered an issue where preventing Vault from starting, the operational logs may show these errors:

2022-12-05T13:52:35.530Z [ERROR] core: mount entry associated with pending removal builtin: name=app-id path=app-id/ status="pending removal" type=auth
2022-12-05T13:52:35.530Z [ERROR] core: shutting down core: error="could not mount \"app-id\": mount entry associated with pending removal builtin"

This is an error that can occur when soon-to-be removed features are enabled when upgrading your version of Vault.

Explanation

This error is because of a deprecated feature in your Vault. There are multiple phases of deprecation, which you can read about here. The four phases of deprecation are: Supported, Deprecated, Pending Removal, andRemoved. The app-id auth method is in the Pending Removal phase.

Pending Removal:This status reflects a feature which has been officially deprecated in this release of Vault. This is the first phase in the process that fundamentally alters the behavior of Vault. The effects are two-fold:

1. After an upgrade, any existing Pending Removal feature (builtin auth/secrets plugins enabled via CLI or API prior to upgrade) will log Error-level messages to the Vault log and cause an immediate shutdown of the Vault core.
2. Any new Pending Removal will fail and log Error-level messages to the Vault log and CLI/API.

In version 1.12.0, there was a new environment variable introduced called VAULT_ALLOW_PENDING_REMOVAL_MOUNTS. This environment variable allows Vault to be started with builtin engines which have the Pending Removal deprecation state. This is a temporary stopgap in place in order to perform an upgrade and disable these engines. Once these engines are marked Removed (in the next major release of Vault), the environment variable will no longer work and a downgrade must be performed in order to remove the offending engines. This environment variable effectively allows all Pending Removal features to be treated as Deprecated.

Once this environment variable is set, Vault will start. You must then work to migrate the deprecated feature to an alternative.