AWS CloudHSM pkcs11: 0x6: CKR_FUNCTION_FAILED

This article describes a scenario where using AWS CloudHSM with Hashicorp Vault results in repeated errors 0x6: CKR_FUNCTION_FAILED.

These errors occur when using  Bring Your Own Key (BYOK) with the following Vault components: Vault HSM PKCS11 Auto-Unseal, Transit secrets engine or PKI secrets engine.

For example, when using transit secrets engine:

vault write transit/encrypt/oaep-kcs plaintext="$(echo 'abc123' | base64)" ;

The output shows:

Error writing data to transit/encrypt/oaep-kcs: Error making API request.

URL: PUT https://192.168.168.232:8200/v1/transit/encrypt/oaep-kcs
Code: 400. Errors:

* unable to encrypt the data: error encrypting plaintext: error initializing encrypt operation: pkcs11: 0x6: CKR_FUNCTION_FAILED

Inspecting the Vault Operational Logs, even at TRACE level, often provides no additional insight into the root cause.

The underlying issue is typically only visible in the HSM / PKCS#11 client logs with PKCS#11 debug level enabled. The CloudHSM client log can be viewed using the command:

tail -f /opt/cloudhsm/run/cloudhsm-pkcs11.log

An error like below will be shown in the log output:

... ERROR [207724] ThreadId(4) [cloudhsm_pkcs11::encryption::rsa_pkcs_oaep::error][][] Key 16278 does not meet the availability requirements - The key must be available on at least 2 HSMs before being used.

The key part of this message is:

"The key must be available on at least 2 HSMs before being used."

This indicates that the key does not meet the availability requirements within the CloudHSM cluster.

Resolution and Workaround

To avoid this issue consider one of the two possible solutions below.

1. Deploy AWS CloudHSM clusters with at least two HSM devices, rather than a single standalone HSM instance. Once the number of HSM instances in the cluster is increased and the key becomes available on multiple HSMs, previously failing algorithms and encryption requests typically begin working as expected.

2. Disable AWS key availability checks as part of the provisioning steps via the  --disable-key-availability-check with the CloudHSM Client SDK. Consult the AWS CloudHSM Client SDK documentation for for further details.

References

• AWS Documentation: What is AWS CloudHSM?

• AWS Documentation: AWS CloudHSM error seen during key availability check

• Vault Documentation: Transit secrets engine - Bring your own key (BYOK)

• Vault Documentation: HSM PKCS11 seal configuration

• Vault API: Transit secrets engine

• Vault API: PKI secrets engine