Configure Vault pkcs#11 provider with Oracle Database Transparent Data Encryption

Avatar
Nestor Reyes
  • Updated

Purpose

The purpose of this document is to provide an end to end configuration example on how to configure the Vault pkcs#11 provider with Oracle Database Transparent Data Encryption.

Note:  Values such as database names, passwords, and usernames are for demonstration purposes only and should not be used in a production environment. Please update the steps to reflect your configuration. 

High level steps are currently documented in the Vault Enterprise Documentation. 

Requirements

To set up Oracle TDE backed by Vault, the following are required:

  • A Database running Oracle 19 Enterprise Edition (Oracle XE can be used for Demo purposes, but is not supported in production).
  • A Vault Enterprise v1.11+ server with Advanced Data Protection for KMIP support.
  • TCP port 5696 open between Vault and Oracle Database. 
  • libvault-pkcs11.so downloaded from releases.hashicorp.com for the operating system running the Oracle database.

Procedure

  1. Download the appropriate vault pkcs11 provider and extract.

  1. Move the libvault-pkcs11.so to /opt/oracle/extapi/64/hsm
    The permissions should be owned by the oracle database user.

  2. Set up Vault KMIP secrets engine per documentation and copy the ca.pem (certificate authority) and cert.pem (client certificate) to the Oracle database server.

Note: The server_hostnames and server_ips config are for configuring the TLS certificates and will be added to the Subject Alternative Name Extension for the KMIP server.

  1. Create the vault-pkcs11.hcl file in /etc per documentation. The values should contain the scope, tls_cert_path, ca_path as well as the IP address to the Vault kmip server/port. Also, be sure to update the locations of the ca.pem and cert.pemwhich were copied in step 3.
  1. On the Oracle Database, check PDB'S are open.
    Example: My PDB is VAULTPDB and is in Read Write State. 
SQL> show pdbs;  
  CON_ID    CON_NAME                       OPEN MODE  RESTRICTED
----------  ------------------------------ ---------- ----------
         2  PDB$SEED                       READ ONLY  NO
         3  VAULTPDB                       READ WRITE NO
  1. Set WALLET_ROOT and TDE_CONFIGURATION parameters on the Oracle database.

    Note: The directory is a root wallet directory and is only required for the purposes of setting TDE_CONFIGURATION. More on these parameters are documented in the Oracle Documentation.

SQL> alter system set wallet_root='/opt/oracle/admin/ORCLCDB/wallet' scope=spfile;
SQL> shutdown immediate;
SQL> startup;
SQL> alter system set TDE_CONFIGURATION="KEYSTORE_CONFIGURATION=HSM" SCOPE=both;


Note: To check this, you can query V$PARAMETER

SQL> SELECT name, value from V$PARAMETER WHERE NAME IN ('wallet_root','tde_configuration');

NAME                           VALUE
------------------------------ --------------------------------------------------
wallet_root                    /opt/oracle/admin/ORCLCDB/wallet
tde_configuration              KEYSTORE_CONFIGURATION=HSM
  1. Check the wallet and encryption keys from the V$ENCRYPTION_WALLET and V$ENCRYPTION_KEYS views. 

You should have a closed HSM wallet and no keys.

Note the con_id field maps to the show pdbs; output in Step 5.

SQL> SELECT wrl_type, status, wallet_type, keystore_mode, con_id FROM V$ENCRYPTION_WALLET;

WRL_TYPE      STATUS                         WALLET_TYPE          KEYSTORE     CON_ID
------------- ------------------------------ -------------------- -------- ----------
HSM           CLOSED                         UNKNOWN              NONE              1
HSM           CLOSED                         UNKNOWN              UNITED            2
HSM           CLOSED                         UNKNOWN              UNITED            3


SQL> SELECT KEY_ID, tag, ACTIVATING_PDBID, CON_ID FROM V$ENCRYPTION_KEYS;

no rows selected
  1. Open the keystore with a password for all containers. Be sure you are in the root container.

    Note: The IDENTIFIED BY "password" is for demo purposes only. Choose a password that will be used to open the wallet. 

SQL>  ALTER SESSION SET CONTAINER = CDB$ROOT;
Session altered.

SQL> ADMINISTER KEY MANAGEMENT SET KEYSTORE OPEN IDENTIFIED BY "password" CONTAINER = ALL;
keystore altered.
  1. Check your wallet. It should be set toOPEN_NO_MASTER_KEY for PDB 3.
SQL> SELECT wrl_type, status, wallet_type, keystore_mode, con_id FROM V$ENCRYPTION_WALLET;

WRL_TYPE       STATUS                         WALLET_TYPE          KEYSTORE     CON_ID
------------- ------------------------------ -------------------- -------- ----------
HSM           OPEN                           HSM                  NONE              1
HSM           OPEN                           HSM                  UNITED            2
HSM           OPEN_NO_MASTER_KEY             HSM                  UNITED            3
  1. Set an encryption key.

Note: This only has to be done once. If you run more than once, you are effectivly executing a re-key.

Note: If you see an ORA-46665 error, its fine. This is likely due to the inability to rekey the PDB$SEED.

SQL> ADMINISTER KEY MANAGEMENT SET ENCRYPTION KEY USING TAG 'default' IDENTIFIED BY "password" CONTAINER = ALL;
ADMINISTER KEY MANAGEMENT SET ENCRYPTION KEY USING TAG 'default' IDENTIFIED BY "password" CONTAINER = ALL
*
ERROR at line 1:
ORA-46665: master keys not activated for all PDBs during REKEY
  1. Check the wallet. It should be open on PDB 3.
SQL> SELECT wrl_type, status, wallet_type, keystore_mode, con_id FROM V$ENCRYPTION_WALLET;

WRL_TYPE       STATUS                         WALLET_TYPE          KEYSTORE     CON_ID
-------------- ------------------------------ -------------------- -------- ----------
HSM            OPEN                           HSM                  NONE              1
HSM            OPEN                           HSM                  UNITED            2
HSM            OPEN                           HSM                  UNITED            3
  1. Check the encryption keys. You should now have an encryption key for your PDB with a tag of default.
SQL> SELECT KEY_ID, tag, ACTIVATING_PDBID, CON_ID FROM V$ENCRYPTION_KEYS;

KEY_ID                              TAG             ACTIVATING_PDBID     CON_ID
----------------------------------- --------------- ---------------- ----------
063F8039B9906F4F57BFD2E4C3052C600E  default                        0          0
062EE7523508ED4F7CBFAA27EF03696F78  default                        1          1
  1. These keys are stored in Vault. Shutting down the database or Vault will close the wallet.

Example when Vault is shut down and wallet closed:

SQL> SELECT KEY_ID, tag, ACTIVATING_PDBID, CON_ID FROM V$ENCRYPTION_KEYS;
SELECT KEY_ID, tag, ACTIVATING_PDBID, CON_ID FROM V$ENCRYPTION_KEYS
*
ERROR at line 1:
ORA-28407: Hardware Security Module failed with PKCS#11 error CKR_DEVICE_ERROR(48)
  1. Once Vault is back up and running, you should see that the wallet is closed and no encryption keys.
SQL> SELECT wrl_type, status, wallet_type, keystore_mode, con_id FROM V$ENCRYPTION_WALLET;

WRL_TYPE       STATUS                         WALLET_TYPE          KEYSTORE     CON_ID
-------------- ------------------------------ -------------------- -------- ----------
HSM            CLOSED                         UNKNOWN              NONE              1
HSM            CLOSED                         UNKNOWN              UNITED            2
HSM            CLOSED                         UNKNOWN              UNITED            3

SQL>  SELECT KEY_ID, tag, ACTIVATING_PDBID, CON_ID FROM V$ENCRYPTION_KEYS;

no rows selected
  1. Open the wallet back up and you should now see encryption keys.
SQL> ADMINISTER KEY MANAGEMENT SET KEYSTORE OPEN IDENTIFIED BY "1234" CONTAINER = ALL;

keystore altered.

SQL> SELECT wrl_type, status, wallet_type, keystore_mode, con_id FROM V$ENCRYPTION_WALLET;

WRL_TYPE       STATUS                         WALLET_TYPE          KEYSTORE     CON_ID
-------------- ------------------------------ -------------------- -------- ----------
HSM            OPEN                           HSM                  NONE              1
HSM            OPEN                           HSM                  UNITED            2
HSM            OPEN                           HSM                  UNITED            3

SQL> SELECT KEY_ID, tag, ACTIVATING_PDBID, CON_ID FROM V$ENCRYPTION_KEYS;

KEY_ID                                   TAG              ACTIVATING_PDBID     CON_ID
---------------------------------------- ---------------- ---------------- ----------
063F8039B9906F4F57BFD2E4C3052C600E       default                         3          3
062EE7523508ED4F7CBFAA27EF03696F78       default                         1          1
  1. Test creating a table in a PDB.
SQL> alter session set container=VAULTPDB;

Session altered.

SQL> connect vault/vaultpasswd@VAULTPDB;
Connected.
SQL> CREATE TABLE test_tde (encrypted_column CHAR(32) ENCRYPT);

Table created.

SQL> describe test_tde;
 Name               Null?     Type
 -----------------  --------  ---------------------------
 ENCRYPTED_COLUMN             CHAR(32) ENCRYPT
  1. This confirms the integration with Vault is working properly.

Note: Anytime the database is shut down or Vault is shutdown, the wallet will have to be re-opened. 

Troubleshooting Tips

  • If you are not able to open the wallet, this could be client certificate related or connection related.
    • Please check using the client cert with openssl back to your Vault KMIP server/port.

  Example from the Oracle Database Server. 

openssl s_client -connect <Vault IP>:5696 -cert cert.pem -CAfile ca.pem

  • If you are not able to connect, then the issue may be with the TLS certs generated on the KMIP server. You may have to add any necessary IP's or hostnames in the Vault KMIP config as mentioned in Step 3.

  • For additional Logging on the Vault PKCS#11 library, set the following Environment Variables on the Oracle Database Server.

    VAULT_LOG_LEVEL
    VAULT_FILE

  • The Oracle Database alert log should be able to provide additional information regarding any returned errors from the Oracle Database Side. The location of the Alert log may vary, so please check with your Oracle DBA. 

Articles in this section

See more

Related articles