Vault Auditor Tool
If you are running at least Vault version 1.3, you can use the vault-auditor tool to extract metrics from the server audit device logs. These metrics are typically useful for metering and billing, but also helpful for getting critical entity and token counts for troubleshooting scenarios.
You can Download vault-auditor binaries for macOS and Linux, and you can use the tool wherever you have access to the audit device log files.
Note: vault-auditor currently supports only audit device logs from Vault versions 1.3 through 1.5.
If you are running Vault v1.9 or later, refer to the version-specific client count metrics details section to learn about key differences in the computation of client usage metrics between versions.
Here is an example interaction with vault-auditor, beginning with help output and concluding with an example where the tool is used to parse a hypothetical directory of audit devices logs.
Get help for vault-auditor with the --help flag.
$ vault-auditor --help
Usage: vault-auditor [--version] [--help] <command> [<args>]
Available commands are:
parse - Parse audit logs to find entity and token counts
version - Prints the vault-auditor version
Currently, the tool features just two sub-commands, parse and version; check the help for the parse sub-command to learn more about how it works.
$ vault-auditor parse --help
Usage: vault-auditor parse [options] <directory>
Parse one or more audit logs inside the given directory to count unique tokens.
$ vault-auditor parse /tmp/audit-logs
Options:
-prefix=<string> String that every line in the audit logs is prefixed with. This should only be necessary if the audit prefix contains a { character.
-start-date=<string> Only count log lines on/after this date. RFC3339, YYYY-MM-DD, YYYY-MM, and Unix epoch time formats are supported.
-end-date=<string> Only count log lines on/before this date. RFC3339, YYYY-MM-DD, YYYY-MM, and Unix epoch time formats are supported.
Note: Options are passed after the parse and the log file is passed at the end of the command. For example with date options:
$ vault-auditor parse -start-date="2022-03-16" -end-date="2022-03-20" /tmp/audit-logs
Output from the command would look like below:
Distinct Entities: 13 Non-Entity Tokens: 17 Total Clients: 30 Total files processed: 1 Date range: 2022-03-17T23:57:05Z - 2022-03-20T00:00:00Z
vault-auditor is also designed to be executed against a directory containing multiple Vault audit device log files. To execute this you need to pass in the full path to a directory containing one or more Vault audit device log files. In this example, the logs are contained in an audit-device-logs directory:
$ vault-auditor parse audit-logs
The parsing of logs will emit progress indicators and when successful, the output resembles this example:
vault-log-0 [=======================================] 100 % 0s
vault-log-1 [=======================================] 100 % 0s
vault-log-2 [=======================================] 100 % 0s
vault-log-3 [=======================================] 100 % 0s
vault-log-4 [=======================================] 100 % 0s
Distinct Entities: 7139
Non-Entity Tokens: 273
Total Clients: 7412
Total files processed: 5
Date range: 2020-10-02T08:36:29Z - 2020-10-07T15:57:50Z
There were a total of 5 logs processed from the audit-logs directory, with a total of 273 non-entity tokens identified, meaning tokens which were created directly from the Token auth method (such as root tokens). The tool also found 7139 distinct entities for which tokens were issued through a Vault auth method. The total clients count is a combination of distinct entities and non-entity tokens. Lastly, the date range represented by the parsed logs is shown.
To check the vault-auditor version, you can:
$ vault-auditor version
Output:
Vault Auditor v1.0.1
Limits of the auditor tool
The vault auditor tool was released along with Vault v1.6.0. Since that time, some improvements were made to the Vault client count computation logic in newer versions of Vault. These changes are reflected only in the usage metrics API and web UI, but are not available in the auditor tool.
This issue may arise if the logs are too large and cannot be read, or you are running an older version than Vault 1.6. We encourage you to upgrade to a newer version of Vault so you can use the API/UI to compute clients. After the upgrade, even if you have only limited historical data on clients (since the upgrade), that data could be extrapolated for future client prediction. Account teams need to look into this on a case-by-case basis.
Not all customers may be on a version greater than Vault version 1.6 that leverages the client count API to display clients via the UI. The auditor tool is available for customers running older Vault versions to compute client counts. The auditor tool does not contain client count computation logic updates (e.g., non-entity token computation logic made in Vault 1.9). In the future, HashiCorp will deprecate the auditor tool.
New updates to the client count logic (made in the API/UI starting Vault 1.8) are not reflected in the Vault auditor tool. HashiCorp do not plan to update the auditor tool to be consistent with the API/UI and will eventually deprecate it. Use this tool as a stop-gap until you upgrade to a newer version of Vault.