Audit and Operational Log Details

Avatar
Brian Shumate
  • Updated
This guide provides an overview of the formats and contents of the audit and operational log outputs in HashiCorp Vault. It includes examples and explanations of the log entries to help you understand the information they provide. Please note that this guide is not an exhaustive reference for all possible log messages.

Vault Audit Log Details

Vault CLI with Token Example

The following are some example audit log entries which demonstrates the request and response logging generated when a user interacts with Vault CLI.

Earlier a token was generated with some specific policies and metadata:

 

$ vault token create \
-policy=sudo \
-policy=surf-admin \
-metadata=loglevel=raw \
-metadata=remote=false \
-metadata=surf=moderate

Key                    Value
---                    -----
token                  hvs.CAESIJT2KM9n2KJqg_E267EHJSY0c4NWtTRnBsNWhzUDkQZQ
token_accessor         bwMAYYBByYBMYs6YbWggfXdA
token_duration         768h
token_renewable        true
token_policies         ["default" "sudo" "surf-admin"]
identity_policies      []
policies               ["default" "sudo" "surf-admin"]
token_meta_loglevel    raw
token_meta_remote      false
token_meta_surf        moderate

 

Now a user is going to login to Vault with the token:

$ vault login hvs.CAESIJT2KM9n2KJqg_E267EHJSY0c4NWtTRnBsNWhzUDkQZQ

Request Log

The above login produces this output in the Vault audit log as the audit log request object:

{
 "time": "2023-04-17T19:48:58.238208382Z",
 "type": "request",
 "auth": {
 "client_token": "hmac-sha256:114e72599d41f7d14c7fc2ba495757195e98d0947405421f7b3be37b94e7f363",
 "accessor": "hmac-sha256:23e9a5bc2a3538252c1d1e8d686267a3ff81730db0da31530863a2760d6771c8",
 "display_name": "token",
 "policies": [
   "default",
   "sudo",
   "surf-admin"
 ],
 "token_policies": [
   "default",
   "sudo",
   "surf-admin"
 ],
 "policy_results": {
   "allowed": true,
   "granting_policies": [
     {
       "name": "default",
       "namespace_id": "root",
       "type": "acl"
     }
   ]
 },
 "metadata": {
   "loglevel": "raw",
   "remote": "false",
   "surf": "moderate"
 },
 "token_type": "service",
 "token_ttl": 2764800,
 "token_issue_time": "2023-04-17T12:47:22-07:00"
 },

 "request": {
 "id": "19073d8c-7567-7ee9-1144-c8ce601ec79d",
 "client_id": "PWa2+llmKWwgQ1Fjaxmh5/v+qc+EntehUSliX0+67DY=",
 "operation": "read",
 "mount_type": "token",
 "client_token": "hmac-sha256:792e3d0261eb8c9ce67afe2ff675da2d8e88703cf6a4d66307ac2117dbdd0eaa",
 "client_token_accessor": "hmac-sha256:23e9a5bc2a3538252c1d1e8d686267a3ff81730db0da31530863a2760d6771c8",
 "namespace": {
   "id": "root"
 },
 "path": "auth/token/lookup-self",
 "remote_address": "10.211.55.12",
 "remote_port": 37312
 }
}

 

The audit log request object fields are described as follows:

  • time: RFC3339 timestamp for the request
  • type: Log entry type; there are currently just two types, request and response and in this case it is request.
  • auth: Authentication details, including:
    • client_token: This is an HMAC of the client’s token ID that can be compared as described in the /sys/audit-hash API documentation
    • accessor: This is an HMAC of the client token accessor that can be compared as described in the /sys/audit-hash API documentation
    • display_name: This is the display name set by the auth method role or explicitly at secret creation time; this is often useful for determining from which auth method mount point or user this request could be related to
    • policies: This will contain a list of policies associated with the client_token
    • policy_results: Contains the set of policies that grant the permissions needed for the request. It is also a more explicit way to detect a request failed due to being unauthorized
    • metadata: This will contain a list of metadata key/value pairs associated with the client_token
  • request: This is the request object, containing the following:
    • id: This is the unique request identifier
    • operation: This is the type of operation which corresponds to path capabilities and is expected to be one of: create, read, update, delete, or list
    • mount_type: Authentication method used for a particular request.
    • client_token: This is an HMAC of the client’s token ID that can be compared as described in the /sys/audit-hash API documentation
    • client_token_accessor: This is an HMAC of the client token accessor that can be compared as described in the /sys/audit-hash API documentation
    • namespace: Namespace in which the request was made
    • path: The requested Vault path for operation
    • remote_address: The IP address of the client making the request
      remote_port: The port used by the client

Audit Response Log

The response to this login is:

{
 "time": "2023-04-17T19:48:58.238890678Z",
 "type": "response",
 "auth": {
   "client_token": "hmac-sha256:114e72599d41f7d14c7fc2ba495757195e98d0947405421f7b3be37b94e7f363",
   "accessor": "hmac-sha256:23e9a5bc2a3538252c1d1e8d686267a3ff81730db0da31530863a2760d6771c8",
   "display_name": "token",
   "policies": [
     "default",
     "sudo",
     "surf-admin"
   ],
 "token_policies": [
   "default",
   "sudo",
   "surf-admin"
 ],
 "policy_results": {
   "allowed": true,
   "granting_policies": [
     {
       "name": "default",
       "namespace_id": "root",
       "type": "acl"
     }
   ]
 },
 "metadata": {
   "loglevel": "raw",
   "remote": "false",
   "surf": "moderate"
 },
 "token_type": "service",
 "token_ttl": 2764800,
 "token_issue_time": "2023-04-17T12:47:22-07:00"
 },

 "request": {
 "id": "19073d8c-7567-7ee9-1144-c8ce601ec79d",
 "client_id": "PWa2+llmKWwgQ1Fjaxmh5/v+qc+EntehUSliX0+67DY=",
 "operation": "read",
 "mount_type": "token",
 "mount_accessor": "auth_token_83c95005",
 "client_token": "hmac-sha256:792e3d0261eb8c9ce67afe2ff675da2d8e88703cf6a4d66307ac2117dbdd0eaa",
 "client_token_accessor": "hmac-sha256:23e9a5bc2a3538252c1d1e8d686267a3ff81730db0da31530863a2760d6771c8",
 "namespace": {
   "id": "root"
 },
 "path": "auth/token/lookup-self",
 "remote_address": "10.211.55.12",
 "remote_port": 37312
 },
 "response": {
   "mount_type": "token",
   "mount_accessor": "auth_token_83c95005",
   "data": {
     "accessor": "hmac-sha256:23e9a5bc2a3538252c1d1e8d686267a3ff81730db0da31530863a2760d6771c8",
     "creation_time": 1681760842,
     "creation_ttl": 2764800,
     "display_name": "hmac-sha256:c2d7ac8eb94123986e52025e81b0f848a4fd68978b8a22721d5a39688728c0dc",
     "entity_id": "hmac-sha256:c86ad62644b04bec20c916705a543e41c17be22d44a6c98d4c280f49b6553e47",
     "expire_time": "2023-05-19T12:47:22.807439692-07:00",
     "explicit_max_ttl": 0,
     "id": "hmac-sha256:114e72599d41f7d14c7fc2ba495757195e98d0947405421f7b3be37b94e7f363",
     "issue_time": "2023-04-17T12:47:22.807444484-07:00",
     "meta": {
       "loglevel": "hmac-sha256:041d1197ed9338cec7b6be78c46cb5b9fae01d27dfa74e348c033ad05dd9bbda",
       "remote": "hmac-sha256:8b8e3faa7570f26f942262a98d493c9673116b87fb50126d991ac76d59384cda",
       "surf": "hmac-sha256:de55d6acb922d16ddf618233d74c01fd02e3a61d1b9b413a582abbbe96adaa5e"
     },
 "num_uses": 0,
 "orphan": false,
 "path": "hmac-sha256:06ef4b78d83d0627a3e0c0e56273a9fed76c42802ab4db4828bc7fbf92461060",
 "policies": [
   "hmac-sha256:bd9c45e381493f5411415e3ef1f5b7979b3806d7c7745b6832e665c795c8a0ef",
   "hmac-sha256:4e4dff4d0d15894a4d005179d3c4a55090bc70d8106eee656cca21eaf8505ee7",
   "hmac-sha256:fe840c80145b419e61e13b62ea5cdcaedb35358c054c83660d9f153894636a7b"
 ],
 "renewable": true,
 "ttl": 2764704,
 "type": "hmac-sha256:56c7d28882a11e98a87e0f70fb7a7a95eea783827cdf293fa4b907fa69bada47"
  }
 }
}

The response objects contains many of the same fields found in the request object and so those are not covered here (please see details the request section above for any unclear fields). There are some additional token specific fields which can be expected in response output depending on the operations and auth methods in question. Those fields are detailed here:

  • creation_time RFC3339 format timestamp of the token’s creation
  • creation_ttl: Token creation TTL in seconds
  • expire_time: RFC3339 format timestamp representing the moment this token will expire
  • explicit_max_ttl: Explicit token maximum TTL value as seconds (‘0’ when not set)
  • issue_time: RFC3339 format timestamp
  • num_uses: It represents the number of times the token associated with the request can be used before it is considered invalid. In the context of the audit log, a "num_uses" value of 0 means that the token can be used an unlimited number of times.
  • orphan: Boolean value representing whether the token is an orphan
  • renewable: Boolean value representing whether the token is renewable
  • ttl: It represents the Time-To-Live (TTL) of the token associated with the request, expressed as an integer in seconds.

BONUS

At the beginning of this guide, there was mention of creating the token used in the audit log request and response log examples. If you are curious about the audit logs of the actual token creation request, well here they are!

Request:

{
 "time": "2023-04-17T19:47:22.786517207Z",
 "type": "request",
 "auth": {
   "client_token": "hmac-sha256:edcfcf1df4278d0ce25c6da75a99df81182afc8fbcb04f7ceba9fef8863431c8",
   "accessor": "hmac-sha256:e26289be3c4792e900198de7c7b6c24ba8f299064266ccdd58233f50f0556eb1",
   "display_name": "token",
   "policies": [
     "root"
   ],
 "token_policies": [
   "root"
 ],
 "policy_results": {
   "allowed": true,
   "granting_policies": [
     {
       "name": "root",
       "namespace_id": "root",
       "type": "acl"
     }
   ]
 },
 "token_type": "service",
 "token_issue_time": "2023-03-03T22:11:34-08:00"
 },
 "request": {
   "id": "07747fab-51fd-5ada-dd8f-52bcde0d1b9d",
   "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
   "operation": "update",
   "mount_type": "token",
   "mount_accessor": "auth_token_83c95005",
   "client_token": "hmac-sha256:edcfcf1df4278d0ce25c6da75a99df81182afc8fbcb04f7ceba9fef8863431c8",
   "client_token_accessor": "hmac-sha256:e26289be3c4792e900198de7c7b6c24ba8f299064266ccdd58233f50f0556eb1",
   "namespace": {
     "id": "root"
   },
   "path": "auth/token/create",
   "data": {
     "display_name": "hmac-sha256:c86ad62644b04bec20c916705a543e41c17be22d44a6c98d4c280f49b6553e47",
     "entity_alias": "hmac-sha256:c86ad62644b04bec20c916705a543e41c17be22d44a6c98d4c280f49b6553e47",
     "explicit_max_ttl": "hmac-sha256:eb9f7899e5b5591c2d76d2ba840b3cc005dd5046b978bd66a3e84d259fb06a90",
     "meta": {
       "loglevel": "hmac-sha256:041d1197ed9338cec7b6be78c46cb5b9fae01d27dfa74e348c033ad05dd9bbda",
       "remote": "hmac-sha256:8b8e3faa7570f26f942262a98d493c9673116b87fb50126d991ac76d59384cda",
       "surf": "hmac-sha256:de55d6acb922d16ddf618233d74c01fd02e3a61d1b9b413a582abbbe96adaa5e"
     },
     "num_uses": "hmac-sha256:fea13c7655a5428a9bbf6f9e25d455e2c79bfe3eee20d65b01c328c6d62d0a69",
     "period": "hmac-sha256:eb9f7899e5b5591c2d76d2ba840b3cc005dd5046b978bd66a3e84d259fb06a90",
     "policies": [
       "hmac-sha256:4e4dff4d0d15894a4d005179d3c4a55090bc70d8106eee656cca21eaf8505ee7",
       "hmac-sha256:fe840c80145b419e61e13b62ea5cdcaedb35358c054c83660d9f153894636a7b"
     ],
   "renewable": true,
   "ttl": "hmac-sha256:eb9f7899e5b5591c2d76d2ba840b3cc005dd5046b978bd66a3e84d259fb06a90",
   "type": "hmac-sha256:56c7d28882a11e98a87e0f70fb7a7a95eea783827cdf293fa4b907fa69bada47"
 },
 "remote_address": "10.211.55.12",
 "remote_port": 37310
 }
}

and the response:

{
 "time": "2023-04-17T19:47:22.815397333Z",
 "type": "response",
 "auth": {
   "client_token": "hmac-sha256:edcfcf1df4278d0ce25c6da75a99df81182afc8fbcb04f7ceba9fef8863431c8",
   "accessor": "hmac-sha256:e26289be3c4792e900198de7c7b6c24ba8f299064266ccdd58233f50f0556eb1",
   "display_name": "token",
   "policies": [
     "root"
   ],
   "token_policies": [
     "root"
   ],
   "policy_results": {
   "allowed": true,
   "granting_policies": [
     {
       "name": "root",
       "namespace_id": "root",
       "type": "acl"
     }
   ]
 },
 "token_type": "service",
 "token_issue_time": "2023-03-03T22:11:34-08:00"
 },
 "request": {
   "id": "07747fab-51fd-5ada-dd8f-52bcde0d1b9d",
   "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
   "operation": "update",
   "mount_type": "token",
   "mount_accessor": "auth_token_83c95005",
   "client_token": "hmac-sha256:edcfcf1df4278d0ce25c6da75a99df81182afc8fbcb04f7ceba9fef8863431c8",
   "client_token_accessor": "hmac-sha256:e26289be3c4792e900198de7c7b6c24ba8f299064266ccdd58233f50f0556eb1",
   "namespace": {
     "id": "root"
   },
   "path": "auth/token/create",
   "data": {
     "display_name": "hmac-sha256:c86ad62644b04bec20c916705a543e41c17be22d44a6c98d4c280f49b6553e47",
     "entity_alias": "hmac-sha256:c86ad62644b04bec20c916705a543e41c17be22d44a6c98d4c280f49b6553e47",
     "explicit_max_ttl": "hmac-sha256:eb9f7899e5b5591c2d76d2ba840b3cc005dd5046b978bd66a3e84d259fb06a90",
     "meta": {
       "loglevel": "hmac-sha256:041d1197ed9338cec7b6be78c46cb5b9fae01d27dfa74e348c033ad05dd9bbda",
       "remote": "hmac-sha256:8b8e3faa7570f26f942262a98d493c9673116b87fb50126d991ac76d59384cda",
       "surf": "hmac-sha256:de55d6acb922d16ddf618233d74c01fd02e3a61d1b9b413a582abbbe96adaa5e"
     },
   "num_uses": "hmac-sha256:fea13c7655a5428a9bbf6f9e25d455e2c79bfe3eee20d65b01c328c6d62d0a69",
   "period": "hmac-sha256:eb9f7899e5b5591c2d76d2ba840b3cc005dd5046b978bd66a3e84d259fb06a90",
   "policies": [
     "hmac-sha256:4e4dff4d0d15894a4d005179d3c4a55090bc70d8106eee656cca21eaf8505ee7",
     "hmac-sha256:fe840c80145b419e61e13b62ea5cdcaedb35358c054c83660d9f153894636a7b"
   ],
   "renewable": true,
   "ttl": "hmac-sha256:eb9f7899e5b5591c2d76d2ba840b3cc005dd5046b978bd66a3e84d259fb06a90",
   "type": "hmac-sha256:56c7d28882a11e98a87e0f70fb7a7a95eea783827cdf293fa4b907fa69bada47"
   },
   "remote_address": "10.211.55.12",
   "remote_port": 37310
 },
 "response": {
   "auth": {
     "client_token": "hmac-sha256:114e72599d41f7d14c7fc2ba495757195e98d0947405421f7b3be37b94e7f363",
     "accessor": "hmac-sha256:23e9a5bc2a3538252c1d1e8d686267a3ff81730db0da31530863a2760d6771c8",
     "display_name": "token",
     "policies": [
       "default",
       "sudo",
       "surf-admin"
     ],
   "token_policies": [
     "default",
     "sudo",
     "surf-admin"
   ],
   "metadata": {
     "loglevel": "raw",
     "remote": "false",
     "surf": "moderate"
   },
   "token_type": "service",
   "token_ttl": 2764800
   },
   "mount_type": "token",
   "mount_accessor": "auth_token_83c95005"
  }
}

 

Vault Operational Log Details

The operational log is derived from an internal logging package such as the go-hclog package, and output is in a single line format which follows a format similar to many popular server tools.

Example Vault operational log entries:

2023-04-28T20:21:32.976Z [INFO] core: security barrier not initialized
2023-04-28T20:21:32.976Z [INFO] core: security barrier initialized: shares=5 threshold=3

The log entries are whitespace separated, and detailed as follows:

  • Timestamp: 2023-04-28T20:26:38.626Z
  • Level: [INFO ]
  • Component: core:
  • Message: security barrier not initialized

These log message fields are further described as follows:

  • Timestamp: RFC3339 timestamp for the log entry
  • Level: Configurable logging levels for Vault’s operational logs, in order from most to least verbose:
    • trace: provides extreme detail from all Vault components including storage backends, auth methods, secrets engines and Enterprise features, such as HSM interaction and replication
    • debug Lower level messages from Vault components which are helpful for QA/test/staging environments, and troubleshooting but generally too verbose for production
    • info typical production level logging of nominal system information messages
    • warn a warning message signifies a problem that does not necessarily impact production operations, but should be further examined and resolved - warnings should be alerted on in monitoring solutions as a heads up to operators.
    • err an error typically signifies conditions which impact operation of Vault and should be immediately investigated to resolution - errors should be alerted on in monitoring solutions as a heads up to operators.
  • Component: The Vault component that is the source of the log message — consists of a range of possible values:
    • audit: Messages related to audit device functionality
    • core: Messages related to Vault core functionality
    • expiration: Messages related to Expiration Manager functionality
    • identity: Messages related to Identity Manager functionality
    • rollback: Messages related to Rollback Manager functionality
    • secrets.TYPE.ID: Messages related to secrets engines of TYPE with the identity of ID
    • storage.cache: Messages related to the storage cache
    • storage.TYPE: Messages related to the storage backend of type TYPE (i.e. storage.consul)
    • replication: Messages related to replication (primary/secondary) functionality
  • Message: Log message body

Note that each Vault component can emit specific log detail and this guide does not attempt to provide an exhaustive reference of all log messages. Generally speaking and for recent Vault versions, logged events can be found throughout the Vault source code with these strings:

  • logger.Trace
  • logger.Debug
  • logger.Info
  • logger.Warn
  • logger.Error

Resources

  1. Audit Devices
  2. vault/audit/format.go
  3. /sys/audit-hash
  4. Capabilities
  5. go-hclog package
  6. -log-level

Articles in this section

See more

Related articles