How to verify a VCS webhook is being received by Terraform Enterprise

Introduction

There is the option of connecting a workspace with a Version Control System as described here in our documentation. 

It will use webhooks to alert Terraform Enterprise of the following

• When someone adds new commits to a branch
• When someone submits a pull request/merge request to a branch

This should trigger a run in Terraform Enterprise for the workspace. If there is no run being started there is an issue somewhere in the communication. 

Use the following steps to verify the webhook message is received by Terraform Enterprise

Expected Outcome

Verification of the webhook message being received or not

Prerequisites 

• Have a Terraform Enterprise admin token
• Have the ability to login to the Terraform Enterprise server using ssh
• Installed a tool called jq for visibility of the API json output. Download here

Procedure

We will investigate a single workspace that has issues of not executing a run after a change on the VCS repository. 

• Get the webhook-url for VCS of the Terraform Enterprise workspace with the following

export TOKEN=<your token>
export hostname=<your_TFE_fqdn>
export organization=<your_organization_name>
export workspace=<workspace_name>

curl \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/vnd.api+json" \
https://${hostname}/api/v2/organizations/${organization}/workspaces/${workspace} | jq '.data.attributes."vcs-repo"."webhook-url"'

• result is something like the following

https://<your_TFE_fqdn>/webhooks/vcs/ed734c1b-0c4f-453b-902f-ecf0b41cf715

• Verify this is the same on the webhook on the repository side. Example of github
  Github -> Repository -> Settings -> Webhooks

• Create a new commit on the repository that would trigger a new webhook to Terraform Enterprise
  
  
• Verify this webhook is being received as the following entry should be found in the nginx logs

For the replicated version of TFE

# Replace this with the unique id of the webhook
# docker logs tfe-nginx 2>&1 | grep <webhook_id>
docker logs tfe-nginx 2>&1 | grep ed734c1b-0c4f-453b-902f-ecf0b41cf715

docker exec terraform-enterprise-tfe-1 grep ed734c1b-0c4f-453b-902f-ecf0b41cf715  /var/log/terraform-enterprise/nginx.log

140.82.115.148 - - [06/Sep/2023:12:33:58 +0000] "POST /webhooks/vcs/ed734c1b-0c4f-453b-902f-ecf0b41cf715 HTTP/1.1" 200 12 "-" "GitHub-Hookshot/435afa9"

Result

If you don't get a result in the nginx log then please verify the following

• Is there something blocking it on the network like a firewall/WAF/Proxy
• Is the certificate used on Terraform Enterprise for TLS trusted by the VCS environment. The following should be executed on your VCS environment and return an OK result. 
•  

curl https://<your_TFE_fqdn>/_health_check

If you do get a result in the nginx logs then please create a ticket with HashiCorp support where we can help you further. 

• Share the results from this KB article as an attachment to the ticket
• Please share a Terraform Support Bundle as described here

Additional Information

• Another article to troubleshoot the VCS webhooks can be found here