Introduction
Problem
When calling a Terraform Cloud/Enterprise API, a user may observe not all data is being returned.
Cause
By default, Terraform Cloud/Enterprise API will only return the first page with 20 items. If the API call returns more than 20 items, then only the first 20 will be returned.
Overview of possible solutions
Solutions:
-
Increase the page size by specifying page[size] query parameter
curl \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/vnd.api+json" \
https://app.terraform.io/api/v2/organizations/my-organization/workspaces?page%5Bsize%5D=100
2. Use a script to loop through all pages
PAGE_NUMBER=1
ENDPOINT="https://app.terraform.io/api/v2/organizations/my-organization/workspaces"
while true; do
RESPONSE=$(curl --header "Authorization: Bearer $TOKEN" --header "Content-Type: application/vnd.api+json" "$ENDPOINT?page%5Bnumber%5D=$PAGE_NUMBER")
echo $RESPONSE
# Check if there are more pages
NEXT_LINK=$(echo "$RESPONSE" | jq -r '.links.next')
if [ "$NEXT_LINK" == "null" ]; then
break
fi
# Increment the page number for the next iteration
PAGE_NUMBER=$((PAGE_NUMBER + 1))
done
Outcome:
After using one of the solutions above, the API should return all the data. If you encounter issues after trying both solutions, please reach out to HashiCorp Support for assistance.