Terraform Enterprise Error: "string exceeds maximum allowed size"

Introduction

This article provides a solution for an error in Terraform Enterprise where run logs exceed the maximum allowed size for a key in the Redis cache.

Problem

When running a plan or apply in a Terraform Enterprise workspace with a very large number of resources, the operation may fail with one of the following errors related to Redis cache limits.

[ERROR] http.stream-upload: failed writing key to cache store { err="ERR string exceeds maximum allowed size (512MB)" obj.key="terraform/logs/plan-ID" ... }

[ERROR] http.stream-download: failed copying data to cache store { err="ERR string exceeds maximum allowed size (512MB)" obj.key="terraform/logs/plan-ID" ... }

Cause

Terraform Enterprise uses Redis to cache various operational data, including the output logs from Terraform runs. Redis has a hard limit of 512MB for the size of any single key or value.

In workspaces that manage a large and complex infrastructure, the resulting log file from a terraform plan or terraform apply can exceed this 512MB limit. When Terraform Enterprise attempts to write this oversized log file to the Redis cache, Redis rejects the write operation, causing the run to fail.

Solutions

Solution 1: Reduce Workspace Complexity

The recommended solution is to reduce the number of resources managed within the affected workspace. You can achieve this by splitting the monolithic configuration into several smaller, more focused workspaces. This approach aligns with infrastructure as code best practices, improving maintainability, reducing the potential for errors, and keeping run times manageable.

Additional Information

For more guidance on structuring your configurations, refer to the official documentation on workspace management and Terraform best practices.