Problem
After upgrading to Terraform Enterprise v202507-1 and later, runs may fail during the plan phase with a log message similar to the following, indicating an issue when uploading artifacts:
Operation failed: failed uploading plan JSON: failed uploading: PUT https://<TFE_HOSTNAME>/_archivist/v1/object/<OBJECT_ID> giving up after 16 attempt(s)
In some instances a message will be logged that indicates an issue downloading artifacts, similar to:
Operation failed: failed setting up Terraform configuration: failed downloading configuration version: GET https://<TFE_HOSTNAME>/_archivist/v1/object/<OBJECT_ID> giving up after 16 attempt(s)
A more detailed error error is logged to the container logs, as well as /var/log/terraform-enterprise/archivist.log
log file in the container.
One example error is AccessDenied
:
{
"@level": "error",
"@message": "writing to archive store failed",
"@module": "archivist.server.http.upload",
"@timestamp": "2025-08-12T01:20:44.035802Z",
"body-size": 779462,
"copied": 779462,
"err": "failed uploading RequestID \"UUID\": operation error S3: PutObject, https response error StatusCode: 403, RequestID: UUID, HostID: , api error AccessDenied: Access Denied",
"obj.compressed": true,
"obj.encrypted": true,
"obj.expire": 1754965825,
"obj.key": "terraform/json-plan/xxx/asmt-yyy",
"obj.mode": "w",
"req.amazon_trace_id": "-",
"req.id": "-"
}
Another example is a SHA256 mismatch:
{
"@level": "error",
"@message": "writing to archive store failed",
"@module": "archivist.server.http.upload",
"@timestamp": "2025-08-14T13:10:43.617381Z",
"body-size": 1443,
"copied": 1443,
"err": "failed uploading RequestID \"ac1c6f29:195922864dc:1f783c:194a\": operation error S3: PutObject, https response error StatusCode: 400, RequestID: ac1c6f29:195922864dc:1f783c:194a, HostID: 9e32f9be08be25a3686d7dd1c8169685b1ea375ea88edd1b9c6bf8708164c585, api error XAmzContentSHA256Mismatch: The Content-SHA256 you specified did not match what we received",
"obj.compressed": true,
"obj.encrypted": true,
"obj.expire": 1755187719,
"obj.key": "terraform/json-plans/xxx/plan-yyy",
"obj.mode": "w",
"req.amazon_trace_id": "-",
"req.id": "-"
}
Prerequisites
-
Terraform Enterprise:
v202507-1 or later
- Storage: External S3-compatible service
Cause
Terraform Enterprise v202507-1
includes an updated SDK for communicating with S3 and S3-compatible object storage services. This update introduces default integrity checks that some S3-compatible services are not compatible with.
The error written to the application logs may differ depending on the specific S3-compatible object service, but can include errors such as:
AccessDenied
MissingContentLength
Unsupported header
XAmzContentSHA256Mismatch
"failed reading object header"
err: "invalid character '\x00' looking for beginning of value"
Solution
Currently, there is no workaround for this issue, and our engineering teams are actively working on a solution for a future release.
Any upgrades performed that encounter this issue will need to be rolled back.