During a run in Terraform Enterprise, the plan or apply might fail with one of these errors: Killed,
connection is shut down, unexpected EOF, or Error: rpc error: code = Canceled desc = context canceled
This is usually indicative of the Terraform configuration consuming more memory than is available on the container than is available on the agent performing the run.
You can confirm this by identifying the time of the suspect run in the Terraform Enterprise UI and then use the journalctl command to see if there are any out of memory termination of processes during that time frame:
journalctl --since "2023-06-05 11:10:10" --until "2023-06-05 12:05:50" | grep "oom-kill"If there are out of memory crashes in this time frame, you'll see output like this:
[ 3012.565547] terraform-provi invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
[ 3041.315832] terraform invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
[ 3042.204040] terraform invoked oom-killer: gfp_mask=0x50, order=0, oom_score_adj=0If there is no output, the initial errors are not caused by out of memory conditions.
For further assistance in resolving Terraform Enterprise out of memory issues, please see this document.