Introduction
This KB article applies to Terraform Enterprise (TFE) v202309-1 when connected to Azure DevOps Server 2020 and 2019.
Problem
After upgrading TFE to v202309-1, which has consolidated services enabled by default, workspaces that are connected to Azure DevOps Server may fail with this error
Failed to ingress slug: Failed to clone: Failed to reach repo using proxy: Failed on ls-remote: exit status 128 Output: Unable to negotiate with <x.x.x.x> port 22: no matching host key type found. Their offer: ssh-rsa fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
Cause
This issue is caused during SSH negotiation as consolidated services uses a newer version of OpenSSH than what may be configured on the Azure DevOps Server. Specifically the SHA-1 hash algorithm has been disabled by default in newer OpenSSH implementations as it is cryptographically broken. Refer here for more details.
Solutions
- Microsoft recently released some patches to align with industry wide changes, refer to 2020 patch and 2019 patch for the details.
- Installing the patch will resolve the issue and Azure DevOps should connect to TFE.
- If there are some restrictions on installing the patch, the workaround is to disable the consolidated services by following below instructions, refer here as well.
- Disable the
consolidated_services_enabled
setting.
- Disable the
replicatedctl app-config set consolidated_services_enabled --value 0
-
- Restart Terraform Enterprise
- replicatedctl app stop
- Wait for the application to stop, check with replicatedctl app status
until "State" transitions to "stopped"
- replicatedctl app start
NOTE: Disabling consolidated services will only be possible until v202401-1, refer to the release notes here.
3. If you are using Terraform Enterprise FDO Docker installation the workaround is
-
- Create this file in your TFE host(s) and change the permission settings
echo 'PubKeyAcceptedAlgorithms=+ssh-rsa' | sudo tee /etc/ssh/tfe_allow_rsa
chmod 644 /etc/ssh/tfe_allow_rsa
-
- Edit your compose.yaml file and add a volume type bind. Save and redeploy your container with docker compose
volumes:
...
- type: bind
source: /etc/ssh/tfe_allow_rsa
target: /etc/ssh/ssh_config.d/tfe_allow_rsa
4. No workaround exists at the moment for Terraform Enterprise FDO on Kubernetes.
Additional Information
- Compatibility issues in ssh versions
- Microsoft Patch for Azure Devops Server
- Disabling Consolidated Services
- Terraform Enterprise FDO