Azure DevOps Server Error: Unable to negotiate with <x.x.x.x> port 22: no matching host key type found. Their offer: ssh-rsa – HashiCorp Help Center

Introduction

This article addresses a connection error between Terraform Enterprise and Azure DevOps Server after upgrading to Terraform Enterprise v202309-1.

Prerequisites

Terraform Enterprise (TFE) version v202309-1
Azure DevOps Server 2020 or 2019

Problem

After upgrading TFE to v202309-1, workspaces connected to an Azure DevOps Server VCS provider may fail with the following error during the slug ingress process.

Failed to ingress slug: Failed to clone: Failed to reach repo using proxy: Failed on ls-remote: exit status 128
Output: Unable to negotiate with <x.x.x.x> port 22: no matching host key type found. Their offer: ssh-rsa
fatal: Could not read from remote repository.Please make sure you have the correct access rights
and the repository exists.

Cause

The error occurs during the SSH key negotiation. Terraform Enterprise v202309-1 enables consolidated services by default, which uses a newer version of OpenSSH. This newer version disables the ssh-rsa host key algorithm, which relies on the SHA-1 hash algorithm, due to security vulnerabilities. Azure DevOps Server may still be offering this older algorithm, causing the negotiation to fail.

Refer to the OpenSSH 8.8 release notes for more details on this change.

Solutions

There are several solutions depending on your environment and security constraints.

Solution 1: Patch Azure DevOps Server

The recommended solution is to update your Azure DevOps Server with the latest patches from Microsoft, which add support for more secure SSH algorithms.

Installing the appropriate patch should resolve the connection issue.

Solution 2: Disable Consolidated Services (Replicated Installations)

If you cannot immediately patch your Azure DevOps Server, you can temporarily disable consolidated services on your Replicated TFE installation. This will revert TFE to its previous SSH client.

Disable the consolidated_services_enabled setting.

$ replicatedctl app-config set consolidated_services_enabled --value 0

Restart Terraform Enterprise for the change to take effect.
```
$ replicatedctl app stop
```
Wait for the application to stop. You can check the status with $ replicatedctl app status until the state shows as stopped.
Start the application.
```
$ replicatedctl app start
```

Note: Disabling consolidated services is a temporary workaround. This option will be removed in TFE version v202401-1 and later, as noted in the v202309-1 release notes.

Solution 3: Re-enable ssh-rsa (Flexible Deployment Options - Docker)

For Terraform Enterprise FDO Docker installations, you can create a custom SSH configuration file to re-enable the ssh-rsa algorithm for the TFE container.

On your TFE host, create a configuration file and set its permissions.

$ echo 'PubKeyAcceptedAlgorithms=+ssh-rsa' | sudo tee /etc/ssh/tfe_allow_rsa
$ sudo chmod 644 /etc/ssh/tfe_allow_rsa

Edit your compose.yaml file to mount this configuration file into the container as a volume. Add the following under the volumes section.
```
volumes:
  ## ...
  - type: bind
    source: /etc/ssh/tfe_allow_rsa
    target: /etc/ssh/ssh_config.d/tfe_allow_rsa
```
Save the file and redeploy your container using docker compose.

Solution 4: No Current Workaround (Flexible Deployment Options - Kubernetes)

There is no workaround available at this time for Terraform Enterprise FDO on Kubernetes installations.

Additional Information