Invalid Resource ID When Authenticating with the Databricks Provider for Azure

Problem

When attempting to create a databricks_metastore resource using the Terraform Databricks provider on Azure, you may encounter an Invalid resource ID error.

Error: cannot create metastore: Invalid resource ID

  with module.databricks,
  on ./main.tf line 7, in resource "databricks_metastore" "example":
     7: resource "databricks_metastore" "example" {

Error: Invalid resource ID

Cause

This error typically occurs when authenticating with a Service Principal in Azure because an incorrect workspace ID was provided in the Databricks provider configuration block. The provider's azure_workspace_resource_id argument requires the full Azure Resource ID, not the shorter numerical workspace_id.

Solutions

Solution 1: Use the Full Azure Resource ID

To resolve this issue, you must provide the complete Azure Resource ID for your Databricks workspace in the azure_workspace_resource_id argument within the provider "databricks" block.

The correct value should follow this format: /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Databricks/workspaces/<WORKSPACE_NAME>.

Update your provider configuration with the correct ID.

provider "databricks" {
  ## The full resource ID is required.
  azure_workspace_resource_id = "/subscriptions/redacted/resourceGroups/example-resources-test/providers/Microsoft.Databricks/workspaces/databricks-test"

  ## Do not use the shorter numerical workspace ID.
  # azure_workspace_resource_id = "3654489030985684"

  host = "abc.123.azuredatabricks.net"
}

Additional Information

• For more details on configuring the Databricks provider, please refer to the official provider documentation.