Fastly plugin crashes when backend variables change

[sethyates]

Terraform Version

Terraform v1.9.6
on linux_amd64
+ provider registry.terraform.io/carlpett/sops v1.1.1
+ provider registry.terraform.io/fastly/fastly v5.13.0
+ provider registry.terraform.io/hashicorp/archive v2.6.0
+ provider registry.terraform.io/hashicorp/google v5.43.1
+ provider registry.terraform.io/hashicorp/google-beta v5.44.0
+ provider registry.terraform.io/hashicorp/null v3.2.3
+ provider registry.terraform.io/hashicorp/random v3.6.3
+ provider registry.terraform.io/pagerduty/pagerduty v3.15.6

Affected Fastly Terraform Resource(s)

• fastly_service_vcl

Terraform Configuration Files

module "app" {
  source = "../../../modules/cdn_run_backend"

  name       = google_cloud_run_v2_service.tokens.name
  project_id = module.integration.project.project_id
  region     = local.region
  prefix     = "oauth"

  depends_on = [google_cloud_run_v2_service.tokens]
}

resource "fastly_service_vcl" "integration" {
  name               = local.connect_domain
  default_ttl        = 3600
  http3              = true
  stale_if_error     = false
  stale_if_error_ttl = 43200

  domain {
    name = local.connect_domain
  }

  dynamic "backend" {
    for_each = [module.app]   # <- have to remove the module.app when the provider crashes

    content {
      name                  = backend.value.name
      address               = backend.value.address
      override_host         = backend.value.override_host
      between_bytes_timeout = backend.value.between_bytes_timeout
      connect_timeout       = backend.value.connect_timeout
      first_byte_timeout    = backend.value.first_byte_timeout
      max_conn              = backend.value.max_conn
      port                  = backend.value.port
      use_ssl               = backend.value.use_ssl
      ssl_check_cert        = backend.value.ssl_check_cert
      ssl_cert_hostname     = backend.value.ssl_cert_hostname
      ssl_sni_hostname      = backend.value.ssl_sni_hostname
      min_tls_version       = backend.value.min_tls_version
      shield                = backend.value.shield
      request_condition     = backend.value.request_condition
    }
  }

  # more irrelevant stuff
}

Expected Behavior

What should have happened?

The plan should be planned or the apply should be applied.

Actual Behavior

What actually happened?

Any change whatsoever to the underlying Cloud Run service causes the Fastly provider to crash. I have to set for_each = [] to get the configuration to apply, then quickly revert and apply. This obviously causes outages because the backend goes away.

╷
│ Error: Request cancelled
│ 
│   with fastly_service_vcl.integration,
│   on cdn.tf line 12, in resource "fastly_service_vcl" "integration":
│   12: resource "fastly_service_vcl" "integration" {
│ 
│ The plugin.(*GRPCProvider).PlanResourceChange request was cancelled.
╵

Stack trace from the terraform-provider-fastly_v5.13.0 plugin:

panic: value is unknown

goroutine 134 [running]:
github.com/hashicorp/go-cty/cty.Value.AsString({{{0xfc4158?, 0xc00026a27b?}}, {0xd48b80?, 0x1598d60?}})
        github.com/hashicorp/[email protected]/cty/value_ops.go:1179 +0x111
github.com/fastly/terraform-provider-fastly/fastly.validateUniqueNames.func1({0xfc3a90?, 0xc000297890?}, 0x0?, {0xce7ea0?, 0xc00011a9b0?})
        github.com/fastly/terraform-provider-fastly/fastly/base_fastly_service.go:178 +0x171
github.com/hashicorp/terraform-plugin-sdk/v2/helper/customdiff.All.func1({0xfc3a90, 0xc000297890}, 0xe7e3df?, {0xce7ea0, 0xc00011a9b0})
        github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/customdiff/compose.go:52 +0xe6
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.schemaMap.Diff(0xc0003d84b0, {0xfc3a90, 0xc000297890}, 0xc000317040, 0xc00071cd70, 0xc000306e00, {0xce7ea0, 0xc00011a9b0}, 0x0)
        github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/schema.go:698 +0x4d4
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).SimpleDiff(0xfc4238?, {0xfc3a90?, 0xc000297890?}, 0xc000317040, 0xd5b740?, {0xce7ea0?, 0xc00011a9b0?})
        github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:990 +0xe5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).PlanResourceChange(0xc0003a26f0, {0xfc3a90?, 0xc000297770?}, 0xc000431c20)
        github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:858 +0xc52
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).PlanResourceChange(0xc000263720, {0xfc3a90?, 0xc000296cf0?}, 0xc0000a3900)
        github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:825 +0x3fc
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_PlanResourceChange_Handler({0xe45d40?, 0xc000263720}, {0xfc3a90, 0xc000296cf0}, 0xc0000a3880, 0x0)
        github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:500 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000151800, {0xfc3a90, 0xc000296c60}, {0xfc8738, 0xc000002180}, 0xc0002fcfc0, 0xc0004a2210, 0x155a8c0, 0x0)
        google.golang.org/[email protected]/server.go:1369 +0xe69
google.golang.org/grpc.(*Server).handleStream(0xc000151800, {0xfc8738, 0xc000002180}, 0xc0002fcfc0)
        google.golang.org/[email protected]/server.go:1780 +0x106f
google.golang.org/grpc.(*Server).serveStreams.func2.1()
        google.golang.org/[email protected]/server.go:1019 +0x91
created by google.golang.org/grpc.(*Server).serveStreams.func2
        google.golang.org/[email protected]/server.go:1030 +0x145

Error: The terraform-provider-fastly_v5.13.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Debug Output (optional)

Please provide a link to a GitHub Gist containing the complete debug output (see Terraform docs). Ensure any sensitive data is <REDACTED>.

https://gist.github.com/sethyates/a9fee5fb5bc85f3e739732437be6814e

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids (optional)

Is there anything atypical about your account or set up that we should know?

[saez0pub]

It seems to crash when the the backend name value is computed

[saez0pub]

Here is a full reproductible example:

$ cat <<EOF > main.tf
resource "random_string" "_" {
  length    = 9
  special   = false
  min_lower = 9
}

resource "fastly_service_vcl" "crash" {
  name               = "crash"

  domain {
    name = "test-fastly"
  }
  backend {
    address = "test"
    name    = random_string._.result
  }
}


provider "fastly" {
  api_key = "testing"
}

terraform {
  backend "local" {}
  required_version = ">= 1.5.0"
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = ">= 3.5.1"
    }
    fastly = {
      source  = "fastly/fastly"
      version = ">= 5.10.0"
    }
  }
}
EOF

$ terraform init
...
$ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform planned the following actions, but then encountered a problem:

  # random_string._ will be created
  + resource "random_string" "_" {
      + id          = (known after apply)
      + length      = 9
      + lower       = true
      + min_lower   = 9
      + min_numeric = 0
      + min_special = 0
      + min_upper   = 0
      + number      = true
      + numeric     = true
      + result      = (known after apply)
      + special     = false
      + upper       = true
    }

Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Error: Request cancelled
│
│   with fastly_service_vcl.crash,
│   on main.tf line 7, in resource "fastly_service_vcl" "crash":
│    7: resource "fastly_service_vcl" "crash" {
│
│ The plugin.(*GRPCProvider).PlanResourceChange request was cancelled.
╵

Stack trace from the terraform-provider-fastly_v5.14.0 plugin:

panic: value is unknown

goroutine 53 [running]:
github.com/hashicorp/go-cty/cty.Value.AsString({{{0x1057bf438?, 0x1400024433b?}}, {0x1056812e0?, 0x105d819e0?}})
	github.com/hashicorp/[email protected]/cty/value_ops.go:1179 +0x108
github.com/fastly/terraform-provider-fastly/fastly.validateUniqueNames.func1({0x1057bed70?, 0x140003da630?}, 0x0?, {0x10561f3c0?, 0x1400000e888?})
	github.com/fastly/terraform-provider-fastly/fastly/base_fastly_service.go:178 +0x13c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/customdiff.All.func1({0x1057bed70, 0x140003da630}, 0x1054776fb?, {0x10561f3c0, 0x1400000e888})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/customdiff/compose.go:52 +0xac
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.schemaMap.Diff(0x140004244b0, {0x1057bed70, 0x140003da630}, 0x140003e8680, 0x14000794a50, 0x140002e2e00, {0x10561f3c0, 0x1400000e888}, 0x0)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/schema.go:698 +0x3c4
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).SimpleDiff(0x1057bf518?, {0x1057bed70?, 0x140003da630?}, 0x140003e8680, 0x105694140?, {0x10561f3c0?, 0x1400000e888?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:990 +0x9c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).PlanResourceChange(0x1400036e7c8, {0x1057bed70?, 0x140003da510?}, 0x140004b3ae0)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:858 +0xa54
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).PlanResourceChange(0x1400023d720, {0x1057bed70?, 0x14000271da0?}, 0x1400048c800)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:825 +0x2c4
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_PlanResourceChange_Handler({0x1057821a0?, 0x1400023d720}, {0x1057bed70, 0x14000271da0}, 0x1400048c780, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:500 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0x14000195000, {0x1057bed70, 0x14000271d10}, {0x1057c3a58, 0x140004fc000}, 0x140003ceb40, 0x14000472210, 0x105d3d860, 0x0)
	google.golang.org/[email protected]/server.go:1369 +0xc84
google.golang.org/grpc.(*Server).handleStream(0x14000195000, {0x1057c3a58, 0x140004fc000}, 0x140003ceb40)
	google.golang.org/[email protected]/server.go:1780 +0xcd0
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	google.golang.org/[email protected]/server.go:1019 +0x8c
created by google.golang.org/grpc.(*Server).serveStreams.func2
	google.golang.org/[email protected]/server.go:1030 +0x160

Error: The terraform-provider-fastly_v5.14.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

[sethyates]

Thanks @saez0pub. Fastly team, is there any update on fixing this?