googleworkspace_chrome_policy errors if policy value exists in state but not in API

[keeleysam]

Terraform Version

1.7.5

Affected Resource(s)

Please list the resources as a list, for example:

• googleworkspace_chrome_policy

Terraform Configuration Files

resource "googleworkspace_chrome_policy" "chrome_users_AllowDinosaurEasterEgg" {
  org_unit_id = local.org_unit_id
  policies {
    schema_name = "chrome.users.AllowDinosaurEasterEgg"
    schema_values = {
      allowDinosaurEasterEgg = jsonencode("TRUE")
    }
  }
}

Debug Output

---[ REQUEST ]---------------------------------------
POST /v1/customers/my_customer/policies/orgunits:batchModify?alt=json&prettyPrint=false HTTP/1.1
Host: chromepolicy.googleapis.com
User-Agent: google-api-go-client/0.5
Content-Length: 228
Content-Type: application/json
X-Goog-Api-Client: gl-go/1.17.9 gdcl/0.79.0
Accept-Encoding: gzip

{
 "requests": [
  {
   "policyTargetKey": {
    "targetResource": "orgunits/xxxxxxx"
   },
   "policyValue": {
    "policySchema": "chrome.users.AllowDinosaurEasterEgg",
    "value": {
     "allowDinosaurEasterEgg": "TRUE"
    }
   },
   "updateMask": "allowDinosaurEasterEgg"
  }
 ]
}

-----------------------------------------------------: timestamp=2024-03-21T18:18:31.038-0400
2024-03-21T18:18:31.062-0400 [INFO]  provider.terraform-provider-googleworkspace_v0.8.2: 2024/03/21 18:18:31 [DEBUG] Google Workspace API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 21 Mar 2024 22:18:31 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{}

Panic Output

n/a

Expected Behavior

Policy value is created in the API and reflected in Terraform state

Actual Behavior

A small percentage of the time (about 5% in my experience, but mostly when doing bulk changes), the policy isn't created in the API, but Terraform believes it has, and it is written to the local state. On the next run, an unexpected number of resolved policies for schema xxxx error is returned as the remote state is fetched via a policies:resolve API call, and an empty dictionary is returned.

Specifically, the state occurs where the policy value previously did not exist (it was inherited), and we're creating it.

Ultimately this is related to an issue with the Chrome Policy API - it should not be returning an HTTP 200 response to a POST if the batchModify didn't actually take place.

However, similar behavior can also occur if a policy value is created by Terraform but then changed to inherited outside of Terraform. Of course, "bad things can occur" can be true if this type of change is done, but this provider already has the wiring to check actual remote state, so it should be able to handle that case and perhaps generate a warning before recreating. If a value is changed outside of Terraform, this provider does have the capability to change it back.

Steps to Reproduce

1. Apply a bunch of changes at once, 20+ and I fairly consistently ran into the problem, in my case all to the same org unit.
2. On the next Terraform run, if one of the resources exists in state but isn't resolved to any value, Terraform will error out.

Step 1 can also be replaced by setting a policy value to inherit outside of Terraform.

Important Factoids

This occurs with both the current state of the hashicorp provider, as well as the https://github.com/Yohan460/terraform-provider-googleworkspace fork.

References

[toddthiel]

The above bug report is a fantastic write up, and captures the issue rather well. Would be good to get some eyes on this issue.