memberKey, notFound on creation of Nth GoogleWorkspace user

[fiq]

Terraform Version

Terraform version: 1.1.0 on linux_amd64

• This has also occured with a slightly different message in Terraform v1.0.10 on darwin_amd64 We did both a terraform init upgrade and started over with a clean tfstate.

Affected Resource(s)

Please list the resources as a list, for example:

• googleworkspace_user
• googleworkspace_group_member

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

See https://gist.github.com/fiq/8ed23041e1657f68b56daa25559082fd
This uses inputs from a csv file with the following structure:

CSV Used: https://gist.github.com/fiq/274fa54c3a6bf33a513c69c4299777e7

Note that the password is currently hard-coded in the main HCL file, ignoring that referenced in the CSV

Debug Output

• Debug output to repeatedly enter a failure state with the error "Error 404: Resource Not Found: memberKey, notFound...with googleworkspace_group_member.members":
  • https://gist.github.com/fiq/c8a7aff59b23a3ace601403cb16437c0
• Debug output when subsequently applying terraform with SEGV signal:
  • https://gist.github.com/fiq/652dbbe2ed188eb1f2a56a87b678fd11

Panic Output

See DEBUG for error. Note there was no specific crash output.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xe96102]

Expected Behavior

Issues when adding a third user ([email protected] from the first row of our CSV File):

• Adding the first line in our CSV (See https://gist.github.com/fiq/274fa54c3a6bf33a513c69c4299777e7) should create a new user
• Removing the user (which works using tf) and then re-adding the user (which breaks) should also result in a new user.

Actual Behavior

• Adding this user to a fresh (previously unfailed) tfstate with two users always fails with a memberKey error (as per debug)
• Removing and adding the user results in the error given in the debug output
• The user can be removed by removing terraform.firsttestuser from the csv file; this works. Re-adding the user results in the memberKey error given above.
• Terraform plan is not applied.

More Surrounding Context

• We had password issues (unsure if this is related) where a user could only login with the password set on account creation.
• Any attempt to change the password after this resulted in the change being affected but it not being possible to login with the account. As though the password was incorrect. The original password was also modified. Note that this used an inline-hash in the tf reducing the risk of a typo.
• During this investigation, we created up to 3 test users.
• This was done by adding additional users to the CSV file. It produces this error which we've seen in a number of situations with this provider: See the output of https://gist.github.com/fiq/c8a7aff59b23a3ace601403cb16437c0
• This creates an error of the form: Error: unexpected error during retries of group_member: googleapi: Error 404: Resource Not Found: memberKey, notFound... with googleworkspace_group_member.members

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

To get a member key error

1. Start in a fresh state, terraform apply with only two users in the file
2. Add third user and terraform apply again.

Although unrelated, note that any attempt to change password via terraform after step 1 resulted in an account which could not be logged into. Note that this was with the password which is harded coded in the users.tf

Important Factoids

Nope.

References

N/A

[megan07]

Hi @fiq , i'm sorry you're running into this. I wanted to provide an update as I've at least found what the problem is. It seems as though when the googleworkspace_group_member is created, the Google Workspace admin API returns id as some random string in this particular case (I've tested this a few times now and it's actually the exact same string, so...I'm wondering if it's maybe hard-coded on their end somewhere?). The provider expects this to be the user's id and then goes to GET that user, but since it's not the correct user id, we get an error.

It's very odd, since the same exact order of events is happening if you were to keep all 3 users in that list and do an apply all at once, which succeeds, so I still need to find what edge case is being hit here.

That being said, the SEGV signal issue you mentioned was reported here and has been resolved. It will go out in the next release.

[megan07]

I found what is happening! It looks like terraform is trying to create the member before the user is created. Sorry, I watched it a little bit closer this time.

Here's how you can make sure that the member depends on the user being created first:

resource "googleworkspace_group_member" "members" {
  for_each = { for user in local.users : user.email => user }
  group_id = googleworkspace_group.terraform-test-group.id
  email    = googleworkspace_user.users[each.value.email].primary_email
}

I'll close this now, but if you see further issues, feel free to open a new issue. Thanks!