No AWS provider error handler on detailed resource read

[raulsh]

Description

There is no throttling error handling when reading resources if "deep" option is enabled. When I enable Terraform logs (TF_LOG=trace), I can see the throttling to Route53. But at no time is this error handled, because at least (to my knowledge) I can't get the error from the terraform library. I know there are other cases, like issue #1645. But I think the problem is not limited to a single resource, but it is where the detailed reading is done.

The problem is in these lines:

driftctl/enumeration/remote/terraform/provider.go

Lines 189 to 208 in 50f8f2d

	var newState cty.Value
	r := retrier.New(retrier.ConstantBackoff(3, 100*time.Millisecond), nil)
	err = r.Run(func() error {
	resp := p.grpcProviders[alias].ReadResource(providers.ReadResourceRequest{
	TypeName: typ,
	PriorState: priorState,
	Private: []byte{},
	ProviderMeta: cty.NullVal(cty.DynamicPseudoType),
	})
	if resp.Diagnostics.HasErrors() {
	return resp.Diagnostics.Err()
	}
	nonFatalErr := resp.Diagnostics.NonFatalErr()
	if resp.NewState.IsNull() && nonFatalErr != nil {
	return errors.Errorf("state returned by ReadResource is nil: %+v", nonFatalErr)
	}
	newState = resp.NewState
	return nil
	})

In my case, because I needed to use it, I just added this inside retry block

if resp.NewState.IsNull() {
	logrus.Info("state returned by ReadResource is nil, retrying ...")
	return errors.Errorf("state returned by ReadResource is nil")
}

I know this is not a good solution, because there may be other reasons why the response is null. But it works.

In some cases, when a resource is not found, driftctl scan fails and can't save an output.

Environment

• OS: WSL2
• driftctl version: 50f8f2d (builded in local)
• terraform version:
• terraform providers versions: [email protected]

How to reproduce

In my case, having a tons of Route53 records.

Possible Solution

1. Implement a exponential backoff (instead of linear) according to AWS recommendations.
2. Handle in retry logic in known cases like throttling or timeout

Additional context