Unable to refresh resources when provider ClientSecret updates

[thecodetinker]

What happened?

Our stack contains Azure AD resources located in a different tenant than the main Azure resources. To access this we log in a Provider with an ID/Password obtained from KeyVault further up the pipeline. The code is roughly like this:

var userTenantProvider = new Provider("UserTenantProvider", new ProviderArgs
{
    TenantId = args.UserTenantId,
    ClientId = args.ClientId,
    ClientSecret = args.ClientSecret
});

var appOptions = new CustomResourceOptions { 
    Parent = this, 
    DeleteBeforeReplace = true, 
    Provider = userTenantProvider 
};

var appRegistrationPassword = new ApplicationPassword("AdministratorAppRegistration-Password", new ApplicationPasswordArgs
{
    ApplicationObjectId = args.ObjectId,
    DisplayName = appRegistrationPasswordName,
    EndDate = "2099-01-01T01:02:03Z",
}, appOptions);

This works fine until the client secret is rotated, at which point refreshing fails. It feels like the refresh doesn't update the provider - possibly reasonable but it breaks this workflow.

Steps to reproduce

1. Create a stack similar to the above, with resources created with a Provider that takes a ClientID and ClientSecret.
2. Run pulumi refresh.
3. Observe that the refresh succeeds.
4. Run pulumi up to create the stack.
5. Update the client secret.
6. Run pulumi refresh again.

Expected Behavior

I expect the provider to use the updated secret I passed in, and the refresh to work.

Actual Behavior

The refresh fails due to the provider still using the expired credential.

Versions used

We use the Pulumi Automation API so there isn't an obvious 'About' but the plugins listed are

Installing Plugins ...
* azure v4.27.0
* azuread v5.6.0
* azure-native v1.52.0
* random v4.3.1

With pulumi automation Nuget package v3.25.1

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[lblackstone]

Hi @thecodetinker -- I've seen similar cases on other providers before, and I believe the solution was to use the --target option to fix the Provider credentials before updating the other resources in a subsequent update.

If the creds are expected to rotate regularly, you might want to consider splitting that resource out into a separate stack so that the Provider lifecycle can be managed separately from the dependent resources.