deleteBeforeReplace with failed create - inconsistent state

[jesse-triplewhale]

What happened?

If I use deleteBeforeReplace: true and the resource deletion succeeds, but the creation fails, the result is inconsistent state. The resource does not exist in the provider, but does exist in pulumi state

Example

new gcp.secretmanager.Secret(
  'testsecret',
  {
    secretId: 'my-test-secret',
    replication: {
      auto: {},
    },
  },
  {
    deleteBeforeReplace: true,
  }
);

2. pulumi up
   resource now exists in pulumi state and in provider
3. change secretId to a taken value, my-existing-secret
4. pulumi up
   resource is successfully deleted, but then not created. resource now exists in pulumi state but not in provider

Output of pulumi about

❯ pulumi about

CLI
Version 3.94.2
Go Version go1.21.3
Go Compiler gc

Plugins
NAME VERSION
clickhouse 0.0.26
cloudflare 5.27.0
datadog 4.28.0
docker 4.5.3
ec 0.8.0
gcp 7.21.1
gcp 7.21.1
google-native 0.32.0
kafka 3.7.3
kubernetes 4.11.0
kubernetes 4.11.0
mongodbatlas 3.15.0
mongodbatlas 3.15.0
nodejs unknown
postgresql 3.11.0
rabbitmq 3.3.2
random 4.16.1
slack 0.4.4

Host
OS darwin
Version 14.4.1
Arch arm64

This project is written in nodejs: executable='/Users/.../.nvm/versions/node/v18.17.1/bin/node' version='v18.17.1'

Current Stack: ...

Found no pending operations associated with shofifi

Backend
Name pulumi.com
URL https://app.pulumi.com/...
User ...
Organizations ...
Token type organization: ...
Token type cloud-build

Pulumi locates its logs in /var/folders/q5/.../T/ by default
warning: Failed to get information about the Pulumi program's dependencies: failed to run "/Users/.../.nvm/versions/node/v18.17.1/bin/npm ls --json --depth=0": exit status 1

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[justinvp]

Thanks for opening the issue and sorry for the trouble!

I was able to reproduce the same behavior with a Bucket resource in AWS. I would expect the resource to be removed from the state after it was deleted (and not remain when the create fails).

This looks like it's potentially the same as #8354 (very old issue).

We'll look into picking this up soon.