Duplicated files generated every time running amplify pull/push

[Dickensli]

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

v22.4.0

Amplify CLI Version

12.12.4

What operating system are you using?

Mac

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Describe the bug

Every time I run amplify pull or amplify push, duplicate files are generated under the #current-cloud-backend directory. For example, my backend folder structure looks like this:

backend
|── api
|── awscloudformation/build
│ └── root-cloudformation-stack.json
|──auth
│ └── build
|── function
|── amplify-meta.json
|── backend-config.json
|── tags.json

However, the #current-cloud-backend directory ends up with:

#current-cloud-backend
|── api
|── api 2
|──awscloudformation/build
| └── root-cloudformation-stack.json
| └── root-cloudformation-stack 2.json
|── auth
│ └── build
│ └── build 2
|── auth 4
|── function
|── amplify-meta.json
|── amplify-meta 2.json
|── backend-config.json
|── backend-config 2.json
|── tags.json
└── tags 2.json

These number-appended folders are usually empty or outdated and vary with each run, e.g. I updated my api's defaultauthentication from API_KEY to AMAZON_COGNITO_USER_POOLS using amplify update api in local; then I ran amplify push, and two amplify-meta.jsons were generated. amplify-meta.json catch the change but amplify-meta 2.json does not.

Sometimes, it gets worse, and the api/ folder disappears. When I check Amplify Backend Studio, I see errors like "can't retrieve data since schema is missing." While I can fix this by running amplify push --force, the issue of duplicated files still persists.

One possible cause is:
I have turned on CI/CD linked to github repo, usually when I have some backend change, I run amplify push, waiting for push to succeed, and then run git add, git commit, git push which triggers CI/CD runs. Could this introduce any conflict between manual amplify push vs. CI/CD push?

my amplify.yml

version: 1
appRoot: ./career-copilot-app/
backend:
  phases:
    build:
      Commands:
        - sudo pip3 install pipenv
        - sudo pip3 install venv
        - amplifyPush --simple
  artifacts:
    baseDirectory: amplify/backend
    files:
      - '**/*'
  cache:
    paths:
      - 'amplify/**/node_modules/**/*'
      - 'amplify/.serverless/**/*'
      - 'amplify/backend/**/build/**/*'
frontend:
  phases:
    preBuild:
      commands:
        - npm ci
        - npm audit fix
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: dist/career-copilot-app/browser
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

Expected behavior

no duplicated files/folders should be generated

Reproduction steps

1. make some local changes in api schema or lambda function src
2. amplify push (duplicated files appear)
3. manually delete all duplicated files under #current-cloud-backend
4. waiting for push to succeed
5. git add
6. git commit
7. git push
8. waiting for CI/CD to finish
9. amplify pull
10. another a few duplicated files appear

Project Identifier

083a411c049aa404823988034e0b09f6

Log output

# Put your logs below this line

Additional information

No response

Before submitting, please confirm:

• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

[ykethan]

Hey @Dickensli, thank you for reaching out. Is the project currently being used in a folder that sync with cloud storage like iCloud or OneDrive?
From the reproduction steps this appears to be similar to #13335

[Dickensli]

Thanks for the quick turnaround! No, I don't have any backup or sync with external storage. This should be more serious issue than seeing duplicated files. Please see this section

Sometimes, it gets worse, and the api/ folder disappears. When I check Amplify Backend Studio, I see errors like "can't retrieve data since schema is missing." While I can fix this by running amplify push --force, the issue of duplicated files still persists.

Moreover, sometimes what I observed is

1. I created a new function/some_lambda_function by running amplify add function
2. I ran amplify push
3. Then after it finished, immediately, I ran amplify pull
4. then cli created function/some_lambda_function and function/some_lambda_function 2 where

• index.py and setup.py were deleted,

• and custom-policies.json, parameter.json, Pipfile and Pipfile.lock were moved from function/some_lambda_function to function/some_lambda_function 2

Overall, the result appears disorganized ...

[ykethan]

Hey @Dickensli, thank you for the information. Could you try the following

1. download the #current-cloud-backend.zip archive, unzip, remove duplicate files, re-zip, and re-upload. While re-zipping do ensure the current-cloud-backend is not being upload as a nested folder.
2. pull the backend into a different empty directory outside the existing project directory path using amplify pull --appId <app-id> --envName <env-name>
3. make a small change such as adding a console log in a function
4. then run amplify push
   Do let us know if this does still create duplicate files.
   On a push Amplify does not make a copy of the directory or files locally, the above enable us to dive deep to the issue.

[Dickensli]

Hi, I deduplicated all files and reupload #current-cloud-backend.zip; running amplify pull --debug --appId dyq20j2b80hyx --envName tuna I got following error

🛑 Failed to pull the backend.

Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/

PullBackendFault: Failed to pull the backend.
    at attachBackend (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend.js:55:15)
    at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/commands/pull.js:64:9)
    at async Object.executeAmplifyCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:194:9)
    at async executePluginModuleCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:139:5)
    at async executeCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:37:9)
    at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:121:5)

File at path: '/Users/dickensli/Desktop/new_amplify/amplify/#current-cloud-backend/amplify-meta.json' does not exist
Error: File at path: '/Users/dickensli/Desktop/new_amplify/amplify/#current-cloud-backend/amplify-meta.json' does not exist
    at JSONUtilities.readJson (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-cli-core/lib/jsonUtilities.js:44:19)
    at AmplifyToolkit.readJsonFile (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/extensions/amplify-helpers/read-json-file.js:6:45)
    at ensureAmplifyMeta (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-provider-awscloudformation/lib/attach-backend.js:126:46)
    at Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-provider-awscloudformation/lib/attach-backend.js:117:36)
    at async Object.attachBackend (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-provider-awscloudformation/lib/index.js:196:3)
    at async queryProvider (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend-steps/a10-queryProvider.js:36:5)
    at async attachBackend (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/attach-backend.js:43:9)
    at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/commands/pull.js:64:9)
    at async Object.executeAmplifyCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:194:9)
    at async executePluginModuleCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:139:5)
    at async executeCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:37:9)
    at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:121:5)

Session Identifier: 718e26cf-46c3-4064-a22b-6b99d0795434

[Dickensli]

Hey I finally got it right. The above error is because I accidentally included nested #current-cloud-backend to zip. You are right, my iCloud backup is the root cause. I did not realize iCloud backed up every file on my laptop... but Thanks! Really helpful!

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[wongmrdev]

Hey I finally got it right. The above error is because I accidentally included nested #current-cloud-backend to zip. You are right, my iCloud backup is the root cause. I did not realize iCloud backed up every file on my laptop... but Thanks! Really helpful!

Could you please comment how the icloud backup corrupted the amplify backend dir?