Data Model deploy failure

[DougalW]

Before opening, please confirm:

• I have searched for duplicate or closed issues.
• I have read the guide for submitting bug reports.
• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

App Id

d2tkk8ivw3yqtf

Region

ap-southeast-2

Environment name

dev

Figma File Version (if applicable)

No response

Amplify CLI Version

10.6.2

If applicable, what version of Node.js are you using?

No response

What operating system are you using?

Mac

Browser type?

Safari 16.3

Describe the bug

Hi, I have a data model consisting of a set of 28 tables, 20 ENUM's and various relationships between the tables (40 are 1:1 and 15 are 1:many).

I am trying to add a new table and relationships via Amplify Studio, and the Data Model editor allows me to do this but when I click Save and Deploy, it says it has deployed the new data model but the new table is not there. When I inspect my Dynamo DB tables, the new table is also not present. When I look at Cloudformation there have been no new stacks run so it looks like the deploy was not submitted.

I have an ERD I can send privately, but would prefer not to share it publicly here.

Expected behavior

When I add a new table or relationship, or update and existing table or relationship, it will be saved and deployed in Amplify Studio Data Modelling.

Reproduction steps

1. Add new table (called 'RiskManagementProcessesDataSet') to existing Data Model
2. Update an existing relationship in the 'Disclosure' table to point to this new table (previous table was called 'Risk Management Processes'
3. Click 'Save and Deploy'

Project Identifier

No response

Additional information

I have repeated these steps and see exactly the same behaviour - no Cloudformation changes and no new tables in DynamoDB, but Amplify Studio shows it has updated successfully.

[ykethan]

Hey @DougalW, 👋 thank you for reaching out; By any chance do you observe a deployment-state.json in the deployment s3 bucket. If yes, could you delete the file and retry the deploy in the data modelling in Studio.
We can find the deployment bucket using the name with the format amplify-<application name>-<env name>-<random number>-deployment in the AWS S3 console.

[DougalW]

Hi ykethan, thanks for getting back so quickly. Deleting deployment-state.json and retrying worked, but I am still getting the same deployment errors on a regular basis and I have had to delete the file three times now to make Amplify deploy my data model.

It is also giving this error a few times:

EMFILE: too many open files, open '/tmp/amplify-ede101ca-49ba-4e45-8507-d7c934a1877e/amplify/backend/api/ClimateDisclosure/build/states/02/resolvers/TransitionPlanningDataSet.GoalsObjectives.auth.1.req.vtl'

But when it occurs I haven't made that many changes to the data model.

I also noticed that when I create a new data model, Amplify doesn't always create a matching form in the UI library. It seemed to stop creating new forms after it had created 32 forms. When I manually create a new form, not all the data models show up in the dialog when it asks if I want to base the form on a data model, so I created the form as blank then mapped it to the data model, using the drop down list which now shows all the available data models.
There seems to be some sort of resource limit or the UI has bugs.

After adding some more tables, I can no longer Save and Deploy my data models.

[ykethan]

Hey @DougalW, thank you for the information. I was able to reproduce the issue, marking this as bug. The Amplify studio and API team has been made aware into the behaviour are tracking this behaviour internally.
Currently, the mitigation step is to pull the application and deploy new API changes utilizing Amplify CLI.

[DougalW]

Many thanks. FYI, the UI Library is now showing 45 forms - it's added 13 overnight following my last deployment. I was able to create the final few tables I needed but I cannot link them into other tables in the UI.

[ykethan]

hey @DougalW, could you provide some additional information on the unable to link behaviour you are experiencing on the UI?
screenshot or a short video would be helpful as well

[DougalW]

here's a video of the behaviour. I'm trying to create a form linked to the Opportunities table from my data model. I can't create the form because it doesn't show up in the menu for selecting tables, but once I create the blank form I can then link it as the Opportunities table shows up in the drop down.

BrokenFormCreate.mov

[ykethan]

hey @DougalW, thank you for the video. I was able to reproduce the issue. When creating forms or linking a model with data model that contains a field with required relationship the model should not appear both the options, which is a bug.
But would also like to announce that a new feature is already rolling out that would allow forms with relationships. The feature should be available in the coming weeks.

I have created a issue to track this behaviour on #924 as well.
Could you retry this when the feature rolls out to ap-southeast-2 and let us know if the issue still appears.

[DougalW]

That's great news. Any idea when the bug fix for the Data Model issue (can't create more tables) will be available? I need to set aside some time to try pushing changes from my local schema - any guidance on how to do this would be appreciated.

[ykethan]

Hey @DougalW, to pull you existing Studio application to your local environment you will need to install Amplify CLI and then pull the app as provided here: https://docs.amplify.aws/console/adminui/extend-cli/
then modify the schema.graphql file at amplify/backend/api/<api-name> location which should contain all your models. https://docs.amplify.aws/cli/graphql/data-modeling/

for the Data Model issue (can't create more tables) behaviour the Amplify studio team are investigating the behaviour. Unfortunately, I do not have a ETA on when this will be released. But the Studio team should provide an update here as soon as a fix gets released.

[DougalW]

Thanks @ykethan I've pulled it down, and was exploring the model again in Amplify so made a few modifications just to see what happens, and it was allowing met to change and extend the data model. I did request an increase in my AppSync limit so perhaps this had some influence?

[ykethan]

Hey @DougalW, if you are able to push new data model using the CLI it should create DynamoDB. By any chance on a push do you see any limit errors?

[DougalW]

Hi @ykethan I renamed a table in my local schema.graphql file and pushed back to Amplify but it failed. Here's the log:

dougalwatt@Dougals-M1-MacBook-Pro-2 seeka % amplify push --appId d2tkk8ivw3yqtf --envName seekadev --allow-destructive-graphql-schema-updates
✅ GraphQL schema compiled successfully.

Edit your schema at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema.graphql or place .graphql files in a directory at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema
⠋ Building resource api/ClimateDisclosure✅ GraphQL schema compiled successfully.

Edit your schema at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema.graphql or place .graphql files in a directory at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema
✔ Successfully pulled backend environment seekadev from the cloud.
⠧ Building resource api/ClimateDisclosure✅ GraphQL schema compiled successfully.

Edit your schema at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema.graphql or place .graphql files in a directory at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema

    Current Environment: seekadev
    
┌──────────┬───────────────────┬───────────┬───────────────────┐
│ Category │ Resource name     │ Operation │ Provider plugin   │
├──────────┼───────────────────┼───────────┼───────────────────┤
│ Api      │ ClimateDisclosure │ Update    │ awscloudformation │
└──────────┴───────────────────┴───────────┴───────────────────┘
✔ Are you sure you want to continue? (Y/n) · yes
✅ GraphQL schema compiled successfully.

Edit your schema at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema.graphql or place .graphql files in a directory at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema
⠙ Building resource api/ClimateDisclosure✅ GraphQL schema compiled successfully.

Edit your schema at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema.graphql or place .graphql files in a directory at /Users/dougalwatt/Documents/seeka/amplify/backend/api/ClimateDisclosure/schema

Deployment failed.
Deploying root stack ClimateDisclosure [ ---------------------------------------- ] 0/2
	amplify-climatedisclosure-see… AWS::CloudFormation::Stack     UPDATE_ROLLBACK_COMPLETE_CLEA… Mon May 29 2023 16:30:55…     
	apiClimateDisclosure           AWS::CloudFormation::Stack     UPDATE_FAILED                  Mon May 29 2023 16:29:34…     
Deploying api ClimateDisclosure [ ===------------------------------------- ] 3/42
	GraphQLAPIDefaultApiKey215A6D… AWS::AppSync::ApiKey           UPDATE_COMPLETE                Mon May 29 2023 16:29:43…     
	GraphQLAPITransformerSchema3C… AWS::AppSync::GraphQLSchema    UPDATE_IN_PROGRESS             Mon May 29 2023 16:29:42…     
	StrategyDevelopmentProcess     AWS::CloudFormation::Stack     CREATE_FAILED                  Mon May 29 2023 16:29:12…     
	DynamoDBAccess71ABE5AE         AWS::IAM::Policy               CREATE_COMPLETE                Mon May 29 2023 16:29:04…     

🛑 The following resources failed to deploy:
Resource Name: ListStrategyDevelopmentProcessResolver (AWS::AppSync::Resolver)
Event Type: create
Reason: Only one resolver is allowed per field. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: 780240a7-dfb4-4133-9c8a-2ae279f4a373; Proxy: null)


Resource Name: SyncStrategyDevelopmentProcessResolver (AWS::AppSync::Resolver)
Event Type: create
Reason: Only one resolver is allowed per field. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: 54e06c53-880a-4a9e-8424-737a3258eb74; Proxy: null)


Resource Name: QuerystrategyDevelopmentProcessesByGovernanceoversightactivitiesIDResolver (AWS::AppSync::Resolver)
Event Type: create
Reason: Only one resolver is allowed per field. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: 0cf116d2-5a82-458f-a629-d371374dc603; Proxy: null)


🛑 Resource is not in the state stackUpdateComplete
Name: ListStrategyDevelopmentProcessResolver (AWS::AppSync::Resolver), Event Type: create, Reason: Only one resolver is allowed per field. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: 780240a7-dfb4-4133-9c8a-2ae279f4a373; Proxy: null)

Name: SyncStrategyDevelopmentProcessResolver (AWS::AppSync::Resolver), Event Type: create, Reason: Only one resolver is allowed per field. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: 54e06c53-880a-4a9e-8424-737a3258eb74; Proxy: null)

Name: QuerystrategyDevelopmentProcessesByGovernanceoversightactivitiesIDResolver (AWS::AppSync::Resolver), Event Type: create, Reason: Only one resolver is allowed per field. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: 0cf116d2-5a82-458f-a629-d371374dc603; Proxy: null)


Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/

Session Identifier: fa85458a-dd3f-4d2d-87e5-6c964c4f17ac

I notice the error is: Reason: Only one resolver is allowed per field.

This is the same error when I try and change the table from the Amplify Studio UI.

When I look in AppSync I can see that the resolver for that class (StrategyDevelopmentProcesses) now is not connected to any resolvers

but in the Amplify Studio UI it shows correctly:

and the table exists in DynamoDB:

I also noticed three other tables that show up in AppSync without resolvers, but their tables do exist in DynamoDB:

• BusinessPlanningHorizon
• CapitalDeploymentFundingProcess
• ClimatePlanningHorizon

To cross-check this, I opened the Amplify Studio UI and created a new table StrategyDevelopmentProcess, deleted the old StrategyDevelopmentProcesses table, and linked GovernanceOversightActivities to the new table, then deployed. It also failed with the same error.

Next I returned to the UI and just created a new StrategyDevelopmentProcess table without changing anything else, and it again failed with the following UI log:

Command failed with exit code 1: amplify update api --headless --allow-destructive-graphql-schema-updates {"errorName":"UnknownFault","errorMessage":"Object type extension 'Query' cannot redeclare field listStrategyDevelopmentProcesses","resolution":"Please report this issue at https://github.com/aws-amplify/amplify-cli/issues and include the project identifier from: 'amplify diagnose --send-report'","link":"https://docs.amplify.aws/cli/project/troubleshooting/"} Amplify command hooks are disabled in the current execution environment. See https://docs.amplify.aws/cli/usage/command-hooks/ for more information. Session Identifier: cb42abc2-b24b-4f59-a76a-c3570c982690

Next I created a totally different table in the Studio UI called "NewStrategyProcessTable" and deployed it with the following error log

Command failed with exit code 1: amplify update api --headless --allow-destructive-graphql-schema-updates {"errorName":"UnknownFault","errorMessage":"Object type extension 'Query' cannot redeclare field listStrategyDevelopmentProcesses","resolution":"Please report this issue at https://github.com/aws-amplify/amplify-cli/issues and include the project identifier from: 'amplify diagnose --send-report'","link":"https://docs.amplify.aws/cli/project/troubleshooting/"} Amplify command hooks are disabled in the current execution environment. See https://docs.amplify.aws/cli/usage/command-hooks/ for more information. Session Identifier: cb42abc2-b24b-4f59-a76a-c3570c982690

This is very odd because the names of the new table is "NewStrategyProcessTable", not "StrategyDevelopmentProcesses".

Then to check it wasn't mixing up table names, I created a new table with a totally different name and no fields, using the Studio UI, and deployed it successfully, then I renamed it to "StratDevProcess" and added the same fields as from the original "StrategyDevelopmentProcesses" table and deployed that successfully. Next I deleted the relationship from the "GovernanceOversightActivitiesDataSet" table to the "StrategyDevelopmentProcesses" table, deployed that change, then linked the "GovernanceOversightActivitiesDataSet" table to the "StratDevProcess" table and successfully deployed that. Lastly I successfully deleted the no-longer-needed "StrategyDevelopmentProcesses" table.

This feels like an AppSync issue - it's clearly out of sync with the backend somehow and seems to confuse similar table names, or it can't handle changing tables with an existing relationship when there are a moderate number of tables.

I'm keen to see your feedback on this issue - seems like a pretty big issue that needs urgent attention as it makes Amplify very hard to use for apps with 30+ tables, which is a pretty common situation.

[DougalW]

This issue is happening again. Just removed a relationship in one table, deployed the change successfully, then added a relationship in another table and deployed, which failed with the 'Too many open files' error. I deleted the deployment-state.json file, re-deployed the model, and this time it appeared to succeed but nothing was changed, and the Data Model UI in Studio showed the model as it was before I made the schema change.

[DougalW]

may also be related to issue #632 when using many:many relationships - seems like the internal 'book-keeping' that amplify performs doesn't account for relationship changes correctly.

[qwikag]

f