-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove serverless functions on soft delete #9438
Remove serverless functions on soft delete #9438
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
This PR implements serverless function cleanup when workflows are soft-deleted, ensuring proper resource management across the system.
- Added
ServerlessFunctionModule
import inworkflow-query-hook.module.ts
to enable serverless function cleanup - Modified
cleanWorkflowsSubEntities
inworkflow-common.workspace-service.ts
to accept workspaceId parameter for scoped operations - Added
deleteServerlessFunctions
method inworkflow-common.workspace-service.ts
for handling serverless function cleanup - Updated post-query hooks to pass workspace ID from auth context to cleanup methods
4 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile
Promise.all( | ||
workflowVersions.map((workflowVersion) => { | ||
workflowVersion.steps?.map(async (step) => { | ||
if (step.type === WorkflowActionType.CODE) { | ||
await this.serverlessFunctionService.deleteOneServerlessFunction( | ||
step.settings.input.serverlessFunctionId, | ||
workspaceId, | ||
); | ||
} | ||
}); | ||
}), | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: Double nested Promise.all/map with async callback will not wait for serverless function deletions to complete. Array.map() with async doesn't return promises that can be awaited
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
greptile is kinda true here.
Promise.all
expects an array of Promise
. Here, we give an Array<Array<Promise>>
. If we want to wait for all the promises to be fulfilled, we must flatten the array with Array#flat
.
Furtheremore, calling Promise.all
is only useful if you want to wait for a set of promises to be fulfilled. Here, we don't wait for this, the function immediately returns—the Promise.all
is not awaited. We can get rid of it.
The problem I can see here is that any failing promise won't be caught. I couldn't find any code snippet in the codebase where we do something similar, but I would do:
workflowVersions.forEach((workflowVersion) => {
workflowVersion.steps?.forEach((step) => {
const cb = () => {
if (step.type === WorkflowActionType.CODE) {
await this.serverlessFunctionService.deleteOneServerlessFunction(
step.settings.input.serverlessFunctionId,
workspaceId,
);
}
}
cb().catch(Logger.catch)
});
})
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uncaught promise rejections make a Node.js process fail. According to my tests, we have a global listener to prevent such crashes, but I would still catch the errors explicitly to provide more context about the error.
As I said, I couldn't see where we did something similar, so we might first want to discuss it broadly with the team.
...y-server/src/modules/workflow/common/workspace-services/workflow-common.workspace-service.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wanted to open a discussion about some topics. I'm unsure the team agreed on a pattern here.
...wenty-server/src/modules/workflow/common/query-hooks/workflow-delete-many.post-query.hook.ts
Outdated
Show resolved
Hide resolved
...twenty-server/src/modules/workflow/common/query-hooks/workflow-delete-one.post-query.hook.ts
Outdated
Show resolved
Hide resolved
Promise.all( | ||
workflowVersions.map((workflowVersion) => { | ||
workflowVersion.steps?.map(async (step) => { | ||
if (step.type === WorkflowActionType.CODE) { | ||
await this.serverlessFunctionService.deleteOneServerlessFunction( | ||
step.settings.input.serverlessFunctionId, | ||
workspaceId, | ||
); | ||
} | ||
}); | ||
}), | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
greptile is kinda true here.
Promise.all
expects an array of Promise
. Here, we give an Array<Array<Promise>>
. If we want to wait for all the promises to be fulfilled, we must flatten the array with Array#flat
.
Furtheremore, calling Promise.all
is only useful if you want to wait for a set of promises to be fulfilled. Here, we don't wait for this, the function immediately returns—the Promise.all
is not awaited. We can get rid of it.
The problem I can see here is that any failing promise won't be caught. I couldn't find any code snippet in the codebase where we do something similar, but I would do:
workflowVersions.forEach((workflowVersion) => {
workflowVersion.steps?.forEach((step) => {
const cb = () => {
if (step.type === WorkflowActionType.CODE) {
await this.serverlessFunctionService.deleteOneServerlessFunction(
step.settings.input.serverlessFunctionId,
workspaceId,
);
}
}
cb().catch(Logger.catch)
});
})
Promise.all( | ||
workflowVersions.map((workflowVersion) => { | ||
workflowVersion.steps?.map(async (step) => { | ||
if (step.type === WorkflowActionType.CODE) { | ||
await this.serverlessFunctionService.deleteOneServerlessFunction( | ||
step.settings.input.serverlessFunctionId, | ||
workspaceId, | ||
); | ||
} | ||
}); | ||
}), | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uncaught promise rejections make a Node.js process fail. According to my tests, we have a global listener to prevent such crashes, but I would still catch the errors explicitly to provide more context about the error.
As I said, I couldn't see where we did something similar, so we might first want to discuss it broadly with the team.
...twenty-server/src/engine/metadata-modules/serverless-function/serverless-function.service.ts
Outdated
Show resolved
Hide resolved
be6daad
to
39bc520
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perfect!
Delete workflow versions serverless functions when soft delete workflow