Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove serverless functions on soft delete #9438

Conversation

martmull
Copy link
Contributor

@martmull martmull commented Jan 7, 2025

Delete workflow versions serverless functions when soft delete workflow

@martmull martmull marked this pull request as ready for review January 7, 2025 14:47
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR implements serverless function cleanup when workflows are soft-deleted, ensuring proper resource management across the system.

  • Added ServerlessFunctionModule import in workflow-query-hook.module.ts to enable serverless function cleanup
  • Modified cleanWorkflowsSubEntities in workflow-common.workspace-service.ts to accept workspaceId parameter for scoped operations
  • Added deleteServerlessFunctions method in workflow-common.workspace-service.ts for handling serverless function cleanup
  • Updated post-query hooks to pass workspace ID from auth context to cleanup methods

4 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile

Comment on lines 120 to 131
Promise.all(
workflowVersions.map((workflowVersion) => {
workflowVersion.steps?.map(async (step) => {
if (step.type === WorkflowActionType.CODE) {
await this.serverlessFunctionService.deleteOneServerlessFunction(
step.settings.input.serverlessFunctionId,
workspaceId,
);
}
});
}),
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Double nested Promise.all/map with async callback will not wait for serverless function deletions to complete. Array.map() with async doesn't return promises that can be awaited

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

greptile is kinda true here.

Promise.all expects an array of Promise. Here, we give an Array<Array<Promise>>. If we want to wait for all the promises to be fulfilled, we must flatten the array with Array#flat.

Furtheremore, calling Promise.all is only useful if you want to wait for a set of promises to be fulfilled. Here, we don't wait for this, the function immediately returns—the Promise.all is not awaited. We can get rid of it.

The problem I can see here is that any failing promise won't be caught. I couldn't find any code snippet in the codebase where we do something similar, but I would do:

  workflowVersions.forEach((workflowVersion) => {
    workflowVersion.steps?.forEach((step) => {
      const cb = () => {
        if (step.type === WorkflowActionType.CODE) {
          await this.serverlessFunctionService.deleteOneServerlessFunction(
            step.settings.input.serverlessFunctionId,
            workspaceId,
          );
        }
      }

      cb().catch(Logger.catch)
    });
  })

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uncaught promise rejections make a Node.js process fail. According to my tests, we have a global listener to prevent such crashes, but I would still catch the errors explicitly to provide more context about the error.

As I said, I couldn't see where we did something similar, so we might first want to discuss it broadly with the team.

Copy link
Contributor

@Devessier Devessier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to open a discussion about some topics. I'm unsure the team agreed on a pattern here.

Comment on lines 120 to 131
Promise.all(
workflowVersions.map((workflowVersion) => {
workflowVersion.steps?.map(async (step) => {
if (step.type === WorkflowActionType.CODE) {
await this.serverlessFunctionService.deleteOneServerlessFunction(
step.settings.input.serverlessFunctionId,
workspaceId,
);
}
});
}),
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

greptile is kinda true here.

Promise.all expects an array of Promise. Here, we give an Array<Array<Promise>>. If we want to wait for all the promises to be fulfilled, we must flatten the array with Array#flat.

Furtheremore, calling Promise.all is only useful if you want to wait for a set of promises to be fulfilled. Here, we don't wait for this, the function immediately returns—the Promise.all is not awaited. We can get rid of it.

The problem I can see here is that any failing promise won't be caught. I couldn't find any code snippet in the codebase where we do something similar, but I would do:

  workflowVersions.forEach((workflowVersion) => {
    workflowVersion.steps?.forEach((step) => {
      const cb = () => {
        if (step.type === WorkflowActionType.CODE) {
          await this.serverlessFunctionService.deleteOneServerlessFunction(
            step.settings.input.serverlessFunctionId,
            workspaceId,
          );
        }
      }

      cb().catch(Logger.catch)
    });
  })

Comment on lines 120 to 131
Promise.all(
workflowVersions.map((workflowVersion) => {
workflowVersion.steps?.map(async (step) => {
if (step.type === WorkflowActionType.CODE) {
await this.serverlessFunctionService.deleteOneServerlessFunction(
step.settings.input.serverlessFunctionId,
workspaceId,
);
}
});
}),
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uncaught promise rejections make a Node.js process fail. According to my tests, we have a global listener to prevent such crashes, but I would still catch the errors explicitly to provide more context about the error.

As I said, I couldn't see where we did something similar, so we might first want to discuss it broadly with the team.

@martmull martmull force-pushed the 9393-soft-delete-workflow-should-delete-serverless-functions-related-to-the-workflow branch from be6daad to 39bc520 Compare January 7, 2025 15:43
@Devessier Devessier self-requested a review January 7, 2025 15:44
Copy link
Contributor

@Devessier Devessier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect!

@martmull martmull enabled auto-merge (squash) January 7, 2025 15:49
@martmull martmull merged commit 0c75b24 into main Jan 7, 2025
21 checks passed
@martmull martmull deleted the 9393-soft-delete-workflow-should-delete-serverless-functions-related-to-the-workflow branch January 7, 2025 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Soft delete workflow should delete serverless functions related to the workflow
3 participants