Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[HOLD] fix(worker): Avoid webpack error on "node:" modules in workflows #1037

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
+20 −12
Draft

[HOLD] fix(worker): Avoid webpack error on "node:" modules in workflows #1037

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
023c27d
fix(worker): Avoid webpack error on "node:" modules in workflows
mjameswh Jan 26, 2023
8039205
- Remove `alias` for things that `externals` can do
mjameswh Jan 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion packages/test/src/mocks/workflows-with-node-dependencies/example/client.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
import dns from 'dns';
import http from 'node:http';

export function exampleHeavyweightFunction(): void {
// Dummy code, only to ensure dependencies on 'dns' and 'node:http' do not get removed.
// This code will actually never run.
dns.resolve('localhost', () => {
/* ignore */
http.get('localhost');
});
}
2 changes: 1 addition & 1 deletion packages/test/src/test-bundler.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ if (RUN_INTEGRATION_TESTS) {
const taskQueue = `${t.title}-${uuid4()}`;
const workflowBundle = await bundleWorkflowCode({
workflowsPath: require.resolve('./mocks/workflows-with-node-dependencies/issue-516'),
ignoreModules: ['dns'],
ignoreModules: ['dns', 'http'],
});
const worker = await Worker.create({
taskQueue,
Expand Down
25 changes: 15 additions & 10 deletions packages/worker/src/workflow/bundler.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ export class WorkflowCodeBundler {
this.payloadConverterPath = payloadConverterPath;
this.failureConverterPath = failureConverterPath;
this.workflowInterceptorModules = workflowInterceptorModules ?? defaultWorflowInterceptorModules;
this.ignoreModules = ignoreModules ?? [];
this.ignoreModules = (ignoreModules ?? []).map((x) => (x.startsWith('node:') ? x.substring('node:'.length) : x));
this.webpackConfigHook = webpackConfigHook ?? ((config) => config);
}

Expand Down Expand Up @@ -183,14 +183,20 @@ exports.importInterceptors = function importInterceptors() {
entry: string,
distDir: string
): Promise<string> {
const captureProblematicModules: Configuration['externals'] = async (data, _callback): Promise<undefined> => {
// Ignore the "node:" prefix if any.
const module: string = data.request?.startsWith('node:')
? data.request.slice('node:'.length)
: data.request ?? '';

if (moduleMatches(module, disallowedModules) && !moduleMatches(module, this.ignoreModules)) {
this.foundProblematicModules.add(module);
const captureProblematicModules: Configuration['externals'] = async (
data,
_callback
): Promise<string | undefined> => {
const hasNodePrefix = data.request?.startsWith('node:');
const module: string = (hasNodePrefix ? data.request?.slice('node:'.length) : data.request) ?? '';
if (hasNodePrefix || moduleMatches(module, disallowedModules) || moduleMatches(module, this.ignoreModules)) {
if (!moduleMatches(module, this.ignoreModules)) {
this.foundProblematicModules.add(module);
}

// Tell webpack to replace that module by an empty object.
// This is functionnally equivalent to `alias: { 'some-module': false }`, but `externals` is called much more early
return 'var {}';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change to return an empty object? Is this better experience in your opinion?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean... Do you mean var false vs var {}?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to return undefined here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see... Actually, THIS is the fix.

By returning var {}, we are telling Webpack to replace imports of that module by an empty object. This is essentially the same as what we do with alias, but externals is called much earlier in the process, so that we can still handle invalid namespaces such as node:.

}

return undefined;
Expand All @@ -204,7 +210,6 @@ exports.importInterceptors = function importInterceptors() {
alias: {
__temporal_custom_payload_converter$: this.payloadConverterPath ?? false,
__temporal_custom_failure_converter$: this.failureConverterPath ?? false,
...Object.fromEntries([...this.ignoreModules, ...disallowedModules].map((m) => [m, false])),
},
},
externals: captureProblematicModules,
Expand Down