Migrating to require(esm) and esm-only

[43081j]

Node is starting to ship an unflagged feature to allow you to require(...) es modules

This means a few things:

• we should be able to migrate many dual packages to esm-only
• we should be able to upgrade dependencies which have since moved to esm-only (but were pinned to keep CJS compatibility)
• we should be able to migrate faux modules (i.e. those with type: "module" that export CJS) to esm-only

Research methods

High impact packages

The npm-esm-vs-cjs repo contains data sets that can help us understand which high impact packages use ESM, and which use CJS (or both).

Libraries

We can search this list for libraries and take the following action(s):

• if a package is esm-only, find high impact consumers of older CJS versions of the package and upgrade them to use the latest ESM version
• if a package is cjs-only, migrate it to esm-only
• if a package is dual (ESM/cjs), migrate it to esm-only

The joyee TODO list

@joyeecheung has helpfully already discovered many packages which publish CJS and ESM (i.e. they are dual packages) but are listed as ESM only.

You can view the list here

This could be a good TODO list for migrating to esm-only.

Considerations

• Not all packages can move yet since they need to support versions of node from before require(esm) was possible. we should create an issue before a PR, to find out if the maintainers are in this situation

Questions

@joyeecheung do you know what the minimum node version is needed for this feature? when we open issues, we will have to ask if its ok to constrain to that version

I suspect many packages will still want older node support, but we will see

[joyeecheung]

do you know what the minimum node version is needed for this feature? when we open issues, we will have to ask if its ok to constrain to that version

Currently the plan is:

1. Test it out a bit with v23, fix bugs found etc., and unflag it to v22 when it looks good enough (I think I have fixed all the bugs found on v23 so far). So far the consensus is that the unflagging can land on v22 in a later semver-minor release (likely this month).
2. Unflagging to v20 is possible (v20.x has it behind flags) but will require more work since the v20 branch is quite different from the main branch now. If we do that, I expect that it will be done before next April (cutoff of v24).
3. Currently, v23 emits experimental warning when require(esm) is hit. On v22, we plan to surpress the warning when the require(esm) comes from node_modules. If it's backported to v20 it will be similar to v22.
4. Aiming towards stabilization (no more experimental warning) by v24. Then we will backport all the bug fixes to the LTS release lines that have the unflagging and remove experimental warning there as well.

Also, the module-sync exports condition can be used as a way for package to feature-detect whether the current Node.js version supports require(esm) and supply the ESM version when it does, and fallback to supply CJS when it loaded by a Node.js version that doesn't support require(esm). Though package authors may have doubts about having more exports conditions, so this is probably only convincing enough when the package already uses exports condition and wish to ship ESM sooner. Note that some packages in the ecosystem already has module exports condition which basically serves the same purpose for bundlers (we tried to just implement that but bundlers have different ESM resolution rules, so simply picking that up would break packages that use resolution rules not supported by Node.js), in those cases it might be a easier sell to duplicate and point another module-sync to what module points to, and just make sure that the ESM being pointed to can run on Node.js just fine. The module-sync exports condition only serves for those who want to ship ESM sooner than all active LTS supports require(esm), and the expectation is that they won't keep it forever - once all Node.js versions they support have require(esm) they can remove all the messy conditions and probably just use default or main.