v0.14.0 rpts2 cache issue when building `umd` with `cjs` #892

MagnusBrzenk · 2020-10-01T16:50:51Z

Current Behavior

Right out of the box, running tsdx build --format umd,cjs,esm causes following error:

> tsdx build --format esm,cjs,umd

✓ Creating entry file 1.1 secs
(typescript) Error: ENOENT: no such file or directory, rename '/Users/magnus/work/javascript/myapp/node_modules/.cache/rollup-plugin-typescript2/rpt2_4c61ae4392b9bd24f4d43d13a3f56419b8b5805d/code/cache_' -> '/Users/magnus/work/javascript/myapp/node_modules/.cache/rollup-plugin-typescript2/rpt2_4c61ae4392b9bd24f4d43d13a3f56419b8b5805d/code/cache'
Error: ENOENT: no such file or directory, rename '/Users/magnus/work/javascript/myapp/node_modules/.cache/rollup-plugin-typescript2/rpt2_4c61ae4392b9bd24f4d43d13a3f56419b8b5805d/code/cache_' -> '/Users/magnus/work/javascript/myapp/node_modules/.cache/rollup-plugin-typescript2/rpt2_4c61ae4392b9bd24f4d43d13a3f56419b8b5805d/code/cache'
    at Object.renameSync (fs.js:660:3)
    at RollingCache.roll (/Users/magnus/work/javascript/myapp/node_modules/tsdx/node_modules/rollup-plugin-typescript2/dist/rollup-plugin-typescript2.cjs.js:24003:12)
    at TsCache.done (/Users/magnus/work/javascript/myapp/node_modules/tsdx/node_modules/rollup-plugin-typescript2/dist/rollup-plugin-typescript2.cjs.js:24845:24)
    at Object._ongenerate (/Users/magnus/work/javascript/myapp/node_modules/tsdx/node_modules/rollup-plugin-typescript2/dist/rollup-plugin-typescript2.cjs.js:29098:21)
    at Object.generateBundle (/Users/magnus/work/javascript/myapp/node_modules/tsdx/node_modules/rollup-plugin-typescript2/dist/rollup-plugin-typescript2.cjs.js:29076:18)
    at /Users/magnus/work/javascript/myapp/node_modules/rollup/dist/shared/node-entry.js:13117:25
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `tsdx build --format esm,cjs,umd`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/magnus/.npm/_logs/2020-10-01T16_45_40_976Z-debug.log

After trying different combinations, I discerned that 'umd' is incompatible with 'cjs'.

Expected behavior

The documentation gives the impression that you can build all three targets simultaneously by running tsdx build --format umd,cjs,esm. If this is not possible to build both 'umd' and 'cjs' types at once then clarify this is in the documentation; otherwise you have a bug here.

Additional context

Your environment

  System:
    OS: macOS 10.15.6
    CPU: (8) x64 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
    Memory: 341.14 MB / 16.00 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 12.16.1 - ~/.nvm/versions/node/v12.16.1/bin/node
    Yarn: 1.22.5 - ~/.nvm/versions/node/v12.16.1/bin/yarn
    npm: 6.13.4 - ~/.nvm/versions/node/v12.16.1/bin/npm
  Browsers:
    Brave Browser: 85.1.14.81
    Chrome: 85.0.4183.121
    Firefox: 81.0
    Safari: 14.0
  npmPackages:
    tsdx: ^0.14.0 => 0.14.0
    typescript: ^4.0.3 => 4.0.3

The text was updated successfully, but these errors were encountered:

agilgur5 · 2020-10-01T23:32:10Z

Based on the logs, sounds like a duplicate of #888. (EDIT: I've now marked that issue as duplicative of this issue because this one has more details now.) Per my response there:

Looks like this is a cache bug upstream that was recently fixed: ezolenko/rollup-plugin-typescript2#243

Per comments there, you can workaround this by deleting the cache. I don't think you need to fully nuke node_modules, think you can just delete node_modules/.cache/rollup-plugin-typescript2/

After trying different combinations, I discerned that 'umd' is incompatible with 'cjs'.

That's an interesting find though upstream in rollup-plugin-typescript2 it has no knowledge of format (I've worked on that codebase several times now), so more likely it's some bad hash and race condition that happens during updating

The documentation gives the impression that you can build all three targets simultaneously

You can, a central piece of TSDX and even Rollup itself, and there are tests for this too. You just appear to have hit a transient cache bug upstream, which the logs point to.

Brings in ezolenko/rollup-plugin-typescript2#243 in order to fix jaredpalmer#888 and jaredpalmer#892.

agilgur5 · 2020-10-12T06:03:51Z

So I tried a few things and confirmed the umd + cjs issue. I also confirmed it was happening with system + cjs. But not esm + cjs for some reason. I also tried ordering it differently, but that didn't change anything (they are parallelized in any case but 🤷 ). I also confirmed umd + system had the issue as well.

The system addition is interesting, as that shouldn't have much difference from the others aside from format. system + umd also knocks the suggestion that it's some cjs issue out 😕 😕

Here's some logs:

$$ tsdx create test-tsdx && cd test-tsdx
[...]

$$ yarn build --format umd
yarn run v1.22.4
$ tsdx build --format umd
✓ Building modules 674 ms
✨  Done in 3.96s.

$$ yarn build --format cjs,umd
yarn run v1.22.4
$ tsdx build --format cjs,umd
✓ Creating entry file 967 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'
[...]

$$ yarn build --format system,umd
yarn run v1.22.4
$ tsdx build --format system,umd
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'
[...]

$$ yarn build --format cjs,esm
yarn run v1.22.4
$ tsdx build --format cjs,esm
✓ Creating entry file 957 ms
✓ Building modules 1.1 secs
✨  Done in 3.86s.

$$ yarn build --format cjs,esm,umd
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd
✓ Creating entry file 954 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'
[...]

$$ yarn build --format cjs,esm,system
yarn run v1.22.4
$ tsdx build --format cjs,esm,system
✓ Creating entry file 963 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'
[...]

$$ yarn build --format cjs,system
yarn run v1.22.4
$ tsdx build --format cjs,system
✓ Creating entry file 969 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'

$$ yarn build --format esm,system
yarn run v1.22.4
$ tsdx build --format esm,system
✓ Building modules 1.2 secs
✨  Done in 3.94s.

$$ yarn build --format esm,umd
yarn run v1.22.4
$ tsdx build --format esm,umd
✓ Building modules 634 ms
✨  Done in 3.36s.

# different ordering
$$ yarn build --format esm,cjs,umd
yarn run v1.22.4
$ tsdx build --format esm,cjs,umd
✓ Creating entry file 980 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'
[...]

# different ordering
$$ yarn build --format esm,umd,cjs
yarn run v1.22.4
$ tsdx build --format esm,umd,cjs
✓ Creating entry file 979 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'
[...]

$$ yarn build --format esm,umd,system
yarn run v1.22.4
$ tsdx build --format esm,umd,system
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_dc5aaa4ea403c2ba50e68d0baecc90e56a88ad12/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_dc5aaa4ea403c2ba50e68d0baecc90e56a88ad12/code/cache'
[...]

$$ yarn build --format umd,system
yarn run v1.22.4
$ tsdx build --format umd,system
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'
[...]

$$ yarn build --format umd,cjs
yarn run v1.22.4
$ tsdx build --format umd,cjs
✓ Creating entry file 962 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/oss/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache_' -
> '[redacted]/oss/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_d8917360bd7c6c37111985035e75a4930df504e4/code/cache'

agilgur5 · 2020-10-12T06:15:56Z

Additionally confirmed that this issue did exist in TSDX v0.13.3, but only for the cjs + system combo, which is much less frequent. Was confused that it hadn't happened with v0.13.3 despite it using the same version of rpts2 as v0.14.0, but this allays some of that. And also makes it seem more of a hashing bug or race condition upstream than a specific format, which would've been confusing too given the lack of knowledge of format.

More logs:

$ yarn add [email protected]
yarn add v1.22.4
[...]
$ tsdx build
✓ Creating entry file 1.3 secs
✓ Building modules 2.2 secs
✨  Done in 17.88s.

$$ yarn build --format umd,cjs
yarn run v1.22.4
$ tsdx build --format umd,cjs
✓ Creating entry file 1 secs
✓ Building modules 1.9 secs
✨  Done in 4.10s.

$$ yarn build --format umd,cjs,esm,system
yarn run v1.22.4
$ tsdx build --format umd,cjs,esm,system
✓ Creating entry file 919 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache'
[...]

$$ yarn build --format umd,esm
yarn run v1.22.4
$ tsdx build --format umd,esm
✓ Building modules 715 ms
✨  Done in 2.68s.

$$ yarn build --format umd,esm,cjs
yarn run v1.22.4
$ tsdx build --format umd,esm,cjs
✓ Creating entry file 924 ms
✓ Building modules 1.4 secs
✨  Done in 3.40s.

$$ yarn build --format umd,system
yarn run v1.22.4
$ tsdx build --format umd,system
✓ Building modules 907 ms
✨  Done in 2.86s.
✔ ~/Desktop/GitHub/oss/test-tsdx (⎈ |services.k8s.dev.ns8.ninja:default) 

$$ yarn build --format cjs,system
yarn run v1.22.4
$ tsdx build --format cjs,system
✓ Creating entry file 915 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache'
[...]

$$ yarn build --format cjs,esm,umd
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd
✓ Creating entry file 924 ms
✓ Building modules 1.4 secs
✨  Done in 3.43s.

$$ yarn build --format umd,system
yarn run v1.22.4
$ tsdx build --format umd,system
✓ Building modules 897 ms
✨  Done in 2.87s.

$$ yarn build --format cjs,system
yarn run v1.22.4
$ tsdx build --format cjs,system
✓ Creating entry file 913 ms
(typescript) Error: ENOENT: no such file or directory, rename '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache_' -
> '[redacted]/test-tsdx/node_modules/.cache/rollup-plugin-typescript2/rpt2_57678ef6f24a79cbc0a37de5d02872e29b66702a/code/cache'
[...]

$$ yarn build --format cjs,esm,umd
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd
✓ Creating entry file 914 ms
✓ Building modules 1.4 secs
✨  Done in 3.34s.

agilgur5 · 2020-10-12T06:31:28Z

Hmmm v0.13.2, v0.13.1, v0.13.0, and v0.12 don't have this issue 🤔 That being said, I'm not sure I'd call this a regression per se, because, per my investigation in #896 and ezolenko/rollup-plugin-typescript2#243, that section of code upstream has been like that for a long time. But TSDX's cache directories and Rollup config (which impacts the hash) have changed over time. 🤔

Side note: the perf decrease when downgrading was noticeable, so that confirms the more recent optimizations I've made help in a variety of codebase sizes 🙂

v0.13.3 consolidated the cache directories for performance reasons (more matching hashes); previously there was a different one for each format, so perhaps that creates more races or makes for an incorrectly matched hash. The problematic cache seems to have an identical hash in all of the logs (only one is different from the rest) 🤔

Logs:

$$ yarn add [email protected]
yarn add v1.22.4
[...]
$ tsdx build
✓ Creating entry file 1.8 secs
✓ Building modules 753 ms
✨  Done in 7.66s.

$$ yarn build --format cjs,esm,umd,system
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd,system
✓ Creating entry file 3.4 secs
✓ Building modules 1.3 secs
✨  Done in 5.88s.

$$ yarn build --format umd,system
yarn run v1.22.4
$ tsdx build --format umd,system
✓ Building modules 944 ms
✨  Done in 4.15s.

$$ yarn build --format umd,cjs,esm
yarn run v1.22.4
$ tsdx build --format umd,cjs,esm
✓ Creating entry file 2.6 secs
✓ Building modules 912 ms
✨  Done in 4.65s.

$$ yarn build --format umd,cjs
yarn run v1.22.4
$ tsdx build --format umd,cjs
✓ Creating entry file 2.1 secs
✓ Building modules 985 ms
✨  Done in 4.18s.

$$ yarn build --format umd,esm
yarn run v1.22.4
$ tsdx build --format umd,esm
✓ Building modules 568 ms
✨  Done in 3.46s.

$$ yarn build --format cjs,system
yarn run v1.22.4
$ tsdx build --format cjs,system
✓ Creating entry file 2.2 secs
✓ Building modules 939 ms
✨  Done in 4.22s.

$$ yarn build --format esm,system
yarn run v1.22.4
$ tsdx build --format esm,system
✓ Building modules 618 ms
✨  Done in 3.43s.

$$ yarn build --format esm,umd,system
yarn run v1.22.4
$ tsdx build --format esm,umd,system
✓ Building modules 1.1 secs
✨  Done in 4.93s.

$$ yarn build --format cjs,umd,system
yarn run v1.22.4
$ tsdx build --format cjs,umd,system
✓ Creating entry file 3 secs
✓ Building modules 1.2 secs
✨  Done in 5.25s.

$$ yarn build --format cjs,esm,umd
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd
✓ Creating entry file 2.6 secs
✓ Building modules 916 ms
✨  Done in 4.63s.

$$ yarn add [email protected]
yarn add v1.22.4
[...]
$ tsdx build
✓ Creating entry file 1.8 secs
✓ Building modules 686 ms
✨  Done in 8.09s.

$$ yarn build --format cjs,esm,umd,system
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd,system
✓ Creating entry file 3.5 secs
✓ Building modules 1.3 secs
✨  Done in 5.91s.

# second time for cache etc
$$ yarn build --format cjs,esm,umd,system
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd,system
✓ Creating entry file 3.5 secs
✓ Building modules 1.3 secs
✨  Done in 5.87s.

$$ yarn add [email protected]
yarn add v1.22.4
[...]
$ tsdx build
✓ Creating entry file 1.9 secs
✓ Building modules 652 ms
✨  Done in 7.44s.

$$ yarn build --format cjs,esm,umd,system
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd,system
✓ Creating entry file 3.4 secs
✓ Building modules 1.2 secs
✨  Done in 5.75s.

# second time for cache etc
$$ yarn build --format cjs,esm,umd,system
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd,system
✓ Creating entry file 3.5 secs
✓ Building modules 1.2 secs
✨  Done in 5.76s.

$$ yarn add [email protected]
[...]
$ tsdx build
✓ Creating entry file 2.4 secs
✓ Building modules 960 ms
✨  Done in 9.50s.

$$ yarn build --format cjs,esm,umd,system
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd,system
✓ Creating entry file 3.7 secs
✓ Building modules 1.4 secs
✨  Done in 6.28s.

# second time for cache etc
$$ yarn build --format cjs,esm,umd,system
yarn run v1.22.4
$ tsdx build --format cjs,esm,umd,system
✓ Creating entry file 3.7 secs
✓ Building modules 1.3 secs
✨  Done in 6.14s.

agilgur5 · 2020-10-13T05:16:39Z

This has been fixed with #896 / ezolenko/rollup-plugin-typescript2#243 and just released in v0.14.1.

Still waiting for a response upstream to understand root cause however.

agilgur5 changed the title ~~umd incompatible with cjs in tsdx build --format umd,cjs~~ v0.14.0 rpts2 cache issue when building umd with cjs Oct 1, 2020

agilgur5 added kind: support Asking for support with something or a specific use case scope: upstream Issue in upstream dependency solution: workaround available There is a workaround available for this issue labels Oct 1, 2020

agilgur5 closed this as completed Oct 1, 2020

agilgur5 added the solution: duplicate This issue or pull request already exists label Oct 1, 2020

tanem added a commit to tanem/tsdx that referenced this issue Oct 6, 2020

fix: update rollup-plugin-typescript2 to fix build issue

f4eb488

Brings in ezolenko/rollup-plugin-typescript2#243 in order to fix jaredpalmer#888 and jaredpalmer#892.

tanem mentioned this issue Oct 6, 2020

fix/deps: upgrade rollup-plugin-typescript2 to fix cache issue #896

Merged

agilgur5 linked a pull request Oct 6, 2020 that will close this issue

fix/deps: upgrade rollup-plugin-typescript2 to fix cache issue #896

Merged

agilgur5 mentioned this issue Oct 7, 2020

stop roll() from failing if the new cache folder is not created ezolenko/rollup-plugin-typescript2#243

Merged

agilgur5 added the problem: stale Issue has not been responded to in some time label Oct 8, 2020

This was referenced Oct 11, 2020

Node.js 13.14.0 'No "exports" main resolved in' @babel/helper-compilation-targets/package.json #900

Closed

Request: provide clear example on creating non-trivial UMD bundles #898

Closed

agilgur5 added kind: bug Something isn't working and removed kind: support Asking for support with something or a specific use case labels Oct 12, 2020

agilgur5 mentioned this issue Oct 12, 2020

v0.14.0 rpts2 cache issue with Yarn workspaces #888

Closed

agilgur5 removed the solution: duplicate This issue or pull request already exists label Oct 12, 2020

agilgur5 added the topic: rollup-plugin-typescript2 Issues and PRs relating to rpts2 label Nov 1, 2020

agilgur5 mentioned this issue Jun 8, 2022

Monorepo: caching logic fails when using yarn workspace ezolenko/rollup-plugin-typescript2#232

Closed

Repository owner locked as resolved and limited conversation to collaborators Jun 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v0.14.0 rpts2 cache issue when building `umd` with `cjs` #892

v0.14.0 rpts2 cache issue when building `umd` with `cjs` #892

MagnusBrzenk commented Oct 1, 2020

agilgur5 commented Oct 1, 2020 •

edited

Loading

agilgur5 commented Oct 12, 2020 •

edited

Loading

agilgur5 commented Oct 12, 2020 •

edited

Loading

agilgur5 commented Oct 12, 2020 •

edited

Loading

agilgur5 commented Oct 13, 2020

v0.14.0 rpts2 cache issue when building umd with cjs #892

v0.14.0 rpts2 cache issue when building umd with cjs #892

Comments

MagnusBrzenk commented Oct 1, 2020

Current Behavior

Expected behavior

Suggested solution(s)

Additional context

Your environment

agilgur5 commented Oct 1, 2020 • edited Loading

agilgur5 commented Oct 12, 2020 • edited Loading

agilgur5 commented Oct 12, 2020 • edited Loading

agilgur5 commented Oct 12, 2020 • edited Loading

agilgur5 commented Oct 13, 2020

v0.14.0 rpts2 cache issue when building `umd` with `cjs` #892

v0.14.0 rpts2 cache issue when building `umd` with `cjs` #892

agilgur5 commented Oct 1, 2020 •

edited

Loading

agilgur5 commented Oct 12, 2020 •

edited

Loading

agilgur5 commented Oct 12, 2020 •

edited

Loading

agilgur5 commented Oct 12, 2020 •

edited

Loading