General Refresh: rewrite as TypeScript with both CommonJS and ESM + tree shaking support #35

CedarMist · 2024-05-01T16:15:23Z

Closes #12
Closes #27
Closes #31
Closes #34

bumps version to 0.0.6
improves README with links to authors, example code etc.
includes the uint32 library
This now includes only the ct32 implementation of AES from the bsaes library
- The vartime implementation has been removed from AES, it was not enabled by default and thus is unused.
Moved to using https://vitest.dev/ for tests & benchmarks
Linting uses biome
Everything is ESM & CommonJS compatibile, with ESM tree-shaking support.
- With a minified build this resolves to 10kb of code
API remains compatible, with full type annotations via TypeScript
CI runs code coverage, we're at 99.8%
Using pnpm instead of npm

CedarMist · 2024-05-01T16:25:22Z

Oops, ran into one problem: "default" is not exported by

import deoxysii from '@oasisprotocol/deoxysii';

Fixed by exporting a namespace, this doesn't affect tree-shakability.

pro-wh

lotta new stuff here. haven't looked through the moved code yet

so we're vendoring in uint32 and bsaes? is it that those libraries didn't have good tree shaking?

scripts/rename-cjs.mjs

package.json

scripts/tsconfig.json

scripts/rename-cjs.mjs

pro-wh · 2024-05-01T17:58:07Z

tsconfig.base.json

+		"useDefineForClassFields": true, // Not enabled by default in `strict` mode unless we bump `target` to ES2022.
+		"noFallthroughCasesInSwitch": true, // Not enabled by default in `strict` mode.
+		"noImplicitReturns": true, // Not enabled by default in `strict` mode.
+		"useUnknownInCatchVariables": true, // TODO: This would normally be enabled in `strict` mode but would require some adjustments to the codebase.


This would normally be enabled in strict mode but would require some adjustments to the codebase.

yet this line doesn't disable it? so are the adjustments made?

tsconfig.base.json

pro-wh · 2024-05-01T18:01:46Z

tsconfig.base.json

+		// Language and environment
+		"moduleResolution": "NodeNext",
+		"module": "NodeNext",
+		"target": "ES2021", // Setting this to `ES2021` enables native support for `Node v16+`: https://github.com/microsoft/TypeScript/wiki/Node-Target-Mapping.


are we still supporting node 16?

No, Node 18+ only - https://endoflife.date/nodejs

Security updates stopped for Node 16 last year, active support ended 1.5 years ago.

https://github.com/microsoft/TypeScript/wiki/Node-Target-Mapping should we target node 18 then?

pro-wh · 2024-05-01T18:22:05Z

package.json

+		"type": "git",
+		"url": "https://github.com/oasisprotocol/deoxysii-js.git"
+	},
+	"license": "MIT",


contains a lot of differently licensed code now

deoxysii.ts & aes.ts are still MIT licensed

The uint32 package doesn't really have a license (and is explicitly noted by the author as something you can do whatever tf you want with it), so that's been noted in an appropriate SPDX-License-Identifier. I believe it would be entirely kosher to strip the license header, add my own copyright and re-license it as MIT if I wanted.

strip the license header

sounds like that's allowed

add my own copyright

I don't think the license controls whether you can do that, and I think you can't do that

re-license it as MIT

I think you have enough permission to redistribute it under MIT, which is more restrictive

It would be fair to add a copyright notice to it given that the changes I made to it are, while trivial, about as much effort as writing it in the first place.

Either way I'm not fussed, please advise on what to do with uint32.ts and the associated uint32.test.ts files.

if it was done by changing the uint32.js albeit extensively, that sounds like it's a derivative work. I've read that it's usually preferred to contribute to a project under its original license, which for the uint32.js library was a custom license. so overall, this deoxysii-js package would have a custom license, of what would be expressed in pseudo-SPDX as (MIT AND uint32.js's license)

biome.json

src/deoxysii.ts

src/TestVectors.json

lukaw3d · 2024-05-01T22:52:01Z

src/uint32.ts

@@ -0,0 +1,267 @@
+// SPDX-License-Identifier: WTFPL


Please split commits so that changes from the original are visible in a commit

Split. Diff from original files is now visible in 36fa971

package.json

pro-wh · 2024-05-01T23:07:38Z

src/uint32.ts

@@ -0,0 +1,267 @@
+// SPDX-License-Identifier: WTFPL


to be clear, this is a relicensing or new work/new license

uint32.js license (https://github.com/fxa/uint32.js?tab=readme-ov-file#license) is:

Do, what You want. This library was designed to be a copy/paste template. It would be nice to hear from You, if You used this library or if You just copy/pasted some source. It would be nice, if you added a "contains code of Franz X Antesberger" or something like that or a ref to the github project to your license information or elsewhere.

WTFPL (https://spdx.org/licenses/WTFPL.html) is:

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
Version 2, December 2004

Copyright (C) 2004 Sam Hocevar [email protected]

Everyone is permitted to copy and distribute verbatim or modified copies of this license document, and changing it is allowed as long as the name is changed.

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

You just DO WHAT THE FUCK YOU WANT TO.

some sentiment in common though

src/deoxysii.ts

CedarMist · 2024-05-02T00:54:46Z

Ok @lukaw3d and @pro-wh

I have incorporated your comments to the best that I'm able to, thank you for the feedback as I'd missed or not considered a few things that you pointed out.

I've also updated the README so it's pretty and includes an example.

lukaw3d · 2024-05-02T01:24:42Z

scripts/rename-cjs.mjs

+/** Renames `.js` files in the /lib/cjs folder to `.cjs` to appease ESM clients. */
+
+import { promises as fs } from "node:fs";
+import url from "node:url";
+
+process.chdir(url.fileURLToPath(`${import.meta.url}/../../dist/_cjs`));
+for (const filename of await fs.readdir(".")) {
+	if (!/\.js$/.test(filename)) continue;
+	const cjs = await fs.readFile(filename, "utf-8");
+	await fs.writeFile(
+		filename.replace(/\.js$/, ".cjs"),
+		cjs.replaceAll(/require\("(.\/[\w\-.]+)\.js"\)/g, 'require("$1.cjs")'),
+	);
+	await fs.unlink(filename);
+}


I tried comparing your previous version + without namespace + using tsup (https://antfu.me/posts/publish-esm-and-cjs#tsup)

32e7271

might be useful some day. The changed code is inside each tgz/src

tsup & unbuild look really good, and I really don't like the rename-cjs hack. I wish there was a simpler way that only needed a single set of files.

Unfortunately there seems to be no safe way to drop CommonJS support

$ pnpm tsup src/index.ts --format cjs,esm --dts --clean --minify --sourcemap CLI Building entry: src/index.ts CLI Using tsconfig: tsconfig.json CLI tsup v8.0.2 CLI Target: es2021 CLI Cleaning output folder CJS Build start ESM Build start ESM dist/index.mjs 10.17 KB ESM dist/index.mjs.map 68.06 KB ESM ⚡️ Build success in 21ms CJS dist/index.js 10.69 KB CJS dist/index.js.map 68.32 KB CJS ⚡️ Build success in 22ms DTS Build start DTS ⚡️ Build success in 723ms DTS dist/index.d.ts 1.17 KB DTS dist/index.d.mts 1.17 KB

tsup is great, so much less hassle, so the project is about 3.7kb gzipped + minified and has no production dependencies, easy to build both CJS & ESM, this is exactly what I wanted.

Just trying to follow the recommendations on https://publint.dev/ (can be installed locally via publint) for due ESM & CJS packaging.

lukaw3d · 2024-05-02T17:23:08Z

package.json

+		"distclean": "pnpm run clean && rm -rf node_modules *.tgz",
+		"build": "pnpm tsup src/index.ts --format cjs,esm --dts --clean",
+		"prepublishOnly": "npm run build",
+		"debug-imports-cjs": "NODE_DEBUG=module,esm bash -c 'pnpm node --experimental-global-webcrypto examples/check.cjs 2> >(grep deoxysii)'",


I don't understand this. Is it just
NODE_DEBUG=module,esm node --experimental-global-webcrypto examples/check.cjs | grep deoxysii
minus the errors?

It greps stderr for logs from module or esm to show only ones related to deoxysii.

Essentially this double checks that check.cjs loads the .js file via the normal module loader, and check.mjs loads the .mjs file via ESM

> @oasisprotocol/[email protected] debug-imports /home/runner/work/deoxysii-js/deoxysii-js > pnpm run debug-imports-cjs && pnpm run debug-imports-esm > @oasisprotocol/[email protected] debug-imports-cjs /home/runner/work/deoxysii-js/deoxysii-js > NODE_DEBUG=module,esm bash -c 'pnpm node --experimental-global-webcrypto examples/check.cjs 2> >(grep deoxysii)' MODULE 2161: Module._load REQUEST /home/runner/work/deoxysii-js/deoxysii-js/.pnpmfile.cjs parent: /home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs MODULE 2161: looking for "/home/runner/work/deoxysii-js/deoxysii-js/.pnpmfile.cjs" in ["/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/dist/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules","/home/runner/node_modules","/home/node_modules","/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/.node_modules","/home/runner/.node_libraries","/opt/hostedtoolcache/node/18.20.2/x64/lib/node"] MODULE 2179: looking for "/home/runner/work/deoxysii-js/deoxysii-js/examples/check.cjs" in ["/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/.node_modules","/home/runner/.node_libraries","/opt/hostedtoolcache/node/18.20.2/x64/lib/node"] MODULE 2179: load "/home/runner/work/deoxysii-js/deoxysii-js/examples/check.cjs" for module "." MODULE 2179: Module._load REQUEST @oasisprotocol/deoxysii parent: . MODULE 2179: looking for "@oasisprotocol/deoxysii" in ["/home/runner/work/deoxysii-js/deoxysii-js/examples/node_modules","/home/runner/work/deoxysii-js/deoxysii-js/node_modules","/home/runner/work/deoxysii-js/node_modules","/home/runner/work/node_modules","/home/runner/node_modules","/home/node_modules","/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/nocjs [class AEAD] Encrypted: Uint8Array(27) [ 233, 24, 247, 21, 97, 209, 159, 253, 5, 151, 186, 13, 204, 42, 227, 216, 204, 121, 32, 176, 232, 24, 16, 216, 130, 102, 206 ] Decrypted: Hello World de_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/.node_modules","/home/runner/.node_libraries","/opt/hostedtoolcache/node/18.20.2/x64/lib/node"] MODULE 2179: load "/home/runner/work/deoxysii-js/deoxysii-js/dist/index.js" for module "/home/runner/work/deoxysii-js/deoxysii-js/dist/index.js" > @oasisprotocol/[email protected] debug-imports-esm /home/runner/work/deoxysii-js/deoxysii-js > NODE_DEBUG=module,esm bash -c 'pnpm node --experimental-global-webcrypto examples/check.mjs 2> >(grep deoxysii)' star [class AEAD] default [class AEAD] AEAD [class AEAD] Encrypted: Uint8Array(27) [ 175, 189, 113, 76, 235, 191, 199, 63, 217, 208, 56, 109, 32, 214, 207, 137, 242, 9, 9, 141, 6, 220, 222, 46, 100, 147, 174 ] Decrypted: Hello World MODULE 2204: Module._load REQUEST /home/runner/work/deoxysii-js/deoxysii-js/.pnpmfile.cjs parent: /home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/dist/pnpm.cjs MODULE 2204: looking for "/home/runner/work/deoxysii-js/deoxysii-js/.pnpmfile.cjs" in ["/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/dist/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules","/home/runner/node_modules","/home/node_modules","/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/bin/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules/pnpm/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/[email protected]/node_modules","/home/runner/setup-pnpm/node_modules/.pnpm/node_modules","/home/runner/.node_modules","/home/runner/.node_libraries","/opt/hostedtoolcache/node/18.20.2/x64/lib/node"] ESM 2222: Storing file:///home/runner/work/deoxysii-js/deoxysii-js/examples/check.mjs (implicit type) in ModuleLoadMap ESM 2222: Translating StandardModule file:///home/runner/work/deoxysii-js/deoxysii-js/examples/check.mjs ESM 2222: Storing file:///home/runner/work/deoxysii-js/deoxysii-js/dist/index.mjs (implicit type) in ModuleLoadMap ESM 2222: Translating StandardModule file:///home/runner/work/deoxysii-js/deoxysii-js/dist/index.mjs

…g support

CedarMist force-pushed the CedarMist/remove-uint32-dependency branch from 0253c58 to 32e18d0 Compare May 1, 2024 16:38

CedarMist self-assigned this May 1, 2024

CedarMist added p:2 Priority: desired feature s:needs review Status: needs review labels May 1, 2024

CedarMist marked this pull request as ready for review May 1, 2024 16:40

CedarMist requested review from kostko and peterjgilbert as code owners May 1, 2024 16:40

CedarMist requested a review from lukaw3d May 1, 2024 16:41

pro-wh reviewed May 1, 2024

View reviewed changes

CedarMist force-pushed the CedarMist/remove-uint32-dependency branch from 513f19d to 65b1f14 Compare May 1, 2024 22:38

CedarMist requested a review from pro-wh May 1, 2024 22:41

lukaw3d reviewed May 1, 2024

View reviewed changes

CedarMist changed the title ~~repackaged as zero-dependency typescript library~~ General Refresh: rewrite as TypeScript with both CommonJS and ESM + tree shaking support, May 1, 2024

pro-wh reviewed May 1, 2024

View reviewed changes

lukaw3d reviewed May 2, 2024

View reviewed changes

src/deoxysii.ts Outdated Show resolved Hide resolved

CedarMist changed the title ~~General Refresh: rewrite as TypeScript with both CommonJS and ESM + tree shaking support,~~ General Refresh: rewrite as TypeScript with both CommonJS and ESM + tree shaking support May 2, 2024

CedarMist force-pushed the CedarMist/remove-uint32-dependency branch 3 times, most recently from 1cae564 to 58cc004 Compare May 2, 2024 00:50

CedarMist force-pushed the CedarMist/remove-uint32-dependency branch 2 times, most recently from a24f397 to 361ddbd Compare May 2, 2024 01:06

lukaw3d reviewed May 2, 2024

View reviewed changes

CedarMist requested a review from lukaw3d May 2, 2024 17:00

lukaw3d reviewed May 2, 2024

View reviewed changes

lukaw3d and others added 4 commits May 3, 2024 00:45

Rename files

185dc68

Copy from oasislabs/bsaes.js

9fa497e

Copy from fxa/uint32.js

62a7b57

Format test vectors

ba2f33b

CedarMist added 2 commits May 3, 2024 00:55

0.0.6: rewrite as TypeScript with both CommonJS and ESM + tree shakin…

559d51d

…g support

moved to tsup with checks for CJS & ESM import styles

98028ea

lukaw3d force-pushed the CedarMist/remove-uint32-dependency branch from 0fbf0b9 to a8efa77 Compare May 2, 2024 23:26

lukaw3d approved these changes May 2, 2024

View reviewed changes

CedarMist force-pushed the CedarMist/remove-uint32-dependency branch from a8efa77 to 98028ea Compare May 3, 2024 10:29

removed unnecessary typescript peer dependencies

f64fedd

CedarMist added s:ready Status: ready to work/merge and removed s:needs review Status: needs review labels May 3, 2024

kostko approved these changes May 7, 2024

View reviewed changes

added release NPMJS.com publishing workflow

8bc5fda

CedarMist merged commit 7018d83 into master May 8, 2024
1 check passed

CedarMist deleted the CedarMist/remove-uint32-dependency branch May 8, 2024 11:23

CedarMist mentioned this pull request May 8, 2024

Add NPM Provenance with publishing via GitHub Actions for releases #38

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

General Refresh: rewrite as TypeScript with both CommonJS and ESM + tree shaking support #35

General Refresh: rewrite as TypeScript with both CommonJS and ESM + tree shaking support #35

CedarMist commented May 1, 2024 •

edited

Loading

CedarMist commented May 1, 2024 •

edited

Loading

pro-wh left a comment

pro-wh May 1, 2024

pro-wh May 1, 2024

CedarMist May 1, 2024

pro-wh May 1, 2024

pro-wh May 1, 2024

CedarMist May 1, 2024

pro-wh May 1, 2024

pro-wh May 1, 2024

CedarMist May 1, 2024 •

edited

Loading

pro-wh May 1, 2024

lukaw3d May 1, 2024

lukaw3d May 2, 2024

pro-wh May 1, 2024 •

edited

Loading

CedarMist commented May 2, 2024

lukaw3d May 2, 2024 •

edited

Loading

CedarMist May 2, 2024 •

edited

Loading

CedarMist May 2, 2024 •

edited

Loading

lukaw3d May 2, 2024

CedarMist May 2, 2024

CedarMist May 2, 2024

General Refresh: rewrite as TypeScript with both CommonJS and ESM + tree shaking support #35

General Refresh: rewrite as TypeScript with both CommonJS and ESM + tree shaking support #35

Conversation

CedarMist commented May 1, 2024 • edited Loading

CedarMist commented May 1, 2024 • edited Loading

pro-wh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CedarMist May 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pro-wh May 1, 2024 • edited Loading

Choose a reason for hiding this comment

CedarMist commented May 2, 2024

lukaw3d May 2, 2024 • edited Loading

Choose a reason for hiding this comment

CedarMist May 2, 2024 • edited Loading

Choose a reason for hiding this comment

CedarMist May 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CedarMist commented May 1, 2024 •

edited

Loading

CedarMist commented May 1, 2024 •

edited

Loading

CedarMist May 1, 2024 •

edited

Loading

pro-wh May 1, 2024 •

edited

Loading

lukaw3d May 2, 2024 •

edited

Loading

CedarMist May 2, 2024 •

edited

Loading

CedarMist May 2, 2024 •

edited

Loading