refactor(shims, extension): Optimize building and testing workflows #37

SMotaal · 2024-08-27T21:49:24Z

This optimizes the building and testing workflows for endoify using vitest.

This PR is now split out into:

chore(shims): Remove extraneous files and update package.json entries #39
- Removes extraneous files from packages/shims/src
  - For consistency — we will keep forgetting them otherwise
- Adds typescript in devDependencies
  - For consistency - needed since feat(shims): Bundle endoify dependencies #34
refactor(shims): Optimize building and testing #40
- Renames endoify.mjs to endoify.js in packages/{shims,extension}
  - For consistency — now all .mjs files are renamed to .js
- Delegates bundling of endoify.js to bundleSource
  - For reliability and predictability — avoids making assumptions and resolving paths inside third-party packages
- Optionally subs ZWJ-prefixed identifiers introduced by bundleSource with conforming CGJ alternatives
  - For conformance — makes it possible to experiment with rollup until endojs/endo#2436 lands
- Explicitly imports ./endoify.js in packages/shims/src/*.test.ts
  - For clarity and flexibility — test files need to be explicit
- Configures vitest to use jsdom with vmThreads for packages/shims
  - For consistency and reduced configuration overhead
refactor(extension): Optimize building and testing #41
- Adds endoify.ts and endoify.test.ts to packages/extension/src
  - For clarity — sibling files importing ./endoified.js
- Explicitly imports ./endoify.js in packages/extension/src/*.test.ts
  - For clarity and flexibility — test files need to be explicit
- Configures vitest to use jsdom with vmThreads for packages/extension
  - For consistency and reduced configuration overhead
- Adds guards against inlining script[src="endoify.ts"] and script[src="endoify.js"] in packages/extension/src/*.html
  - For efficacy and efficiency — parsing before vs after a failure makes no difference in speed, only in accuracy

packages/extension/vite.config.ts

SMotaal · 2024-08-27T22:28:09Z

packages/shims/scripts/bundle.js

+  if (!process.argv.includes('--without-rollup-transform')) {
+    source = endoScriptIdentifierTransformPlugin({
+      scopedRoot: path.resolve(rootDir, '../..'),
+    }).transform(source, specifier).code;
+  }


May be better to opt into the transform than to opt out.

Suggested change

if (!process.argv.includes('--without-rollup-transform')) {

source = endoScriptIdentifierTransformPlugin({

scopedRoot: path.resolve(rootDir, '../..'),

}).transform(source, specifier).code;

}

if (process.argv.includes('--with-rollup-transform')) {

source = endoScriptIdentifierTransformPlugin({

scopedRoot: path.resolve(rootDir, '../..'),

}).transform(source, specifier).code;

}

I'm in favour of keeping it in for now. The transform is deliberate, speedy and includes opt-in options for validation and debugging for if and when they would be needed.

Some of the cons is that this is still an extra step, and that it uses regular expressions instead of AST to rewrite the bundled sources, which I am arguing against myself.

Ideally, we could upstream the replacements to @endojs/endo and avoiding needing this step all together.

Either way, the flag needs to be renamed to --without-rollup-parser-optimizations or --with-rollup-parser-optimizations or something more reflective of what is actually taking place 🤷

I drafted endojs/endo#2436 today to see if this can be upstreamed!

SMotaal · 2024-08-27T22:44:11Z

packages/shims/src/endoify.js

Correct with proper git rm and git mv before landing to avoid incorrectly aliasing file history!

For context: From this diff, it seems that the file operations are being wrongly inferred. Instead, it seems to infer that apply-lock.mjs is being renamed to endoify.js, and that endoify.mjs is being deleted. If this lands, the history for those files will likely be misaligned with what we are explicitly intending in this PR.

Nope, history is still not picked up sadly!

Possible solution:

Remove apply-lockdown.mjs on main.

Rebase against main.

grypez · 2024-08-28T04:41:33Z

packages/shims/src/vitest-environment-endoified.ts

this is wonderful news

…kflows" This reverts commit 26d3c5e.

rekmarks

This PR makes a lot of distinct changes, which makes it difficult to understand. It would benefit from being split into multiple, smaller PRs. Here's one possible way to slice this that would make the changes more digestible:

Changes specific to /shims
Changes specific to tests
Changes specific to building the extension

…es (#39) This is the first PR to land instead of #37 before #40 and #41: - Removes extraneous files from `packages/shims/src` - For consistency — we will keep forgetting them otherwise - Adds `typescript` in `devDependencies` - For consistency - needed since #34

@grypez

This is the second PR to land instead of #37 after #39 and before #41: - Renames `endoify.mjs` to `endoify.js` in `packages/{shims,extension}` - For consistency — now all `.mjs` files are renamed to `.js` - Delegates bundling of `endoify.js` to `bundleSource` - For reliability and predictability — avoids making assumptions and resolving paths inside third-party packages - Optionally subs `ZWJ`-prefixed identifiers introduced by `bundleSource` with conforming `CGJ` alternatives - For conformance — makes it possible to [experiment with `rollup`](https://www.smotaal.io/experimental/#/experimental/modules/shim/README.md) until [endojs/endo#2436](endojs/endo#2436) rolls out. - Explicitly imports `./endoify.js` in `packages/shims/src/*.test.ts` - For clarity and flexibility — test files need to be explicit - Configures `vitest` to use `jsdom` with `vmThreads` for `packages/shims` - For consistency and reduced configuration overhead — thanks to @grypez's exploration putting us on this path of least resistance with `vitest`

This is the third PR to land instead of #37 and after #39 and #40: - Adds `endoify.ts` and `endoify.test.ts` to `packages/extension/src` - For clarity — sibling files importing `./endoified.js` - Explicitly imports `./endoify.js` in `packages/extension/src/*.test.ts` - For clarity and flexibility — test files need to be explicit - Configures `vitest` to use `jsdom` with `vmThreads` for `packages/extension` - For consistency and reduced configuration overhead - Adds guards against inlining `script[src="endoify.ts"]` and `script[src="endoify.js"]` in `packages/extension/src/*.html` - For efficacy and efficiency — parsing before vs after a failure makes no difference in speed, only in accuracy --------- Co-authored-by: grypez <143971198+grypez@users.noreply.github.com>

refactor(shims, extension): Optimize building and testing workflows

26d3c5e

SMotaal marked this pull request as ready for review August 27, 2024 22:23

SMotaal requested a review from a team as a code owner August 27, 2024 22:23

SMotaal commented Aug 27, 2024

View reviewed changes

packages/extension/vite.config.ts Outdated Show resolved Hide resolved

SMotaal commented Aug 27, 2024

View reviewed changes

SMotaal requested review from FUDCo and grypez August 27, 2024 22:29

SMotaal commented Aug 27, 2024

View reviewed changes

grypez reviewed Aug 28, 2024

View reviewed changes

packages/shims/src/vitest-environment-endoified.ts

Copy link

Contributor

grypez Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is wonderful news

SMotaal reacted with heart emoji

SMotaal added 5 commits August 28, 2024 04:58

Revert "refactor(shims, extension): Optimize building and testing wor…

ac39daf

…kflows" This reverts commit 26d3c5e.

chore: delete apply-lockdown.mjs

5991a8b

chore: Remove extraneous exports from shims

ff23674

chore: Rename endoify.mjs to endoify.js

96f4c4c

chore: Reapply intended changes with history intact

9fc70c1

rekmarks requested changes Aug 28, 2024

View reviewed changes

rekmarks mentioned this pull request Aug 28, 2024

chore: Reconfigure @typescript-eslint/explicit-function-return-type #38

Merged

SMotaal mentioned this pull request Aug 28, 2024

chore(shims): Remove extraneous files and update package.json entries #39

Merged

SMotaal marked this pull request as draft August 28, 2024 18:09

This was referenced Aug 28, 2024

refactor(shims): Optimize building and testing #40

Merged

refactor(extension): Optimize building and testing #41

Merged

SMotaal closed this Aug 28, 2024

SMotaal deleted the smotaal/test-endoify-vitest branch September 16, 2024 14:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(shims, extension): Optimize building and testing workflows #37

refactor(shims, extension): Optimize building and testing workflows #37

Uh oh!

SMotaal commented Aug 27, 2024 •

edited

Loading

Uh oh!

Uh oh!

SMotaal Aug 27, 2024 •

edited

Loading

Uh oh!

SMotaal Aug 28, 2024

Uh oh!

SMotaal Aug 27, 2024 •

edited

Loading

Uh oh!

SMotaal Aug 28, 2024

Uh oh!

grypez Aug 28, 2024

Uh oh!

rekmarks left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

refactor(shims, extension): Optimize building and testing workflows #37

refactor(shims, extension): Optimize building and testing workflows #37

Uh oh!

Conversation

SMotaal commented Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

SMotaal Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SMotaal Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

SMotaal Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SMotaal Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

grypez Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

rekmarks left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

SMotaal commented Aug 27, 2024 •

edited

Loading

SMotaal Aug 27, 2024 •

edited

Loading

SMotaal Aug 27, 2024 •

edited

Loading