feat(api): proper esm support and codegen overhauls #754

erunion · 2023-10-11T23:28:30Z

🚥 Resolves RM-7990

🧰 Changes

Changes api install to no longer prompt for a language or target to install. JS is now the only export target.
Overhauls codegen internals to now use tsup to compile our constructed TS code into a JS SDK that can be used in CJS and ESM environments.
- We were previously handling this with a lot of very touchy ts-morph hacks that didn't really support ESM environments.
Refactored codegen to now store generated SDKs within a src/ directory.
- Doing this mostly for aesthetic reasons now that we're going to have a dist/ directory post-tsup compilation.
Upgrades ts-morph to the latest release. chore(deps): bump ts-morph from 17.0.1 to 20.0.0 #752

🧬 QA & Testing

Can you create an SDK?
Can you load the SDK from a .cjs, .mjs, .js, and .ts example file?

erunion · 2023-10-16T18:27:01Z

packages/api/src/codegen/languages/typescript/index.ts

-             * `ts-morph` unfortunately doesn't give us any options for programatically doing this
-             * so we need to resort to modifying the emitted JS code.
-             */
-            code = code


goodbye forever to this mess

packages/api/package.json

packages/api/src/codegen/languages/typescript/index.ts

erunion · 2023-10-16T18:30:19Z

packages/api/src/codegen/languages/typescript/index.ts

-    const sourceFile = this.project.createSourceFile('schemas.ts', '');
-    const schemasDir = this.project.createDirectory('schemas');
+  private createSchemasFile(sourceDirectory: Directory) {
+    const sourceFile = sourceDirectory.createSourceFile('schemas.ts', '');


These sourceDirectory changes mean that these files we're creating here (schemas.ts and schemas/) are now being created in a src/ directory.

* chore: minor tsconfig cleanup * fix: point to correct api-core pkg version * fix: install TS as dev-dep * fix: add this to silence the JSON import warning * refactor: slight cleanup

kanadgupta

mostly cleanup suggestions but overall very happy with this! love how much logic we get to clean up thanks to tsup 🥳

kanadgupta · 2023-10-16T20:33:58Z

packages/api/package.json

    "ssri": "^10.0.1",
-    "ts-morph": "^17.0.1",
+    "ts-morph": "^20.0.0",
+    "tsup": "^7.2.0",


could we remove this as a dep since it's being installed in the codegen'd directory now?

kanadgupta · 2023-10-16T20:35:52Z

packages/api/src/codegen/codegenerator.ts

+  requiredPackages!: Record<
+    string,
+    {
+      reason: string;


as a follow-up to the work I added in #759, i was thinking it might make sense to have a devDep flag that splits out our required deps (e.g., @readme/api-core) from our required dev-deps (e.g., typescript)

Suggested change

reason: string;

devDependency: boolean;

reason: string;

will do this in another PR since it's going to require a little bit of work

packages/api/src/codegen/factory.ts

kanadgupta · 2023-10-16T20:41:21Z

packages/api/test/__fixtures__/sdk/optional-payload/package.json

might wanna regenerate these now that this file has devDependencies

packages/api/test/codegen/languages/typescript/index.test.ts

kanadgupta · 2023-10-16T20:46:34Z

packages/api/test/codegen/languages/typescript/index.test.ts

-      //   });
-      // });
-
-      it('should be able to make an API request (JS + ESM)', async () => {


what if we added a test in this file that imports the generated cjs file and ensures that that works?

thats going to require building a dist for these fixtures and keeping that dist in git. i want to move these fixtures into @api/test-utils at some point, will rework this then

packages/api/src/codegen/languages/typescript/index.ts

kanadgupta · 2023-10-16T20:53:28Z

packages/api/src/codegen/languages/typescript/index.ts

+            }
+          : {}),
+      },
+      dependencies,


per this comment, might be good to add the dev-deps here so we're not running npm i several times

kanadgupta · 2023-10-16T20:54:21Z

packages/api/src/codegen/languages/typescript/index.ts

+  async compile(storage: Storage, opts: InstallerOptions = {}): Promise<void> {
+    const installDir = storage.getIdentifierStorageDir();
+
+    await execa('npm', ['install', 'tsup', 'typescript', '-D'], {


per this comment, might be good to remove this and take care of the dev-dep installation in the earlier install command so we're not running npm i several times

This reverts commit 3b82df8.

Co-authored-by: Kanad Gupta <8854718+kanadgupta@users.noreply.github.com>

feat(api): wip of proper esm support and codegen overhauls

8e5558d

erunion changed the title ~~feat(api): wip of proper esm support and codegen overhauls~~ feat(api): proper esm support and codegen overhauls Oct 11, 2023

erunion mentioned this pull request Oct 11, 2023

refactor(api): minor refactors to how some internal language classes are used #755

Merged

Merge branch 'main' into refactor/codegen

0db62a3

erunion added enhancement New feature or request refactor Issues about tackling technical debt area:core Issues related to `core`, which is the package that powers the SDKs at runtime area:api Issues related to the `api` CLI, which builds the SDKs labels Oct 11, 2023

erunion mentioned this pull request Oct 12, 2023

chore(deps): bump ts-morph from 17.0.1 to 20.0.0 #752

Closed

erunion added 3 commits October 11, 2023 17:14

docs: documenting why we're exporting createSDK the way we are

84872e6

chore(deps): bumping ts-morph

e2e1259

fix: overhauling schemas to now be stored in a schemas/ directory

6463a02

erunion mentioned this pull request Oct 12, 2023

fix(api): nestling generated schemas into a new schemas/ directory #756

Merged

erunion added 8 commits October 12, 2023 16:41

Merge branch 'main' into refactor/codegen

ccbdc40

fix: storage system didn't properly create subdirectories

654194e

fix: broken test

9ade11b

fix: flaky test in ci

8e11923

fix: bringing back some tests

d33e655

feat: refactoring codegen'd sdks to be placed into a src/ directory

7d7b4cc

fix: pinning installed sdk dependencies to known working versions

d4e5d5a

fix: cjs compatibility with our types file being exported in index

330a3a1

erunion commented Oct 16, 2023

View reviewed changes

fix: removing some dead code

488041e

erunion commented Oct 16, 2023

View reviewed changes

packages/api/package.json Outdated Show resolved Hide resolved

erunion commented Oct 16, 2023

View reviewed changes

packages/api/src/codegen/languages/typescript/index.ts Outdated Show resolved Hide resolved

erunion commented Oct 16, 2023

View reviewed changes

erunion marked this pull request as ready for review October 16, 2023 18:31

erunion requested a review from kanadgupta October 16, 2023 18:31

fix: compatibility with node 20 on npm run debug:bin

9fd0370

erunion force-pushed the refactor/codegen branch from 9f45fd2 to 9fd0370 Compare October 16, 2023 19:33

v7.0.0-alpha.5

b1e705d

kanadgupta mentioned this pull request Oct 16, 2023

feat: esm + tsup #734

Closed

7 tasks

kanadgupta and others added 2 commits October 16, 2023 13:32

fix: minor tsup codegen fixes (#759)

87f7319

* chore: minor tsconfig cleanup * fix: point to correct api-core pkg version * fix: install TS as dev-dep * fix: add this to silence the JSON import warning * refactor: slight cleanup

Update packages/api/src/codegen/languages/typescript/index.ts

594710c

erunion added this to the v7 milestone Oct 16, 2023

kanadgupta suggested changes Oct 16, 2023

View reviewed changes

kanadgupta and others added 5 commits October 16, 2023 16:00

Revert "refactor: slight cleanup"

c7c7f55

This reverts commit 3b82df8.

Update packages/api/test/codegen/languages/typescript/index.test.ts

e681ee7

Co-authored-by: Kanad Gupta <8854718+kanadgupta@users.noreply.github.com>

Update packages/api/test/codegen/languages/typescript/index.test.ts

6a66d22

Co-authored-by: Kanad Gupta <8854718+kanadgupta@users.noreply.github.com>

fix: pr feedback

7173c80

fix: ts ignore typo

d591696

erunion requested a review from kanadgupta October 16, 2023 21:16

kanadgupta approved these changes Oct 16, 2023

View reviewed changes

erunion merged commit e739d30 into main Oct 16, 2023

erunion deleted the refactor/codegen branch October 16, 2023 21:23

erunion mentioned this pull request Oct 16, 2023

feat(codegen): required devDeps are now listed in package.json #761

Merged

feat(api): proper esm support and codegen overhauls #754

feat(api): proper esm support and codegen overhauls #754

Uh oh!

Conversation

erunion commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧰 Changes

🧬 QA & Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kanadgupta left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kanadgupta Oct 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kanadgupta Oct 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

erunion commented Oct 11, 2023 •

edited

Loading

kanadgupta Oct 16, 2023 •

edited

Loading

kanadgupta Oct 16, 2023 •

edited

Loading