From 2987882566648bb5f4782cdb6f9f53f5b6d519b0 Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 8 Nov 2020 16:05:55 -0800 Subject: [PATCH 01/14] doc: esm documentation consolidation and reordering --- doc/api/cli.md | 4 +- doc/api/esm.md | 323 ++++++++++++++++++++++++++----------------------- 2 files changed, 174 insertions(+), 153 deletions(-) diff --git a/doc/api/cli.md b/doc/api/cli.md index 484695a347428d..0cab81d64cac12 100644 --- a/doc/api/cli.md +++ b/doc/api/cli.md @@ -234,7 +234,7 @@ Enable experimental JSON support for the ES Module loader. added: v9.0.0 --> -Specify the `module` of a custom [experimental ECMAScript Module loader][]. +Specify the `module` of a custom experimental [ECMAScript Module loader][]. `module` may be either a path to a file, or an ECMAScript Module name. ### `--experimental-modules` @@ -1642,6 +1642,7 @@ $ node --max-old-space-size=1536 index.js ``` [Chrome DevTools Protocol]: https://chromedevtools.github.io/devtools-protocol/ +[ECMAScript Module loader]: esm.md#esm_loaders [REPL]: repl.md [ScriptCoverage]: https://chromedevtools.github.io/devtools-protocol/tot/Profiler#type-ScriptCoverage [Source Map]: https://sourcemaps.info/spec.html @@ -1662,7 +1663,6 @@ $ node --max-old-space-size=1536 index.js [debugger]: debugger.md [debugging security implications]: https://nodejs.org/en/docs/guides/debugging-getting-started/#security-implications [emit_warning]: process.md#process_process_emitwarning_warning_type_code_ctor -[experimental ECMAScript Module loader]: esm.md#esm_experimental_loaders [jitless]: https://v8.dev/blog/jitless [libuv threadpool documentation]: https://docs.libuv.org/en/latest/threadpool.html [remote code execution]: https://www.owasp.org/index.php/Code_Injection diff --git a/doc/api/esm.md b/doc/api/esm.md index 7770297ad7af94..83ce392d6d8227 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -116,42 +116,62 @@ e.g. `'path'` in `import { sep } from 'path'`. Specifiers are also used in There are four types of specifiers: -* _Bare specifiers_ like `'some-package'`. They refer to an entry point of a - package by the package name. - -* _Deep import specifiers_ like `'some-package/lib/shuffle.mjs'`. They refer to - a path within a package prefixed by the package name. - * _Relative specifiers_ like `'./startup.js'` or `'../config.mjs'`. They refer - to a path relative to the location of the importing file. + to a path relative to the location of the importing file. _The file extension + is always mandatory for these._ + +* _Bare specifiers_ like `'some-package'` or `'some-package/shuffle'`. They can + refer to the main entry point of a package by the package name, or a + specific feature module within a package prefixed by the package name as per + the examples respectively. _The file extension is optional for these._ * _Absolute specifiers_ like `'file:///opt/nodejs/config.js'`. They refer directly and explicitly to a full path. -Bare specifiers, and the bare specifier portion of deep import specifiers, are -strings; but everything else in a specifier is a URL. +Bare specifier resolutions are handled by the [Node.js module resolution +algorithm][]. All other specifier resolutions are always only resolved with +the standard relative [URL](https://url.spec.whatwg.org/) resolution semantics. -`file:`, `node:`, and `data:` URLs are supported. A specifier like -`'https://example.com/app.js'` may be supported by browsers but it is not -supported in Node.js. +Like in CommonJS, module files within packages can be accessed by appending a +path to the package name; unless the package’s [`package.json`][] contains an +[`"exports"`][] field, in which case files within packages can only be accessed +via the paths defined in [`"exports"`][]. -Specifiers may not begin with `/` or `//`. These are reserved for potential -future use. The root of the current volume may be referenced via `file:///`. +For details on these package resolution rules that apply to bare specifiers in +the Node.js module resolution, see the [packages documentation](packages.md). -#### `node:` Imports +### Mandatory file extensions - +A file extension must be provided when using the `import` keyword to resolve +relative or absolute specifiers. Directory indexes (e.g. `'./startup/index.js'`) +must also be fully specified. + +This behavior matches how `import` behaves in browser environments, assuming a +typically configured server. -`node:` URLs are supported as a means to load Node.js builtin modules. This -URL scheme allows for builtin modules to be referenced by valid absolute URL -strings. +### URLs + +ES modules are resolved and cached as URLs. This means that files containing +special characters such as `#` and `?` need to be escaped. + +`file:`, `node:`, and `data:` URL schemes are supported. A specifier like +`'https://example.com/app.js'` is not supported natively in Node.js unless using +a [custom HTTPS loader][]. + +#### `file:` URLs + +Modules are loaded multiple times if the `import` specifier used to resolve +them has a different query or fragment. ```js -import fs from 'node:fs/promises'; +import './foo.mjs?query=1'; // loads ./foo.mjs with query of "?query=1" +import './foo.mjs?query=2'; // loads ./foo.mjs with query of "?query=2" ``` +The volume root may be referenced via `/`, `//` or `file:///`. Given the +differences between URL and path resolution (such as percent encoding details), +it is recommended to use [url.pathToFileURL][] when importing a path. + #### `data:` Imports -The `import.meta` metaproperty is an `Object` that contains the following -property: +`node:` URLs are supported as an alternative means to load Node.js builtin +modules. This URL scheme allows for builtin modules to be referenced by valid +absolute URL strings. -* `url` {string} The absolute `file:` URL of the module. +```js +import fs from 'node:fs/promises'; +``` -## Differences between ES modules and CommonJS +## Builtin modules -### Mandatory file extensions +[Core modules][] provide named exports of their public API. A +default export is also provided which is the value of the CommonJS exports. +The default export can be used for, among other things, modifying the named +exports. Named exports of builtin modules are updated only by calling +[`module.syncBuiltinESMExports()`][]. -A file extension must be provided when using the `import` keyword. Directory -indexes (e.g. `'./startup/index.js'`) must also be fully specified. +```js +import EventEmitter from 'events'; +const e = new EventEmitter(); +``` -This behavior matches how `import` behaves in browser environments, assuming a -typically configured server. +```js +import { readFile } from 'fs'; +readFile('./foo.txt', (err, source) => { + if (err) { + console.error(err); + } else { + console.log(source); + } +}); +``` -### No `NODE_PATH` +```js +import fs, { readFileSync } from 'fs'; +import { syncBuiltinESMExports } from 'module'; -`NODE_PATH` is not part of resolving `import` specifiers. Please use symlinks -if this behavior is desired. +fs.readFileSync = () => Buffer.from('Hello, ESM'); +syncBuiltinESMExports(); -### No `require`, `exports`, `module.exports`, `__filename`, `__dirname` +fs.readFileSync === readFileSync; +``` -These CommonJS variables are not available in ES modules. +## `import.meta` -`require` can be imported into an ES module using [`module.createRequire()`][]. +* {Object} -Equivalents of `__filename` and `__dirname` can be created inside of each file -via [`import.meta.url`][]. +The `import.meta` meta property is an `Object` that contains the following +properties. -```js -import { fileURLToPath } from 'url'; -import { dirname } from 'path'; +### `import.meta.url` -const __filename = fileURLToPath(import.meta.url); -const __dirname = dirname(__filename); +* {string} The absolute `file:` URL of the module. + +This is defined exactly the same as it is in browsers providing the URL of the +current module file. + +This enables useful patterns such as relative file loading: + +```js +import { readFileSync } from 'fs'; +const buffer = readFileSync(new URL('./data.proto', import.meta.url)); ``` -### No `require.resolve` +### `import.meta.resolve(specifier[, parent])` -Former use cases relying on `require.resolve` to determine the resolved path -of a module can be supported via `import.meta.resolve`, which is experimental -and supported via the `--experimental-import-meta-resolve` flag: +> Stability: 1 - Experimental + +* `specifier` {string} The module specifier to resolve relative to `parent`. +* `parent` {string|URL} The absolute parent module URL to resolve from. If none + is specified, the value of `import.meta.url` is used as the default. +* Returns: {Promise} + ```js -(async () => { - const dependencyAsset = await import.meta.resolve('component-lib/asset.css'); -})(); +const dependencyAsset = await import.meta.resolve('component-lib/asset.css'); ``` `import.meta.resolve` also accepts a second argument which is the parent module from which to resolve from: + ```js -(async () => { - // Equivalent to import.meta.resolve('./dep') - await import.meta.resolve('./dep', import.meta.url); -})(); +await import.meta.resolve('./dep', import.meta.url); ``` This function is asynchronous because the ES module resolver in Node.js is -asynchronous. With the introduction of [Top-Level Await][], these use cases -will be easier as they won't require an async function wrapper. - -### No `require.extensions` +asynchronous. -`require.extensions` is not used by `import`. The expectation is that loader -hooks can provide this workflow in the future. +## `import()` expressions -### No `require.cache` - -`require.cache` is not used by `import`. It has a separate cache. - -### URL-based paths - -ES modules are resolved and cached based upon -[URL](https://url.spec.whatwg.org/) semantics. This means that files containing -special characters such as `#` and `?` need to be escaped. - -Modules are loaded multiple times if the `import` specifier used to resolve -them has a different query or fragment. - -```js -import './foo.mjs?query=1'; // loads ./foo.mjs with query of "?query=1" -import './foo.mjs?query=2'; // loads ./foo.mjs with query of "?query=2" -``` - -For now, only modules using the `file:` protocol can be loaded. +[Dynamic `import()`][] is supported in both CommonJS and ES modules. In CommonJS +modules it can be used to load ES modules. ## Interoperability with CommonJS ### `require` -`require` always treats the files it references as CommonJS. This applies -whether `require` is used the traditional way within a CommonJS environment, or -in an ES module environment using [`module.createRequire()`][]. +`require` always treats the files it references as CommonJS. -To include an ES module into CommonJS, use [`import()`][]. +Using `require` to load an ES module is not supported because ES modules have +asynchronous execution. Instead, use use [`import()`][] to load an ES module +from a CommonJS module. ### `import` statements @@ -290,30 +317,7 @@ When importing [CommonJS modules](#esm_commonjs_namespaces), the available, provided by static analysis as a convenience for better ecosystem compatibility. -Additional experimental flags are available for importing -[Wasm modules](#esm_experimental_wasm_modules) or -[JSON modules](#esm_experimental_json_modules). For importing native modules or -JSON modules unflagged, see [`module.createRequire()`][]. - -The _specifier_ of an `import` statement (the string after the `from` keyword) -can either be an URL-style relative path like `'./file.mjs'` or a package name -like `'fs'`. - -Like in CommonJS, files within packages can be accessed by appending a path to -the package name; unless the package’s [`package.json`][] contains an -[`"exports"`][] field, in which case files within packages need to be accessed -via the path defined in [`"exports"`][]. - -```js -import { sin, cos } from 'geometry/trigonometry-functions.mjs'; -``` - -### `import()` expressions - -[Dynamic `import()`][] is supported in both CommonJS and ES modules. It can be -used to include ES module files from CommonJS code. - -## CommonJS Namespaces +### CommonJS Namespaces CommonJS modules consist of a `module.exports` object which can be of any type. @@ -396,41 +400,7 @@ Named exports detection covers many common export patterns, reexport patterns and build tool and transpiler outputs. See [cjs-module-lexer][] for the exact semantics implemented. -## Builtin modules - -[Core modules][] provide named exports of their public API. A -default export is also provided which is the value of the CommonJS exports. -The default export can be used for, among other things, modifying the named -exports. Named exports of builtin modules are updated only by calling -[`module.syncBuiltinESMExports()`][]. - -```js -import EventEmitter from 'events'; -const e = new EventEmitter(); -``` - -```js -import { readFile } from 'fs'; -readFile('./foo.txt', (err, source) => { - if (err) { - console.error(err); - } else { - console.log(source); - } -}); -``` - -```js -import fs, { readFileSync } from 'fs'; -import { syncBuiltinESMExports } from 'module'; - -fs.readFileSync = () => Buffer.from('Hello, ESM'); -syncBuiltinESMExports(); - -fs.readFileSync === readFileSync; -``` - -## CommonJS, JSON, and native modules +### CommonJS, JSON, and native modules CommonJS, JSON, and native modules can be used with [`module.createRequire()`][]. @@ -448,7 +418,47 @@ const cjs = require('./cjs.cjs'); cjs === 'cjs'; // true ``` -## Experimental JSON modules +### Differences between ES modules and CommonJS + +#### No `NODE_PATH` + +`NODE_PATH` is not part of resolving `import` specifiers. Please use symlinks +if this behavior is desired. + +#### No `require`, `exports`, `module.exports`, `__filename`, `__dirname` + +These CommonJS variables are not available in ES modules. + +`require` can be imported into an ES module using [`module.createRequire()`][]. + +Equivalents of `__filename` and `__dirname` can be created inside of each file +via [`import.meta.url`][]. + +```js +import { fileURLToPath } from 'url'; +import { dirname } from 'path'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = dirname(__filename); +``` + +#### No `require.resolve` + +Former use cases relying on `require.resolve` to determine the resolved path +of a module can be supported via [`import.meta.resolve`][]. + +#### No `require.extensions` + +`require.extensions` is not used by `import`. The expectation is that loader +hooks can provide this workflow in the future. + +#### No `require.cache` + +`require.cache` is not used by `import`. It has a separate cache. + +## JSON modules + +> Stability: 1 - Experimental Currently importing JSON modules are only supported in the `commonjs` mode and are loaded using the CJS loader. [WHATWG JSON modules specification][] are @@ -478,7 +488,9 @@ node index.mjs # fails node --experimental-json-modules index.mjs # works ``` -## Experimental Wasm modules +## Wasm modules + +> Stability: 1 - Experimental Importing Web Assembly modules is supported under the `--experimental-wasm-modules` flag, allowing any `.wasm` files to be @@ -502,7 +514,9 @@ node --experimental-wasm-modules index.mjs would provide the exports interface for the instantiation of `module.wasm`. -## Experimental top-level `await` +## Top-level `await` + +> Stability: 1 - Experimental The `await` keyword may be used in the top level (outside of async functions) within modules as per the [ECMAScript Top-Level `await` proposal][]. @@ -526,7 +540,9 @@ console.log(five); // Logs `5` node b.mjs # works ``` -## Experimental loaders +## Loaders + +> Stability: 1 - Experimental **Note: This API is currently being redesigned and will still change.** @@ -1237,6 +1253,8 @@ _internal_, _conditions_) ### Customizing ESM specifier resolution algorithm +> Stability: 1 - Experimental + The current specifier resolution does not support all default behavior of the CommonJS loader. One of the behavior differences is automatic resolution of file extensions and the ability to import directories that have an index @@ -1267,8 +1285,8 @@ success! [ECMAScript-modules implementation]: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md [ES Module Integration Proposal for Web Assembly]: https://github.com/webassembly/esm-integration [Node.js EP for ES Modules]: https://github.com/nodejs/node-eps/blob/master/002-es-modules.md +[Node.js Module Resolution Algorithm]: #esm_resolver_algorithm_specification [Terminology]: #esm_terminology -[Top-Level Await]: https://github.com/tc39/proposal-top-level-await [WHATWG JSON modules specification]: https://html.spec.whatwg.org/#creating-a-json-module-script [`"exports"`]: packages.md#packages_exports [`"type"`]: packages.md#packages_type @@ -1279,7 +1297,8 @@ success! [`data:` URLs]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs [`export`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export [`import()`]: #esm_import_expressions -[`import.meta.url`]: #esm_import_meta +[`import.meta.url`]: #esm_import_meta_url +[`import.meta.resolve`]: #esm_import_meta_resolve [`import`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import [`module.createRequire()`]: module.md#module_module_createrequire_filename [`module.syncBuiltinESMExports()`]: module.md#module_module_syncbuiltinesmexports @@ -1288,6 +1307,8 @@ success! [`string`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String [`util.TextDecoder`]: util.md#util_class_util_textdecoder [cjs-module-lexer]: https://github.com/guybedford/cjs-module-lexer/tree/1.0.0 +[custom https loader]: #esm_https_loader [special scheme]: https://url.spec.whatwg.org/#special-scheme [the official standard format]: https://tc39.github.io/ecma262/#sec-modules [transpiler loader example]: #esm_transpiler_loader +[url.pathToFileURL]: url.md#url_url_pathtofileurl_path From f4cc179834ef97a92b14ee6b18438de9b8c1999e Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 8 Nov 2020 16:28:08 -0800 Subject: [PATCH 02/14] fixup link --- doc/api/esm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 83ce392d6d8227..44ef78460c4ced 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -1298,7 +1298,7 @@ success! [`export`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export [`import()`]: #esm_import_expressions [`import.meta.url`]: #esm_import_meta_url -[`import.meta.resolve`]: #esm_import_meta_resolve +[`import.meta.resolve`]: #esm_import_meta_resolve_specifier_parent [`import`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import [`module.createRequire()`]: module.md#module_module_createrequire_filename [`module.syncBuiltinESMExports()`]: module.md#module_module_syncbuiltinesmexports From b44219c0042d02c5a967e1e47b6578dc3469a5ae Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 8 Nov 2020 16:39:02 -0800 Subject: [PATCH 03/14] Apply suggestions from code review Co-authored-by: Antoine du Hamel --- doc/api/esm.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 44ef78460c4ced..dc56b2877eaa45 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -114,7 +114,7 @@ The _specifier_ of an `import` statement is the string after the `from` keyword, e.g. `'path'` in `import { sep } from 'path'`. Specifiers are also used in `export from` statements, and as the argument to an `import()` expression. -There are four types of specifiers: +There are three types of specifiers: * _Relative specifiers_ like `'./startup.js'` or `'../config.mjs'`. They refer to a path relative to the location of the importing file. _The file extension @@ -130,7 +130,7 @@ There are four types of specifiers: Bare specifier resolutions are handled by the [Node.js module resolution algorithm][]. All other specifier resolutions are always only resolved with -the standard relative [URL](https://url.spec.whatwg.org/) resolution semantics. +the standard relative [URL][] resolution semantics. Like in CommonJS, module files within packages can be accessed by appending a path to the package name; unless the package’s [`package.json`][] contains an @@ -169,7 +169,7 @@ import './foo.mjs?query=2'; // loads ./foo.mjs with query of "?query=2" ``` The volume root may be referenced via `/`, `//` or `file:///`. Given the -differences between URL and path resolution (such as percent encoding details), +differences between [URL][] and path resolution (such as percent encoding details), it is recommended to use [url.pathToFileURL][] when importing a path. #### `data:` Imports From ff48150c42bb226a1cd8093da843673301af022c Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 8 Nov 2020 16:39:27 -0800 Subject: [PATCH 04/14] fixup: add url link --- doc/api/esm.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/api/esm.md b/doc/api/esm.md index dc56b2877eaa45..b77be44cea68dd 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -1287,6 +1287,7 @@ success! [Node.js EP for ES Modules]: https://github.com/nodejs/node-eps/blob/master/002-es-modules.md [Node.js Module Resolution Algorithm]: #esm_resolver_algorithm_specification [Terminology]: #esm_terminology +[URL]: https://url.spec.whatwg.org/ [WHATWG JSON modules specification]: https://html.spec.whatwg.org/#creating-a-json-module-script [`"exports"`]: packages.md#packages_exports [`"type"`]: packages.md#packages_type From 6b9be0a990e83c607b535300cfc9d7cd7887b46c Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 8 Nov 2020 16:40:51 -0800 Subject: [PATCH 05/14] add experimental anchors --- doc/api/esm.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/doc/api/esm.md b/doc/api/esm.md index b77be44cea68dd..5936801fdb538e 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -456,6 +456,8 @@ hooks can provide this workflow in the future. `require.cache` is not used by `import`. It has a separate cache. + + ## JSON modules > Stability: 1 - Experimental @@ -488,6 +490,8 @@ node index.mjs # fails node --experimental-json-modules index.mjs # works ``` + + ## Wasm modules > Stability: 1 - Experimental @@ -514,6 +518,8 @@ node --experimental-wasm-modules index.mjs would provide the exports interface for the instantiation of `module.wasm`. + + ## Top-level `await` > Stability: 1 - Experimental @@ -540,6 +546,8 @@ console.log(five); // Logs `5` node b.mjs # works ``` + + ## Loaders > Stability: 1 - Experimental From 3e0a9e342611d8ed12df2851ca15be9fff367a3d Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 8 Nov 2020 16:46:40 -0800 Subject: [PATCH 06/14] lint fix --- doc/api/esm.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 5936801fdb538e..68737c183024da 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -169,8 +169,8 @@ import './foo.mjs?query=2'; // loads ./foo.mjs with query of "?query=2" ``` The volume root may be referenced via `/`, `//` or `file:///`. Given the -differences between [URL][] and path resolution (such as percent encoding details), -it is recommended to use [url.pathToFileURL][] when importing a path. +differences between [URL][] and path resolution (such as percent encoding +details), it is recommended to use [url.pathToFileURL][] when importing a path. #### `data:` Imports From abf52d369df92b36b169843fc87a11c5d6c7d261 Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 8 Nov 2020 16:47:31 -0800 Subject: [PATCH 07/14] remove "#" from ids --- doc/api/esm.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 68737c183024da..f86d1ad7e2835f 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -71,9 +71,9 @@ Expect major changes in the implementation including interoperability support, specifier resolution, and default behavior. - - - + + + ## Enabling @@ -456,7 +456,7 @@ hooks can provide this workflow in the future. `require.cache` is not used by `import`. It has a separate cache. - + ## JSON modules @@ -490,7 +490,7 @@ node index.mjs # fails node --experimental-json-modules index.mjs # works ``` - + ## Wasm modules @@ -518,7 +518,7 @@ node --experimental-wasm-modules index.mjs would provide the exports interface for the instantiation of `module.wasm`. - + ## Top-level `await` @@ -546,7 +546,7 @@ console.log(five); // Logs `5` node b.mjs # works ``` - + ## Loaders From e38101ed34d3f04a67a554ddea009731fabe6787 Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 8 Nov 2020 18:43:59 -0800 Subject: [PATCH 08/14] clarify optional extensions for bare specifiers --- doc/api/esm.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index f86d1ad7e2835f..0aab8dd3508cb1 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -118,12 +118,13 @@ There are three types of specifiers: * _Relative specifiers_ like `'./startup.js'` or `'../config.mjs'`. They refer to a path relative to the location of the importing file. _The file extension - is always mandatory for these._ + is always necessary for these._ * _Bare specifiers_ like `'some-package'` or `'some-package/shuffle'`. They can refer to the main entry point of a package by the package name, or a specific feature module within a package prefixed by the package name as per - the examples respectively. _The file extension is optional for these._ + the examples respectively. _Including the file extension is only necessary + for packages without an [`"exports"`][] field._ * _Absolute specifiers_ like `'file:///opt/nodejs/config.js'`. They refer directly and explicitly to a full path. From c4258d2a2dc13ffa7bf05a6ba06f664f39633b38 Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Wed, 11 Nov 2020 13:28:04 -0800 Subject: [PATCH 09/14] differences reworking, dynamic import reorder --- doc/api/esm.md | 103 +++++++++++++++++++++++++++++-------------------- 1 file changed, 61 insertions(+), 42 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 0aab8dd3508cb1..a35226205022a3 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -246,6 +246,11 @@ syncBuiltinESMExports(); fs.readFileSync === readFileSync; ``` +## `import()` expressions + +[Dynamic `import()`][] is supported in both CommonJS and ES modules. In CommonJS +modules it can be used to load ES modules. + ## `import.meta` * {Object} @@ -276,6 +281,9 @@ const buffer = readFileSync(new URL('./data.proto', import.meta.url)); is specified, the value of `import.meta.url` is used as the default. * Returns: {Promise} +Provides a module-relative resolution function scoped to each module, returning +the URL string. + ```js const dependencyAsset = await import.meta.resolve('component-lib/asset.css'); @@ -290,34 +298,29 @@ await import.meta.resolve('./dep', import.meta.url); ``` This function is asynchronous because the ES module resolver in Node.js is -asynchronous. - -## `import()` expressions - -[Dynamic `import()`][] is supported in both CommonJS and ES modules. In CommonJS -modules it can be used to load ES modules. +allowed to be asynchronous. ## Interoperability with CommonJS -### `require` - -`require` always treats the files it references as CommonJS. - -Using `require` to load an ES module is not supported because ES modules have -asynchronous execution. Instead, use use [`import()`][] to load an ES module -from a CommonJS module. - ### `import` statements An `import` statement can reference an ES module or a CommonJS module. -`import` statements are permitted only in ES modules. For similar functionality -in CommonJS, see [`import()`][]. +`import` statements are permitted only in ES modules, but dynamic [`import()`][] +expressions are supported in CommonJS for loading ES modules. When importing [CommonJS modules](#esm_commonjs_namespaces), the `module.exports` object is provided as the default export. Named exports may be available, provided by static analysis as a convenience for better ecosystem compatibility. +### `require` + +The CommonJS module `require` always treats the files it references as CommonJS. + +Using `require` to load an ES module is not supported because ES modules have +asynchronous execution. Instead, use use [`import()`][] to load an ES module +from a CommonJS module. + ### CommonJS Namespaces CommonJS modules consist of a `module.exports` object which can be of any type. @@ -401,52 +404,67 @@ Named exports detection covers many common export patterns, reexport patterns and build tool and transpiler outputs. See [cjs-module-lexer][] for the exact semantics implemented. -### CommonJS, JSON, and native modules +### Differences between ES modules and CommonJS + +#### No `require`, `exports` or `module.exports` + +In most cases, the ES module `import` can be used to load CommonJS modules. -CommonJS, JSON, and native modules can be used with +If needed, a `require` function can be constructed within an ES module using [`module.createRequire()`][]. -```js -// cjs.cjs -module.exports = 'cjs'; +#### No `__filename` or `__dirname` -// esm.mjs -import { createRequire } from 'module'; +These CommonJS variables are not available in ES modules. -const require = createRequire(import.meta.url); +`__filename` and `__dirname` use cases can be replicated via +[`import.meta.url`][]. -const cjs = require('./cjs.cjs'); -cjs === 'cjs'; // true -``` +#### No JSON Module Loading -### Differences between ES modules and CommonJS +JSON imports are still experimental and only supported via the +`--experimental-json-modules` flag. -#### No `NODE_PATH` +Local JSON files can be loaded relative to `import.meta.url` with `fs` directly: -`NODE_PATH` is not part of resolving `import` specifiers. Please use symlinks -if this behavior is desired. + +```js +import { promises as fs } from 'fs'; -#### No `require`, `exports`, `module.exports`, `__filename`, `__dirname` +const json = JSON.parse(await fs.readFile('./data.json', import.meta.url)); +``` -These CommonJS variables are not available in ES modules. +Alterantively `module.createRequire()` can be used. -`require` can be imported into an ES module using [`module.createRequire()`][]. +#### No Native Module Loading -Equivalents of `__filename` and `__dirname` can be created inside of each file -via [`import.meta.url`][]. +Native modules are not currently supported with ES module imports. + +They can be loaded directly with `process.dlopen`: ```js +import process from 'process'; import { fileURLToPath } from 'url'; -import { dirname } from 'path'; -const __filename = fileURLToPath(import.meta.url); -const __dirname = dirname(__filename); +const module = { exports: {} }; +process.dlopen(module, fileURLToPath(new URL('./local.node', import.meta.url))); ``` +Alternatively `module.createRequire()` can be used. + #### No `require.resolve` -Former use cases relying on `require.resolve` to determine the resolved path -of a module can be supported via [`import.meta.resolve`][]. +Relative resolution can be handled via `new URL('./local', import.meta.url)`. + +For a complete `require.resolve` replacement, there is a flagged experimental +[`import.meta.resolve`][] API. + +Alternatively `module.createRequire()` can be used. + +#### No `NODE_PATH` + +`NODE_PATH` is not part of resolving `import` specifiers. Please use symlinks +if this behavior is desired. #### No `require.extensions` @@ -455,7 +473,8 @@ hooks can provide this workflow in the future. #### No `require.cache` -`require.cache` is not used by `import`. It has a separate cache. +`require.cache` is not used by `import` as the ES module loader has its own +separate cache. From 24761117e81dc3445fdb8fec8e933e622ec398b5 Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Wed, 11 Nov 2020 16:10:00 -0800 Subject: [PATCH 10/14] fixup json example --- doc/api/esm.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index a35226205022a3..fde841ad20a69e 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -429,9 +429,8 @@ Local JSON files can be loaded relative to `import.meta.url` with `fs` directly: ```js -import { promises as fs } from 'fs'; - -const json = JSON.parse(await fs.readFile('./data.json', import.meta.url)); +import { readFile } from 'fs/promises'; +const json = JSON.parse(await readFile(new URL('./dat.json', import.meta.url))); ``` Alterantively `module.createRequire()` can be used. From eadbbf0a735d5042295b424d8c0e1aa8d021f1b0 Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Thu, 12 Nov 2020 08:28:24 -0800 Subject: [PATCH 11/14] remove dlopen example, createRequire mention first --- doc/api/esm.md | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index fde841ad20a69e..7fa200366d1f0f 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -439,17 +439,7 @@ Alterantively `module.createRequire()` can be used. Native modules are not currently supported with ES module imports. -They can be loaded directly with `process.dlopen`: - -```js -import process from 'process'; -import { fileURLToPath } from 'url'; - -const module = { exports: {} }; -process.dlopen(module, fileURLToPath(new URL('./local.node', import.meta.url))); -``` - -Alternatively `module.createRequire()` can be used. +The can instead be loaded with `module.createRequire()` or `process.dlopen`. #### No `require.resolve` From 7723459b22ddc5416dc771f08e3952fdb2030414 Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Thu, 12 Nov 2020 08:43:08 -0800 Subject: [PATCH 12/14] add reference links --- doc/api/esm.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 7fa200366d1f0f..9251785df52088 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -439,7 +439,8 @@ Alterantively `module.createRequire()` can be used. Native modules are not currently supported with ES module imports. -The can instead be loaded with `module.createRequire()` or `process.dlopen`. +The can instead be loaded with [`module.createRequire()`][] or +[`process.dlopen`][]. #### No `require.resolve` @@ -1321,6 +1322,7 @@ success! [`module.createRequire()`]: module.md#module_module_createrequire_filename [`module.syncBuiltinESMExports()`]: module.md#module_module_syncbuiltinesmexports [`package.json`]: packages.md#packages_node_js_package_json_field_definitions +[`process.dlopen`]: process.md#process_process_dlopen_module_filename_flags [`transformSource` hook]: #esm_transformsource_source_context_defaulttransformsource [`string`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String [`util.TextDecoder`]: util.md#util_class_util_textdecoder From af41f91517cffe0685631c775c98347d1976b3d0 Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Fri, 13 Nov 2020 07:12:13 -0800 Subject: [PATCH 13/14] update process.dlopen docs --- doc/api/process.md | 27 ++++++++++++--------------- 1 file changed, 12 insertions(+), 15 deletions(-) diff --git a/doc/api/process.md b/doc/api/process.md index f2dd0c18d5cc00..a5db7e1d8e71a0 100644 --- a/doc/api/process.md +++ b/doc/api/process.md @@ -885,31 +885,29 @@ changes: * `filename` {string} * `flags` {os.constants.dlopen} **Default:** `os.constants.dlopen.RTLD_LAZY` -The `process.dlopen()` method allows to dynamically load shared -objects. It is primarily used by `require()` to load -C++ Addons, and should not be used directly, except in special -cases. In other words, [`require()`][] should be preferred over -`process.dlopen()`, unless there are specific reasons. +The `process.dlopen()` method allows dynamically loading shared objects. It is +primarily used by `require()` to load C++ Addons, and should not be used +directly, except in special cases. In other words, [`require()`][] should be +preferred over `process.dlopen()` unless there are specific reasons such as +custom dlopen flags or loading from ES modules. The `flags` argument is an integer that allows to specify dlopen behavior. See the [`os.constants.dlopen`][] documentation for details. -If there are specific reasons to use `process.dlopen()` (for instance, -to specify dlopen flags), it's often useful to use [`require.resolve()`][] -to look up the module's path. +An important requirement when calling `process.dlopen()` is that the `module` +instance must be passed. Functions exported by the C++ Addon will then be +accessible via `module.exports`. -An important drawback when calling `process.dlopen()` is that the `module` -instance must be passed. Functions exported by the C++ Addon will be accessible -via `module.exports`. - -The example below shows how to load a C++ Addon, named as `binding`, +The example below shows how to load a C++ Addon, named `local.node`, that exports a `foo` function. All the symbols will be loaded before the call returns, by passing the `RTLD_NOW` constant. In this example the constant is assumed to be available. ```js const os = require('os'); -process.dlopen(module, require.resolve('binding'), +const path = require('path'); +const module = { exports: {} }; +process.dlopen(module, path.join(__dirname, 'local.node'), os.constants.dlopen.RTLD_NOW); module.exports.foo(); ``` @@ -2678,7 +2676,6 @@ cases: [`readable.read()`]: stream.md#stream_readable_read_size [`require()`]: globals.md#globals_require [`require.main`]: modules.md#modules_accessing_the_main_module -[`require.resolve()`]: modules.md#modules_require_resolve_request_options [`subprocess.kill()`]: child_process.md#child_process_subprocess_kill_signal [`v8.setFlagsFromString()`]: v8.md#v8_v8_setflagsfromstring_flags [debugger]: debugger.md From f51933945a6a6a282283e5786c44965aa612693b Mon Sep 17 00:00:00 2001 From: Guy Bedford Date: Sun, 15 Nov 2020 06:31:59 -0800 Subject: [PATCH 14/14] Apply suggestions from code review Co-authored-by: Rich Trott --- doc/api/esm.md | 2 +- doc/api/process.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 9251785df52088..e631f6c3926162 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -134,7 +134,7 @@ algorithm][]. All other specifier resolutions are always only resolved with the standard relative [URL][] resolution semantics. Like in CommonJS, module files within packages can be accessed by appending a -path to the package name; unless the package’s [`package.json`][] contains an +path to the package name unless the package’s [`package.json`][] contains an [`"exports"`][] field, in which case files within packages can only be accessed via the paths defined in [`"exports"`][]. diff --git a/doc/api/process.md b/doc/api/process.md index a5db7e1d8e71a0..d368b0aa573fa8 100644 --- a/doc/api/process.md +++ b/doc/api/process.md @@ -895,11 +895,11 @@ The `flags` argument is an integer that allows to specify dlopen behavior. See the [`os.constants.dlopen`][] documentation for details. An important requirement when calling `process.dlopen()` is that the `module` -instance must be passed. Functions exported by the C++ Addon will then be +instance must be passed. Functions exported by the C++ Addon are then accessible via `module.exports`. The example below shows how to load a C++ Addon, named `local.node`, -that exports a `foo` function. All the symbols will be loaded before +that exports a `foo` function. All the symbols are loaded before the call returns, by passing the `RTLD_NOW` constant. In this example the constant is assumed to be available.