From 7e10a12f381abef7b88ddec274a92e0d889e9f1a Mon Sep 17 00:00:00 2001 From: Chris Holdgraf Date: Tue, 18 May 2021 23:28:45 -0700 Subject: [PATCH 1/4] Refactoring documentation --- CHANGELOG.md | 34 +- README.md | 11 +- docs/api/directive.rst | 7 - docs/api/index.md | 11 +- docs/{using/use_api.md => api/parsers.md} | 230 +++----- docs/api/{renderers.rst => reference.rst} | 31 +- docs/api/renderers.md | 146 ++++++ docs/api/sphinx_parser.rst | 14 - docs/conf.py | 9 + docs/explain/index.md | 83 +++ docs/index.md | 118 ++--- docs/sphinx/faq.md | 16 + docs/sphinx/index.md | 15 + docs/sphinx/intro.md | 423 +++++++++++++++ docs/sphinx/reference.md | 110 ++++ docs/sphinx/roles-and-directives.md | 23 + docs/{using => sphinx}/snippets/include-md.md | 0 .../snippets/include-rst.rst | 0 docs/{using/howto.md => sphinx/use.md} | 69 ++- docs/{using => syntax}/img/fun-fish.png | Bin docs/syntax/reference.md | 264 ++++++++++ docs/{using => syntax}/syntax-optional.md | 168 +++++- docs/{using => syntax}/syntax.md | 494 ++---------------- docs/using/faq.md | 64 --- docs/using/intro.md | 328 ------------ example-include.md | 2 +- myst_parser/mocking.py | 2 +- setup.cfg | 2 + 28 files changed, 1523 insertions(+), 1151 deletions(-) delete mode 100644 docs/api/directive.rst rename docs/{using/use_api.md => api/parsers.md} (58%) rename docs/api/{renderers.rst => reference.rst} (74%) create mode 100644 docs/api/renderers.md delete mode 100644 docs/api/sphinx_parser.rst create mode 100644 docs/explain/index.md create mode 100644 docs/sphinx/faq.md create mode 100644 docs/sphinx/index.md create mode 100644 docs/sphinx/intro.md create mode 100644 docs/sphinx/reference.md create mode 100644 docs/sphinx/roles-and-directives.md rename docs/{using => sphinx}/snippets/include-md.md (100%) rename docs/{using => sphinx}/snippets/include-rst.rst (100%) rename docs/{using/howto.md => sphinx/use.md} (76%) rename docs/{using => syntax}/img/fun-fish.png (100%) create mode 100644 docs/syntax/reference.md rename docs/{using => syntax}/syntax-optional.md (78%) rename docs/{using => syntax}/syntax.md (57%) delete mode 100644 docs/using/faq.md delete mode 100644 docs/using/intro.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 74f8b4b9..faede6d6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,7 +45,7 @@ In particular for users, this update alters the parsing of tables to be consiste Add "tasklist" to the `myst_enable_extensions` configuration to enable. - See [the optional syntax guide](docs/using/syntax-optional.md#task-lists) for further information. + See [the optional syntax guide](docs/syntax/syntax-optional.md#task-lists) for further information. - The **`sub-ref`** role has been added for use identical to ReST's `|name|` syntax. @@ -55,10 +55,10 @@ In particular for users, this update alters the parsing of tables to be consiste > {sub-ref}`today` | {sub-ref}`wordcount-words` words | {sub-ref}`wordcount-minutes` min read ``` - See [the roles syntax guide](docs/using/syntax.md) for further information. + See [the roles syntax guide](docs/syntax/syntax.md) for further information. - The **`dmath_double_inline`** configuration option allows display math (i.e. `$$`) within an inline context. - See [the math syntax guide](docs/using/syntax.md#math-shortcuts) for further information. + See [the math syntax guide](docs/syntax/syntax-optional.md#math-shortcuts) for further information. ### Remove v0.13 deprecations ā€¼ļø @@ -110,7 +110,7 @@ A warning (of type `myst.nested_header`) is now emitted when this occurs. - āœØ NEW: Add warning types `myst.subtype`: All parsing warnings are assigned a type/subtype, and also the messages are appended with them. These warning types can be suppressed with the sphinx `suppress_warnings` config option. - See [How-to suppress warnings](docs/using/howto.md) for more information. + See [How-to suppress warnings](howto/warnings) for more information. ## 0.13.3 - 2021-01-20 @@ -133,14 +133,14 @@ Minor fixes: This is the *content* ``` -: See [the optional syntax guide](docs/using/syntax-optional.md) for further information. +: See [the optional syntax guide](docs/syntax/syntax-optional.md) for further information. šŸ‘Œ IMPROVE: Footnotes : If the label is an integer, then it will always use this integer for the rendered label (i.e. they are manually numbered). : Add `myst_footnote_transition` configuration, to turn on/off transition line. : Add `footnotes` class to transition `
` in HTML. -: See [the syntax guide](docs/using/syntax.md) for further information. +: See [the syntax guide](docs/syntax/syntax.md) for further information. šŸ‘Œ IMPROVE: `substitution` extension logic @@ -176,7 +176,7 @@ An inline image: {{ key }} ## 0.13.0 - 2020-12-18 This release makes some major updates to the optional syntaxes. -For full details see [Optional MyST Syntaxes](docs/using/syntax-optional.md). +For full details see [Optional MyST Syntaxes](docs/syntax/syntax-optional.md). ### šŸ—‘ Deprecations @@ -261,7 +261,7 @@ Iā€™m an inline image: It is enabled in your `conf.py` *via* `myst_heading_anchors = 2` (sets maximum heading level). - See [the documentation here](docs/using/syntax-optional.md#auto-generated-header-anchors). + See [the documentation here](docs/syntax/syntax-optional.md#auto-generated-header-anchors). šŸ› FIX: doc reference resolution for singlehtml/latex. : These reference resolutions are passed to the "missing-reference" event, and require the `node["refdoc"]` attribute to be available, which was missing for `[text](./path/to/file.md)` type references. @@ -269,7 +269,7 @@ Iā€™m an inline image: ## 0.12.7 - 2020-08-31 āœØ NEW: Want to include your README.md in the documentation? -: See [including a file from outside the docs folder](docs/using/howto.md). +: See [including a file from outside the docs folder](howto/include-readme). (šŸ‘Œ added `relative-docs` option in 0.12.8) @@ -278,7 +278,7 @@ Iā€™m an inline image: āœØ NEW: Add Markdown figure syntax : Setting `myst_figure_enable = True` in your sphinx `conf.py`, combines the above two extended syntaxes, to create a fully Markdown compliant version of the `figure` directive. - See [Markdown Figures](docs/using/syntax-optional.md#markdown-figures) for details. + See [Markdown Figures](docs/syntax/syntax-optional.md#markdown-figures) for details. (šŸ‘Œ formatting of caption improved in 0.12.6) @@ -296,18 +296,18 @@ Term 1 : Definition ``` -See the [Definition Lists documentation](https://myst-parser.readthedocs.io/en/latest/using/syntax-optional.html#definition-lists) for further details. +See the [Definition Lists documentation](https://myst-parser.readthedocs.io/en/latest/syntax/syntax-optional.html#definition-lists) for further details. šŸ‘Œ IMPROVE: mathjax_config override. : Only `mathjax_config["tex2jax"]` will now be overridden, in order to not interfere with other user configurations, such as adding TeX macros. The configuration name has also changed from `myst_override_mathjax` to `myst_update_mathjax`. - See [Mathjax and math parsing](https://myst-parser.readthedocs.io/en/latest/using/syntax.html#mathjax-and-math-parsing) for further details. + See [Mathjax and math parsing](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#mathjax-and-math-parsing) for further details. ## 0.12.2 - 2020-08-25 āœØ NEW: Add the `eval-rst` directive -: This directive parses its contents as ReStructuredText, which integrates back into the rest of the document, e.g. for cross-referencing. See [this documentation](https://myst-parser.readthedocs.io/en/latest/using/syntax.html#how-directives-parse-content) for further explanation. +: This directive parses its contents as ReStructuredText, which integrates back into the rest of the document, e.g. for cross-referencing. See [this documentation](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#how-directives-parse-content) for further explanation. In particular, this addition solves some outstanding user requests: @@ -324,7 +324,7 @@ See the [Definition Lists documentation](https://myst-parser.readthedocs.io/en/l ### ā€¼ļø BREAKING -If you are using math in your documents, be sure to read the updated [Math syntax guide](https://myst-parser.readthedocs.io/en/latest/using/syntax.html#math-shortcuts)! +If you are using math in your documents, be sure to read the updated [Math syntax guide](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#math-shortcuts)! In particular, the Mathjax configuration is now overridden, such that LaTeX environments will only be rendered if `myst_amsmath_enable=True` is set. The `myst_math_delimiters` option has also been removed (please open an issue if you would like brackets math parsing to be re-implemented). @@ -341,7 +341,7 @@ More configuration options for math parsing (see [MyST configuration options](ht ### Added āœØ -- `` tag parsing to sphinx representation, see [the image syntax guide](https://myst-parser.readthedocs.io/en/latest/using/syntax.html#images) +- `` tag parsing to sphinx representation, see [the image syntax guide](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#images) ### Improved šŸ‘Œ @@ -366,10 +366,10 @@ More configuration options for math parsing (see [MyST configuration options](ht ::: ``` - See [the syntax guide section](https://myst-parser.readthedocs.io/en/latest/using/syntax.html#admonition-directives-special-syntax-optional) for details. + See [the syntax guide section](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#admonition-directives-special-syntax-optional) for details. * Direct parsing of [amsmath](https://ctan.org/pkg/amsmath) LaTeX equations (optional). - See [the syntax guide section](https://myst-parser.readthedocs.io/en/latest/using/syntax.html#direct-latex-math-optional) for details. + See [the syntax guide section](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#direct-latex-math-optional) for details. ### Breaking ā€¼ļø diff --git a/README.md b/README.md index 1436fefd..874badaa 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,16 @@ [![Conda][conda-badge]][conda-link] [![PyPI - Downloads][install-badge]][install-link] -An extended [CommonMark](https://commonmark.org)-compliant parser, with bridges to [Docutils](https://docutils.sourceforge.io) and [Sphinx](https://www.sphinx-doc.org). -## Usage +**MyST is a rich and extensible flavor of Markdown meant for technical documentation and publishing**. + +MyST is a flavor of markdown that is designed for simplicity, flexibility, and extensibility. +This repository serves as the reference implementation of MyST Markdown, as well as a collection of tools to support working with MyST in Python and Sphinx. +It contains an extended [CommonMark](https://commonmark.org)-compliant parser using [`markdown-it-py`](https://markdown-it-py.readthedocs.io/), as well as a [Sphinx](https://www.sphinx-doc.org) extension that allows you to write MyST Markdown in Sphinx. + +[**See the MyST Parser documentation for more information**](https://myst-parser.readthedocs.io/en/latest/). + +## Installation To install the MyST parser, run the following in a [Conda environment](https://docs.conda.io) (recommended): diff --git a/docs/api/directive.rst b/docs/api/directive.rst deleted file mode 100644 index 7f3b2d51..00000000 --- a/docs/api/directive.rst +++ /dev/null @@ -1,7 +0,0 @@ -.. _api/directive: - -Directive Parsing ------------------ - -.. automodule:: myst_parser.parse_directives - :members: diff --git a/docs/api/index.md b/docs/api/index.md index 4ac04b81..c646a562 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -1,11 +1,14 @@ (api/main)= -# MyST-Parser API reference +# MyST-Parser Python API + +MyST-Parser also has a Python API *via* the `myst_parser` package. +This allows you to parse MyST Markdown and render various outputs with Python. ```{toctree} :maxdepth: 2 -directive.rst -renderers.rst -sphinx_parser.rst +parsers.md +renderers.md +reference.rst ``` diff --git a/docs/using/use_api.md b/docs/api/parsers.md similarity index 58% rename from docs/using/use_api.md rename to docs/api/parsers.md index 79a81e1f..669839fe 100644 --- a/docs/using/use_api.md +++ b/docs/api/parsers.md @@ -1,18 +1,17 @@ -# Python API - -MyST-Parser also has a Python API *via* the `myst_parser` package. +# Parse MyST Markdown ```{seealso} -- The [markdown-it-py](https://github.com/executablebooks/markdown-it-py) package -- {ref}`The MyST-Parser API reference ` +- The MyST Parser package heavily uses the the [markdown-it-py](https://github.com/executablebooks/markdown-it-py) package. +- {ref}`The MyST-Parser API reference ` contains a more complete reference of this API. ``` -The raw text is first parsed to syntax 'tokens', -then these are converted to other formats using 'renderers'. +## Parsing and rendering helper functions + +The MyST Parser comes bundled with some helper functions to quickly parse MyST Markdown and render its output. -## Quick-Start +### Parse MyST Markdown to HTML -The simplest way to understand how text will be parsed is using: +For example, the following code parses markdown and renders as HTML: ```python from myst_parser.main import to_html @@ -25,6 +24,10 @@ to_html("some *text*") ``` +### Parse MyST Markdown to docutils + +The following function renders your text as **docutils objects** (for example, for use with the Sphinx ecosystem): + ```python from myst_parser.main import to_docutils print(to_docutils("some *text*").pformat()) @@ -38,13 +41,19 @@ print(to_docutils("some *text*").pformat()) text ``` +### Parse MyST Markdown as `markdown-it` tokens + +The MyST Parser uses `markdown-it-py` tokens as an intermediate representation of your text. +Normally these tokens are then *rendered* into various outputs. +If you'd like direct access to the tokens, use the `to_tokens` function. +Here's an example of its use: + ```python from pprint import pprint from myst_parser.main import to_tokens for token in to_tokens("some *text*"): - print(token) - print() + print(token, "\n") ``` @@ -57,10 +66,23 @@ Token(type='paragraph_close', tag='p', nesting=-1, attrs=None, map=None, level=0 ``` -# The Parser +Each token is an abstract representation of a piece of MyST Markdown syntax. + +## Use the parser object for more control + +The MyST Parser is actually a `markdown-it-py` parser with several extensions pre-enabled that support the MyST syntax. +If you'd like more control over the parsing process, then you can directly use a `markdown-it-py` parser with MyST syntax extensions loaded. +:::{seealso} +[`markdown-it-py`](https://markdown-it-py.readthedocs.io/) is an extensible Python parser and renderer for flavors of markdown. +It is inspired heavily by the [`markdown-it`](https://github.com/markdown-it/markdown-it) Javascript package. +See the documentation of these tools for more information. +::: -The `default_parser` function loads a standard markdown-it parser with the default syntax rules for MyST. +### Load a parser + +To load one of these parsers for your own use, use the `default_parser` function. +Below we'll create such a parser and show that it is an instance of a `markdown-it-py` parser: ```python from myst_parser.main import default_parser, MdParserConfig @@ -75,6 +97,12 @@ markdown_it.main.MarkdownIt() ``` +### List the active rules + +We can list the **currently active rules** for this parser. +Each rules maps onto a particular markdown syntax, and a Token. +To list the active rules, use the `get_active_rules` method: + ```python pprint(parser.get_active_rules()) ``` @@ -118,6 +146,11 @@ pprint(parser.get_active_rules()) ``` +### Parse and render markdown + +Once we have a Parser instance, we can use it to parse some markdown. +Use the `render` function to do so: + ```python parser.render("*abc*") ``` @@ -128,7 +161,10 @@ parser.render("*abc*") ``` -Any of these rules can be disabled: +### Disable and enable rules + +You can disable and enable rules for a parser using the `disable` and `enable` methods. +For example, below we'll disable the `emphasis` rule (which is what detected the `*abc*` syntax above) and re-render the text: ```python parser.disable("emphasis").render("*abc*") @@ -140,7 +176,11 @@ parser.disable("emphasis").render("*abc*") ``` -`renderInline` turns off any block syntax rules. +As you can see, the parser no longer detected the `**` syntax as requiring an _emphasis_. + +### Turn off all block-level syntax + +If you'd like to use your parser *only* for in-line content, you may turn off all block-level syntax with the `renderInline` method: ```python parser.enable("emphasis").renderInline("- *abc*") @@ -152,12 +192,13 @@ parser.enable("emphasis").renderInline("- *abc*") ``` -## The Token Stream - +## The Token Stream +When you parse markdown with the MyST Parser, the result is a flat stream of **Tokens**. +These are abstract representations of each type of syntax that the parser has detected. -The text is parsed to a flat token stream: +For example, below we'll show the token stream for some simple markdown: ```python from myst_parser.main import to_tokens @@ -190,7 +231,10 @@ Here's some *text* ``` -Inline type tokens contain the inline tokens as children: +Note that these tokens are **flat**, although some of the tokens refere to one another (for example, Tokens with `_open` and `_close` represent the start/end of blocks). + +Tokens of type `inline` will have a `children` attribute that contains a list of the Tokens that they contain. +For example: ```python tokens[6] @@ -202,7 +246,12 @@ Token(type='inline', tag='', nesting=0, attrs=None, map=[3, 4], level=3, childre ``` -The sphinx renderer first converts the token to a nested structure, collapsing the opening/closing tokens into single tokens: +### Rendering tokens + +The list of Token objects can be rendered to a number of different outputs. +This involves first processing the Tokens, and then defining how each should be rendered in an output format (e.g., HTML or Docutils). + +For example, the sphinx renderer first converts the token to a nested structure, collapsing the opening/closing tokens into single tokens: ```python from markdown_it.token import nest_tokens @@ -232,142 +281,5 @@ Token(type='paragraph_close', tag='p', nesting=-1, attrs=None, map=None, level=0 ``` -## Renderers - -The `myst_parser.docutils_renderer.DocutilsRenderer` converts a token directly to the `docutils.document` representation of the document, converting roles and directives to a `docutils.nodes` if a converter can be found for the given name. - -````python -from myst_parser.main import to_docutils - -document = to_docutils(""" -Here's some *text* - -1. a list - -> a quote - -{emphasis}`content` - -```{sidebar} my sidebar -content -``` -""") - -print(document.pformat()) -```` - -```xml - - - Here's some - - text - - - - a list - - - a quote - - - content - - - my sidebar - <paragraph> - content -``` - - -The `myst_parser.sphinx_renderer.SphinxRenderer` builds on the `DocutilsRenderer` to add sphinx specific nodes, e.g. for cross-referencing between documents. - -To use the sphinx specific roles and directives outside of a `sphinx-build`, they must first be loaded with the `in_sphinx_env` option. - -````python -document = to_docutils(""" -Here's some *text* - -1. a list - -> a quote - -{ref}`target` - -```{glossary} my gloassary -name - definition -``` -""", - in_sphinx_env=True) -print(document.pformat()) -```` - -```xml -<document source="notset"> - <paragraph> - Here's some - <emphasis> - text - <enumerated_list> - <list_item> - <paragraph> - a list - <block_quote> - <paragraph> - a quote - <paragraph> - <pending_xref refdoc="mock_docname" refdomain="std" refexplicit="False" reftarget="target" reftype="ref" refwarn="True"> - <inline classes="xref std std-ref"> - target - <glossary> - <definition_list classes="glossary"> - <definition_list_item> - <term ids="term-my-gloassary"> - my gloassary - <index entries="('single',\ 'my\ gloassary',\ 'term-my-gloassary',\ 'main',\ None)"> - <term ids="term-name"> - name - <index entries="('single',\ 'name',\ 'term-name',\ 'main',\ None)"> - <definition> - <paragraph> - definition -``` - - -You can also set Sphinx configuration *via* `sphinx_conf`. This is a dictionary representation of the contents of the Sphinx `conf.py`. - -```{warning} -This feature is only meant for simple testing. -It will fail for extensions that require the full -Sphinx build process and/or access to external files. -``` - -`````python -document = to_docutils(""" -````{tabs} - -```{tab} Apples - -Apples are green, or sometimes red. -``` -```` -""", - in_sphinx_env=True, - conf={"extensions": ["sphinx_tabs.tabs"]} -) -print(document.pformat()) -````` - -```xml -<document source="notset"> - <container classes="sphinx-tabs"> - <container> - <a classes="item"> - <container> - <paragraph> - Apples - <container classes="ui bottom attached sphinx-tab tab segment sphinx-data-tab-0-0 active"> - <paragraph> - Apples are green, or sometimes red. -``` +It then renders each token to a Sphinx-based docutils object. +See [the renderers section](renderers.md) for more information about rendering tokens. diff --git a/docs/api/renderers.rst b/docs/api/reference.rst similarity index 74% rename from docs/api/renderers.rst rename to docs/api/reference.rst index 8806fd77..431fdb73 100644 --- a/docs/api/renderers.rst +++ b/docs/api/reference.rst @@ -1,4 +1,14 @@ -.. _api/renderers: +============= +API Reference +============= + +.. _api/directive: + +Directive Parsing Reference +--------------------------- + +.. automodule:: myst_parser.parse_directives + :members: MyST Renderers -------------- @@ -69,3 +79,22 @@ Additional Methods .. autofunction:: myst_parser.sphinx_renderer.minimal_sphinx_app .. autofunction:: myst_parser.sphinx_renderer.mock_sphinx_env + + + +.. _api/sphinx_parser: + +Sphinx Parser Reference +----------------------- + +This class builds on the :py:class:`~myst_parser.sphinx_renderer.SphinxRenderer` +to generate a parser for Sphinx, using the :ref:`Sphinx parser API <sphinx:parser-api>`: + +.. autoclass:: myst_parser.sphinx_parser.MystParser + :members: supported, parse + :undoc-members: + :member-order: bysource + :show-inheritance: + :exclude-members: __init__ + +.. _api/renderers: diff --git a/docs/api/renderers.md b/docs/api/renderers.md new file mode 100644 index 00000000..bdc9d60f --- /dev/null +++ b/docs/api/renderers.md @@ -0,0 +1,146 @@ +# Render outputs + +There are a few different ways to render MyST Parser tokens into different outputs. +This section covers a few common ones. + +## The `docutils` renderer + +The `myst_parser.docutils_renderer.DocutilsRenderer` converts a token directly to the `docutils.document` representation of the document, converting roles and directives to a `docutils.nodes` if a converter can be found for the given name. + +````python +from myst_parser.main import to_docutils + +document = to_docutils(""" +Here's some *text* + +1. a list + +> a quote + +{emphasis}`content` + +```{sidebar} my sidebar +content +``` +""") + +print(document.pformat()) +```` + +```xml +<document source="notset"> + <paragraph> + Here's some + <emphasis> + text + <enumerated_list> + <list_item> + <paragraph> + a list + <block_quote> + <paragraph> + a quote + <paragraph> + <emphasis> + content + <sidebar> + <title> + my sidebar + <paragraph> + content +``` + +## The Sphinx renderer + +The `myst_parser.sphinx_renderer.SphinxRenderer` builds on the `DocutilsRenderer` to add sphinx specific nodes, e.g. for cross-referencing between documents. + +To use the sphinx specific roles and directives outside of a `sphinx-build`, they must first be loaded with the `in_sphinx_env` option. + +````python +document = to_docutils(""" +Here's some *text* + +1. a list + +> a quote + +{ref}`target` + +```{glossary} my gloassary +name + definition +``` +""", + in_sphinx_env=True) +print(document.pformat()) +```` + +```xml +<document source="notset"> + <paragraph> + Here's some + <emphasis> + text + <enumerated_list> + <list_item> + <paragraph> + a list + <block_quote> + <paragraph> + a quote + <paragraph> + <pending_xref refdoc="mock_docname" refdomain="std" refexplicit="False" reftarget="target" reftype="ref" refwarn="True"> + <inline classes="xref std std-ref"> + target + <glossary> + <definition_list classes="glossary"> + <definition_list_item> + <term ids="term-my-gloassary"> + my gloassary + <index entries="('single',\ 'my\ gloassary',\ 'term-my-gloassary',\ 'main',\ None)"> + <term ids="term-name"> + name + <index entries="('single',\ 'name',\ 'term-name',\ 'main',\ None)"> + <definition> + <paragraph> + definition +``` + +### Set Sphinx configuration for testing + +You can also set Sphinx configuration *via* `sphinx_conf`. This is a dictionary representation of the contents of the Sphinx `conf.py`. + +```{warning} +This feature is only meant for simple testing. +It will fail for extensions that require the full +Sphinx build process and/or access to external files. +``` + +`````python +document = to_docutils(""" +````{tabs} + +```{tab} Apples + +Apples are green, or sometimes red. +``` +```` +""", + in_sphinx_env=True, + conf={"extensions": ["sphinx_tabs.tabs"]} +) +print(document.pformat()) +````` + +```xml +<document source="notset"> + <container classes="sphinx-tabs"> + <container> + <a classes="item"> + <container> + <paragraph> + Apples + <container classes="ui bottom attached sphinx-tab tab segment sphinx-data-tab-0-0 active"> + <paragraph> + Apples are green, or sometimes red. +``` diff --git a/docs/api/sphinx_parser.rst b/docs/api/sphinx_parser.rst deleted file mode 100644 index a16bdba6..00000000 --- a/docs/api/sphinx_parser.rst +++ /dev/null @@ -1,14 +0,0 @@ -.. _api/sphinx_parser: - -Sphinx Parser -------------- - -This class builds on the :py:class:`~myst_parser.sphinx_renderer.SphinxRenderer` -to generate a parser for Sphinx, using the :ref:`Sphinx parser API <sphinx:parser-api>`: - -.. autoclass:: myst_parser.sphinx_parser.MystParser - :members: supported, parse - :undoc-members: - :member-order: bysource - :show-inheritance: - :exclude-members: __init__ diff --git a/docs/conf.py b/docs/conf.py index 28fbb65f..57a156db 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -28,6 +28,8 @@ "sphinx.ext.viewcode", "sphinxcontrib.bibtex", "sphinx_panels", + "sphinxext.rediraffe", + "sphinxcontrib.mermaid", ] # Add any paths that contain templates here, relative to this directory. @@ -79,6 +81,13 @@ myst_dmath_double_inline = True panels_add_bootstrap_css = False bibtex_bibfiles = ["examples/references.bib"] +rediraffe_redirects = { + "using/intro.md": "sphinx/intro.md", + "using/use_api.md": "api/index.md", + "using/syntax.md": "syntax/syntax.md", + "using/syntax-optional.md": "syntax/syntax-optional.md", + "using/reference.md": "syntax/reference.md", +} def run_apidoc(app): diff --git a/docs/explain/index.md b/docs/explain/index.md new file mode 100644 index 00000000..2528973a --- /dev/null +++ b/docs/explain/index.md @@ -0,0 +1,83 @@ +# Discussion and explanation + +These sections discuss high-level questions about the MyST ecosystem, and explain a few decisions made in the project. + + +## Why did we create MyST markdown? + +While markdown is ubiquitous, it is not powerful enough for writing modern, +fully-featured documentation. Some flavors of markdown support features needed for this, +but there is no community standard around various syntactic choices for these features. + +Sphinx is a documentation generation framework written in Python. It heavily-utilizes +reStructuredText syntax, which is another markup language for writing documents. In +particular, Sphinx defines two extension points that are extremely useful: +**{ref}`in-line roles<sphinx:rst-roles-alt>`** and **{ref}`block-level directives <sphinx:rst-directives>`**. + +**This project is an attempt at combining the simplicity and readability of Markdown +with the power and flexibility of reStructuredText and the Sphinx platform.** It +starts with the [CommonMark markdown specification][commonmark], and selectively adds a few extra +syntax pieces to utilize the most powerful parts of reStructuredText. + +```{note} +The CommonMark community has been discussing an "official" extension syntax for many +years now (for example, see +[this seven-year-old thread about directives](https://talk.commonmark.org/t/generic-directives-plugins-syntax/444) as well as +[this more recent converstaion](https://talk.commonmark.org/t/support-for-extension-token/2771), +and [this comment listing several more threads on this topic](https://talk.commonmark.org/t/extension-terminology-and-rules/1233)). + +We have chosen a "roles and directives" syntax that seems reasonable and follows other +common conventions in Markdown flavors. However, if the CommonMark community ever +decides on an "official" extension syntax, we will likely utilize this syntax for +MyST. +``` + +## The relationship between MyST, reStructuredText, and Sphinx + +MyST markdown provides a markdown equivalent of the reStructuredText syntax, +meaning that you can do anything in MyST that you can do with reStructuredText. + +The Sphinx documentation engine supports a number of different input types. By default, +Sphinx reads **reStructuredText** (`.rst`) files. Sphinx uses a **parser** to parse input files +into its own internal document model (which is provided by a core Python project, +[docutils](https://docutils.sourceforge.io/)). + +Developers can *extend Sphinx* to support other kinds of input files. Any content file +can be read into the Sphinx document structure, provided that somebody writes a +**parser** for that file. Once a content file has been parsed into Sphinx, it behaves +nearly the same way as any other content file, regardless of the language in which it +was written. + +The MyST-parser is a Sphinx parser for the MyST markdown language. +When you use it, Sphinx will know how to parse content files that contain MyST markdown (by default, Sphinx will assume any files ending in `.md` are written in MyST markdown). Once a document has been parsed into Sphinx, it behaves the same way regardless of whether it has been written in rST or MyST markdown. + +``` +myst markdown (.md) ------> myst parser ---+ + | + +-->Sphinx document (docutils) + | +reStructuredText (.rst) --> rst parser ----+ +``` + +For example, here's how you'd write a `toctree` directive in MyST markdown: + +```` +```{toctree} +My page name <page1> +page2 +``` +```` + +and here's the same in rST: + +``` +.. toctree:: + + My page name <page1> + page2 +``` + +They will both behave the same in Sphinx. + + +[commonmark]: https://commonmark.org/ diff --git a/docs/index.md b/docs/index.md index a44a5d60..f35ef928 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,90 +3,80 @@ [![PyPI][pypi-badge]][pypi-link] [![Conda][conda-badge]][conda-link] -**A fully-functional markdown flavor and parser for Sphinx.** +**MyST is a rich and extensible flavor of Markdown meant for technical documentation and publishing**. -MyST allows you to write Sphinx documentation entirely in markdown. -MyST markdown provides a markdown equivalent of the reStructuredText syntax, -meaning that you can do anything in MyST that you can do with reStructuredText. -It is an attempt to have the best of both worlds: the flexibility -and extensibility of Sphinx with the simplicity and readability of Markdown. +MyST is a flavor of markdown that is designed for simplicity, flexibility, and extensibility. +Here are a few key features of MyST Markdown: -MyST has the following main features: +{fa}`check,text-success mr-1` It is [a superset of CommonMark markdown][commonmark]. +: Any CommonMark document is also MyST-compliant. -{fa}`check,text-success mr-1` [A markdown parser for Sphinx](parse-with-sphinx) -: You can write your entire {doc}`Sphinx documentation <sphinx:usage/quickstart>` in Markdown. +{fa}`check,text-success mr-1` It [extends CommonMark with rich writing syntax](extended-block-tokens). +: These make it easier to write MyST and extend its functionality to technical documentation (for example, line commenting and footnotes). -{fa}`check,text-success mr-1` [Call Sphinx directives and roles from within Markdown](syntax/directives) -: Allowing you to extend your document via Sphinx extensions. +{fa}`check,text-success mr-1` It provides [extension points with directives and roles](syntax/directives). +: Any MyST parser can define new functionality via a standard syntax extension point (this is kind of like defining functions with markdown). -{fa}`check,text-success mr-1` [Extended Markdown syntax for useful rST features](extended-block-tokens) -: Such as line commenting and footnotes. +This MyST Parser serves as a reference implementation for MyST Markdown, and comes bundled with several tools that help you use MyST in your work. +It comes bundled with a few tools to help you use MyST with Python: -{fa}`check,text-success mr-1` [A Sphinx-independent parser of MyST markdown](using/use_api) -: This can be extended to add new functionality and outputs for MyST. +[A Python parser of MyST markdown](api/index) +: This can be extended to add new functionality and outputs for MyST, and is built on top of [`markdown-it-py`](https://markdown-it-py.readthedocs.io/). -{fa}`check,text-success mr-1` [A superset of CommonMark markdown][commonmark] -: Any CommonMark markdown (such as Jupyter Notebook markdown) is natively supported by the MyST parser. +[A markdown parser for Sphinx](sphinx/index) +: You can write your entire {doc}`Sphinx documentation <sphinx:usage/quickstart>` in Markdown, including [**roles** and **directives**](syntax/directives). + This allows you to extend MyST's functionality in Sphinx via Sphinx extensions. -You may use MyST markdown **in addition to** using reStructuredText in Sphinx. -See {doc}`using/intro` to get started. +:::{seealso} +For some examples of how MyST can be used, check out the [Jupyter Book project](https://jupyterbook.org), which uses MyST Markdown heavily, as well as [the Jupyter Book gallery](https://gallery.jupyterbook.org) which contains a list of books built by others, all using MyST. +::: -## Site contents +## Get started + +These pages cover step-by-step instructions to get started with MyST Markdown. ```{toctree} ---- -maxdepth: 2 -caption: Using MyST Markdown ---- -using/intro.md -using/syntax.md -using/syntax-optional.md -using/howto.md -using/faq.md -using/use_api.md +:maxdepth: 2 +sphinx/intro.md ``` +## MyST Syntax + +These sections cover the syntax that makes up MyST Markdown, some common use-cases that it supports, as well as a few extensions that allow you to enable new features with MyST. + ```{toctree} ---- -maxdepth: 2 -caption: Reference and contributing ---- +:caption: MyST Syntax +:maxdepth: 2 +syntax/syntax +syntax/syntax-optional +syntax/reference +``` + +## Topic Guides + +Topic guides cover particular tools, use-cases, and functionality in the MyST ecosystem. + +```{toctree} +:maxdepth: 2 +:caption: Topic Guides +sphinx/index.md +api/index.md +explain/index.md +``` + +## About the project + +These sections cover "meta" information about the MyST Markdown project. + +```{toctree} +:maxdepth: 2 +:caption: About the project examples/index.md develop/index.md -api/index.md develop/_changelog.md GitHub repo <https://github.com/executablebooks/myst-parser> ``` -## Why MyST markdown? - -While markdown is ubiquitous, it is not powerful enough for writing modern, -fully-featured documentation. Some flavors of markdown support features needed for this, -but there is no community standard around various syntactic choices for these features. - -Sphinx is a documentation generation framework written in Python. It heavily-utilizes -reStructuredText syntax, which is another markup language for writing documents. In -particular, Sphinx defines two extension points that are extremely useful: -**{ref}`in-line roles<sphinx:rst-roles-alt>`** and **{ref}`block-level directives <sphinx:rst-directives>`**. - -**This project is an attempt at combining the simplicity and readability of Markdown -with the power and flexibility of reStructuredText and the Sphinx platform.** It -starts with the [CommonMark markdown specification][commonmark], and selectively adds a few extra -syntax pieces to utilize the most powerful parts of reStructuredText. - -```{note} -The CommonMark community has been discussing an "official" extension syntax for many -years now (for example, see -[this seven-year-old thread about directives](https://talk.commonmark.org/t/generic-directives-plugins-syntax/444) as well as -[this more recent converstaion](https://talk.commonmark.org/t/support-for-extension-token/2771), -and [this comment listing several more threads on this topic](https://talk.commonmark.org/t/extension-terminology-and-rules/1233)). - -We have chosen a "roles and directives" syntax that seems reasonable and follows other -common conventions in Markdown flavors. However, if the CommonMark community ever -decides on an "official" extension syntax, we will likely utilize this syntax for -MyST. -``` - ## Acknowledgements The MyST markdown language and MyST parser are both supported by the open community, diff --git a/docs/sphinx/faq.md b/docs/sphinx/faq.md new file mode 100644 index 00000000..a799b168 --- /dev/null +++ b/docs/sphinx/faq.md @@ -0,0 +1,16 @@ +# Common errors and questions + +These are common issues and gotchas that people may experience when using the MyST Sphinx extension. + +## What markup language should I use inside directives? + +If you need to parse content *inside* of another block of content (for example, the +content inside a **note directive**), note that the MyST parser will be used for this +nested parsing as well. + +## Why doesn't my role/directive recognize markdown link syntax? + +There are some roles/directives that _hard-code_ syntax into +their behavior. For example, many roles allow you to supply titles for links like so: +`` {role}`My title <myref>` ``. While this looks like reStructuredText, the role may +be explicitly expecting the `My title <myref>` structure, and so MyST will behave the same way. diff --git a/docs/sphinx/index.md b/docs/sphinx/index.md new file mode 100644 index 00000000..216e08af --- /dev/null +++ b/docs/sphinx/index.md @@ -0,0 +1,15 @@ +# MyST with Sphinx + +The MyST Parser comes bundled with a Sphinx extension that allows you to use write Sphinx documentation entirely in MyST (or, in a combination of rST and MyST). +The following sections cover some major functionality of the Sphinx extension. + +:::{seealso} +For an introduction to MyST with Sphinx, see [](intro.md). +::: + +```{toctree} +use.md +roles-and-directives.md +reference.md +faq.md +``` diff --git a/docs/sphinx/intro.md b/docs/sphinx/intro.md new file mode 100644 index 00000000..7c7f46b3 --- /dev/null +++ b/docs/sphinx/intro.md @@ -0,0 +1,423 @@ +(intro/get-started)= +# Get started with MyST in Sphinx + +This page describes how to get started with the MyST parser, with a focus on enabling it in the Sphinx documentation engine. + +## Install the MyST Parser + +[![PyPI][pypi-badge]][pypi-link] +[![Conda][conda-badge]][conda-link] + +Installing the MyST parser provides access to two tools: + +* A Python library that can parse MyST markdown, and render it to a number of output formats (in particular, `docutils` format for use with Sphinx). +* A Sphinx extension that that utilizes the above tool in order to parse MyST Markdown in your documentation. + +To install the MyST parser, run the following in a +[Conda environment](https://docs.conda.io) (recommended): + +```bash +conda install -c conda-forge myst-parser +``` + +or + +```bash +pip install myst-parser +``` + +[pypi-badge]: https://img.shields.io/pypi/v/myst-parser.svg +[pypi-link]: https://pypi.org/project/myst-parser +[conda-badge]: https://anaconda.org/conda-forge/myst-parser/badges/version.svg +[conda-link]: https://anaconda.org/conda-forge/myst-parser + +(parse-with-sphinx)= +## Enable MyST in Sphinx + +Sphinx is a documentation generator for building a website or book from multiple source documents and assets. To get started with Sphinx, see their [Quickstart Guide](https://www.sphinx-doc.org/en/master/usage/quickstart.html). This guide assumes that you've already got a pre-existing Sphinx site that builds properly. + +To use the MyST parser in Sphinx, simply add the following to your `conf.py` file: + +```python +extensions = ["myst_parser"] +``` + +This will activate the MyST Parser extension, causing all documents with the `.md` extension to be parsed as MyST. + +:::{admonition} You can use both MyST and reStructuredText +:class: tip + +Activating the MyST parser will simply *enable* parsing markdown files with MyST, and the rST parser that ships with Sphinx will still work the same way. +Files ending with `.md` will be parsed as MyST, and files ending in `.rst` will be parsed as reStructuredText. +::: + +(intro/writing)= +## Write your first markdown document + +Now that you've enabled the `myst-parser` in Sphinx, you can write MyST markdown in a file that ends with `.md` extension for your pages. + +:::{note} +MyST markdown is a mixture of two flavors of markdown: + +It supports all the syntax of **[CommonMark Markdown](https://commonmark.org/)** at its +base. This is a community standard flavor of markdown used across many projects. + +In addition, it includes **[several extensions](../syntax/syntax.md) to CommonMark**. +These add extra syntax features for technical writing, such as the roles and directives used by Sphinx. +::: + +To start off, create an empty file called `myfile.md` and give it a markdown title and text. + +```md +# My nifty title + +Some **text**! +``` + +In the "main document" of your Sphinx project (the landing page of your Sphinx documentation), include `myfile.md` in a `toctree` directive so that it is included in your documentation: + +```rst +.. toctree:: + + myfile.md +``` + +Now build your site: + +```bash +make html +``` + +and navigate to your landing page. +You should see a link to the page generated from `myfile.md`. +Clicking that link should take you to your rendered Markdown! + +## Extend markdown with a directive + +The most important functionality available with MyST markdown is writing **directives**. +Directives are kind-of like functions that are designed for writing content. +Sphinx and reStructuredText use directives extensively. +Here's how a directive looks in MyST markdown: + +````{margin} Alternative options syntax +If you've got a lot of options for your directive, or have a value that is really +long (e.g., that spans multiple lines), then you can also wrap your options in +`---` lines and write them as YAML. For example: + +```yaml +--- +key1: val1 +key2: | + val line 1 + val line 2 +--- +``` +```` + +```` +```{directivename} <directive arguments> +:optionname: <valuename> + +<directive content> +``` +```` + +For those who are familiar with reStructuredText, you can find [a mapping from MyST directive syntax to rST syntax here](syntax/directives). + + +As seen above, there are four main parts to consider when writing directives. + +* **the directive name** is kind of like the function name. Different names trigger + different functionality. They are wrapped in `{}` brackets. +* **directive arguments** come just after the directive name. They can be used + to trigger behavior in the directive. +* **directive options** come just after the first line of the directive. They also + control behavior of the directive. +* **directive content** is markdown that you put inside the directive. The directive + often displays the content in a special way. + +For example, add an **`admonition`** directive to your markdown page, like so: + + +````md +# My nifty title + +Some **text**! + +```{admonition} Here's my title +:class: warning + +Here's my admonition content +``` +```` + +Re-build your Sphinx site and you should see the new admonition box show up. + +As you can see, we've used each of the four pieces described above to configure this +directive. Here's how the directive looks when rendered: + +```{admonition} Here's my title +:class: warning + +Here's my admonition content +``` + +:::{seealso} +For more information about using directives with MyST, see {ref}`syntax/directives`. +::: + +(sphinx/intro:reference)= +## Reference a section label with a role + +Roles are another core Sphinx tool. They behave similarly to directives, but are given +in-line with text instead of in a separate block. They have the following form: + +```md +{rolename}`role content` +``` + +Roles are a bit more simple than directives, though some roles allow for more complex syntax inside their content area. +For example, the `ref` role is used to make references to other sections of your documentation, and allows you to specify the displayed text as well as the reference itself within the role: + +```md +{ref}`My displayed text <my-ref>` +``` + +For example, let's add a **section reference** to your markdown file. +To do this, we'll first need to add a **label** to a section of your page. +To do so, use the following structure: + +```md +(label-name)= +## Some header +``` + +Add this to your markdown file from above, like so: + +````md +# My nifty title + +Some **text**! + +```{admonition} Here's my title +:class: warning + +Here's my admonition content +``` + +(section-two)= +## Here's another section + +And some more content. +```` + +Because your new section has a label (`section-two`), you can reference it with the `ref` role. +Add it to your markdown file like so: + + +```md +(label-name)= +## Some header +``` + +Add this to your markdown file from above, like so: + +````md +# My nifty title + +Some **text**! + +```{admonition} Here's my title +:class: warning + +Here's my admonition content +``` + +(section-two)= +## Here's another section + +And some more content. + +And here's {ref}`a reference to this section <section-two>`. +I can also reference the section {ref}`section-two` without specifying my title. +```` + +Re-build your documentation and you should see the references automatically inserted. +Here's an example of how the `ref` roles look in the final output: + +Here's a reference to {ref}`sphinx/intro:reference`. + +:::{seealso} +For more information about roles, see {ref}`syntax/roles`. +::: + +## Add a comment using extra MyST syntax + +There are many other kinds of syntax in MyST to make writing more productive and enjoyable. +Let's play around with a couple of options. + +First, try writing a **comment**. +This can be done by adding a line starting with `%` to your markdown file. +For example, try adding a comment to your markdown file, like so: + +````md +# My nifty title + +Some **text**! + +```{admonition} Here's my title +:class: warning + +Here's my admonition content +``` + +(section-two)= +## Here's another section + +And some more content. + +% This comment won't make it into the outputs! +And here's {ref}`a reference to this section <section-two>`. +I can also reference the section {ref}`section-two` without specifying my title. +```` + +Re-build your documentation - the comment should _not_ be present in the output. + +## Extending MyST via configuration + +Thus far we have covered the basic MyST syntax with Sphinx. +However, there are a few ways that you can _extend_ this base syntax and get new functionality. +The first is to enable some "out of the box" extensions with the MyST parser. +These add new syntax that aren't part of "core MyST" but that are useful nonetheless (and may become part of core MyST one day). + +Let's extend the base MyST syntax to enable **fences for directives**. +This allows you to define a directive with `:::` in addition to ` ``` `. +This is useful for directives that have markdown in their content. +By using `:::`, a non-MyST markdown renderer will still be able to render what is inside (instead of displaying it as a code block). + +To activate extensions, add a list to your `conf.py` file that contains the extensions you'd like to activate. +For example, to activate the "colon code fences" extension, add the following to your `conf.py` file: + +```python +myst_enable_extensions = [ + "colon_fence", +] +``` + +You may now use `:::` to define directives. +For example, modify your markdown file like so: + +````md +# My nifty title + +Some **text**! + +```{admonition} Here's my title +:class: warning + +Here's my admonition content +``` + +(section-two)= +## Here's another section + +And some more content. + +% This comment won't make it into the outputs! +And here's {ref}`a reference to this section <section-two>`. +I can also reference the section {ref}`section-two` without specifying my title. + +:::{note} +And here's a note with a colon fence! +::: +```` + +It should render as a "note block" in your output when you build your site. + +## Install a new Sphinx extension and use its functionality + +The other way to extend MyST in Sphinx is to install Sphinx extensions that define new directives. +Directives are kind of like "functions" in Sphinx, and installing a new package can add new directives to use in your content. + +For example, let's install the `sphinxcontib.mermaid` extension, which will allow us to generate [Mermaid diagrams](https://mermaid-js.github.io/mermaid/#/) with MyST. + +First, install `sphinxcontrib.mermaid`: + +```shell +pip install sphinxcontrib-mermaid +``` + +Next, add it to your list of extensions in `conf.py`: + +```python +extensions = [ + "myst_parser", + "sphinxcontrib.mermaid", +] +``` + +Now, add a **mermaid directive** to your markdown file. +For example: + +````md +# My nifty title + +Some **text**! + +```{admonition} Here's my title +:class: warning + +Here's my admonition content +``` + +(section-two)= +## Here's another section + +And some more content. + +% This comment won't make it into the outputs! +And here's {ref}`a reference to this section <section-two>`. +I can also reference the section {ref}`section-two` without specifying my title. + +:::{note} +And here's a note with a colon fence! +::: + +And finally, here's a cool mermaid diagram! + +```{mermaid} +sequenceDiagram + participant Alice + participant Bob + Alice->John: Hello John, how are you? + loop Healthcheck + John->John: Fight against hypochondria + end + Note right of John: Rational thoughts <br/>prevail... + John-->Alice: Great! + John->Bob: How about you? + Bob-->John: Jolly good! +``` +```` + +When you build your documentation, you should see something like this: + +```{mermaid} +sequenceDiagram + participant Alice + participant Bob + Alice->John: Hello John, how are you? + loop Healthcheck + John->John: Fight against hypochondria + end + Note right of John: Rational thoughts <br/>prevail... + John-->Alice: Great! + John->Bob: How about you? + Bob-->John: Jolly good! +``` + +## Next steps - Learn more about MyST Syntax + +In this tutorial we've covered some of the basics of MyST Markdown, how to enable and use it with Sphinx, and how to extend it for new use-cases. +There is much more functionality in MyST (and in the Sphinx ecosystem) that we haven't covered here. +For more information, see the [documentation on MyST Syntax](../syntax/syntax.md) and the [documentation about using MyST with Sphinx](../sphinx/index.md). diff --git a/docs/sphinx/reference.md b/docs/sphinx/reference.md new file mode 100644 index 00000000..f9d0e645 --- /dev/null +++ b/docs/sphinx/reference.md @@ -0,0 +1,110 @@ +(sphinx/config-options)= +# Sphinx configuration options + +You can control the behaviour of the MyST parser in Sphinx by modifying your `conf.py` file. +To do so, use the keywords beginning `myst_`. + +`````{list-table} +:header-rows: 1 + +* - Option + - Default + - Description +* - `myst_commonmark_only` + - `False` + - If `True` convert text as strict CommonMark (all options below are then ignored). Note that strict CommonMark is unable to parse any directives, including the `toctree` directive, thus limiting MyST parser to single-page documentations. Use in conjunction with [sphinx-external-toc](https://github.com/executablebooks/sphinx-external-toc) Sphinx extension to counter this limitation. +* - `myst_disable_syntax` + - () + - List of markdown syntax elements to disable, see the [markdown-it parser guide](markdown_it:using). +* - `myst_enable_extensions` + - `["dollarmath"]` + - Enable Markdown extensions, [see here](../syntax/syntax-optional.md) for details. +* - `myst_url_schemes` + - `None` + - [URI schemes](https://en.wikipedia.org/wiki/List_of_URI_schemes) that will be recognised as external URLs in `[](scheme:loc)` syntax, or set `None` to recognise all. + Other links will be resolved as internal cross-references. +* - `myst_heading_anchors` + - `None` + - Enable auto-generated heading anchors, up to a maximum level, [see here](syntax/header-anchors) for details. +* - `myst_heading_slug_func` + - `None` + - Use the specified function to auto-generate heading anchors, [see here](syntax/header-anchors) for details. +* - `myst_substitutions` + - `{}` + - A mapping of keys to substitutions, used globally for all MyST documents when the "substitution" extension is enabled. +* - `myst_html_meta` + - `{}` + - A mapping of keys to HTML metadata, used globally for all MyST documents. See [](syntax/html_meta). +* - `myst_footnote_transition` + - `True` + - Place a transition before any footnotes. +* - `myst_words_per_minute` + - `200` + - Reading speed used to calculate `` {sub-ref}`wordcount-minutes` `` +````` + +List of extensions: + +- "amsmath": enable direct parsing of [amsmath](https://ctan.org/pkg/amsmath) LaTeX equations +- "colon_fence": Enable code fences using `:::` delimiters, [see here](syntax/colon_fence) for details +- "deflist" +- "dollarmath": Enable parsing of dollar `$` and `$$` encapsulated math +- "html_admonition": Convert `<div class="admonition">` elements to sphinx admonition nodes, see the [HTML admonition syntax](syntax/html-admonition) for details +- "html_image": Convert HTML `<img>` elements to sphinx image nodes, see the [image syntax](syntax/images) for details +- "linkify": automatically identify "bare" web URLs and add hyperlinks +- "replacements": automatically convert some common typographic texts +- "smartquotes": automatically convert standard quotations to their opening/closing variants +- "substitution": substitute keys, see the [substitutions syntax](syntax/substitutions) for details +- "tasklist": add check-boxes to the start of list items, see the [tasklist syntax](syntax/tasklists) for details + +Math specific, when `"dollarmath"` activated, see the [Math syntax](syntax/math) for more details: + +`````{list-table} +:header-rows: 1 + +* - Option + - Default + - Description +* - `myst_dmath_double_inline` + - `False` + - Allow display math (i.e. `$$`) within an inline context +* - `myst_dmath_allow_labels` + - `True` + - Parse `$$...$$ (label)` syntax +* - `myst_dmath_allow_space` + - `True` + - If False then inline math will only be parsed if there are no initial/final spaces, + e.g. `$a$` but not `$ a$` or `$a $` +* - `myst_dmath_allow_digits` + - `True` + - If False then inline math will only be parsed if there are no initial/final digits, + e.g. `$a$` but not `1$a$` or `$a$2` (this is useful for using `$` as currency) +* - `myst_amsmath_enable` + - `False` + - Enable direct parsing of [amsmath LaTeX environments](https://ctan.org/pkg/amsmath) +* - `myst_update_mathjax` + - `True` + - If using [sphinx.ext.mathjax](https://www.sphinx-doc.org/en/master/usage/extensions/math.html#module-sphinx.ext.mathjax) (the default) then `mathjax_config` will be updated, + to ignore `$` delimiters and LaTeX environments, which should instead be handled by + `myst_dmath_enable` and `myst_amsmath_enable` respectively. +````` + +## Disable markdown syntax for the parser + +If you'd like to either enable or disable custom markdown syntax, use `myst_disable_syntax`. +Anything in this list will no longer be parsed by the MyST parser. + +For example, to disable the `emphasis` in-line syntax, use this configuration: + +```python +myst_disable_syntax = ["emphasis"] +``` + +emphasis syntax will now be disabled. For example, the following will be rendered +*without* any italics: + +```md +*emphasis is now disabled* +``` + +For a list of all the syntax elements you can disable, see the [markdown-it parser guide](markdown_it:using). diff --git a/docs/sphinx/roles-and-directives.md b/docs/sphinx/roles-and-directives.md new file mode 100644 index 00000000..63c86a31 --- /dev/null +++ b/docs/sphinx/roles-and-directives.md @@ -0,0 +1,23 @@ +# Special roles and directives + +This section contains information about special roles and directives that come bundled with the MyST Parser Sphinx extension. + +## Insert the date and reading time + +```{versionadded} 0.14.0 +The `sub-ref` role and word counting. +``` + +You may insert the "last updated" date and estimated reading time into your document via substitution definitions, which can be accessed *via* the `sub-ref` role. + +For example: + +```markdown +> {sub-ref}`today` | {sub-ref}`wordcount-words` words | {sub-ref}`wordcount-minutes` min read +``` + +> {sub-ref}`today` | {sub-ref}`wordcount-words` words | {sub-ref}`wordcount-minutes` min read + +`today` is replaced by either the date on which the document is parsed, with the format set by [`today_fmt`](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-today_fmt), or the `today` variable if set in the configuration file. + +The reading speed is computed using the `myst_words_per_minute` configuration (see the [Sphinx configuration options](sphinx/config-options)). diff --git a/docs/using/snippets/include-md.md b/docs/sphinx/snippets/include-md.md similarity index 100% rename from docs/using/snippets/include-md.md rename to docs/sphinx/snippets/include-md.md diff --git a/docs/using/snippets/include-rst.rst b/docs/sphinx/snippets/include-rst.rst similarity index 100% rename from docs/using/snippets/include-rst.rst rename to docs/sphinx/snippets/include-rst.rst diff --git a/docs/using/howto.md b/docs/sphinx/use.md similarity index 76% rename from docs/using/howto.md rename to docs/sphinx/use.md index 7c69fe09..cb22ca65 100644 --- a/docs/using/howto.md +++ b/docs/sphinx/use.md @@ -1,6 +1,10 @@ -# How-To Guides +# Sphinx extension usage guide -This page describes several common uses of MyST parser and how to accomplish them. +These sections describe some common scenarios and use-cases for writing MyST with Sphinx. + +:::{seealso} +For an introduction to using MyST with Sphinx, see [](intro.md). +::: (howto/include-rst)= ## Include rST files into a Markdown file @@ -28,6 +32,10 @@ To include rST, we must first "wrap" the directive in the [eval-rst directive](s .. include:: snippets/include-rst.rst ``` +## Use MyST in Jupyter Notebooks + +The [MyST-NB](https://myst-nb.readthedocs.io) tool provides a Sphinx extension for parsing **Jupyter Notebooks written with MyST Markdown**. It includes features like automatically executing notebooks during documentation builds, storing notebook cell outputs in order to insert them elsewhere in your documentation, and more. See the [MyST-NB documentation](https://myst-nb.readthedocs.io) for more information. + (howto/include-readme)= ## Include a file from outside the docs folder (like README.md) @@ -108,32 +116,6 @@ This expects docstrings to be written in reStructuredText. We hope to support Markdown in the future, see [GitHub issue #228](https://github.com/executablebooks/MyST-Parser/issues/228). ``` -## Show backticks inside raw markdown blocks - -If you'd like to show backticks inside of your markdown, you can do so by nesting them -in backticks of a greater length. Markdown will treat the outer-most backticks as the -edges of the "raw" block and everything inside will show up. For example: - -``` `` `hi` `` ``` will be rendered as: `` `hi` `` - -and - -````` -```` -``` -hi -``` -```` -````` - -will be rendered as: - -```` -``` -hi -``` -```` - (howto/autosectionlabel)= ## Automatically create targets for section headers @@ -197,3 +179,34 @@ Add to your `conf.py`: ```python suppress_warnings = ["myst.header"] ``` + + +## Sphinx-specific page front matter + +Sphinx intercepts front matter and stores them within the global environment +(as discussed [in the deflists documentation](https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html)). +There are certain front-matter keys (or their translations) that are also recognised specifically by docutils and parsed to inline Markdown: + +- `author` +- `authors` +- `organization` +- `address` +- `contact` +- `version` +- `revision` +- `status` +- `date` +- `copyright` +- `dedication` +- `abstract` + +A classic use-case is to specify 'orphan' documents, that are not specified in any toctrees. +For example, inserting the following syntax at the top of a page will cause Sphinx to treat it as an orphan page: + +```md +--- +orphan: true +--- + +This is an orphan document, not specified in any toctrees. +``` diff --git a/docs/using/img/fun-fish.png b/docs/syntax/img/fun-fish.png similarity index 100% rename from docs/using/img/fun-fish.png rename to docs/syntax/img/fun-fish.png diff --git a/docs/syntax/reference.md b/docs/syntax/reference.md new file mode 100644 index 00000000..c3264a23 --- /dev/null +++ b/docs/syntax/reference.md @@ -0,0 +1,264 @@ +# MyST Syntax Reference + +This page serves as a reference for the syntax that makes of MyST Markdown. + +:::{seealso} +For more description and explanation of MyST syntax, see the [syntax guide](syntax.md). +::: + +## Block (Multi-line) Tokens + +Block tokens span multiple lines of content. They are broken down into two sections: + +- {ref}`extended-block-tokens` contains *extra* tokens that are not in CommonMark. +- {ref}`commonmark-block-tokens` contains CommonMark tokens that also work, for reference. + +In addition to these summaries of block-level syntax, see {ref}`extra-markdown-syntax`. + +:::{note} +Because MyST markdown was inspired by functionality that exists in reStructuredText, +we have shown equivalent rST syntax for many MyST markdown features below. +::: + +(extended-block-tokens)= +### Extended block tokens + +`````{list-table} +:header-rows: 1 +:widths: 10 20 20 + +* - Token + - Description + - Example +* - FrontMatter + - A YAML block at the start of the document enclosed by `---` + - ```yaml + --- + key: value + --- + ``` +* - Directives + - enclosed in 3 or more backticks followed by the directive name wrapped + in curly brackets `{}`. See {ref}`syntax/directives` for more details. + - ````md + ```{directive} + :option: value + + content + ``` + ```` +* - Math + - `$$` (default) or `\[`...`\]` characters wrapping multi-line math, or even direct [amsmath](https://ctan.org/pkg/amsmath) LaTeX equations (optional). + See {ref}`syntax/math` for more information. + - ```latex + $$ + a=1 + $$ + ``` +* - Table + - Standard markdown table style, with pipe separation. + - ```md + | a | b | + | :--- | ---: | + | c | d | + ``` +* - LineComment + - A commented line. See {ref}`syntax/comments` for more information. + - ```latex + % this is a comment + ``` +* - BlockBreak + - Define blocks of text. See {ref}`syntax/blockbreaks` for more information. + - ```md + +++ {"meta": "data"} + ``` +* - Footnote + - A definition for a referencing footnote, that is placed at the bottom of the document. + See {ref}`syntax/footnotes` for more details. + - ```md + [^ref]: Some footnote text + ``` +* - Admonitions (optional) + - An alternative approach for admonition style directives only, which has the benefit of allowing the content to be rendered in standard markdown editors. + See [admonition directives](syntax/admonitions) for more details. + - ````md + :::{note} + *content* + ::: + ```` +````` + +(commonmark-block-tokens)= +### CommonMark tokens + +`````{list-table} +:header-rows: 1 +:widths: 10 20 20 + +* - Token + - Description + - Example +* - HTMLBlock + - Any valid HTML (rendered in HTML output only) + - ```html + <p>some text</p> + ``` +* - BlockCode + - indented text (4 spaces or a tab) + - ```md + included as literal *text* + ``` +* - Heading + - Level 1-6 headings, denoted by number of `#` + - ```md + ### Heading level 3 + ``` +* - SetextHeading + - Underlined header (using multiple `=` or `-`) + - ```md + Header + ====== + ``` +* - Quote + - quoted text + - ```md + > this is a quote + ``` +* - CodeFence + - enclosed in 3 or more backticks with an optional language name + - ````md + ```python + print('this is python') + ``` + ```` +* - ThematicBreak + - Creates a horizontal line in the output + - ```md + --- + ``` +* - List + - bullet points or enumerated. + - ```md + - item + - nested item + 1. numbered item + ``` +* - LinkDefinition + - A substitution for an inline link, which can have a reference target (no spaces), and an optional title (in `"`) + - ```md + [key]: https://www.google.com "a title" + ``` +* - Paragraph + - General inline text + - ```md + any *text* + ``` +````` + +## Span (Inline) Tokens + +Span (or inline) tokens are defined on a single line of content. They are broken down into two +sections below: + +- {ref}`extended-span-tokens` contains *extra* tokens that are not in CommonMark. +- {ref}`commonmark-span-tokens` contains CommonMark tokens that also work, for reference. + +In addition to these summaries of inline syntax, see {ref}`extra-markdown-syntax`. + +(extended-span-tokens)= +### Extended inline tokens + +`````{list-table} +:header-rows: 1 +:widths: 10 20 20 + +* - Token + - Description + - Example +* - Role + - See {ref}`syntax/roles` for more + information. + - ```md + {rolename}`interpreted text` + ``` +* - Target + - Precedes element to target, e.g. header. See + {ref}`syntax/targets` for more information. + - ```md + (target)= + ``` +* - Math + - `$` (default) or `\(`...`\)` enclosed math. See + {ref}`syntax/math` for more information. + - ```latex + $a=1$ or $$a=1$$ + ``` +* - FootReference + - Reference a footnote. See {ref}`syntax/footnotes` for more details. + - ```md + [^abc] + ``` +````` + +(commonmark-span-tokens)= +### CommonMark inline tokens + +`````{list-table} +:header-rows: 1 +:widths: 10 20 20 + +* - Token + - Description + - Example +* - HTMLSpan + - Any valid HTML (rendered in HTML output only) + - ```html + <p>some text</p> + ``` +* - EscapeSequence + - Escaped symbols (to avoid them being interpreted as other syntax elements) + - ```md + \* + ``` +* - AutoLink + - Link that is shown in final output + - ```md + <http://www.google.com> + ``` +* - InlineCode + - Literal text + - ```md + `a=1` + ``` +* - LineBreak + - Soft or hard (ends with spaces or backslash) + - ```md + A hard break\ + ``` +* - Image + - Link to an image. + You can also use HTML syntax, to include image size etc, [see here](syntax/images) for details + - ```md + ![alt](src "title") + ``` +* - Link + - Reference `LinkDefinitions` + - ```md + [text](target "title") or [text][key] + ``` +* - Strong + - Bold text + - ```md + **strong** + ``` +* - Emphasis + - Italic text + - ```md + *emphasis* + ``` +* - RawText + - Any text + - ```md + any text + ``` +````` diff --git a/docs/using/syntax-optional.md b/docs/syntax/syntax-optional.md similarity index 78% rename from docs/using/syntax-optional.md rename to docs/syntax/syntax-optional.md index 5621ae8b..2249898e 100644 --- a/docs/using/syntax-optional.md +++ b/docs/syntax/syntax-optional.md @@ -19,7 +19,7 @@ substitutions: # Optional MyST Syntaxes MyST-Parser is highly configurable, utilising the inherent "plugability" of the [markdown-it-py](markdown_it:index) parser. -The following syntaxes are optional (disabled by default) and can be enabled *via* the sphinx `conf.py` (see also [](intro/config-options)). +The following syntaxes are optional (disabled by default) and can be enabled *via* the sphinx `conf.py` (see also [](sphinx/config-options)). Their goal is generally to add more *Markdown friendly* syntaxes; often enabling and rendering [markdown-it-py plugins](markdown_it:md/plugins) that extend the [CommonMark specification](https://commonmark.org/). To enable all the syntaxes explained below: @@ -74,6 +74,170 @@ text | converted (syntax/linkify)= + +(syntax/math)= +## Math shortcuts + +Math is parsed by adding to the `myst_enable_extensions` list option, in the sphinx `conf.py` [configuration file](https://www.sphinx-doc.org/en/master/usage/configuration.html) one or both of: + +- `"dollarmath"` (added by default) for parsing of dollar `$` and `$$` encapsulated math. +- `"amsmath"` (off by default) for direct parsing of [amsmath LaTeX environments](https://ctan.org/pkg/amsmath). + +These options enable their respective Markdown parser plugins, as detailed in the [markdown-it plugin guide](markdown_it:md/plugins). + +:::{important} +`myst_dmath_enable=True` and `myst_amsmath_enable=True` are deprecated, and replaced by `myst_enable_extensions = ["dollarmath", "amsmath"]` +::: + +### Dollar delimited math + +Enabling dollar math will parse the following syntax: + +- Inline math: `$...$` +- Display (block) math: `$$...$$` + +Additionally if `myst_dmath_allow_labels=True` is set (the default): + +- Display (block) math with equation label: `$$...$$ (1)` + +For example, `$x_{hey}=it+is^{math}$` renders as $x_{hey}=it+is^{math}$. +This is equivalent to writing: + +```md +{math}`x_{hey}=it+is^{math}` +``` + +:::{admonition} Escaping Dollars +:class: tip + +Math can be escaped (negated) by adding a `\` before the first symbol, e.g. `\$a$` renders as \$a\$. +Escaping can also be used inside math, e.g. `$a=\$3$` renders as $a=\$3$. + +Conversely `\\` will negate the escaping, so `\\$a$` renders as \\$a$. +::: + +Block-level math can be specified with `$$` signs that wrap the math block you'd like to parse. +For example: + +```latex +$$ + \begin{eqnarray} + y & = & ax^2 + bx + c \\ + f(x) & = & x^2 + 2xy + y^2 + \end{eqnarray} +$$ +``` + +becomes + +$$ + \begin{eqnarray} + y & = & ax^2 + bx + c \\ + f(x) & = & x^2 + 2xy + y^2 + \end{eqnarray} +$$ + +This is equivalent to the following directive: + +````md +```{math} + \begin{eqnarray} + y & = & ax^2 + bx + c \\ + f(x) & = & x^2 + 2xy + y^2 + \end{eqnarray} +``` +```` + +You can also add labels to block equations: + +```latex +$$ +e = mc^2 +$$ (eqn:best) + +This is the best equation {eq}`eqn:best` +``` + +$$ +e = mc^2 +$$ (eqn:best) + +This is the best equation {eq}`eqn:best` + +There are a few other options available to control dollar math parsing: + +`myst_dmath_allow_space=False`, will cause inline math to only be parsed if there are no initial / final spaces, e.g. `$a$` but not `$ a$` or `$a $`. + +`myst_dmath_allow_digits=False`, will cause inline math to only be parsed if there are no initial / final digits, e.g. `$a$` but not `1$a$` or `$a$2`. + +These options can both be useful if you also wish to use `$` as a unit of currency. + +```{versionadded} 0.14.0 +`myst_dmath_double_inline` option +``` + +To allow display math (i.e. `$$`) within an inline context, set `myst_dmath_double_inline = True` (`False` by default). +This allows for example: + +```latex +Hence, for $\alpha \in (0, 1)$, +$$ + \mathbb P (\alpha \bar{X} \ge \mu) \le \alpha; +$$ +i.e., $[\alpha \bar{X}, \infty)$ is a lower 1-sided $1-\alpha$ confidence bound for $\mu$. +``` + +Hence, for $\alpha \in (0, 1)$, +$$ + \mathbb P (\alpha \bar{X} \ge \mu) \le \alpha; +$$ +i.e., $[\alpha \bar{X}, \infty)$ is a lower 1-sided $1-\alpha$ confidence bound for $\mu$. + +### Math in other block elements + +Math will also work when nested in other block elements, like lists or quotes: + +```md +- A list +- $$ a = 1 $$ + +> A block quote +> $$ a = 1 $$ +``` + +- A list +- $$ a = 1 $$ + +> A block quote +> $$ a = 1 $$ + +### Direct LaTeX Math + +Want to use [amsmath](https://ctan.org/pkg/amsmath) LaTeX directly, with no dollars? +See [the extended syntax option](syntax/amsmath). + +(syntax/mathjax)= +### Mathjax and math parsing + +When building HTML using the [sphinx.ext.mathjax](https://www.sphinx-doc.org/en/master/usage/extensions/math.html#module-sphinx.ext.mathjax) extension (enabled by default), +Myst-Parser injects the `tex2jax_ignore` (MathJax v2) and `mathjax_ignore` (MathJax v3) classes in to the top-level section of each MyST document, and adds the following default MathJax configuration: + +MathJax version 2 (see [the tex2jax preprocessor](https://docs.mathjax.org/en/v2.7-latest/options/preprocessors/tex2jax.html#configure-tex2jax): + +```javascript +MathJax.Hub.Config({"tex2jax": {"processClass": "tex2jax_process|mathjax_process|math"}}) +``` + +MathJax version 3 (see [the document options](https://docs.mathjax.org/en/latest/options/document.html?highlight=ignoreHtmlClass#the-configuration-block)): + +```javascript +window.MathJax = {"options": {"processHtmlClass": "tex2jax_process|mathjax_process|math"}} +``` + +This ensurea that MathJax processes only math, identified by the `dollarmath` and `amsmath` extensions, or specified in `math` directives. + +To change this behaviour, set a custom regex, for identifying HTML classes to process, like `myst_mathjax_classes="math|myclass"`, or set `update_mathjax=False` to inhibit this override and process all HTML elements. + ## Linkify Adding `"linkify"` to `myst_enable_extensions` (in the sphinx `conf.py` [configuration file](https://www.sphinx-doc.org/en/master/usage/configuration.html)) will automatically identify "bare" web URLs and add hyperlinks: @@ -316,7 +480,7 @@ To change the slug function, set `myst_heading_slug_func` in your `conf.py` to a You can inspect the links that will be created using the command-line tool: ```console -$ myst-anchors -l 2 docs/using/syntax-optional.md +$ myst-anchors -l 2 docs/syntax/syntax-optional.md <h1 id="optional-myst-syntaxes"></h1> <h2 id="admonition-directives"></h2> <h2 id="auto-generated-header-anchors"></h2> diff --git a/docs/using/syntax.md b/docs/syntax/syntax.md similarity index 57% rename from docs/using/syntax.md rename to docs/syntax/syntax.md index 34c0e791..0f25c14f 100644 --- a/docs/using/syntax.md +++ b/docs/syntax/syntax.md @@ -17,7 +17,8 @@ Below is a summary of the syntax 'tokens' parsed, and further details of a few major extensions from the CommonMark flavor of markdown. :::{seealso} -For an introduction to writing Directives and Roles with MyST markdown, see {ref}`intro/writing`. +- For an introduction to writing Directives and Roles with MyST markdown, see {ref}`intro/writing`. +- Check out the [MyST-Markdown VS Code extension](https://marketplace.visualstudio.com/items?itemName=ExecutableBookProject.myst-highlight), for MyST extended syntax highlighting. ::: MyST builds on the tokens defined by markdown-it, to extend the syntax @@ -25,263 +26,6 @@ described in the [CommonMark Spec](https://spec.commonmark.org/0.29/), which the % TODO link to markdown-it documentation -## Block Tokens Summary - -Block tokens span multiple lines of content. They are broken down into two sections: - -- {ref}`extended-block-tokens` contains *extra* tokens that are not in CommonMark. -- {ref}`commonmark-block-tokens` contains CommonMark tokens that also work, for reference. - -In addition to these summaries of block-level syntax, see {ref}`extra-markdown-syntax`. - -:::{note} -Because MyST markdown was inspired by functionality that exists in reStructuredText, -we have shown equivalent rST syntax for many MyST markdown features below. -::: - -(extended-block-tokens)= -### Extended block tokens - -`````{list-table} -:header-rows: 1 -:widths: 10 20 20 - -* - Token - - Description - - Example -* - FrontMatter - - A YAML block at the start of the document enclosed by `---` - - ```yaml - --- - key: value - --- - ``` -* - Directives - - enclosed in 3 or more backticks followed by the directive name wrapped - in curly brackets `{}`. See {ref}`syntax/directives` for more details. - - ````md - ```{directive} - :option: value - - content - ``` - ```` -* - Math - - `$$` (default) or `\[`...`\]` characters wrapping multi-line math, or even direct [amsmath](https://ctan.org/pkg/amsmath) LaTeX equations (optional). - See {ref}`syntax/math` for more information. - - ```latex - $$ - a=1 - $$ - ``` -* - Table - - Standard markdown table style, with pipe separation. - - ```md - | a | b | - | :--- | ---: | - | c | d | - ``` -* - LineComment - - A commented line. See {ref}`syntax/comments` for more information. - - ```latex - % this is a comment - ``` -* - BlockBreak - - Define blocks of text. See {ref}`syntax/blockbreaks` for more information. - - ```md - +++ {"meta": "data"} - ``` -* - Footnote - - A definition for a referencing footnote, that is placed at the bottom of the document. - See {ref}`syntax/footnotes` for more details. - - ```md - [^ref]: Some footnote text - ``` -* - Admonitions (optional) - - An alternative approach for admonition style directives only, which has the benefit of allowing the content to be rendered in standard markdown editors. - See [admonition directives](syntax/admonitions) for more details. - - ````md - :::{note} - *content* - ::: - ```` -````` - -(commonmark-block-tokens)= -### CommonMark tokens - -`````{list-table} -:header-rows: 1 -:widths: 10 20 20 - -* - Token - - Description - - Example -* - HTMLBlock - - Any valid HTML (rendered in HTML output only) - - ```html - <p>some text</p> - ``` -* - BlockCode - - indented text (4 spaces or a tab) - - ```md - included as literal *text* - ``` -* - Heading - - Level 1-6 headings, denoted by number of `#` - - ```md - ### Heading level 3 - ``` -* - SetextHeading - - Underlined header (using multiple `=` or `-`) - - ```md - Header - ====== - ``` -* - Quote - - quoted text - - ```md - > this is a quote - ``` -* - CodeFence - - enclosed in 3 or more backticks with an optional language name - - ````md - ```python - print('this is python') - ``` - ```` -* - ThematicBreak - - Creates a horizontal line in the output - - ```md - --- - ``` -* - List - - bullet points or enumerated. - - ```md - - item - - nested item - 1. numbered item - ``` -* - LinkDefinition - - A substitution for an inline link, which can have a reference target (no spaces), and an optional title (in `"`) - - ```md - [key]: https://www.google.com "a title" - ``` -* - Paragraph - - General inline text - - ```md - any *text* - ``` -````` - -## Span (Inline) Tokens Summary - -Span (or inline) tokens are defined on a single line of content. They are broken down into two -sections below: - -- {ref}`extended-span-tokens` contains *extra* tokens that are not in CommonMark. -- {ref}`commonmark-span-tokens` contains CommonMark tokens that also work, for reference. - -In addition to these summaries of inline syntax, see {ref}`extra-markdown-syntax`. - -(extended-span-tokens)= -### Extended inline tokens - -`````{list-table} -:header-rows: 1 -:widths: 10 20 20 - -* - Token - - Description - - Example -* - Role - - See {ref}`syntax/roles` for more - information. - - ```md - {rolename}`interpreted text` - ``` -* - Target - - Precedes element to target, e.g. header. See - {ref}`syntax/targets` for more information. - - ```md - (target)= - ``` -* - Math - - `$` (default) or `\(`...`\)` enclosed math. See - {ref}`syntax/math` for more information. - - ```latex - $a=1$ or $$a=1$$ - ``` -* - FootReference - - Reference a footnote. See {ref}`syntax/footnotes` for more details. - - ```md - [^abc] - ``` -````` - -(commonmark-span-tokens)= -### CommonMark inline tokens - -`````{list-table} -:header-rows: 1 -:widths: 10 20 20 - -* - Token - - Description - - Example -* - HTMLSpan - - Any valid HTML (rendered in HTML output only) - - ```html - <p>some text</p> - ``` -* - EscapeSequence - - Escaped symbols (to avoid them being interpreted as other syntax elements) - - ```md - \* - ``` -* - AutoLink - - Link that is shown in final output - - ```md - <http://www.google.com> - ``` -* - InlineCode - - Literal text - - ```md - `a=1` - ``` -* - LineBreak - - Soft or hard (ends with spaces or backslash) - - ```md - A hard break\ - ``` -* - Image - - Link to an image. - You can also use HTML syntax, to include image size etc, [see here](syntax/images) for details - - ```md - ![alt](src "title") - ``` -* - Link - - Reference `LinkDefinitions` - - ```md - [text](target "title") or [text][key] - ``` -* - Strong - - Bold text - - ```md - **strong** - ``` -* - Emphasis - - Italic text - - ```md - *emphasis* - ``` -* - RawText - - Any text - - ```md - any text - ``` -````` - (syntax/directives)= ## Directives - a block-level extension point @@ -540,7 +284,7 @@ This text is **standard** _Markdown_ ## Roles - an in-line extension point Roles are similar to directives - they allow you to define arbitrary new -functionality in Sphinx, but they are used *in-line*. To define an in-line +functionality, but they are used *in-line*. To define an in-line role, use the following form: ````{list-table} @@ -608,27 +352,6 @@ How roles parse this content depends on the author that created the role. (syntax/roles/special)= -### Special roles - -```{versionadded} 0.14.0 -The `sub-ref` role and word counting. -``` - -For all MyST documents, the date and word-count are made available by substitution definitions, -which can be accessed *via* the `sub-ref` role. - -For example: - -```markdown -> {sub-ref}`today` | {sub-ref}`wordcount-words` words | {sub-ref}`wordcount-minutes` min read -``` - -> {sub-ref}`today` | {sub-ref}`wordcount-words` words | {sub-ref}`wordcount-minutes` min read - -`today` is replaced by either the date on which the document is parsed, with the format set by [`today_fmt`](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-today_fmt), or the `today` variable if set in the configuration file. - -The reading speed is computed using the `myst_words_per_minute` configuration (see the [Sphinx configuration options](intro/config-options)). - (extra-markdown-syntax)= ## Extra markdown syntax @@ -665,190 +388,13 @@ header-rows: 1 - `.. _mytarget:` ```` -(syntax/math)= - -## Math shortcuts - -Math is parsed by adding to the `myst_enable_extensions` list option, in the sphinx `conf.py` [configuration file](https://www.sphinx-doc.org/en/master/usage/configuration.html) one or both of: - -- `"dollarmath"` (added by default) for parsing of dollar `$` and `$$` encapsulated math. -- `"amsmath"` (off by default) for direct parsing of [amsmath LaTeX environments](https://ctan.org/pkg/amsmath). - -These options enable their respective Markdown parser plugins, as detailed in the [markdown-it plugin guide](markdown_it:md/plugins). - -:::{important} -`myst_dmath_enable=True` and `myst_amsmath_enable=True` are deprecated, and replaced by `myst_enable_extensions = ["dollarmath", "amsmath"]` -::: - -### Dollar delimited math - -Enabling dollar math will parse the following syntax: - -- Inline math: `$...$` -- Display (block) math: `$$...$$` - -Additionally if `myst_dmath_allow_labels=True` is set (the default): - -- Display (block) math with equation label: `$$...$$ (1)` - -For example, `$x_{hey}=it+is^{math}$` renders as $x_{hey}=it+is^{math}$. -This is equivalent to writing: - -```md -{math}`x_{hey}=it+is^{math}` -``` - -:::{admonition} Escaping Dollars -:class: tip - -Math can be escaped (negated) by adding a `\` before the first symbol, e.g. `\$a$` renders as \$a\$. -Escaping can also be used inside math, e.g. `$a=\$3$` renders as $a=\$3$. - -Conversely `\\` will negate the escaping, so `\\$a$` renders as \\$a$. -::: - -Block-level math can be specified with `$$` signs that wrap the math block you'd like to parse. -For example: - -```latex -$$ - \begin{eqnarray} - y & = & ax^2 + bx + c \\ - f(x) & = & x^2 + 2xy + y^2 - \end{eqnarray} -$$ -``` - -becomes - -$$ - \begin{eqnarray} - y & = & ax^2 + bx + c \\ - f(x) & = & x^2 + 2xy + y^2 - \end{eqnarray} -$$ - -This is equivalent to the following directive: - -````md -```{math} - \begin{eqnarray} - y & = & ax^2 + bx + c \\ - f(x) & = & x^2 + 2xy + y^2 - \end{eqnarray} -``` -```` - -You can also add labels to block equations: - -```latex -$$ -e = mc^2 -$$ (eqn:best) - -This is the best equation {eq}`eqn:best` -``` - -$$ -e = mc^2 -$$ (eqn:best) - -This is the best equation {eq}`eqn:best` - -There are a few other options available to control dollar math parsing: - -`myst_dmath_allow_space=False`, will cause inline math to only be parsed if there are no initial / final spaces, e.g. `$a$` but not `$ a$` or `$a $`. - -`myst_dmath_allow_digits=False`, will cause inline math to only be parsed if there are no initial / final digits, e.g. `$a$` but not `1$a$` or `$a$2`. - -These options can both be useful if you also wish to use `$` as a unit of currency. - -```{versionadded} 0.14.0 -`myst_dmath_double_inline` option -``` - -To allow display math (i.e. `$$`) within an inline context, set `myst_dmath_double_inline = True` (`False` by default). -This allows for example: - -```latex -Hence, for $\alpha \in (0, 1)$, -$$ - \mathbb P (\alpha \bar{X} \ge \mu) \le \alpha; -$$ -i.e., $[\alpha \bar{X}, \infty)$ is a lower 1-sided $1-\alpha$ confidence bound for $\mu$. -``` - -Hence, for $\alpha \in (0, 1)$, -$$ - \mathbb P (\alpha \bar{X} \ge \mu) \le \alpha; -$$ -i.e., $[\alpha \bar{X}, \infty)$ is a lower 1-sided $1-\alpha$ confidence bound for $\mu$. - -### Math in other block elements - -Math will also work when nested in other block elements, like lists or quotes: - -```md -- A list -- $$ a = 1 $$ - -> A block quote -> $$ a = 1 $$ -``` - -- A list -- $$ a = 1 $$ - -> A block quote -> $$ a = 1 $$ - -### Direct LaTeX Math - -Want to use [amsmath](https://ctan.org/pkg/amsmath) LaTeX directly, with no dollars? -See [the extended syntax option](syntax/amsmath). - -(syntax/mathjax)= -### Mathjax and math parsing - -When building HTML using the [sphinx.ext.mathjax](https://www.sphinx-doc.org/en/master/usage/extensions/math.html#module-sphinx.ext.mathjax) extension (enabled by default), -Myst-Parser injects the `tex2jax_ignore` (MathJax v2) and `mathjax_ignore` (MathJax v3) classes in to the top-level section of each MyST document, and adds the following default MathJax configuration: - -MathJax version 2 (see [the tex2jax preprocessor](https://docs.mathjax.org/en/v2.7-latest/options/preprocessors/tex2jax.html#configure-tex2jax): - -```javascript -MathJax.Hub.Config({"tex2jax": {"processClass": "tex2jax_process|mathjax_process|math"}}) -``` - -MathJax version 3 (see [the document options](https://docs.mathjax.org/en/latest/options/document.html?highlight=ignoreHtmlClass#the-configuration-block)): - -```javascript -window.MathJax = {"options": {"processHtmlClass": "tex2jax_process|mathjax_process|math"}} -``` - -This ensurea that MathJax processes only math, identified by the `dollarmath` and `amsmath` extensions, or specified in `math` directives. - -To change this behaviour, set a custom regex, for identifying HTML classes to process, like `myst_mathjax_classes="math|myclass"`, or set `update_mathjax=False` to inhibit this override and process all HTML elements. - (syntax/frontmatter)= ## Front Matter This is a YAML block at the start of the document, as used for example in [jekyll](https://jekyllrb.com/docs/front-matter/). -Sphinx intercepts these data and stores them within the global environment -(as discussed [here](https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html)). -The following keys, or their translation (dependent on the set language), are also recognised specifically by docutils and parsed to inline Markdown: author, authors, organization, address, contact, version, revision, status, date, copyright, dedication, abstract. -A classic use-case is to specify 'orphan' documents, that are not specified in any toctrees. -For example, inserting the following syntax at the top of a page will cause Sphinx to treat it as an orphan page: - -```md ---- -orphan: true ---- - -This is an orphan document, not specified in any toctrees. -``` :::{seealso} Top-matter is also used for the [substitution syntax extension](syntax/substitutions), @@ -1020,10 +566,10 @@ is equivalent to using the [any inline role](https://www.sphinx-doc.org/en/maste {any}`my text <header_target>` ``` -but can also accept "nested" syntax (like bold text) and will recognise document paths that include extensions (e.g. `using/syntax` or `using/syntax.md`) +but can also accept "nested" syntax (like bold text) and will recognise document paths that include extensions (e.g. `syntax/syntax` or `syntax/syntax.md`) Using the same example, see this ref: [](syntax/targets), here is a reference back to the top of -this page: [my text with **nested** $\alpha$ syntax](example_syntax), and here is a reference to another page (`[](intro.md)`): [](intro.md). +this page: [my text with **nested** $\alpha$ syntax](example_syntax), and here is a reference to another page (`[](../sphinx/intro.md)`): [](../sphinx/intro.md). ```{note} If you wish to have the target's title inserted into your text, you can @@ -1123,3 +669,33 @@ This is because, in the current implementation, they may not be available to ref By default, a transition line (with a `footnotes` class) will be placed before any footnotes. This can be turned off by adding `myst_footnote_transition = False` to the config file. + + +## Code blocks + + +### Show backticks inside raw markdown blocks + +If you'd like to show backticks inside of your markdown, you can do so by nesting them +in backticks of a greater length. Markdown will treat the outer-most backticks as the +edges of the "raw" block and everything inside will show up. For example: + +``` `` `hi` `` ``` will be rendered as: `` `hi` `` + +and + +````` +```` +``` +hi +``` +```` +````` + +will be rendered as: + +```` +``` +hi +``` +```` diff --git a/docs/using/faq.md b/docs/using/faq.md deleted file mode 100644 index 3966c29d..00000000 --- a/docs/using/faq.md +++ /dev/null @@ -1,64 +0,0 @@ -# Frequently Asked Questions - -This page covers some frequently-asked questions about the MyST markdown language, -as well as the MyST parser for Sphinx. If you're just getting started, see {doc}`intro`. - -## What is the relationship between MyST, reStructuredText, and Sphinx? - -MyST markdown provides a markdown equivalent of the reStructuredText syntax, -meaning that you can do anything in MyST that you can do with reStructuredText. - -Sphinx has its own internal document model, and by default it knows how to convert -reStructuredText into that document model (with an rST parser). The MyST markdown language -defines markdown syntax that has all of the functionality that Sphinx provides (most -notable, directives, roles, and labels). The MyST parser is a Sphinx extension that can -parse this flavor of markdown into Sphinx's internal document model. Once a document -has been parsed into Sphinx, it behaves the same way regardless of whether it has -been written in rST or MyST markdown. - -``` -myst markdown (.md) ------> myst parser ---+ - | - +-->Sphinx document (docutils) - | -reStructuredText (.rst) --> rst parser ----+ -``` - -For example, here's how you'd write a `toctree` directive in MyST markdown: - -```` -```{toctree} -My page name <page1> -page2 -``` -```` - -and here's the same in rST: - -``` -.. toctree:: - - My page name <page1> - page2 -``` - -They will both behave the same in Sphinx. - -## What markup language should I use inside directives? - -If you need to parse content *inside* of another block of content (for example, the -content inside a **note directive**), note that the MyST parser will be used for this -nested parsing as well. - -## Why doesn't my role/directive recognize markdown link syntax? - -There are some roles/directives that _hard-code_ syntax into -their behavior. For example, many roles allow you to supply titles for links like so: -`` {role}`My title <myref>` ``. While this looks like reStructuredText, the role may -be explicitly expecting the `My title <myref>` structure, and so MyST will behave the same way. - -## How fast is the `myst-parser`? - -MyST-Parser uses the fastest, __*CommonMark compliant*__, parser written in Python! - -See the [markdown-it-py performance documentation](markdown_it:md/performance). diff --git a/docs/using/intro.md b/docs/using/intro.md deleted file mode 100644 index a6ba732c..00000000 --- a/docs/using/intro.md +++ /dev/null @@ -1,328 +0,0 @@ -(intro/get-started)= -# Getting Started - -This page describes how to get started with the MyST parser, with a focus on enabling -it in the Sphinx documentation engine. - -## Installation - -[![PyPI][pypi-badge]][pypi-link] -[![Conda][conda-badge]][conda-link] - -Installing the MyST parser provides access to two tools: - -* A MyST-to-docutils parser and renderer. -* A Sphinx parser that utilizes the above tool in building your documentation. - -To install the MyST parser, run the following in a -[Conda environment](https://docs.conda.io) (recommended): - -```bash -conda install -c conda-forge myst-parser -``` - -or - -```bash -pip install myst-parser -``` - -[pypi-badge]: https://img.shields.io/pypi/v/myst-parser.svg -[pypi-link]: https://pypi.org/project/myst-parser -[conda-badge]: https://anaconda.org/conda-forge/myst-parser/badges/version.svg -[conda-link]: https://anaconda.org/conda-forge/myst-parser - -(parse-with-sphinx)= -## Enable MyST in Sphinx - -Sphinx is a documentation generator for building a website or book from multiple source documents and assets. To get started with Sphinx, see their [Quickstart Guide](https://www.sphinx-doc.org/en/master/usage/quickstart.html). - -To use the MyST parser in Sphinx, simply add: `extensions = ["myst_parser"]` to your `conf.py` and all documents with the `.md` extension will be parsed as MyST. - -Naturally this site is generated with Sphinx and MyST! - -:::{admonition} You can use both MyST and reStructuredText -:class: tip - -Activating the MyST parser will simply *enable* parsing markdown files with MyST, and the rST parser that ships with Sphinx by default will still work the same way. -You can have combinations of both markdown and rST files in your documentation, and Sphinx will choose the right parser based on each file's extension. -Sphinx features like cross-references will work just fine between the pages. - -You can even inject raw rST into Markdown files! (see [this explanation](syntax/directives/parsing)) -::: - -:::{admonition} Want to add Jupyter Notebooks to your documentation? -:class: seealso - -See also [MyST-NB](https://myst-nb.readthedocs.io), our complimentary parser and execution engine, -for ipynb and text-based notebooks. -::: - -## How does MyST parser relate to Sphinx? - -The Sphinx documentation engine supports a number of different input types. By default, -Sphinx reads **reStructuredText** (`.rst`) files. Sphinx uses a **parser** to parse input files -into its own internal document model (which is provided by a core Python project, -[docutils](https://docutils.sourceforge.io/)). - -Developers can *extend Sphinx* to support other kinds of input files. Any content file -can be read into the Sphinx document structure, provided that somebody writes a -**parser** for that file. Once a content file has been parsed into Sphinx, it behaves -nearly the same way as any other content file, regardless of the language in which it -was written. - -The MyST-parser is a Sphinx parser for the MyST markdown language. When you use it, -Sphinx will know how to parse content files that contain MyST markdown (by default, -Sphinx will assume any files ending in `.md` are written in MyST markdown). - -:::{note} -Sphinx will still be able to parse files written in `.rst`. Activating this parser -simply adds another parser, and Sphinx will still be able to use its default parser -for `.rst` files. -::: - -(intro/writing)= -## Writing MyST in Sphinx - -Once you've enabled the `myst-parser` in Sphinx, it will be able to parse your MyST -markdown documents. This means that you can use the `.md` extension for your pages, -and write MyST markdown in these pages. - -:::{note} -MyST markdown is a mixture of two flavors of markdown: - -It supports all the syntax of **[CommonMark Markdown](https://commonmark.org/)** at its -base. This is a community standard flavor of markdown used across many projects. - -In addition, it includes **several extensions to CommonMark** -(often described as [MyST Markdown syntax](syntax)). These add extra syntax features -designed to work with the Sphinx ecosystem (and inspired by reStructuredText) -::: - -:::{tip} -If you want to parse your files as only **strict** CommonMark (no extensions), then you can set the `conf.py` option `myst_commonmark_only=True`. -::: - -The following sections cover a few core syntax patterns in MyST markdown, you can -find a more exhaustive list in {doc}`syntax`. - -### Block-level directives with MyST markdown - -The most important functionality available with MyST markdown is writing **directives**. -Directives are kind-of like functions that are designed for writing content. Sphinx -and reStructuredText use directives extensively. Here's how a directive looks in -MyST markdown: - -````{margin} Alternative options syntax -If you've got a lot of options for your directive, or have a value that is really -long (e.g., that spans multiple lines), then you can also wrap your options in -`---` lines and write them as YAML. For example: - -```yaml ---- -key1: val1 -key2: | - val line 1 - val line 2 ---- -``` -```` - -```` -```{directivename} <directive arguments> -:optionname: <valuename> - -<directive content> -``` -```` - -````{admonition} MyST vs. rST -:class: warning -For those who are familiar with reStructuredText, here is the equivalent in rST: - -```rst -.. directivename: <directive-arguments> - :optionname: <valuename> - - <directive content> -``` - -Note that almost all documentation in the Sphinx ecosystem is written with -reStructuredText (MyST is only a few months old). -That means you'll likely see examples that have rST structure. You can modify any rST to work with MyST. Use this page, and [the syntax page](./syntax.md) to help guide you. -```` - -As seen above, there are four main parts to consider when writing directives. - -* **the directive name** is kind of like the function name. Different names trigger - different functionality. They are wrapped in `{}` brackets. -* **directive arguments** come just after the directive name. They can be used - to trigger behavior in the directive. -* **directive options** come just after the first line of the directive. They also - control behavior of the directive. -* **directive content** is markdown that you put inside the directive. The directive - often displays the content in a special way. - -For example, here's an **`admonition`** directive: - -```` -```{admonition} Here's my title -:class: warning - -Here's my admonition content -``` -```` - -As you can see, we've used each of the four pieces described above to configure this -directive. Here's how it looks when rendered: - -```{admonition} Here's my title -:class: warning - -Here's my admonition content -``` - -For more information about using directives with MyST, see {ref}`syntax/directives`. - -### In-line roles with MyST Markdown - -Roles are another core Sphinx tool. They behave similarly to directives, but are given -in-line with text instead of in a separate block. They have the following form: - -```md -{rolename}`role content` -``` - -For those who are familiar with reStructuredText, here is the equivalent in rST: - -```rst -:rolename:`role content` -``` - -As you can see, roles are a bit more simple than directives, though some roles allow -for more complex syntax inside their content area. For example, the `ref` role is used -to make references to other sections of your documentation, and allows you to specify -the displayed text as well as the reference itself within the role: - -```md -{ref}`My displayed text <my-ref>` -``` - -For example, the following reference role: `` {ref}`Check out this reference <syntax/roles>` `` -will be rendered as {ref}`Check out this reference <syntax/roles>`. - -For more information about roles, see {ref}`syntax/roles`. - -:::{tip} -Check out the [MyST-Markdown VS Code extension](https://marketplace.visualstudio.com/items?itemName=ExecutableBookProject.myst-highlight), -for MyST extended syntax highlighting. -::: - -(intro/config-options)= -# Sphinx configuration options - -You can control the behaviour of the MyST parser in Sphinx by modifying your `conf.py` file. -To do so, use the keywords beginning `myst_`. - -`````{list-table} -:header-rows: 1 - -* - Option - - Default - - Description -* - `myst_commonmark_only` - - `False` - - If `True` convert text as strict CommonMark (all options below are then ignored). Note that strict CommonMark is unable to parse any directives, including the `toctree` directive, thus limiting MyST parser to single-page documentations. Use in conjunction with [sphinx-external-toc](https://github.com/executablebooks/sphinx-external-toc) Sphinx extension to counter this limitation. -* - `myst_disable_syntax` - - () - - List of markdown syntax elements to disable, see the [markdown-it parser guide](markdown_it:using). -* - `myst_enable_extensions` - - `["dollarmath"]` - - Enable Markdown extensions, [see here](syntax-optional) for details. -* - `myst_url_schemes` - - `("http", "https", "mailto", "ftp")` - - [URI schemes](https://en.wikipedia.org/wiki/List_of_URI_schemes) that will be recognised as external URLs in `[](scheme:loc)` syntax, or set `None` to recognise all. - Other links will be resolved as internal cross-references. -* - `myst_heading_anchors` - - `None` - - Enable auto-generated heading anchors, up to a maximum level, [see here](syntax/header-anchors) for details. -* - `myst_heading_slug_func` - - `None` - - Use the specified function to auto-generate heading anchors, [see here](syntax/header-anchors) for details. -* - `myst_substitutions` - - `{}` - - A mapping of keys to substitutions, used globally for all MyST documents when the "substitution" extension is enabled. -* - `myst_html_meta` - - `{}` - - A mapping of keys to HTML metadata, used globally for all MyST documents. See [](syntax/html_meta). -* - `myst_footnote_transition` - - `True` - - Place a transition before any footnotes. -* - `myst_words_per_minute` - - `200` - - Reading speed used to calculate `` {sub-ref}`wordcount-minutes` `` -````` - -List of extensions: - -- "amsmath": enable direct parsing of [amsmath](https://ctan.org/pkg/amsmath) LaTeX equations -- "colon_fence": Enable code fences using `:::` delimiters, [see here](syntax/colon_fence) for details -- "deflist" -- "dollarmath": Enable parsing of dollar `$` and `$$` encapsulated math -- "html_admonition": Convert `<div class="admonition">` elements to sphinx admonition nodes, see the [HTML admonition syntax](syntax/html-admonition) for details -- "html_image": Convert HTML `<img>` elements to sphinx image nodes, see the [image syntax](syntax/images) for details -- "linkify": automatically identify "bare" web URLs and add hyperlinks -- "replacements": automatically convert some common typographic texts -- "smartquotes": automatically convert standard quotations to their opening/closing variants -- "substitution": substitute keys, see the [substitutions syntax](syntax/substitutions) for details -- "tasklist": add check-boxes to the start of list items, see the [tasklist syntax](syntax/tasklists) for details - -Math specific, when `"dollarmath"` activated, see the [Math syntax](syntax/math) for more details: - -`````{list-table} -:header-rows: 1 - -* - Option - - Default - - Description -* - `myst_dmath_double_inline` - - `False` - - Allow display math (i.e. `$$`) within an inline context -* - `myst_dmath_allow_labels` - - `True` - - Parse `$$...$$ (label)` syntax -* - `myst_dmath_allow_space` - - `True` - - If False then inline math will only be parsed if there are no initial/final spaces, - e.g. `$a$` but not `$ a$` or `$a $` -* - `myst_dmath_allow_digits` - - `True` - - If False then inline math will only be parsed if there are no initial/final digits, - e.g. `$a$` but not `1$a$` or `$a$2` (this is useful for using `$` as currency) -* - `myst_update_mathjax` - - `True` - - If using [sphinx.ext.mathjax](https://www.sphinx-doc.org/en/master/usage/extensions/math.html#module-sphinx.ext.mathjax) (the default) then `mathjax_config` will be updated to only process specific HTML classes. -* - `myst_mathjax_classes` - - `"tex2jax_process|mathjax_process|math"` - - A regex for the HTML classes that MathJax will process -````` - -## Disable markdown syntax for the parser - -If you'd like to either enable or disable custom markdown syntax, use `myst_disable_syntax`. -Anything in this list will no longer be parsed by the MyST parser. - -For example, to disable the `emphasis` in-line syntax, use this configuration: - -```python -myst_disable_syntax = ["emphasis"] -``` - -emphasis syntax will now be disabled. For example, the following will be rendered -*without* any italics: - -```md -*emphasis is now disabled* -``` - -For a list of all the syntax elements you can disable, see the [markdown-it parser guide](markdown_it:using). diff --git a/example-include.md b/example-include.md index dd535173..4d23ee51 100644 --- a/example-include.md +++ b/example-include.md @@ -1,2 +1,2 @@ -[Used in how-to](docs/using/howto.md) +[Used in how-to](docs/sphinx/use.md) ![alt](docs/_static/logo-wide.svg) diff --git a/myst_parser/mocking.py b/myst_parser/mocking.py index 0ae0bb27..e4889661 100644 --- a/myst_parser/mocking.py +++ b/myst_parser/mocking.py @@ -267,7 +267,7 @@ def __getattr__(self, name: str): msg = ( f"{cls} has not yet implemented attribute '{name}'. " "You can parse RST directly via the `{eval-rst}` directive: " - "https://myst-parser.readthedocs.io/en/latest/using/syntax.html#how-directives-parse-content" # noqa: E501 + "https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#how-directives-parse-content" # noqa: E501 ) else: # The requested `name` is not a docutils Body element diff --git a/setup.cfg b/setup.cfg index 3aabb51c..d95029f4 100644 --- a/setup.cfg +++ b/setup.cfg @@ -63,6 +63,8 @@ rtd = sphinx-book-theme~=0.1.0 sphinx-panels~=0.5.2 sphinxcontrib-bibtex~=2.1 + sphinxext-rediraffe~=0.2 + sphinxcontrib.mermaid~=0.6.3 # left in for back-compatability sphinx = testing = From f0b16d59ae2f077523fca81ec4e15ed7fc1031cc Mon Sep 17 00:00:00 2001 From: Chris Holdgraf <choldgraf@berkeley.edu> Date: Fri, 21 May 2021 01:28:55 -0700 Subject: [PATCH 2/4] Addressing comments --- docs/explain/index.md | 3 +- docs/index.md | 64 +++++++++++++++++++++++++++---------------- 2 files changed, 42 insertions(+), 25 deletions(-) diff --git a/docs/explain/index.md b/docs/explain/index.md index 2528973a..1ac48703 100644 --- a/docs/explain/index.md +++ b/docs/explain/index.md @@ -1,8 +1,7 @@ -# Discussion and explanation +# Background and explanation These sections discuss high-level questions about the MyST ecosystem, and explain a few decisions made in the project. - ## Why did we create MyST markdown? While markdown is ubiquitous, it is not powerful enough for writing modern, diff --git a/docs/index.md b/docs/index.md index f35ef928..f3447869 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,27 +5,42 @@ **MyST is a rich and extensible flavor of Markdown meant for technical documentation and publishing**. -MyST is a flavor of markdown that is designed for simplicity, flexibility, and extensibility. -Here are a few key features of MyST Markdown: - -{fa}`check,text-success mr-1` It is [a superset of CommonMark markdown][commonmark]. -: Any CommonMark document is also MyST-compliant. - -{fa}`check,text-success mr-1` It [extends CommonMark with rich writing syntax](extended-block-tokens). -: These make it easier to write MyST and extend its functionality to technical documentation (for example, line commenting and footnotes). - -{fa}`check,text-success mr-1` It provides [extension points with directives and roles](syntax/directives). -: Any MyST parser can define new functionality via a standard syntax extension point (this is kind of like defining functions with markdown). - -This MyST Parser serves as a reference implementation for MyST Markdown, and comes bundled with several tools that help you use MyST in your work. -It comes bundled with a few tools to help you use MyST with Python: - -[A Python parser of MyST markdown](api/index) -: This can be extended to add new functionality and outputs for MyST, and is built on top of [`markdown-it-py`](https://markdown-it-py.readthedocs.io/). - -[A markdown parser for Sphinx](sphinx/index) -: You can write your entire {doc}`Sphinx documentation <sphinx:usage/quickstart>` in Markdown, including [**roles** and **directives**](syntax/directives). - This allows you to extend MyST's functionality in Sphinx via Sphinx extensions. +MyST is a flavor of markdown that is designed for simplicity, flexibility, and extensibility. Here are a few major features + +:::{panels} +:container: +full-width text-center +:column: col-4 px-2 py-2 +:card: + +**[CommonMark compliant](commonmark-block-tokens)** āœ” +^^^ +MyST is a superset of [CommonMark markdown][commonmark]. Any CommonMark document is also MyST-compliant. +--- + +**[Extra syntax for authoring](extended-block-tokens)** āœ +^^^ +MyST extends CommonMark with [syntax meant for scholarly writing and technical documentation](extended-block-tokens). + +--- +**[Extendable syntax](syntax/directives)** šŸš€ +^^^ +MyST provides [roles](syntax/roles) and [directives](syntax/directives), allowing you to extend MyST's functionality. + +--- +**[Compatible with Sphinx](sphinx/index.md)** šŸ“„ +^^^ +MyST is inspired by Sphinx, and comes with [its own Sphinx parser](sphinx/index.md). [Write your Sphinx docs in markdown](sphinx:usage/quickstart)! + +--- +**[Hackable with Python](api/index.md)** šŸ +^^^ +This MyST parser is built on top of the [`markdown-it-py` package][markdown-it-py], an pluggable Python parser for Markdown. + +--- +**[Hackable with Javascript][markdown-it-myst]** šŸŒ +^^^ +The [Javascript parser][markdown-it-myst] builds on [markdown-it][markdown-it], and allows you to parse MyST in websites. +::: :::{seealso} For some examples of how MyST can be used, check out the [Jupyter Book project](https://jupyterbook.org), which uses MyST Markdown heavily, as well as [the Jupyter Book gallery](https://gallery.jupyterbook.org) which contains a list of books built by others, all using MyST. @@ -59,9 +74,10 @@ Topic guides cover particular tools, use-cases, and functionality in the MyST ec ```{toctree} :maxdepth: 2 :caption: Topic Guides +explain/index.md sphinx/index.md api/index.md -explain/index.md +develop/index.md ``` ## About the project @@ -72,7 +88,6 @@ These sections cover "meta" information about the MyST Markdown project. :maxdepth: 2 :caption: About the project examples/index.md -develop/index.md develop/_changelog.md GitHub repo <https://github.com/executablebooks/myst-parser> ``` @@ -96,3 +111,6 @@ The MyST markdown language and MyST parser are both supported by the open commun [conda-link]: https://anaconda.org/conda-forge/myst-parser [black-link]: https://github.com/ambv/black [github-badge]: https://img.shields.io/github/stars/executablebooks/myst-parser?label=github +[markdown-it-py]: https://markdown-it-py.readthedocs.io/ +[markdown-it-myst]: https://github.com/executablebooks/markdown-it-myst +[markdown-it]: https://markdown-it.github.io/ From 024c545887f4b5eaf8bcb855e1eab49a70338106 Mon Sep 17 00:00:00 2001 From: Chris Holdgraf <choldgraf@berkeley.edu> Date: Sun, 6 Jun 2021 14:33:16 -0700 Subject: [PATCH 3/4] More docs updating --- CHANGELOG.md | 14 ++--- docs/_static/custom.css | 11 ++++ docs/conf.py | 3 +- docs/index.md | 62 ++++++++++++------- docs/sphinx/reference.md | 2 +- docs/sphinx/use.md | 4 ++ .../{syntax-optional.md => optional.md} | 2 +- 7 files changed, 66 insertions(+), 32 deletions(-) create mode 100644 docs/_static/custom.css rename docs/syntax/{syntax-optional.md => optional.md} (99%) diff --git a/CHANGELOG.md b/CHANGELOG.md index faede6d6..a245672e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,7 +45,7 @@ In particular for users, this update alters the parsing of tables to be consiste Add "tasklist" to the `myst_enable_extensions` configuration to enable. - See [the optional syntax guide](docs/syntax/syntax-optional.md#task-lists) for further information. + See [the optional syntax guide](docs/syntax/optional.md#task-lists) for further information. - The **`sub-ref`** role has been added for use identical to ReST's `|name|` syntax. @@ -58,7 +58,7 @@ In particular for users, this update alters the parsing of tables to be consiste See [the roles syntax guide](docs/syntax/syntax.md) for further information. - The **`dmath_double_inline`** configuration option allows display math (i.e. `$$`) within an inline context. - See [the math syntax guide](docs/syntax/syntax-optional.md#math-shortcuts) for further information. + See [the math syntax guide](docs/syntax/optional.md#math-shortcuts) for further information. ### Remove v0.13 deprecations ā€¼ļø @@ -133,7 +133,7 @@ Minor fixes: This is the *content* </div> ``` -: See [the optional syntax guide](docs/syntax/syntax-optional.md) for further information. +: See [the optional syntax guide](docs/syntax/optional.md) for further information. šŸ‘Œ IMPROVE: Footnotes @@ -176,7 +176,7 @@ An inline image: {{ key }} ## 0.13.0 - 2020-12-18 This release makes some major updates to the optional syntaxes. -For full details see [Optional MyST Syntaxes](docs/syntax/syntax-optional.md). +For full details see [Optional MyST Syntaxes](docs/syntax/optional.md). ### šŸ—‘ Deprecations @@ -261,7 +261,7 @@ Iā€™m an inline image: <img src="img/fun-fish.png" height="20px"> It is enabled in your `conf.py` *via* `myst_heading_anchors = 2` (sets maximum heading level). - See [the documentation here](docs/syntax/syntax-optional.md#auto-generated-header-anchors). + See [the documentation here](docs/syntax/optional.md#auto-generated-header-anchors). šŸ› FIX: doc reference resolution for singlehtml/latex. : These reference resolutions are passed to the "missing-reference" event, and require the `node["refdoc"]` attribute to be available, which was missing for `[text](./path/to/file.md)` type references. @@ -278,7 +278,7 @@ Iā€™m an inline image: <img src="img/fun-fish.png" height="20px"> āœØ NEW: Add Markdown figure syntax : Setting `myst_figure_enable = True` in your sphinx `conf.py`, combines the above two extended syntaxes, to create a fully Markdown compliant version of the `figure` directive. - See [Markdown Figures](docs/syntax/syntax-optional.md#markdown-figures) for details. + See [Markdown Figures](docs/syntax/optional.md#markdown-figures) for details. (šŸ‘Œ formatting of caption improved in 0.12.6) @@ -296,7 +296,7 @@ Term 1 : Definition ``` -See the [Definition Lists documentation](https://myst-parser.readthedocs.io/en/latest/syntax/syntax-optional.html#definition-lists) for further details. +See the [Definition Lists documentation](https://myst-parser.readthedocs.io/en/latest/syntax/optional.html#definition-lists) for further details. šŸ‘Œ IMPROVE: mathjax_config override. : Only `mathjax_config["tex2jax"]` will now be overridden, in order to not interfere with other user configurations, such as adding TeX macros. diff --git a/docs/_static/custom.css b/docs/_static/custom.css new file mode 100644 index 00000000..118c5a83 --- /dev/null +++ b/docs/_static/custom.css @@ -0,0 +1,11 @@ +.bg-myst-one { + background-color: #52d16f3b; +} + +.bg-myst-two { + background-color: #e7dd7b73; +} + +.bg-myst-three { + background-color: #e7b07b96; +} diff --git a/docs/conf.py b/docs/conf.py index 57a156db..6f235f23 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -85,7 +85,7 @@ "using/intro.md": "sphinx/intro.md", "using/use_api.md": "api/index.md", "using/syntax.md": "syntax/syntax.md", - "using/syntax-optional.md": "syntax/syntax-optional.md", + "using/syntax-optional.md": "syntax/optional.md", "using/reference.md": "syntax/reference.md", } @@ -161,3 +161,4 @@ def run_apidoc(app): def setup(app): """Add functions to the Sphinx setup.""" # app.connect("builder-inited", run_apidoc) + app.add_css_file("custom.css") diff --git a/docs/index.md b/docs/index.md index f3447869..b4efe306 100644 --- a/docs/index.md +++ b/docs/index.md @@ -42,37 +42,59 @@ This MyST parser is built on top of the [`markdown-it-py` package][markdown-it-p The [Javascript parser][markdown-it-myst] builds on [markdown-it][markdown-it], and allows you to parse MyST in websites. ::: -:::{seealso} -For some examples of how MyST can be used, check out the [Jupyter Book project](https://jupyterbook.org), which uses MyST Markdown heavily, as well as [the Jupyter Book gallery](https://gallery.jupyterbook.org) which contains a list of books built by others, all using MyST. -::: +## Find the right documentation resources + +This documentation is organized into a few major sections. **Tutorials** are step-by-step introductory guides to MyST Markdown. **Topic Guides** cover specific areas in more depth, and are organized as discrete "how-to" sections. **Reference** sections describe the API/syntax/etc of the MyST Parser in detail. + +In addition, here are a few pointers to help you get started. + +:::{panels} +:container: full-width +:column: col-4 p-2 +--- +:header: bg-myst-one +**Get started with MyST** +^^^ +**[](sphinx/intro.md)**: a step-by-step tutorial. + +**[](syntax/syntax.md)**: discusses major MyST syntax components. + +**[The Sphinx guide](sphinx/index.md)**: how to use MyST with your Sphinx documentation. +--- +:header: bg-myst-two -## Get started +**Learn more about MyST** +^^^ +**[](syntax/optional.md)**: additional syntax you can enable for extra features. -These pages cover step-by-step instructions to get started with MyST Markdown. +**[The Python API guide](api/index.md)**: parsing and rendering MyST with Python. + +**[](explain/index.md)**: background understanding and discussions of MyST markdown. +--- +:header: bg-myst-three + +**Get inspired** +^^^ +**[Jupyter Book](https://jupyterbook.org)**: An open source project for building beautiful, publication-quality books and documents from computational material, built on top of the MyST Parser. + +**[The Jupyter Book gallery](https://gallery.jupyterbook.org)**: examples of documents built with MyST. +::: ```{toctree} -:maxdepth: 2 +:hidden: sphinx/intro.md ``` -## MyST Syntax - -These sections cover the syntax that makes up MyST Markdown, some common use-cases that it supports, as well as a few extensions that allow you to enable new features with MyST. - ```{toctree} :caption: MyST Syntax -:maxdepth: 2 +:hidden: syntax/syntax -syntax/syntax-optional +syntax/optional syntax/reference ``` -## Topic Guides - -Topic guides cover particular tools, use-cases, and functionality in the MyST ecosystem. - ```{toctree} -:maxdepth: 2 +:hidden: :caption: Topic Guides explain/index.md sphinx/index.md @@ -80,12 +102,8 @@ api/index.md develop/index.md ``` -## About the project - -These sections cover "meta" information about the MyST Markdown project. - ```{toctree} -:maxdepth: 2 +:hidden: :caption: About the project examples/index.md develop/_changelog.md diff --git a/docs/sphinx/reference.md b/docs/sphinx/reference.md index f9d0e645..1d80b99a 100644 --- a/docs/sphinx/reference.md +++ b/docs/sphinx/reference.md @@ -18,7 +18,7 @@ To do so, use the keywords beginning `myst_`. - List of markdown syntax elements to disable, see the [markdown-it parser guide](markdown_it:using). * - `myst_enable_extensions` - `["dollarmath"]` - - Enable Markdown extensions, [see here](../syntax/syntax-optional.md) for details. + - Enable Markdown extensions, [see here](../syntax/optional.md) for details. * - `myst_url_schemes` - `None` - [URI schemes](https://en.wikipedia.org/wiki/List_of_URI_schemes) that will be recognised as external URLs in `[](scheme:loc)` syntax, or set `None` to recognise all. diff --git a/docs/sphinx/use.md b/docs/sphinx/use.md index cb22ca65..3612940c 100644 --- a/docs/sphinx/use.md +++ b/docs/sphinx/use.md @@ -210,3 +210,7 @@ orphan: true This is an orphan document, not specified in any toctrees. ``` + +## Migrate pre-existing rST into MyST + +If you've already got some reStructuredText files that you'd like to convert into MyST Markdown, try the [`rst-to-myst`](https://github.com/executablebooks/rst-to-myst) tool, which allows you to convert single rST files to MyST markdown documents. diff --git a/docs/syntax/syntax-optional.md b/docs/syntax/optional.md similarity index 99% rename from docs/syntax/syntax-optional.md rename to docs/syntax/optional.md index 2249898e..f450363e 100644 --- a/docs/syntax/syntax-optional.md +++ b/docs/syntax/optional.md @@ -480,7 +480,7 @@ To change the slug function, set `myst_heading_slug_func` in your `conf.py` to a You can inspect the links that will be created using the command-line tool: ```console -$ myst-anchors -l 2 docs/syntax/syntax-optional.md +$ myst-anchors -l 2 docs/syntax/optional.md <h1 id="optional-myst-syntaxes"></h1> <h2 id="admonition-directives"></h2> <h2 id="auto-generated-header-anchors"></h2> From c5232a87f958d77db66ea06d8acf96f179e79153 Mon Sep 17 00:00:00 2001 From: Chris Holdgraf <choldgraf@berkeley.edu> Date: Sun, 13 Jun 2021 10:05:56 -0700 Subject: [PATCH 4/4] Fix changelog links --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a245672e..1980938c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,7 +12,7 @@ Instead of removing all `$` processing for the whole project, during MyST document parsing, the top-level section is now given the classes `tex2jax_ignore` and `mathjax_ignore` (turning off default MathJax processing of all HTML elements) and MathJax is then configured to process elements with the `tex2jax_process|mathjax_process|math` classes. -See [the math syntax guide](docs/using/syntax.md#math-shortcuts) for further information. +See [the math syntax guide](docs/syntax/optional.md#math-shortcuts) for further information. ### Set URL scheme defaults ā€¼ļø @@ -23,7 +23,7 @@ Set `myst_url_schemes = None`, to revert to the previous default. ### Added `myst_heading_slug_func` option šŸ‘Œ -Use this option to specify a custom function to auto-generate heading anchors (see [Auto-generated header anchors](docs/using/syntax-optional.md#auto-generated-header-anchors)). +Use this option to specify a custom function to auto-generate heading anchors (see [Auto-generated header anchors](docs/syntax/optional.md#auto-generated-header-anchors)). Thanks to [@jpmckinney](https://github.com/jpmckinney)!