-
Notifications
You must be signed in to change notification settings - Fork 1
[snippet_var_resolver] Add plugin to resolve variables inside of snippets #36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
3d0a9e9
add variables in snippets resolver plugin
eshaben 2b5ada6
bump plugin version
eshaben 2ee0772
handle malformed and non-mapping YAML
eshaben e4d1526
correct comment
eshaben ea7ca69
remove unused imports
eshaben df50b85
remove unused function
eshaben 80153a3
test last-file-wins precedence for overlapping include_yaml keys
eshaben File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # Snippet Var Resolver Plugin | ||
|
|
||
| The Snippet Var Resolver plugin resolves `{{ variable }}` placeholders that survive into the rendered HTML of a page after pymdownx.snippets has injected snippet content. | ||
|
|
||
| The problem it solves: mkdocs-macros resolves Jinja2 variables during the `on_page_markdown` event, but pymdownx.snippets injects snippet file content later during markdown-to-HTML conversion — after macros has already run. Any `{{ variable }}` references inside snippet files therefore appear unresolved in the final HTML. | ||
|
|
||
| This plugin runs on the `on_page_content` event (after snippets have been injected) and replaces any remaining `{{ variable }}` patterns using the `include_yaml` files listed in the macros plugin config. | ||
|
|
||
| ## 🔹 Usage | ||
|
|
||
| Add the plugin **after `macros`** in your `mkdocs.yml`: | ||
|
|
||
| ```yaml | ||
| plugins: | ||
| - macros: | ||
| include_yaml: | ||
| - polkadot-docs/variables.yml | ||
| - snippet_var_resolver | ||
| ``` | ||
|
|
||
| No additional configuration is required. The plugin reads its variable sources directly from the macros plugin config. | ||
|
|
||
| ## 🔹 Configuration | ||
|
|
||
| This plugin has no configuration options. It self-configures from the surrounding plugin setup. | ||
|
|
||
| ## 🔹 Notes | ||
|
|
||
| - Variable lookup supports dotted paths — e.g. `{{ dependencies.zombienet.version }}` resolves into nested YAML structures. | ||
| - Unknown placeholders (keys not found in any variable source) are left untouched, so unrelated Jinja2 syntax in the HTML is not affected. | ||
| - If multiple `include_yaml` files define the same top-level key, the last file wins (same behaviour as mkdocs-macros). | ||
| - The plugin runs on the `on_page_content` event, so it processes the rendered HTML of each page rather than the Markdown source. |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| import logging | ||
| import re | ||
|
|
||
| import yaml | ||
| from mkdocs.config.defaults import MkDocsConfig | ||
| from mkdocs.plugins import BasePlugin | ||
| from mkdocs.structure.files import Files | ||
| from mkdocs.structure.pages import Page | ||
|
|
||
| log = logging.getLogger("mkdocs.plugins.snippet_var_resolver") | ||
|
|
||
| PLACEHOLDER_PATTERN = re.compile(r"{{\s*([A-Za-z0-9_.-]+)\s*}}") | ||
|
|
||
|
|
||
| def get_value_from_path(data, path): | ||
| """Dotted key lookup into a nested dict (e.g. 'dependencies.foo.version').""" | ||
| value = data | ||
| for key in path.split("."): | ||
| if not isinstance(value, dict) or key not in value: | ||
| return None | ||
| value = value[key] | ||
| return value | ||
|
|
||
|
|
||
| class SnippetVarResolverPlugin(BasePlugin): | ||
| config_scheme = () | ||
|
|
||
| def on_config(self, config: MkDocsConfig) -> MkDocsConfig: | ||
| self._variables = {} | ||
|
|
||
| # Load variables from macros plugin's include_yaml files | ||
| macros = config["plugins"].get("macros") | ||
| if macros: | ||
| for yaml_path in macros.config.get("include_yaml", []): | ||
| self._load_yaml_file(yaml_path, config) | ||
| else: | ||
| log.debug("[snippet_var_resolver] macros plugin not found, skipping include_yaml") | ||
|
|
||
| if self._variables: | ||
| log.debug(f"[snippet_var_resolver] loaded {len(self._variables)} top-level variable keys") | ||
| else: | ||
| log.warning("[snippet_var_resolver] no variables loaded — {{ }} patterns will not be resolved") | ||
|
|
||
| return config | ||
|
|
||
| def _load_yaml_file(self, yaml_path: str, config: MkDocsConfig) -> None: | ||
| from pathlib import Path | ||
|
|
||
| # Resolve relative to project root, then docs_dir, then as an absolute path | ||
| candidates = [ | ||
| Path(config["docs_dir"]).parent / yaml_path, | ||
| Path(config["docs_dir"]) / yaml_path, | ||
| Path(yaml_path), | ||
| ] | ||
| for path in candidates: | ||
| if path.exists(): | ||
| try: | ||
| with open(path, "r", encoding="utf-8") as f: | ||
| data = yaml.safe_load(f) | ||
| except yaml.YAMLError as exc: | ||
| log.warning(f"[snippet_var_resolver] unable to parse {path}: {exc}") | ||
| return | ||
| if not isinstance(data, dict): | ||
| log.warning(f"[snippet_var_resolver] expected a YAML mapping in {path}, got {type(data).__name__} — skipping") | ||
| return | ||
| self._variables.update(data) | ||
| log.debug(f"[snippet_var_resolver] loaded variables from {path}") | ||
| return | ||
|
|
||
| log.warning(f"[snippet_var_resolver] variables file not found: {yaml_path}") | ||
|
|
||
| def on_page_content( | ||
| self, html: str, page: Page, config: MkDocsConfig, files: Files | ||
| ) -> str: | ||
| if not self._variables: | ||
| return html | ||
|
|
||
| def replacer(match): | ||
| key_path = match.group(1) | ||
| value = get_value_from_path(self._variables, key_path) | ||
| if value is not None: | ||
| return str(value) | ||
| return match.group(0) | ||
|
|
||
| resolved = PLACEHOLDER_PATTERN.sub(replacer, html) | ||
|
|
||
| if resolved != html: | ||
| log.debug(f"[snippet_var_resolver] resolved variables in {page.file.src_path}") | ||
|
|
||
| return resolved |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
195 changes: 195 additions & 0 deletions
195
tests/snippet_var_resolver/test_snippet_var_resolver.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,195 @@ | ||
| from unittest.mock import MagicMock | ||
|
|
||
| import yaml | ||
|
|
||
| from plugins.snippet_var_resolver.plugin import SnippetVarResolverPlugin, get_value_from_path | ||
|
|
||
|
|
||
| def make_plugin(variables=None): | ||
| plugin = SnippetVarResolverPlugin() | ||
| plugin.load_config({}) | ||
| plugin._variables = variables or {} | ||
| return plugin | ||
|
|
||
|
|
||
| def resolve(html, variables=None): | ||
| plugin = make_plugin(variables) | ||
| return plugin.on_page_content(html, page=MagicMock(), config=None, files=None) | ||
|
|
||
|
|
||
|
|
||
| class TestGetValueFromPath: | ||
| def test_top_level_key(self): | ||
| assert get_value_from_path({"version": "1.0"}, "version") == "1.0" | ||
|
|
||
| def test_dotted_path(self): | ||
| data = {"dependencies": {"foo": {"version": "2.3"}}} | ||
| assert get_value_from_path(data, "dependencies.foo.version") == "2.3" | ||
|
|
||
| def test_missing_key_returns_none(self): | ||
| assert get_value_from_path({"a": "b"}, "missing") is None | ||
|
|
||
| def test_partially_missing_path_returns_none(self): | ||
| data = {"dependencies": {"foo": "bar"}} | ||
| assert get_value_from_path(data, "dependencies.foo.version") is None | ||
|
|
||
| def test_empty_path_returns_none(self): | ||
| assert get_value_from_path({"a": "b"}, "") is None | ||
|
|
||
|
|
||
| class TestVariableResolution: | ||
| def test_simple_variable_replaced(self): | ||
| out = resolve("<p>Version: {{ version }}</p>", {"version": "1.2.3"}) | ||
| assert out == "<p>Version: 1.2.3</p>" | ||
|
|
||
| def test_dotted_path_replaced(self): | ||
| variables = {"deps": {"zombienet": {"version": "v1.3.0"}}} | ||
| out = resolve("<p>{{ deps.zombienet.version }}</p>", variables) | ||
| assert out == "<p>v1.3.0</p>" | ||
|
|
||
| def test_unknown_variable_left_intact(self): | ||
| out = resolve("<p>{{ unknown.key }}</p>", {"version": "1.0"}) | ||
| assert out == "<p>{{ unknown.key }}</p>" | ||
|
|
||
| def test_multiple_variables_in_one_page(self): | ||
| variables = {"name": "Polkadot", "version": "1.0"} | ||
| out = resolve("<p>{{ name }} {{ version }}</p>", variables) | ||
| assert out == "<p>Polkadot 1.0</p>" | ||
|
|
||
| def test_mixed_known_and_unknown(self): | ||
| out = resolve("<p>{{ name }} and {{ missing }}</p>", {"name": "Polkadot"}) | ||
| assert out == "<p>Polkadot and {{ missing }}</p>" | ||
|
|
||
| def test_no_variables_loaded_returns_html_unchanged(self): | ||
| html = "<p>{{ version }}</p>" | ||
| out = resolve(html, {}) | ||
| assert out == html | ||
|
|
||
| def test_whitespace_variants_in_placeholder(self): | ||
| variables = {"version": "1.0"} | ||
| assert "1.0" in resolve("<p>{{version}}</p>", variables) | ||
| assert "1.0" in resolve("<p>{{ version }}</p>", variables) | ||
|
|
||
| def test_variable_in_href(self): | ||
| variables = {"repo": {"url": "https://github.com/example/repo"}} | ||
| out = resolve('<a href="{{ repo.url }}">link</a>', variables) | ||
| assert 'href="https://github.com/example/repo"' in out | ||
|
|
||
| def test_non_string_value_cast_to_string(self): | ||
| out = resolve("<p>{{ count }}</p>", {"count": 42}) | ||
| assert out == "<p>42</p>" | ||
|
|
||
| def test_no_placeholders_returns_html_unchanged(self): | ||
| html = "<p>No variables here.</p>" | ||
| assert resolve(html, {"version": "1.0"}) == html | ||
|
|
||
|
|
||
| class TestOnConfig: | ||
| def test_loads_variables_from_include_yaml(self, tmp_path): | ||
| # Plugin resolves yaml paths relative to docs_dir parent, so place | ||
| # variables.yml at tmp_path and set docs_dir to tmp_path/docs. | ||
| (tmp_path / "variables.yml").write_text(yaml.dump({"version": "3.0", "name": "Test"})) | ||
|
|
||
| plugin = SnippetVarResolverPlugin() | ||
| plugin.load_config({}) | ||
| config = _build_config(tmp_path, include_yaml=["variables.yml"]) | ||
|
|
||
| plugin.on_config(config) | ||
| assert plugin._variables.get("version") == "3.0" | ||
| assert plugin._variables.get("name") == "Test" | ||
|
|
||
| def test_merges_multiple_yaml_files(self, tmp_path): | ||
| (tmp_path / "a.yml").write_text(yaml.dump({"key_a": "val_a"})) | ||
| (tmp_path / "b.yml").write_text(yaml.dump({"key_b": "val_b"})) | ||
|
|
||
| plugin = SnippetVarResolverPlugin() | ||
| plugin.load_config({}) | ||
| config = _build_config(tmp_path, include_yaml=["a.yml", "b.yml"]) | ||
|
|
||
| plugin.on_config(config) | ||
| assert plugin._variables.get("key_a") == "val_a" | ||
| assert plugin._variables.get("key_b") == "val_b" | ||
|
|
||
|
eshaben marked this conversation as resolved.
|
||
| def test_malformed_yaml_does_not_raise(self, tmp_path): | ||
| (tmp_path / "bad.yml").write_text("key: [unclosed") | ||
|
|
||
| plugin = SnippetVarResolverPlugin() | ||
| plugin.load_config({}) | ||
| config = _build_config(tmp_path, include_yaml=["bad.yml"]) | ||
|
|
||
| plugin.on_config(config) # should not raise | ||
| assert plugin._variables == {} | ||
|
|
||
| def test_non_mapping_yaml_does_not_raise(self, tmp_path): | ||
| (tmp_path / "list.yml").write_text(yaml.dump(["item1", "item2"])) | ||
|
|
||
| plugin = SnippetVarResolverPlugin() | ||
| plugin.load_config({}) | ||
| config = _build_config(tmp_path, include_yaml=["list.yml"]) | ||
|
|
||
| plugin.on_config(config) # should not raise | ||
| assert plugin._variables == {} | ||
|
|
||
| def test_later_yaml_file_wins_on_conflict(self, tmp_path): | ||
| (tmp_path / "a.yml").write_text(yaml.dump({"version": "1.0"})) | ||
| (tmp_path / "b.yml").write_text(yaml.dump({"version": "2.0"})) | ||
|
|
||
| plugin = SnippetVarResolverPlugin() | ||
| plugin.load_config({}) | ||
| config = _build_config(tmp_path, include_yaml=["a.yml", "b.yml"]) | ||
|
|
||
| plugin.on_config(config) | ||
| assert plugin._variables.get("version") == "2.0" | ||
|
|
||
| def test_missing_yaml_file_does_not_raise(self, tmp_path): | ||
| plugin = SnippetVarResolverPlugin() | ||
| plugin.load_config({}) | ||
| config = _build_config(tmp_path, include_yaml=["nonexistent.yml"]) | ||
|
|
||
| plugin.on_config(config) # should not raise | ||
| assert plugin._variables == {} | ||
|
|
||
| def test_no_macros_plugin_does_not_raise(self, tmp_path): | ||
| plugin = SnippetVarResolverPlugin() | ||
| plugin.load_config({}) | ||
|
|
||
| config = MagicMock() | ||
| config.__getitem__ = lambda self, key: { | ||
| "plugins": {}, | ||
| "extra": {}, | ||
| "docs_dir": str(tmp_path / "docs"), | ||
| }[key] | ||
| config.get = lambda key, default=None: { | ||
| "plugins": {}, | ||
| "extra": {}, | ||
| "docs_dir": str(tmp_path / "docs"), | ||
| }.get(key, default) | ||
|
|
||
| plugin.on_config(config) # should not raise | ||
| assert plugin._variables == {} | ||
|
|
||
|
|
||
| # --- helpers --- | ||
|
|
||
| def _make_macros(include_yaml=None): | ||
| macros = MagicMock() | ||
| macros.config = {"include_yaml": include_yaml or []} | ||
| return macros | ||
|
|
||
|
|
||
| def _build_config(tmp_path, include_yaml=None, extra=None): | ||
| macros = _make_macros(include_yaml) | ||
| docs_dir = str(tmp_path / "docs") | ||
|
|
||
| config = MagicMock() | ||
| config.__getitem__ = lambda self, key: { | ||
| "plugins": {"macros": macros}, | ||
| "extra": extra or {}, | ||
| "docs_dir": docs_dir, | ||
| }[key] | ||
| config.get = lambda key, default=None: { | ||
| "plugins": {"macros": macros}, | ||
| "extra": extra or {}, | ||
| "docs_dir": docs_dir, | ||
| }.get(key, default) | ||
| return config | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.