Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions config/changelog.example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -212,9 +212,13 @@ bundle:
output_directory: docs/releases
# Whether to resolve (copy contents) by default
resolve: true
# When true, PR/issue links to repos marked private in assembler.yml are rewritten in bundle output
# (requires resolve: true). Option-based: --sanitize-private-links / --no-sanitize-private-links.
sanitize_private_links: false
# PR/issue link allowlist: when set (including []), only links to these owner/repo pairs are kept
# in bundle output; others are rewritten to '# PRIVATE:' sentinels (requires resolve: true).
# When omitted, no link filtering is applied.
# Add your repository and any others whose PR/issue links should appear in published docs.
# link_allow_repos:
# - elastic/elasticsearch
# - elastic/kibana
# Optional: default GitHub repo name applied to all profiles that do not specify their own.
# Used by the {changelog} directive to generate correct PR/issue links when the product ID
# differs from the GitHub repository name. Can be overridden per profile.
Expand All @@ -238,9 +242,6 @@ bundle:
# output: "elasticsearch-{version}.yaml"
# # Optional: override the products array written to the bundle output.
# # output_products: "elasticsearch {version}"
# # Optional: override bundle.sanitize_private_links for this profile only (requires bundle.resolve: true).
# sanitize_private_links: true

# Example: GitHub release profile (fetches PR list directly from a GitHub release)
# Use when you want to bundle or remove changelogs based on a published GitHub release.
# elasticsearch-gh-release:
Expand Down
11 changes: 6 additions & 5 deletions docs/changelog/changelog-private-link-sanitization.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
type: feature
title: Add bundle-time private link sanitization and changelog directive link visibility
title: Add bundle-time link allowlist for PR and issue references
products:
- product: docs-builder
target: 0.100.0
Expand All @@ -9,7 +9,8 @@ areas:
prs:
- https://github.com/elastic/docs-builder/pull/3002
description: |
Adds opt-in `bundle.sanitize_private_links` in changelog configuration (and per-profile override),
with `--sanitize-private-links` and `--no-sanitize-private-links` on option-based changelog bundle.
Rewrites PR/issue references targeting private `assembler.yml` repos to quoted `# PRIVATE` sentinels
when resolve is true. The `{changelog}` directive gains `:link-visibility:` (`auto`, `keep-links`, `hide-links`).
Replaces `bundle.sanitize_private_links` with explicit `bundle.link_allow_repos` (`owner/repo` list).
When set (including an empty list), PR/issue references not in the allowlist are rewritten to quoted
`# PRIVATE:` sentinels when the bundle is resolved. `bundle.repo` must be a single repository (no `+` syntax).
Optional assembler.yml warnings when an allowlisted repo is missing or marked private.
The `{changelog}` directive retains `:link-visibility:` (`auto`, `keep-links`, `hide-links`).
29 changes: 9 additions & 20 deletions docs/cli/changelog/bundle.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,6 @@ The `--input-products` option determines which changelog files are gathered for
: Each occurrence can be either comma-separated issues ( `--issues "https://github.com/owner/repo/issues/123,456"`) or a file path (for example `--issues /path/to/file.txt`).
: When using a file, every line must be a fully-qualified GitHub issue URL such as `https://github.com/owner/repo/issues/123`. Bare numbers and short forms are not allowed in files.

`--no-sanitize-private-links`
: Optional: Explicitly turn off the `sanitize_private_links` option if it's specified in the changelog configuration file.

`--no-resolve`
: Optional: Explicitly turn off the `resolve` option if it's specified in the changelog configuration file.

Expand Down Expand Up @@ -140,13 +137,6 @@ The `--input-products` option determines which changelog files are gathered for
: Optional: Copy the contents of each changelog file into the entries array.
: By default, the bundle contains only the file names and checksums.

`--sanitize-private-links`
: Optional: Turn on [private link sanitization](#private-link-sanitization).
: Pull requests and issues that target repositories marked `private: true` in the `references` section of `assembler.yml` are rewritten as quoted `# PRIVATE:` sentinel strings in the bundle file.
: This option requires a resolved bundle: use `--resolve` or set `bundle.resolve: true` in the `changelog.yml`.
: If sanitization is enabled and the bundle is not resolved, the command fails.
: When you omit this option, it defaults to `bundle.sanitize_private_links` in your changelog configuration file, which defaults to `false`.

## Output files

Both modes use the same ordered fallback to determine where to write the bundle. The first value that is set wins:
Expand Down Expand Up @@ -284,27 +274,26 @@ rules:
- "Monitoring"
```

## Private link sanitization [private-link-sanitization]
## PR and issue link allowlist [link-allowlist]

A changelog in a public repository might contain links to pull requests or issues in private repositories.
To prevent that information from appearing in the documentation, use `bundle.sanitize_private_links` in the changelog configuration file (or a product-specific profile override) or the `--sanitize-private-links` command option.
A changelog in a public repository might contain links to pull requests or issues in repositories that should not appear in published documentation.

This feature relies on the [`assembler.yml`](/configure/site/content.md) file and the existence of `private: true` to determine which repo links should be sanitized.
Every repository that appears in a PR or issue link must be listed under `assembler.yml` `references`. References to unknown repositories fail the command so you can fix the registry.
Repos are assumed to be `private: false` unless you specify otherwise.
Set `bundle.link_allow_repos` in `changelog.yml` to an explicit list of `owner/repo` strings (for example, `elastic/elasticsearch`). When this key is present (including as an empty list), PR and issue references are filtered at bundle time: only links whose resolved repository is in the list are kept; others are rewritten to quoted `# PRIVATE:` sentinel strings in the bundle YAML.

:::{important}
When you use these options, you must also set `bundle.resolve: true` or specify `--resolve`.
Unresolved bundles that only store `file:` pointers do not get this rewrite; if you need private link sanitization, you must use a resolved bundle.
`bundle.link_allow_repos` requires a **resolved** bundle. Set `bundle.resolve: true` or pass `--resolve`. Unresolved bundles that only store `file:` pointers are not rewritten.
:::

The `changelog bundle`, `changelog gh-release`, and `changelog bundle-amend` commands rewrite PR and issue references that **target** private repositories into quoted sentinel strings such as `"# PRIVATE: …"` in the bundle file.
The changelog directive and `changelog render` command then omit these sentinels from the documentation.
When [`assembler.yml`](/configure/site/content.md) is available, docs-builder emits **warnings** (non-fatal) if an allowlisted repo is missing from `references` or is marked `private: true`, so you can verify the registry before publishing.

The `changelog bundle`, `changelog gh-release`, and `changelog bundle-amend` commands apply the same rules. The changelog directive and `changelog render` command omit `# PRIVATE:` sentinels from rendered documentation.

:::{warning}
Sentinel values are omitted from rendered documentation but remain in bundle files; they are not cryptographic redaction.
:::

`bundle.repo` must name a **single** GitHub repository (do not use `repo1+repo2` merged-repo syntax).

## Option-based examples

### Bundle by report or URL list
Expand Down
9 changes: 4 additions & 5 deletions docs/contribute/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -713,8 +713,8 @@ Top-level `bundle` fields:
|---|---|
| `repo` | Default GitHub repository name applied to all profiles. Falls back to product ID if not set at any level. |
| `owner` | Default GitHub repository owner applied to all profiles. |
| `resolve` | When `true`, embeds full changelog entry content in the bundle (same as `--resolve`). Required when `sanitize_private_links` is enabled. |
| `sanitize_private_links` | When `true`, rewrites PR/issue references that target private repositories (per `assembler.yml` `references`) to quoted `# PRIVATE:` sentinel strings in bundle YAML. Requires `resolve: true` and a non-empty `references` section in `assembler.yml`. Default `false`. Refer to [Private link sanitization at bundle time](/cli/changelog/bundle.md#private-link-sanitization). |
| `resolve` | When `true`, embeds full changelog entry content in the bundle (same as `--resolve`). Required when `link_allow_repos` is set. |
| `link_allow_repos` | When set (including an empty list), only PR/issue links whose resolved repository is in this `owner/repo` list are kept; others are rewritten to `# PRIVATE:` sentinels in bundle YAML. When absent, no link filtering is applied. Requires `resolve: true`. Refer to [PR and issue link allowlist](/cli/changelog/bundle.md#link-allowlist). |

Profile configuration fields in `bundle.profiles`:

Expand All @@ -727,7 +727,6 @@ Profile configuration fields in `bundle.profiles`:
| `repo` | Optional. Overrides `bundle.repo` for this profile only. Required when `source: github_release` is used and no `bundle.repo` is set. |
| `owner` | Optional. Overrides `bundle.owner` for this profile only. |
| `hide_features` | List of feature IDs to embed in the bundle as hidden. |
| `sanitize_private_links` | Optional. Overrides `bundle.sanitize_private_links` for this profile. |

Example profile configuration:

Expand Down Expand Up @@ -1048,7 +1047,7 @@ The `--hide-features` option on the `render` command and the `hide-features` fie

A changelog can reference multiple pull requests and issues in the `prs` and `issues` array fields.

To comment out the private links in all changelogs in your bundles, refer to [changelog bundle](/cli/changelog/bundle.md#private-link-sanitization).
To comment out links that are not in your allowlist in all changelogs in your bundles, refer to [changelog bundle](/cli/changelog/bundle.md#link-allowlist).

If you are working in a private repo and do not want any pull request or issue links to appear (even if they target a public repo), you also have the option to configure link visibiblity in the [changelog directive](/syntax/changelog.md) and [changelog render](/cli/changelog/render.md) command.

Expand Down Expand Up @@ -1296,7 +1295,7 @@ docs-builder changelog remove elasticsearch-release 9.2.0 --dry-run
The command automatically discovers `changelog.yml` by checking `./changelog.yml` then `./docs/changelog.yml` relative to your current directory.
If no configuration file is found, the command returns an error with advice to create one or to run from the directory where the file exists.

The `output`, `output_products`, `hide_features`, `sanitize_private_links`, and `resolve` fields are bundle-specific and are always ignored for removal (along with other bundle-only settings that do not affect which changelog files match the filter).
The `output`, `output_products`, `hide_features`, `link_allow_repos`, and `resolve` fields are bundle-specific and are always ignored for removal (along with other bundle-only settings that do not affect which changelog files match the filter).
Which other fields are used depends on the profile type:

- Standard profiles: only the `products` field is used. The `repo` and `owner` fields are ignored (they only affect bundle output metadata).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,11 @@ public record BundleConfiguration
public string? Owner { get; init; }

/// <summary>
/// When true, PR/issue references targeting repositories marked <c>private: true</c> in
/// <c>assembler.yml</c> are rewritten to sentinel values at bundle time (requires <see cref="Resolve"/>).
/// When set (including an empty list), PR/issue references whose resolved <c>owner/repo</c> is not listed
/// are rewritten to <c># PRIVATE:</c> sentinels at bundle time. When absent, no link filtering is applied.
/// Requires <see cref="Resolve"/>.
/// </summary>
public bool SanitizePrivateLinks { get; init; }
public IReadOnlyList<string>? LinkAllowRepos { get; init; }

/// <summary>
/// Named bundle profiles for different release scenarios.
Expand Down Expand Up @@ -104,9 +105,4 @@ public record BundleProfile
/// Mutually exclusive with <see cref="Products"/>.
/// </summary>
public string? Source { get; init; }

/// <summary>
/// When set, overrides <see cref="BundleConfiguration.SanitizePrivateLinks"/> for this profile.
/// </summary>
public bool? SanitizePrivateLinks { get; init; }
}
Original file line number Diff line number Diff line change
Expand Up @@ -134,22 +134,14 @@ public async Task<bool> AmendBundle(IDiagnosticsCollector collector, AmendBundle
if (_configLoader != null)
changelogConfig = await _configLoader.LoadChangelogConfiguration(collector, null, ctx);

var sanitizePrivateLinks = changelogConfig?.Bundle?.SanitizePrivateLinks == true;
Bundle? parentBundleForSanitize = null;
AssemblyConfiguration? assemblyForSanitize = null;
var linkAllowRepos = changelogConfig?.Bundle?.LinkAllowRepos;
var linkAllowlistActive = linkAllowRepos != null;
Bundle? parentBundleForAllowlist = null;

if (sanitizePrivateLinks)
if (linkAllowlistActive)
{
if (configurationContext == null)
{
collector.EmitError(
string.Empty,
"Private link sanitization requires assembler configuration. Run docs-builder with a valid configuration context.");
return false;
}

if (parentBundleFromInfer != null)
parentBundleForSanitize = parentBundleFromInfer;
parentBundleForAllowlist = parentBundleFromInfer;
else
{
var (ok, loaded) = await TryDeserializeParentBundleAsync(
Expand All @@ -160,61 +152,47 @@ public async Task<bool> AmendBundle(IDiagnosticsCollector collector, AmendBundle
if (!ok)
return false;
ArgumentNullException.ThrowIfNull(loaded);
parentBundleForSanitize = loaded;
parentBundleForAllowlist = loaded;
}

ArgumentNullException.ThrowIfNull(parentBundleForSanitize);
if (!parentBundleForSanitize.IsResolved)
ArgumentNullException.ThrowIfNull(parentBundleForAllowlist);
if (!parentBundleForAllowlist.IsResolved)
{
collector.EmitError(
string.Empty,
"Private link sanitization requires the parent bundle to be resolved (inline entry content). " +
"Re-create the bundle with resolve enabled, or disable bundle.sanitize_private_links.");
"bundle.link_allow_repos requires the parent bundle to be resolved (inline entry content). " +
"Re-create the bundle with resolve enabled, or remove bundle.link_allow_repos.");
return false;
}

var assemblyYaml = configurationContext.ConfigurationFileProvider.AssemblerFile.ReadToEnd();
try
{
assemblyForSanitize = AssemblyConfiguration.Deserialize(assemblyYaml, skipPrivateRepositories: false);
}
catch (Exception ex) when (ex is not (OutOfMemoryException or StackOverflowException))
{
collector.EmitError(
string.Empty,
$"Failed to parse assembler configuration YAML: {ex.Message}",
ex);
return false;
}

var owner = parentBundleForSanitize.Products.Count > 0 ? parentBundleForSanitize.Products[0].Owner ?? "elastic" : "elastic";
var repo = parentBundleForSanitize.Products.Count > 0 ? parentBundleForSanitize.Products[0].Repo : null;
if (!PrivateChangelogLinkSanitizer.TrySanitizeBundle(
var owner = parentBundleForAllowlist.Products.Count > 0 ? parentBundleForAllowlist.Products[0].Owner ?? "elastic" : "elastic";
var repo = parentBundleForAllowlist.Products.Count > 0 ? parentBundleForAllowlist.Products[0].Repo : null;
if (!LinkAllowlistSanitizer.TryApplyBundle(
collector,
parentBundleForSanitize,
assemblyForSanitize,
parentBundleForAllowlist,
linkAllowRepos!,
owner,
repo,
out _,
out var parentHadUnsanitizedLinks))
out var parentHadAllowlistChanges))
return false;

if (parentHadUnsanitizedLinks)
if (parentHadAllowlistChanges)
{
collector.EmitError(
string.Empty,
"Private link sanitization requires the parent bundle to already reflect sanitized PR/issue references. " +
"Re-create the parent bundle with bundle.sanitize_private_links enabled and resolve enabled, " +
"or disable bundle.sanitize_private_links for amend.");
"bundle.link_allow_repos requires the parent bundle to already reflect filtered PR/issue references. " +
"Re-create the parent bundle with the same bundle.link_allow_repos and resolve enabled, " +
"or remove bundle.link_allow_repos for amend.");
return false;
}
}

if (sanitizePrivateLinks && !shouldResolve)
if (linkAllowlistActive && !shouldResolve)
{
collector.EmitError(
string.Empty,
"Private link sanitization requires resolved amend content. Use --resolve or ensure the original bundle is resolved, or disable bundle.sanitize_private_links.");
"bundle.link_allow_repos requires resolved amend content. Use --resolve or ensure the original bundle is resolved, or remove bundle.link_allow_repos.");
return false;
}

Expand All @@ -236,23 +214,38 @@ public async Task<bool> AmendBundle(IDiagnosticsCollector collector, AmendBundle
};

var bundleForWrite = amendBundle;
if (sanitizePrivateLinks && shouldResolve)
if (linkAllowlistActive && shouldResolve)
{
ArgumentNullException.ThrowIfNull(parentBundleForSanitize);
ArgumentNullException.ThrowIfNull(assemblyForSanitize);
var owner = parentBundleForSanitize.Products.Count > 0 ? parentBundleForSanitize.Products[0].Owner ?? "elastic" : "elastic";
var repo = parentBundleForSanitize.Products.Count > 0 ? parentBundleForSanitize.Products[0].Repo : null;
ArgumentNullException.ThrowIfNull(parentBundleForAllowlist);
var owner = parentBundleForAllowlist.Products.Count > 0 ? parentBundleForAllowlist.Products[0].Owner ?? "elastic" : "elastic";
var repo = parentBundleForAllowlist.Products.Count > 0 ? parentBundleForAllowlist.Products[0].Repo : null;

if (!PrivateChangelogLinkSanitizer.TrySanitizeBundle(
if (!LinkAllowlistSanitizer.TryApplyBundle(
collector,
amendBundle,
assemblyForSanitize,
linkAllowRepos!,
owner,
repo,
out var sanitized,
out _))
return false;
bundleForWrite = sanitized;

if (configurationContext != null && linkAllowRepos!.Count > 0)
{
try
{
var assemblyYaml = configurationContext.ConfigurationFileProvider.AssemblerFile.ReadToEnd();
var assembly = AssemblyConfiguration.Deserialize(assemblyYaml, skipPrivateRepositories: false);
LinkAllowlistSanitizer.EmitAssemblerDiagnostics(collector, linkAllowRepos!, assembly);
}
catch (Exception ex) when (ex is not (OutOfMemoryException or StackOverflowException))
{
collector.EmitWarning(
string.Empty,
$"Could not load assembler.yml for bundle.link_allow_repos diagnostics: {ex.Message}");
}
}
}

// Serialize and write the amend file
Expand Down
Loading
Loading