[8.3] [8.3] Adding Documents for v8.3.3 Pre-Built Detection Rules Integration Release (backport #2905) #2909
Conversation
|
Documentation previews: |
|
Failing because backporting is going to 8.3 and 8.4 references exist. Ultimately, we may need have to specify to only backport to the latest branch the documents reference. 10:03:25 INFO:build_docs:asciidoctor: ERROR: detections/prebuilt-rules/prebuilt-rules-downloadable-updates.asciidoc: line 94: include file not found: /tmp/docsbuild/MM7_7ZsqRT/security-docs/docs/detections/prebuilt-rules/downloadable-packages/8-4-1/prebuilt-rules-8-4-1-summary.asciidoc
10:03:25 INFO:build_docs:asciidoctor: WARNING: detections/prebuilt-rules/downloadable-packages/8-3-2/prebuilt-rules-8-3-2-summary.asciidoc: line 3: id assigned to block already in use: prebuilt-rule-8-3-2-prebuilt-rules-8-3-2-summary
10:03:25 INFO:build_docs:asciidoctor: ERROR: index.asciidoc: line 76: include file not found: /tmp/docsbuild/MM7_7ZsqRT/security-docs/docs/detections/prebuilt-rules/downloadable-packages/8-4-1/prebuilt-rules-8-4-1-appendix.asciidoc
10:03:25 INFO:build_docs:asciidoctor: WARNING: invalid reference: prebuilt-rule-8-4-1-prebuilt-rules-8-4-1-summary
10:03:25 INFO:build_docs:Since we are releasing a package for each version -3 from main, we may need to rethink how we approach updates to this repository as a result. Merging into main all of the packages is fine. When backporting, we can only backport to +1 of the earliest package version. Example below. OOB Packages (Packages - v8.6.1, v8.5.1, v8.4.2, v8.3.3): This adds complexity to the doc updates. We may also want to limit the documentation scope to only matching minor stacks. For the 8.5 branch, we should only display v8.5.x packages as customers will not receive 8.6 packages and Fleet automatically installs the latest package, therefore previous is obsolete. As a stop-gap prior to https://github.com/elastic/ia-trade-team/issues/17 (re-factor), we may need to only apply package documents to the compatible branch as so... OOB Packages (Packages - v8.6.1, v8.5.1, v8.4.2, v8.3.3): As shown, we can still merge everything into master but the specific branch versions would only reflect the latest compatible package just as Fleet would install. The potential issue is the target branch would no longer receive previous branch packages but I think these can still be added, it just adds complexity to the PRs. At the moment, it appears there are no rule version history links for OOB packages, this only exists for filesystem rules in Kibana, which is no longer available as of 8.6. This would be nice detail, but we may save this for the pre-built detection rules security docs refactor.
Regarding Detection Rules code for the CLI command that generates these documents, it appears the "new" vs "update" is dependent on logic in detection rules and is not dependent on the security docs repository. As a result, if we only pushed package docs to a specific branch, it will still determine if these are new or not. TRaDE will need to be careful when running our integration docs generation CLI command as we compare diffs from the specified tags. This should work well when targeting specific branches (i.e. comparing 8.3.3 to 8.3.4) to tell what rules have been updated and which are new. @dev_group.command('build-integration-docs')
@click.argument('registry-version')
@click.option('--pre', required=True, help='Tag for pre-existing rules')
@click.option('--post', required=True, help='Tag for rules post updates')
@click.option('--directory', '-d', type=Path, required=True, help='Output directory to save docs to')
@click.option('--force', '-f', is_flag=True, help='Bypass the confirmation prompt')
@click.option('--remote', '-r', default='origin', help='Override the remote from "origin"')
@click.pass_context
def build_integration_docs(ctx: click.Context, registry_version: str, pre: str, post: str, directory: Path, force: bool,
remote: Optional[str] = 'origin') -> IntegrationSecurityDocs:
"""Build documents from two git tags for an integration package."""
if not force:
if not click.confirm(f'This will refresh tags and may overwrite local tags for: {pre} and {post}. Continue?'):
ctx.exit(1)
rules_changes = get_release_diff(pre, post, remote)
docs = IntegrationSecurityDocs(registry_version, directory, True, *rules_changes)
package_dir = docs.generate()
click.echo(f'Generated documents saved to: {package_dir}')
updated, new, deprecated = rules_changes
click.echo(f'- {len(updated)} updated rules')
click.echo(f'- {len(new)} new rules')
click.echo(f'- {len(deprecated)} deprecated rules')
return docsShort-Term SolutionTo not become potentially blocker in this repository.
I am concerned about that branches will diverge from main, since it relies on backporting and therefore I am unaware if this temporary solution is feasible at the moment. |
1 participant
This is an automatic backport of pull request #2905 done by Mergify.
Cherry-pick of 3d0edd6 has failed:
To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally
Mergify commands and options
More conditions and actions can be found in the documentation.
You can also trigger Mergify actions by commenting on this pull request:
@Mergifyio refreshwill re-evaluate the rules@Mergifyio rebasewill rebase this PR on its base branch@Mergifyio updatewill merge the base branch into this PR@Mergifyio backport <destination>will backport this PR on<destination>branchAdditionally, on Mergify dashboard you can:
Finally, you can contact us on https://mergify.com