Skip to content

Try to fix parallel docs builds with subdirs#1154

Merged
jimklimov merged 12 commits intonetworkupstools:masterfrom
jimklimov:fix-paral-docs
Nov 1, 2021
Merged

Try to fix parallel docs builds with subdirs#1154
jimklimov merged 12 commits intonetworkupstools:masterfrom
jimklimov:fix-paral-docs

Conversation

@jimklimov
Copy link
Copy Markdown
Member

@jimklimov jimklimov commented Nov 1, 2021

Picks up from #1152 to allow actual parallel builds (and distchecks) with each piece of documentation rolling in a dedicated uniquely named directory while it is being generated. This allows to avoid the conflict of requesting same-named targets from different rules, as well as of a2x using same-named helper files to build different docs, ruling it out on the OS/filesystem level. And to gain parallelism instead of safely evading it with earlier trickery attempts.

Subject also touched in #505 and #418 discussions.

…l times (mentioned in different list variables)
…t several times (mentioned in different list variables)"

This reverts commit 29ae5a8f6e5524d689ebaf2abdf508528288b010.
…l times (mentioned in different list variables)
…ilt several times (mentioned in different list variables)
…omponent docs"

This reverts commit 0b9ac2500ddbc7bd11a639805d43cda9259bfb1f.
…n up the destination if needed and make the workdir removable if needed (a2x can copy system/EXTRA_DIST resources with their attrs)
@jimklimov
Copy link
Copy Markdown
Member Author

TODO: If the different make implementations in CI farm (GNU, BSD, Sun) all do not choke on this change of recipe for at least sequential builds, next step is to activate parallel rules in Jenkinsfile-dynamatrix to gain the benefit on CI farm. Or maybe, temporarily, to add such scenarios as "permitted to fail", to see if this change causes any intermittent/racey non-100% faults.

@jimklimov
Copy link
Copy Markdown
Member Author

jimklimov commented Nov 1, 2021

For a rough benchmark, on a laptop this branch passed make clean -k; make distcheck -j15 in 2m4s (there are far less than 15 cores, so this was more to check for collisions), and a sequential make clean -k; make distcheck in 4m18s. Both succeeded. (C/C++ overhead was negligible thanks to ccache; so this was mostly configure, make and asciidoc/a2x CPU time).

Master branch passed seq build in 4m5s at the same timeframe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant