Docusaurus2 upgrade for master

[317brian]

Description

This PR updates master to Docusaurus 2. It backs up the previous website directory to website_old. We can delete it after we’re ready to merge this PR.

You can see a build preview here: https://druid-n5fxtlvhm-317brian.vercel.app/docs/latest/design/. It is completely unstyled and uses OOTB Docusaurus2 settings since this repo only acts as the source of truth for docs. apache/druid-website-src is what builds the prod site.

Related PRs:

• Separate PR that also upgrades the 26.0.0 release branch: 26.0.0: upgrade the branch to docusaurus2 #14417
• Upgrades druid-website-src to Docusaurus 2. Docusaurus2 conversion druid-website-src#394
• Proposes a new publishing process: Proposed publishing process for Docusaurus 2 druid-website-src#392

Benefits

• No longer EOL backend!
• More nesting in the side nav
• Support for extended Markdown (MDX), which lets you add Javascript to Markdown pages.

Changed

• The Markdown parser is stricter for Docusaurus2. Impacted docs have already been updated.
• The format of redirects. To add a new redirect, follow the ones that have already been converted in website/redirects.json in this PR.
• Sidebar syntax
  • subcategory -> category
  • ids -> items
• The tab syntax used for things like showing the curl and Python equivalent API calls.
• The new config file for the site settings is website/docusaurus.config.js
• Minor change but this PR also deletes docs/operations/getting-started.md. There was a redirect that was written for it. The redirect never gets generated though because the page existed. Deleting the file allows for the redirect to get built.)

Stayed the same

• Install Docusaurus with npm/yarn install and run the site with npm/yarn start.
• All the website pages, like the home page and "Downloads" page are still in apache/druid-website-src.
• The source of truth for the CSS and styling of the prod site is apache/druid-website-src. I didn’t see a need to style this repo since the main concern would be the Markdown rendering correctly, and they both use the same Markdown parser.
• druid-website-src still builds the production site
• This repo is the source of truth for the docs, sidebar, and redirects.

Need help on the following:

The link checking script and broken link behavior. I copied the link checking script over and updated the path to actually point at the build output from npm run build, but the output is different depending on how the filepath is declared:

• const entries = fg.sync(['./build/docs/**/*.html']) results in:There are 13232 issues. That's definitely wrong

• const entries = fg.sync(['./build/docs/latest/*.html']) results in 1 issue: Could not find '/docs/latest/design/' linked from './build/docs/latest/index.html. I think the issue is that it's not going through the directories recursively, so it can't find the file. But if we do the ** instead of latest, we get 13k issues.

Neither of these are the actual link I broke intentionally to test the link checker, which Docusaurus 2 detects when you run npm run build:

[INFO] Docs markdown link couldn't be resolved: (./bob.md) in "/Users/brian/my-druid-fork/docs/ingestion/ingestion-spec.md" for version current

In the meantime, we can configure Docusaurus 2 to do a one of 2 things on broken links when someone runs the site or tries to build it:

We can either set it to log a command line msg like so for npm run start/build:

[WARNING] Docs markdown link couldn't be resolved: (../ingestion/in.md) in "/Users/brian/my-druid-fork/docs/tutorials/index.md" for version current

We went with this option. See #14411 (comment)
Or we can surface an error in the rendered site:

The user can close the error and navigate around the site still.

---

This PR has:

• been self-reviewed.

Thanks @ektravel, @writer-jill, @techdocsmith, and @vtlim for helping get this together.

[317brian]

@vtlim and @techdocsmith can you review and approve this PR:

Check out the PR and do npm start and npm run build in website to verify the docs build

Should we add a GHA step that just runs npm run build to do the link checking built into Docusaurus 2 and do a markdown syntax check on prs?

I guess i should also update https://github.com/apache/druid/blob/master/distribution/asf-release-process-guide.md#update-druidapacheorg in this PR

[317brian]

Updated asf-release-process-guide.md too, so that's also ready for review.

[317brian]

f6ec159 (#14411) works as intended. The CI failed with the errant link in the output. Will revert the fake link I put in.

[clintropolis]

why do we still need 'website_old' path and small set of files? since this was already merged into a release branch it seems safe to just dump it

[techdocsmith]

LGTM

[xvrl]

@techdocsmith @317brian it looks like this removed the website/pom.xml but I couldn't find an explanation for that. This seems to have broken dependabot, because the module is still referenced in one of the maven profiles in the main pom.xml. It's fine if we want to remove the maven build for the website, but can we at least add a comment to explain why we removed it? Thanks!

[317brian]

Hi @xvrl, this removed the pom.xml for the website directory cause the Druid docs for staging/production aren't built in this repo anymore: #15113 (comment)

I forgot to do that piece of the followup. I'll put up a PR that deletes the profile.

[xvrl]

no worries @317brian, I already have a PR up to remove the module here #15540