GenPipes Usage boost via LLMS.txt

[shaloo]

Now-a-days, content is discovered not just via humans through organic searches but also via bots/AI/LLMs. The latter is on the rise by significant numbers published by Google/OpenAI.

Many documentation sites are designed to be accessible to humans and organic bot searches/scraping, as well as LLMs. To enable this, GenPipes RTD docs must auto-generate the LLMS.txt file, which most tooling uses to answer user queries, especially about genomics and gene sequencing. We must enable these tools to locate and use GenPipes in their answers to humans seeking advanced gene sequencing processes and tools.

I propose that we use https://sphinx-llms-txt.readthedocs.io/en/latest/ and enable GenPipes RTD docs for LLMs. This should boost our GA numbers as well.

cc: @pphector

[pphector]

Let me ask how others in the team feel about this and I will reply. I think I would be in favour of doing this, but I want to make sure everyone else agrees.