[Task]: Improve project documentation on apache-beam pypi page.

[tvalentyn]

What needs to happen?

Current project summary is a very short paragraph: https://pypi.org/project/apache-beam/. We could add more content and useful pointers to that page, perhaps similar to https://pypi.org/project/scipy/.

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[rakeshcusat]

@tvalentyn / @damccorm should this be referring readme.md file instead.

from setuptools import setup

# read the contents of your README file
from pathlib import Path
this_directory = Path(__file__).parent
long_description = (this_directory / "README.md").read_text()

setup(
    name='an_example_package',
    # other arguments omitted
    long_description=long_description,
    long_description_content_type='text/markdown'
)

Scipy uses something similar. In that case there will be only one source of truth and get update automatically as soon as the readme file is updated.

What do you guys think?

[liferoad]

Good idea.

[tvalentyn]

Ah, so this is coming from setup.py itself:

beam/sdks/python/setup.py

Line 94 in 76b0df9

PACKAGE_LONG_DESCRIPTION = '''

.

I think that's a good idea. @rakeshcusat would you be interested in sending a PR?

[rakeshcusat]

@tvalentyn / @liferoad on the second thought, I realized there are a lot of stuff in Readme.md file which are not relevant to Python. So directly getting the content from readme.md file may not work. It would be better if we just change this section and include all the relevant information.

I would be more than happy to send the PR once we agree on the approach

Edit1:

The third approach is to have a readme.md file at Python directory level and have only content which is relevant to PyPi and Python SDK and use the above code snippet in the setup.py file. This could be more cleaner approach. What do you think?

Edit2:

here is the PR: #34329

Also, need help to check the changes locally.

[rakeshcusat]

.take-issue

[rakeshcusat]

@tvalentyn / @liferoad can you please check this PR #34329 and let me know if I am heading in right direction.

[rakeshcusat]

@tvalentyn let's close this ticket.