Make Python dependencies required by importers optional

[ligurio]

otava has a number of data importers: PostgreSQL, BigQuery, Graphite etc. ¹. All these importers (except CSV) implemented using 3rd-party Python modules, and all these 3rd-party modules must be installed to make otava work. Currently, a list of dependencies includes 74 Python modules:

asn1crypto==1.5.1     
attrs==25.3.0 
autoflake==1.7.8                   
backports.zoneinfo==0.2.1                                                                                                                                     
cachetools==5.5.2
certifi==2025.1.31
cfgv==3.4.0        
charset-normalizer==3.4.1
dateparser==1.2.0
decorator==5.2.1  
distlib==0.3.9             
expandvars==0.6.5
filelock==3.16.1 
flake8==4.0.1     
google-api-core==2.24.2
google-auth==2.38.0
google-cloud-bigquery==3.30.0    
google-cloud-core==2.4.3
google-crc32c==1.5.0
google-resumable-media==2.7.2
googleapis-common-protos==1.69.1
grpcio==1.70.0
grpcio-status==1.70.0
identify==2.6.1 
idna==3.10      
importlib_metadata==8.5.0
iniconfig==2.0.0 
isort==5.13.2          
mccabe==0.6.1
more-itertools==8.14.0
nodeenv==1.9.1
numpy==1.24.0                      
-e git+https://github.com/apache/otava/@2f3369f447133594cf827978ca0e558df064b7bf#egg=otava
packaging==24.2  
pg8000==1.31.2   
platformdirs==4.3.6
pluggy==1.5.0
pre-commit==3.5.0
proto-plus==1.26.1
protobuf==5.29.3           
py==1.11.0  
py-cpuinfo==9.0.0
pyasn1==0.6.1     
pyasn1_modules==0.4.1
pycodestyle==2.8.0
pyflakes==2.4.0                  
pystache==0.6.7
pytest==6.2.5
pytest-benchmark==4.0.0
python-dateutil==2.9.0.post0
pytz==2021.1
PyYAML==6.0.2
regex==2024.11.6
requests==2.32.3
rsa==4.9
ruamel.yaml==0.17.21
ruamel.yaml.clib==0.2.8
ruff==0.6.9
scipy==1.9.3
scramp==1.4.5
signal-processing-algorithms==1.3.5
six==1.17.0
slack_sdk==3.34.0
structlog==19.2.0
tabulate==0.8.10
toml==0.10.2
tomli==2.2.1
tox==3.28.0
typing-extensions==3.10.0.2
tzlocal==5.2
urllib3==2.2.3
validators==0.18.2
virtualenv==20.29.3
zipp==3.20.2

So if someone uses a single importer, he must install all other 3rd-party modules for the rest of importers. Otherwise, Otava will not work. I propose to make Python modules, required by data importers, optional. This will allow reducing a list of dependencies.

Footnotes

1. https://github.com/apache/otava/blob/master/docs/README.md#data-sources ↩

[Gerrrr]

Hey @ligurio! This is an awesome idea, thanks for proposing this!

Thinking out loud, it would be great if:

1. Data importers could check during initialization if dependencies are present and throw a user-friendly error if they are absent.
2. We could group optional dependencies such that a user could configure its installation likegimme all dependencies for BigQuery and PostgreSQL.

[Nortsx]

I could potentially take this task, but @Gerrrr will we be using poetry in the future?

[Gerrrr]

Hey @Nortsx ! Thank you for you interest in the project!

I would love us to migrate to uv, so there is a dependency on this decision. My plan there is to open a proposal for discussion right after we finish with the first release, so hopefully in 1.5-2 weeks.