Publish this tool on PyPI

[JohnGiorgi]

We need to properly package Saber, and publish it on PyPI so that it can be pip install saber. I don't know how to do this, so I need to do my homework first and then publish it. See here for some inspiration.

Resources

• this blog

[cthoyt]

I know quite a bit about this and would be happy to help. Packaging tools that have exotically specified dependencies like this is tricky though, and I've never had good luck using dependency_links. Do you think it's possible to find more streamlined distributions (hopefully via PyPI) of the keras contrib repo and the NeuroalCoref medium model?

[JohnGiorgi]

@cthoyt Oh wow, well that would be great!

This has been the main thing holding me back from trying to stick Saber on PyPI in the first place. The documentation of both keras-contrib and NeuralCoref both instruct you to use pip install on some GitHub URL.

If there is a more streamlined distribution for either of these tools, I do not know about it. I have even opened an issue on the keras-contrib repo asking for help installing it with a setup.py file. None of advice did the trick for me.

[cthoyt]

PR is all done. It looks like it should work now!

[cthoyt]

However, it looks like pip install saber has already been snagged...

Update: I've dealt with packages whose names on pip and the thing they install are not the same. I would strongly recommend against doing that. It's a huge problem for users.

That doesn't mean that the project name has to be changed nor the repository name on GitHub (in fact, I think your acronym is very cool). What I might suggest is changing the folder in the repository where all the code lives to something else like saber_ner and also change the name in setup.py to match. Of course a new name would be up to you.

[JohnGiorgi]

Okay so it looks like the PyPI FAQs mention something about claiming names from abandoned projects. A bit more digging reveals what is considered an abandoned project.

I would say the saber project currently onl PyPI satisfies the second two criteria (no activity within 12 months and no homepage listed).

It is just not clear how (or if?) I can make a request for the name saber. I could always email the owner of the current saber project and ask if they are willing to remove their project from PyPi (the last activity was in 2014 after all). I emailed the author... now we wait! the author got back to me, saber is now free for us to use!

If that doesn't work / isn't a good idea, I will try to think of a new name as you suggest. I would probably go with saber_nlp or saber_ie.

(in fact, I think your acronym is very cool)

Thanks! I was pretty proud of it haha

[cthoyt]

That’s really great to hear! Definitely claim it yourself ASAP. The best way would be to make an account on PyPI, then run the tox -e release command I added for you, which takes care of building and uploading to PyPI. Even if there are some issues, you can always release a new version later

[JohnGiorgi]

@cthoyt Okay I ran into this issue after running tox -e release:

release create: /Users/johngiorgi/Documents/Masters/Class/natural_language_computing/project/saber/.tox/release
release installdeps: wheel, setuptools, twine >= 1.5.0
release installed: bleach==3.1.0,certifi==2018.11.29,chardet==3.0.4,docutils==0.14,idna==2.8,pkginfo==1.5.0.1,Pygments==2.3.1,readme-render
er==24.0,requests==2.21.0,requests-toolbelt==0.8.0,saber==0.0.1,six==1.12.0,tqdm==4.29.1,twine==1.12.1,urllib3==1.24.1,webencodings==0.5.1
release run-test-pre: PYTHONHASHSEED='1259105692'
release runtests: commands[0] | python setup.py -q sdist bdist_wheel
no previously-included directories found matching 'notebooks'

no previously-included directories found matching 'img'
release runtests: commands[1] | twine upload --skip-existing 'dist/*'
Enter your username: johnmgiorgi
Enter your password:
Uploading distributions to https://upload.pypi.org/legacy/
Uploading saber-0.1.0-py3-none-any.whl
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 91.9M/91.9M [01:14<00:00, 1.29MB/s]
NOTE: Try --verbose to see response content.
HTTPError: 400 Client Error: Invalid value for requires_dist. Error: Can't have direct dependency: 'keras-contrib @ git+https://www.github.
com/keras-team/keras-contrib.git' for url: https://upload.pypi.org/legacy/
ERROR: InvocationError for command '/Users/johngiorgi/Documents/Masters/Class/natural_language_computing/project/saber/.tox/release/bin/twi
ne upload --skip-existing dist/*' (exited with code 1)
_________________________________________________________________ summary _________________________________________________________________
ERROR:   release: commands failed

Any suggestions? I will try to figure it out in the meantime!

[cthoyt]

Oh dear... Maybe one of sdist or bdist_wheel doesn't accept this type of requirement :/ Maybe you can try removing those one at a time? You'll have to delete the build and dist folders in the mean time. I might try it myself on test PyPI and see if I can get it working later.

[JohnGiorgi]

I think you are correct, if I remove keras-contrib @ git+https://www.github.com/keras-team/keras-contrib.git from setup.py and run tox -e release I get the same error, just about en-cored-md this time:

release installed: bleach==3.1.0,certifi==2018.11.29,chardet==3.0.4,docutils==0.14,idna==2.8,pkginfo==1.5.0.1,Pygments==2.3.1,readme-renderer==24.0,requests==2.21.0,requests-tool
belt==0.8.0,saber==0.1.0,six==1.12.0,tqdm==4.29.1,twine==1.12.1,urllib3==1.24.1,webencodings==0.5.1
release run-test-pre: PYTHONHASHSEED='2477164563'
release runtests: commands[0] | python setup.py -q sdist bdist_wheel
no previously-included directories found matching 'notebooks'

no previously-included directories found matching 'img'
release runtests: commands[1] | twine upload --skip-existing 'dist/*'
Enter your username: johnmgiorgi
Enter your password:
Uploading distributions to https://upload.pypi.org/legacy/
Uploading saber-0.1.0-py3-none-any.whl
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 91.9M/91.9M [01:07<00:00, 1.43MB/s]
NOTE: Try --verbose to see response content.
HTTPError: 400 Client Error: Invalid value for requires_dist. Error: Can't have direct dependency: 'en-coref-md @ https://github.com/huggingface/neuralcoref-models/releases/downl
oad/en_coref_md-3.0.0/en_coref_md-3.0.0.tar.gz' for url: https://upload.pypi.org/legacy/
ERROR: InvocationError for command '/Users/johngiorgi/Documents/Masters/Class/natural_language_computing/project/saber/.tox/release/bin/twine upload --skip-existing dist/*' (exit
ed with code 1)
____________________________________________________________________________________ summary _____________________________________________________________________________________
ERROR:   release: commands failed