Error executing action `install` on resource 'python_package[setuptools]'

[f9n]

$ chef generate cookbook install-python
$ cd install-python
$ vim recipes/default.rb
python_runtime '3'
$ vim .kitchen.yml
...
platforms:
  - name: ubuntu-16.04
...
$ vim metadata.rb   # Add poise python
...
depends 'poise-python'
...

$ kitchen converge ubuntu
.....
Installing chef 
       installing with dpkg...
       Selecting previously unselected package chef.
(Reading database ... 65145 files and directories currently installed.)
       Preparing to unpack .../cache/chef_14.5.33-1_amd64.deb ...
       Unpacking chef (14.5.33-1) ...
       Setting up chef (14.5.33-1) ...
       Thank you for installing Chef!
       Transferring files to <default-ubuntu-1604>
       Starting Chef Client, version 14.5.33
       Creating a new client identity for default-ubuntu-1604 using the validator key.
       resolving cookbooks for run list: ["install-python::default"]
       Synchronizing Cookbooks:
         - poise-python (1.7.0)
         - install-python (0.1.0)
         - poise-languages (2.1.2)
         - poise (2.8.2)
         - poise-archive (1.5.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 1 resources
       Recipe: install-python::default
         * python_runtime[3] action install
           * poise_languages_system[python3.5] action install
             - install version 3.5.2-2ubuntu0~16.04.4 of package python3.5-dev
           * apt_package[python3.5, python3.5-dev] action nothing (skipped due to action :nothing)
         
         * python_runtime_pip[3] action install
           - Bootstrapping pip latest from https://bootstrap.pypa.io/get-pip.py
         * python_package[setuptools] action install
           
           ================================================================================
           Error executing action `install` on resource 'python_package[setuptools]'
           ================================================================================
           
           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
           STDOUT: 
           STDERR: The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
           Traceback (most recent call last):
             File "<stdin>", line 42, in <module>
           AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
           ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
           Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1
           
           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:329:in `pip_command'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:353:in `pip_outdated'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:221:in `check_package_versions'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:200:in `load_current_resource'
           /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in `action_install'
           
           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb
           
           138:         python_package 'setuptools' do
           139:           parent_python new_resource
           140:           version setuptools_version if setuptools_version.is_a?(String)
           141:         end
           142:       end
           
           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:138:in `install_setuptools'
           
           python_package("setuptools") do
             package_name "setuptools"
             action [:install]
             default_guard_interpreter :default
             declared_type :python_package
             cookbook_name "install-python"
             parent_python python_runtime[3]
             timeout 900
           end
           
           System Info:
           ------------
           chef_version=14.5.33
           platform=ubuntu
           platform_version=16.04
           ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
           program_name=/opt/chef/bin/chef-client
           executable=/opt/chef/bin/chef-client
           
         
         ================================================================================
         Error executing action `install` on resource 'python_runtime[3]'
         ================================================================================
         
         Mixlib::ShellOut::ShellCommandFailed
         ------------------------------------
         python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
         ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
         STDOUT: 
         STDERR: The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
         Traceback (most recent call last):
           File "<stdin>", line 42, in <module>
         AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
         ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
         Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1
         
         Cookbook Trace:
         ---------------
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:329:in `pip_command'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:353:in `pip_outdated'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:221:in `check_package_versions'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:200:in `load_current_resource'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in `action_install'
         
         Resource Declaration:
         ---------------------
         # In /tmp/kitchen/cache/cookbooks/install-python/recipes/default.rb
         
           7: python_runtime '3'
         
         Compiled Resource:
         ------------------
         # Declared in /tmp/kitchen/cache/cookbooks/install-python/recipes/default.rb:7:in `from_file'
         
         python_runtime("3") do
           action [:install]
           updated true
           updated_by_last_action true
           default_guard_interpreter :default
           subresources [python_runtime_pip[3], python_package[setuptools], python_package[wheel]]
           declared_type :python_runtime
           cookbook_name "install-python"
           recipe_name "default"
           get_pip_url "https://bootstrap.pypa.io/get-pip.py"
           pip_version true
           setuptools_version true
           version "3"
           virtualenv_version true
           wheel_version true
         end
         
         System Info:
         ------------
         chef_version=14.5.33
         platform=ubuntu
         platform_version=16.04
         ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
         program_name=/opt/chef/bin/chef-client
         executable=/opt/chef/bin/chef-client
         
       
       Running handlers:
       [2018-10-05T12:52:36+00:00] ERROR: Running exception handlers
       [2018-10-05T12:52:36+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2018-10-05T12:52:36+00:00] ERROR: Exception handlers complete
       [2018-10-05T12:52:36+00:00] ERROR: Exception handlers complete
       Chef Client failed. 3 resources updated in 22 seconds
       [2018-10-05T12:52:36+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2018-10-05T12:52:36+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2018-10-05T12:52:36+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2018-10-05T12:52:36+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2018-10-05T12:52:36+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: python_runtime[3] (install-python::default line 7) had an error: Mixlib::ShellOut::ShellCommandFailed: python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       STDOUT: 
       STDERR: The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
       Traceback (most recent call last):
         File "<stdin>", line 42, in <module>
       AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
       ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1
       [2018-10-05T12:52:36+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: python_runtime[3] (install-python::default line 7) had an error: Mixlib::ShellOut::ShellCommandFailed: python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       STDOUT: 
       STDERR: The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
       Traceback (most recent call last):
         File "<stdin>", line 42, in <module>
       AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
       ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <default-ubuntu-1604>.  Please see .kitchen/logs/default-ubuntu-1604.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

I don't understand, it worked in the past.

[devsibwarra]

https://pypi.org/project/pip/#history shows 18.1 was released a couple hours ago. Rolling back to 18.0 via pip install pip==18.0 resolved this error for me

[knightorc]

Interestingly enough. Does not impact CentOS 6.x but does impact CentOS 7.x

[fozboz]

Doing

python_runtime '3' do
  pip_version '18.0'
end

does not work because of pypa/pip#5857 which is probably the cause of original issue, given that they occurred at the same time.

[nxf5025]

Any workaround we can implement into the actual cookbook?

[coderanger]

You can try setting the get_pip_url property to an older version of the script, either one you host yourself or one of the legacy scripts like https://bootstrap.pypa.io/3.3/get-pip.py.

[jeffbyrnes]

This works for me:

python_runtime '3' do
  get_pip_url 'https://github.com/pypa/get-pip/raw/f88ab195ecdf2f0001ed21443e247fb32265cabb/get-pip.py'
  pip_version '18.0'
end

[coderanger]

Hopefully this will be fixed upstream shortly. I'll monitor the upstream ticket and see how it goes.

[aungger]

I am getting the same thing for python_runtime '2', any tips for this? plz and ty!

[coderanger]

@aungger Same fix, the current default get-pip.py script is broken.

[eriepasquare]

if your dependency on poise-python is through a wrapped cookbook and you can't change the pip_version or get_pip_url parameters of the resource... we had success implementing the following workaround in our wrapper cookbook:

edit_resource(:python_runtime, '2') do
  pip_version '18.0'
  get_pip_url 'https://github.com/pypa/get-pip/raw/f88ab195ecdf2f0001ed21443e247fb32265cabb/get-pip.py'
end

[nkovacne]

If you're using a virtualenv resource, you also needed to set the pip_version and get_pip_url in the python_virtualenv resource:

python_virtualenv '/opt/venv' do
  pip_version '18.0'
  get_pip_url 'https://github.com/pypa/get-pip/raw/f88ab195ecdf2f0001ed21443e247fb32265cabb/get-pip.py'
  action :create
end

[hartmantis]

Update: They merged a fix to get-pip but it seems to only resolve the part where get_pip_url had to be specified. So

python_virtualenv '/opt/venv' do
  pip_version '18.0'
end

works now, but blows up without the pip_version when it tries to install setuptools.

[lopezm1]

confirmed, overriding the attribute will force poise-python to use pip 18.0 and our cookbooks now work as expected

"poise-python": {
      "options": {
        "pip_version": "18.0"
      }	      
}