ArgumentError: could not find a temporary directory

[jessereynolds]

Error Report

I am encountering the following issue with bundler 2.1.4 and 1.17.3 when executing in a ruby:2.5.8 docker container on Ubuntu 18 with kernel 4.15.0-101-generic. The same project does not exhibit the error on an equivalent container built and run on CentOS 7 with kernel 3.10.0-1127.el7.x86_64.

Questions

Please fill out answers to these questions, it'll help us figure out
why things are going wrong.

• What did you do?
  I ran the command /usr/local/bundle/bin/bundle install --without system_tests --path /bundle --jobs 4
• What did you expect to happen?
  I expected Bundler to install the bundle without errors.
• What happened instead?
  Instead, ruby failed to find a tmpdir and caused bundler to raise this exception and exit without installing any gems.
• Have you tried any solutions posted on similar issues in our issue tracker, stack overflow, or google?
  I tried:
  • updating ruby from 2.5.7 to 2.5.8
  • updating bundler from 1.17.3 to 2.1.4
  • alternative docker host (problem occurs on ubuntu bionic but not centos 7.x)
• Have you read our issues document, https://github.com/bundler/bundler/blob/master/doc/contributing/ISSUES.md?
  Yes

Backtrace

ArgumentError: could not find a temporary directory
  /usr/local/lib/ruby/2.5.0/tmpdir.rb:35:in `tmpdir'
  /usr/local/lib/ruby/2.5.0/tmpdir.rb:112:in `tmpdir'
  /usr/local/lib/ruby/2.5.0/tmpdir.rb:120:in `create'
  /usr/local/lib/ruby/2.5.0/tmpdir.rb:87:in `mktmpdir'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/compact_index_client/updater.rb:31:in `update'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/compact_index_client.rb:98:in `update'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/compact_index_client.rb:85:in `update_and_parse_checksums!'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/fetcher/compact_index.rb:75:in `available?'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/fetcher/compact_index.rb:16:in `call'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/fetcher/compact_index.rb:16:in `block in compact_index_request'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/fetcher.rb:162:in `use_api'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:388:in `block in api_fetchers'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:388:in `select'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:388:in `api_fetchers'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:393:in `block in remote_specs'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/index.rb:11:in `build'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:392:in `remote_specs'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:88:in `specs'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:273:in `block (2 levels) in index'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:271:in `each'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:271:in `block in index'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/index.rb:11:in `build'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:268:in `index'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:258:in `resolve'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:170:in `specs'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:158:in `resolve_remotely!'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/installer.rb:303:in `resolve_if_needed'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/installer.rb:84:in `block in run'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/process_lock.rb:12:in `block in lock'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/process_lock.rb:9:in `open'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/process_lock.rb:9:in `lock'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/installer.rb:73:in `run'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/installer.rb:25:in `install'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli/install.rb:66:in `run'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli.rb:256:in `block in install'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/settings.rb:124:in `temporary'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli.rb:255:in `install'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli.rb:30:in `dispatch'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli.rb:24:in `start'
  /usr/local/bundle/gems/bundler-2.1.4/exe/bundle:46:in `block in <top (required)>'
  /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
  /usr/local/bundle/gems/bundler-2.1.4/exe/bundle:34:in `<top (required)>'
  /usr/local/bundle/bin/bundle:23:in `load'
  /usr/local/bundle/bin/bundle:23:in `<main>'

Environment

Bundler       2.1.4
  Platforms   ruby, x86_64-linux
Ruby          2.5.8p224 (2020-03-31 revision 67882) [x86_64-linux]
  Full Path   /usr/local/bin/ruby
  Config Dir  /usr/local/etc
RubyGems      3.0.3
  Gem Home    /bundle/ruby/2.5.0
  Gem Path    /bundle/ruby/2.5.0
  User Home   /root
  User Path   /root/.gem/ruby/2.5.0
  Bin Dir     /bundle/ruby/2.5.0/bin
OpenSSL       
  Compiled    OpenSSL 1.1.1d  10 Sep 2019
  Loaded      OpenSSL 1.1.1d  10 Sep 2019
  Cert File   /usr/lib/ssl/cert.pem
  Cert Dir    /usr/lib/ssl/certs
Tools         
  Git         2.20.1
  RVM         not installed
  rbenv       not installed
  chruby      not installed

Bundler Build Metadata

Built At          2020-01-05
Git SHA           32a4159325
Released Version  true

Bundler settings

path
  Set for your local app (/usr/local/bundle/config): "/bundle"
jobs
  Set for your local app (/usr/local/bundle/config): 4
without
  Set for your local app (/usr/local/bundle/config): [:system_tests]
app_config
  Set via BUNDLE_APP_CONFIG: "/usr/local/bundle"
silence_root_warning
  Set via BUNDLE_SILENCE_ROOT_WARNING: true

Gemfile

Gemfile

source ENV['GEM_SOURCE'] || 'https://rubygems.org'
def location_for(place_or_version, fake_version = nil)
  git_url_regex = %r{\A(?<url>(https?|git)[:@][^#]*)(#(?<branch>.*))?}
  file_url_regex = %r{\Afile:\/\/(?<path>.*)}
  if place_or_version && (git_url = place_or_version.match(git_url_regex))
    [fake_version, { git: git_url[:url], branch: git_url[:branch], require: false }].compact
  elsif place_or_version && (file_url = place_or_version.match(file_url_regex))
    ['>= 0', { path: File.expand_path(file_url[:path]), require: false }]
  else
    [place_or_version, { require: false }]
  end
end
ruby_version_segments = Gem::Version.new(RUBY_VERSION.dup).segments
minor_version = ruby_version_segments[0..1].join('.')
group :development do
  gem "fast_gettext", '1.1.0',                                   require: false if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.1.0')
  gem "fast_gettext",                                            require: false if Gem::Version.new(RUBY_VERSION.dup) >= Gem::Version.new('2.1.0')
  gem "json_pure", '<= 2.0.1',                                   require: false if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.0.0')
  gem "json", '= 1.8.1',                                         require: false if Gem::Version.new(RUBY_VERSION.dup) == Gem::Version.new('2.1.9')
  gem "json", '= 2.0.4',                                         require: false if Gem::Requirement.create('~> 2.4.2').satisfied_by?(Gem::Version.new(RUBY_VERSION.dup))
  gem "json", '= 2.1.0',                                         require: false if Gem::Requirement.create(['>= 2.5.0', '< 2.7.0']).satisfied_by?(Gem::Version.new(RUBY_VERSION.dup))
  gem "rb-readline", '= 0.5.5',                                  require: false, platforms: [:mswin, :mingw, :x64_mingw]
  gem "puppet-module-posix-default-r#{minor_version}", '~> 0.4', require: false, platforms: [:ruby]
  gem "puppet-module-posix-dev-r#{minor_version}", '~> 0.4',     require: false, platforms: [:ruby]
  gem "puppet-module-win-default-r#{minor_version}", '~> 0.4',   require: false, platforms: [:mswin, :mingw, :x64_mingw]
  gem "puppet-module-win-dev-r#{minor_version}", '~> 0.4',       require: false, platforms: [:mswin, :mingw, :x64_mingw]
end
puppet_version = ENV['PUPPET_GEM_VERSION']
facter_version = ENV['FACTER_GEM_VERSION']
hiera_version = ENV['HIERA_GEM_VERSION']
gems = {}
gems['puppet'] = location_for(puppet_version)
# If facter or hiera versions have been specified via the environment
# variables
gems['facter'] = location_for(facter_version) if facter_version
gems['hiera'] = location_for(hiera_version) if hiera_version
if Gem.win_platform? && puppet_version =~ %r{^(file:///|git://)}
  # If we're using a Puppet gem on Windows which handles its own win32-xxx gem
  # dependencies (>= 3.5.0), set the maximum versions (see PUP-6445).
  gems['win32-dir'] =      ['<= 0.4.9', require: false]
  gems['win32-eventlog'] = ['<= 0.6.5', require: false]
  gems['win32-process'] =  ['<= 0.7.5', require: false]
  gems['win32-security'] = ['<= 0.2.5', require: false]
  gems['win32-service'] =  ['0.8.8', require: false]
end
gems.each do |gem_name, gem_params|
  gem gem_name, *gem_params
end
# Evaluate Gemfile.local and ~/.gemfile if they exist
extra_gemfiles = [
  "#{__FILE__}.local",
  File.join(Dir.home, '.gemfile'),
]
extra_gemfiles.each do |gemfile|
  if File.file?(gemfile) && File.readable?(gemfile)
    eval(File.read(gemfile), binding)
  end
end
# vim: syntax=ruby

Gemfile.lock

<No /tmp/cba/odpeng-odpengsudo/Gemfile.lock found>

[deivid-rodriguez]

Could you create repro steps inside a docker container so that this can be reproduced independently of your environment? Thanks!

[deivid-rodriguez]

Or point to a public repo where this reproduces.

[jessereynolds]

@deivid-rodriguez yep will do. I think it will matter what version of the linux kernel, or linux distro, your docker server is running on, or something environmental on the host.

[jessereynolds]

@deivid-rodriguez well I have this repo that I use to build the container, and then GitLab CI is executing tests from its .gitlab-ci.yaml which is breaking on the ubuntu 18 server. I also noticed that the gitllab-runner version is 13.0.0, whereas the working one on my CentOS host is 12.10.1 so that could be to do with it also.

So, I think this will be reproducable on the current gitlab-runner 13.0.0.

https://github.com/jessereynolds/puppet-module-testing-container

Run:

• scripts/build_image.sh
• then have gitlab-ci execute the jobs in .gitlab-ci.yaml

I also added commands in the .gitlab-ci.yaml to test writing to /tmp dir and it works:

$ touch /tmp/aaaaaaaa
$ ls -la /tmp
total 16
drwxrwxrwx 4 root root 4096 May 21 15:24 .
drwxr-xr-x 1 root root 4096 May 21 15:24 ..
-rw-r--r-- 1 root root    0 May 21 15:24 aaaaaaaa

[jessereynolds]

OK this looks to definitely not be a bundler issue as I'm also getting the error doing this in the same container:

$ ruby -e 'require "tmpdir" ; puts Dir.tmpdir ; puts Dir.mktmpdir'
/usr/local/lib/ruby/2.5.0/tmpdir.rb:35:in `tmpdir': could not find a temporary directory (ArgumentError)
	from -e:1:in `<main>'

/tmp mount point:

/dev/sda2 on /tmp type ext4 (rw,relatime,data=ordered)

A little more debugging... this script:

#!/usr/bin/env ruby

require "tmpdir"
dir = File.expand_path("/tmp")
puts dir.inspect
stat = File.stat(dir)
puts "directory? #{stat.directory?}"
puts "writable? #{stat.writable?}"
puts "world_writable? #{stat.world_writable?}"
puts "sticky? #{stat.sticky?}"
puts Dir.tmpdir
puts Dir.mktmpdir

produces:

/usr/local/lib/ruby/2.5.0/tmpdir.rb:35:in `tmpdir': could not find a temporary directory (ArgumentError)
	from scripts/debug_gitlab_runner_ruby_tmpdir.rb:11:in `<main>'
"/tmp"
directory? true
writable? true
world_writable? 511
sticky? false

Those are the tests of a tmpdir's acceptability in Dir.tmpdir in ruby 2.5.x as per:

        if stat = File.stat(dir) and stat.directory? and stat.writable? and
            (!stat.world_writable? or stat.sticky?)

So it's the fact that /tmp is both world_writeable and not sticky that makes it invalid.

On the working runner the output is:

"/tmp"
 directory? true
 writable? true
 world_writable? 511
 sticky? true
 /tmp
 /tmp/d20200522-26-gl4mb6

[jessereynolds]

My workaround is to set TMPDIR env var to /var/tmp for all pipelines on the new gitlab 13 runner.

[jessereynolds]

Created an issue against gitlab runner https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25799

[deivid-rodriguez]

Wow, nice investigation. I believe an issue should also be reported to https://github.com/ruby/tmpdir to print a better error message in this case.

[deivid-rodriguez]

Better late than never, I raised an issue about this at ruby/tmpdir#2.