Gem::DependencyResolutionError trigger from gem install

While trying to gem install on locally built chef.gem and its dependencies chef-utils.gem and chef-config.gem, I was seeing the following error:

Running `gem install /Users/powell/projects/chef/pkg/chef-18.0.160.gem` failed with the following output:

ERROR:  While executing gem ... (Gem::DependencyResolutionError)
    conflicting dependencies chef-utils (< 19, >= 16.0) and chef-utils (= 18.0.160)
  Activated chef-utils-18.0.160
  which does not match conflicting dependency (< 19, >= 16.0)

  Conflicting dependency chains:
    chef (= 18.0.160), 18.0.160 activated, depends on
    chef-utils (= 18.0.160), 18.0.160 activated

  versus:
    chef (= 18.0.160), 18.0.160 activated, depends on
    ohai (~> 18.0), 18.0.14 activated, depends on
    chef-utils (< 19, >= 16.0)

  Gems matching chef-utils (< 19, >= 16.0):
    chef-utils-18.0.160

Note that the “conflict” is (<19, >= 16.0) vs. (= 18.0.160). That shouldn’t be a problem, right? 18 is definitely less than 19 and greater than or equal to 16!

Ok, so let’s find the source of the error… I’m using rbenv with ruby 3.1.2 installed, so the rubygems root is at ~/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems so we’ll search for a DependencyResolutionError from there. (Using the_silver_searcher for searching or apt install silversearcher-ag on Ubuntu)

Tracing symptoms

Tracing down several layers, we get to requirement.rb and Gem::Requirement#satisfied_by? From there, I added a tap block and output some code on the result from requirements.all?

  ##
  # True if +version+ satisfies this Requirement.

  def satisfied_by?(version)
    raise ArgumentError, "Need a Gem::Version: #{version.inspect}" unless
      Gem::Version === version
    requirements.all? {|op, rv| OPS[op].call version, rv }.tap do |result|
      # tap block for debugging requirements
      unless result 
        puts "Version #{version}" # what version are we trying
        puts caller[0..10] # how are we getting here??
        # what are our requirements
        requirements.each do |op, rv|
          puts "#{op} #{rv} #{OPS[op].call version, rv}"
        end
      end
    end
  end

Now we go back and rake install 2>&1 | gvim - to see the output of the debugging. Most notable is the versions being validated

 Version 17.10.0
<internal:kernel>:90:in `tap'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/requirement.rb:244:in `satisfied_by?'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/dependency.rb:238:in `match?'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:57:in `block in find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:56:in `each'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:56:in `find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:54:in `block in find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:53:in `map'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:53:in `find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/best_set.rb:30:in `find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/installer_set.rb:175:in `find_all'
= 18.0.162 false
Version 17.10.0
<internal:kernel>:90:in `tap'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/requirement.rb:244:in `satisfied_by?'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/dependency.rb:238:in `match?'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:57:in `block in find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:56:in `each'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:56:in `find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:54:in `block in find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:53:in `map'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:53:in `find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/best_set.rb:30:in `find_all'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/installer_set.rb:175:in `find_all'
= 18.0.162 false
Version 16.18.0
<internal:kernel>:90:in `tap'
/home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/requirement.rb:244:in `satisfied_by?'

We are validating two sets of conditions: <19, >=16.0 and = 18.0.162. It is the latter condition that is tripping us up. The versions being used to fulfill the dependencies for gem install that are in the gemspec are remote versions and chef-utils stops at 17.10.0 on rubygems

Since our gemspec is looking for an exact version match on a version that’s not published, we can’t use gem install yet… in this case, the version is pinned to use the same version as the dependency.

  s.add_dependency "chef-config", "= #{Chef::VERSION}"
  s.add_dependency "chef-utils", "= #{Chef::VERSION}"

Conclusion / Workarounds / Solutions

The Gem::DependencyResolutionError is a bit misleading in this case, but looking through the resolution mechanism, there’s not an easy path to determine that the resolution failed due to the exact version match not existing. This will happen on any gem install in which the gem you’re installing is looking for versions that don’t match dependencies that are published (whether RubyGems or internal gem server.) You don’t have to be looking for an exact match. If you’re looking for >= 18 and only 17 and earlier is published, you’ll still have a problem.

Clearly, if you’re juggling multiple “internal” gems at the same time, your development process will likely run into this if you try to gem install the gem. How do you avoid it? bundle install doesn’t have this limitation, as you can look for candidate gems locally and/or specify local repos. Install the dependencies first and then bundle install on the parent project. Once you’re preparing to launch the gem, you can publish prerelease versions of the dependencies and pin to those.