Gem::DependencyResolutionError trigger from gem install
While trying to gem install on locally built chef.gem and its dependencies chef-utils.gem and chef-config.gem, I was seeing the following error:
Running `gem install /Users/powell/projects/chef/pkg/chef-18.0.160.gem` failed with the following output:
ERROR: While executing gem ... (Gem::DependencyResolutionError)
conflicting dependencies chef-utils (< 19, >= 16.0) and chef-utils (= 18.0.160)
Activated chef-utils-18.0.160
which does not match conflicting dependency (< 19, >= 16.0)
Conflicting dependency chains:
chef (= 18.0.160), 18.0.160 activated, depends on
chef-utils (= 18.0.160), 18.0.160 activated
versus:
chef (= 18.0.160), 18.0.160 activated, depends on
ohai (~> 18.0), 18.0.14 activated, depends on
chef-utils (< 19, >= 16.0)
Gems matching chef-utils (< 19, >= 16.0):
chef-utils-18.0.160
Note that the “conflict” is (<19, >= 16.0) vs. (= 18.0.160). That shouldn’t be a problem, right? 18 is definitely less than 19 and greater than or equal to 16!
Ok, so let’s find the source of the error… I’m using rbenv with ruby 3.1.2 installed, so the rubygems root is at ~/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems so we’ll search for a DependencyResolutionError from there. (Using the_silver_searcher for searching or apt install silversearcher-ag on Ubuntu)
Tracing symptoms
Tracing down several layers, we get to requirement.rb and Gem::Requirement#satisfied_by? From there, I added a tap block and output some code on the result from requirements.all?
##
# True if +version+ satisfies this Requirement.
def satisfied_by?(version)
raise ArgumentError, "Need a Gem::Version: #{version.inspect}" unless
Gem::Version === version
requirements.all? {|op, rv| OPS[op].call version, rv }.tap do |result|
# tap block for debugging requirements
unless result
puts "Version #{version}" # what version are we trying
puts caller[0..10] # how are we getting here??
# what are our requirements
requirements.each do |op, rv|
puts "#{op} #{rv} #{OPS[op].call version, rv}"
end
end
end
end
Now we go back and rake install 2>&1 | gvim - to see the output of the debugging. Most notable is the versions being validated
Version 17.10.0 <internal:kernel>:90:in `tap' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/requirement.rb:244:in `satisfied_by?' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/dependency.rb:238:in `match?' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:57:in `block in find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:56:in `each' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:56:in `find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:54:in `block in find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:53:in `map' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:53:in `find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/best_set.rb:30:in `find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/installer_set.rb:175:in `find_all' = 18.0.162 false Version 17.10.0 <internal:kernel>:90:in `tap' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/requirement.rb:244:in `satisfied_by?' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/dependency.rb:238:in `match?' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:57:in `block in find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:56:in `each' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/api_set.rb:56:in `find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:54:in `block in find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:53:in `map' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/composed_set.rb:53:in `find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/best_set.rb:30:in `find_all' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/resolver/installer_set.rb:175:in `find_all' = 18.0.162 false Version 16.18.0 <internal:kernel>:90:in `tap' /home/thomas/.rbenv/versions/3.1.2/lib/ruby/3.1.0/rubygems/requirement.rb:244:in `satisfied_by?'
We are validating two sets of conditions: <19, >=16.0 and = 18.0.162. It is the latter condition that is tripping us up. The versions being used to fulfill the dependencies for gem install that are in the gemspec are remote versions and chef-utils stops at 17.10.0 on rubygems
Since our gemspec is looking for an exact version match on a version that’s not published, we can’t use gem install yet… in this case, the version is pinned to use the same version as the dependency.
s.add_dependency "chef-config", "= #{Chef::VERSION}"
s.add_dependency "chef-utils", "= #{Chef::VERSION}"
Conclusion / Workarounds / Solutions
The Gem::DependencyResolutionError is a bit misleading in this case, but looking through the resolution mechanism, there’s not an easy path to determine that the resolution failed due to the exact version match not existing. This will happen on any gem install in which the gem you’re installing is looking for versions that don’t match dependencies that are published (whether RubyGems or internal gem server.) You don’t have to be looking for an exact match. If you’re looking for >= 18 and only 17 and earlier is published, you’ll still have a problem.
Clearly, if you’re juggling multiple “internal” gems at the same time, your development process will likely run into this if you try to gem install the gem. How do you avoid it? bundle install doesn’t have this limitation, as you can look for candidate gems locally and/or specify local repos. Install the dependencies first and then bundle install on the parent project. Once you’re preparing to launch the gem, you can publish prerelease versions of the dependencies and pin to those.