XML Handler Bypassing ParseError Handling Rack Middleware on Rails 6

In a project using actionpack-xml_parser, and implementing a modified version of Catching Invalid JSON Parse Errors with Rack Middleware to also handle XML errors, an upgrade to Rails 6 broke the handling of the ParseError. Interestingly enough, JSON was being handled as expected.

TL;DR: The short answer to this is that config/initializers/wrap_parameters.rb only had json mentioned as a format. This can be remedied in the initializer or on a per controller basis. The entire app fix was just adding :xml to the initializer:

ActiveSupport.on_load(:action_controller) do
  wrap_parameters format: [:json, :xml]

In looking through the stack traces/call stack for where the parse error occurs (ActionDispatch::Http::Parameters#parse_formatted_parameters) using the caller array in the gem source and backtrace in the rescue, I was able to catch that ActionController::Metal::ParamsWrapper#_wrapper_enabled? was called for JSON, but not for XML. Ultimately, I was able to track down the difference to ActionController::Metal::ParamsWrapper#_wrapper_formats?, which was only returning [:json], which led to the wrap_parameters functionality in Rails.

rspec-rails tricks: Hacking Your Routes for a Spec

I don’t know that this is so much a reminder of “What can be done with rspec-rails” as a note that, “If you do this will rspec-rails, you will also need to undo it.”

Today’s note: If you hack the application routes for a controller test, you have to reload routes.

The original problem

This all came down to a spec that was attempting to test a method in ApplicationController. In our Rails 3 setup (rspec-core and rspec-rails 3.7.0), the tests only needed a monkey patched ApplicationController to happen prior to the test:

before do
  class ApplicationController
    def dummy_action

it 'example do test' do
  get :dummy_action #etc...
  # validate dummy_method_actually_under_test did the thing

The fix

In Rails 4 with the same gem versions, we’d end up with the “No route matches…” which could be remedied by redefining the routes.

before do
  class ApplicationController
    def dummy_action
  Rails.application.routes.draw do
    get 'application/dummy_action/:id', to: 'application#dummy_action'

Great! The test passes now! (Insert philosophical argument about whether the person writing the test should have tested a method in this way.)

Or not…

..until you run the rest of the suite, of course. Now *every* subsequent controller test has a “No route matches” issue. (Insert philosophical argument about whether you should be writing controller tests.)

This should serve as a periodic reminder to clean up after your tests, which in this case is:

after do

RailsSettings vs parallel_tests

RailsSetting stubbing problem

I have an area of functionality that is controlled by a Settings class that is a subclass of RailsSettings::CachedSettings (rails-settings-cached gem) and changing that setting for tests was interfering with the parallel_tests gem when I ran my rspec tests.

I tried namespacing the cache, making TTL 0, invalidating the cache, monkeypatching the lookup… Every convoluted solution possible… to try and get tests to play nicely together.

The humbling thing about programming is how simple actual solutions are compared the things the brain comes up with.

The solution? Wrap the access to the Setting in a layer of abstraction (which I had partially done already.)

class ThatFunctionality
  def self.enabled?
  def self.enabled=(value)

And then in tests:

before do
  allow(ThatFunctionality).to receive(:enabled?).and_return(true) # or false

Trying again to stub RailsSettings

The class method approach is explicit, but once a large set of settings is needed, it can get unwieldy. If you haven’t found a better strategy for wrapping settings values, you can go back to stubbing the :[] method. The caveat is that you need to save the original Settings behavior for any settings that you want to leave unchanged. By stubbing the :[] first to passthrough to .and_call_original, you can add additional stubs on top.

# a module included
module StubSettings
def stub_setting(setting_name, setting_value)
allow(Setting).to receive(:[]).with(setting_name).and_return(setting_value)
def stub_setting_assignment
allow(Setting).to receive(:[]=) { |setting, value| stub_setting(setting, setting) }
# in spec_helper.rb
config.before(:each) do
# detect if Settings being set without you knowing about it
allow(Setting).to receive(:[]=)
.and_raise("stub_setting(setting_name, setting_value) to set the value of a Setting instead")
# passthrough read-only settings calls (necessary if you don't want .with to block other settings
allow(Setting).to receive(:[]).and_call_original
# in rspec example
describe 'settings will be set in code invoked from here' do
# automatically stubs assigned value to new stub
before { stub_setting_assignment }
# stubbing a setting value
describe 'setting to be forced to a value' do
before { stub_setting('some_toggle_setting', true) }

view raw
hosted with ❤ by GitHub

Rails 4, phantom ArgumentError: wrong number of arguments (0 for 1) on UserSession.find for AuthLogic

This was a total pain to locate, as the exceptions being raised were pointed to UserSession.find in Authlogic::Session::Persistence.

gems in question:

  activerecord-session_store 0.1.1
  authlogic 3.4.6
  activerecord 4.2.3
  rails 4.2.3

I noticed that the only meaningful difference between the two environments was the following instance variable in Rails.application.config:

irb> pp Rails.application.config

Commenting out this initialization in the environment allowed the application to work again.

   config.logger = ActiveSupport::TaggedLogging.new(Syslog::Logger.new('name of log'))

Searching on “usersession syslog rails 4” results in the following issue as the third search result: Version 0.1.1 breaks Syslog::Logger setups There is a fix in master, but it doesn’t seem to have been published. I had promoted activerecord-session_store to 0.1.1 because DEPRECATION WARNING: `#quietly` is deprecated in rails-4.2.0.beta4, but it looks like locking the gem at 0.1 will work otherwise except for noisy tests.

Hooking in an LDAP Test Server to Cucumber Tests

I managed to get a custom Devise strategy with LDAP working, but had no clear way of automating tests. I wanted to validate if I still had to keep the password fresh in the database, and needed to be able to write scenarios around that in case someone attempted to refactor out the code.

After trying to incorporate the spec strategy used in the development devise_ldap_authenticatable and failing, I found a ruby wrapper of ApacheDS called ladle that looked like it would serve my purposes.

I included in gem in my test group in my Gemfile:

  gem 'ladle'

At the top of my features/env.rb file for configuring cucumber, I turned off admin binding (wanted the connection as simple as possible):

::Devise.ldap_use_admin_to_bind = false

I then created an @ldap tag for my LDAP-dependent features than would start and stop the LDAP server in those instances. (Again, in my features/env.rb… probably need to clean that up.)

Around('@ldap') do |scenario, block|
  $ladle ||= Ladle::Server.new(
    :ldif => "spec/ldap/test_users.ldif",
    :domain => "dc=example,dc=org",
    :quiet => true

I then created an the spec/ldap/test_users.ldif (from following the example in the ladle project).

version: 1

dn: ou=people,dc=example,dc=org
objectClass: top
objectClass: organizationalUnit
ou: people

dn: uid=eadmin,ou=people,dc=example,dc=org
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
cn: Example Admin
sn: Admin
givenName: Example
mail: eadmin@example.com
uid: eadmin
# Password is "b44b44bl@cksh33p!"
userpassword: {SHA}Aedq5WHQSxglvJSfpX0kgdGRdHk=

I generated the password with:

  slappasswd -h {SHA} -s b44b44bl@cksh33p!

One stupid mistake that I did in the process was kicking off two Ladle servers (with slightly different parameters). In one iteration, I couldn’t bind to the user. Another, the server using the test file failed to start. Be aware that Ladle will run happily with default parameters, but that they won’t be much use to you.

If you want to test your configuration file:

require 'net/ldap'
require 'ladle'

$ladle ||= Ladle::Server.new(
  :ldif => "spec/ldap/test_users.ldif",
  :domain => "dc=example,dc=org",
  :quiet => true

ldap = Net::LDAP.new(host: 'localhost',
    :port => 3897,
filter = Net::LDAP::Filter.eq('mail', 'eadmin@example.com') # or ('uid', 'eadmin') 

ldap.search(:base => 'ou=people,dc=example,dc=org', :filter => filter) do |entry|
  ldap.auth(entry.dn, 'b44b44bl@cksh33p!') # or whatever your password is

  entry.each do |attribute, values|
    puts "   #{attribute}:"
    values.each do |value|
      puts "      --->#{value}"

devise_ldap_authentication for your domain email on top of database_authenticatable

I have a devise user model named LoginUser whose authentication key is :login. I want normal users of the system to be database_authenticatable.

However, I want to be able to authenticate previously added users via internal LDAP. Furthermore, I didn’t want the underlying database_authenticatable password to be used or to expire on me (also using devise_security_extensions). Most of the work is in the LocalOverride custom strategy’s authenticate! method, with a few other hooks (such as default strategy added to devise.yml).


To allow all other strategies to be used, but still trap our domains for one-off LDAP auth, I added devise :ldap_authenticatable to a singleton class inherited from the user loaded by the custom strategy.

Also, removed the other two “fails” from the code. Not necessary and will result in a “Failed to Login” message for too many other Devise-related Unauthorized events.

In config/initializers/local_override.rb:

module Devise
  module Strategies
    class LocalOverride < Authenticatable
      def valid?

      def authenticate!
        if params[:login_user]
          user = LoginUser.find_by_login(params[:login_user][:login])
          # trap our domain only
          if params[:login_user][:login] =~ /@example.com/
            # fail! halts the authentication chain completely
            return fail! unless ::Devise::LDAP::Adapter.valid_login?(params[:login_user][:login])
            class << user
              # make use of ldap_authenticatable for custom strategy only
              devise :ldap_authenticatable
            return fail! unless user.valid_ldap_authentication?(params[:login_user][:password])
            # use the after_ldap_authentication hook
            return success!(user)

Warden::Strategies.add(:local_override, Devise::Strategies::LocalOverride)

In config/initializers/devise.rb:

  # use local_override as default strategy
  config.warden do |manager|
    manager.default_strategies(:scope => :login_user).unshift :local_override

In config/models/login_user.rb:

class LoginUser < ActiveRecord::Base
  devise :database_authenticatable,
         :recoverable, :trackable, :secure_validatable,
         :authentication_keys => [:login]

  def after_ldap_authentication
    # force fresh password every log in
    self.password = self.password_confirmation = Random.new.bytes(47)

defaults:  &defaults
  host: our.ldap
  port: 636
  attribute: mail
  base: dc=IDENT,o=Orgname
  admin_password: adminpassw0rd
  ssl: sslmethod

See local_override.rb for original tip that got me there.

Monkey patching if you need to force an error in delayed_job

I wanted to force an error condition in my cucumber tests, but the code ran through a DelayedJob, so I couldn’t redefine the code running under the .delay chain, because the DelayedJob worker will reload the normal code base for its processing. I finally realized that simply patching the delay instance method to return self for the class in question would bypass the delayed job.

def force_query_error
  ObjectDelayed.class_eval do
    alias :old_delay :delay
    alias :old_query :query

    def delay

    def query
      raise 'monkey'

def restore_query
  ObjectDelayed.class_eval do
    remove_method :query
    remove_method :delay
    alias :query :old_query
    alias :delay :old_delay

This is not to be confused will actually allowing DelayedJob to do its thing:

When(/^I wait for processing of jobs$/) do
  Delayed::Worker.new(:quiet => false).work_off
  Timeout::timeout(10) do
    until Delayed::Job.count == 0 do

On Mac OSX Lion: “ERROR: Error installing ruby-oci8: ERROR: Failed to build gem native extension”

Running Mac OSX Lion, trying to bundle install a project that included the ruby-oci8 gem, I received the following error:

Building native extensions.  This could take a while...
ERROR:  Error installing ruby-oci8:
	ERROR: Failed to build gem native extension.

The fix:

Fortunately, the 64-bit client for Oracle has now been updated for Lion/Mountain Lion. Download them from Instant Client Downloads for Mac OSX (Intel x86). Downloading “Basic Lite” and the “SDK” under Version 11.2… (64-bit) worked for me.

Unzip the files to ~/instantclient_11_2 so that the sdk directory and libs are directory beneath it:

$ ls ~/instantclient_11_2 
BASIC_LITE_README    genezi               libclntsh.dylib.11.1 libocci.dylib.11.1   libocijdbc11.dylib   ojdbc6.jar           uidrvci
adrci                libclntsh.dylib      libnnz11.dylib       libociicus.dylib     ojdbc5.jar           sdk                  xstreams.jar

Add the following to my .zlogin or .bash_login

export DYLD_LIBRARY_PATH=~/instantclient_11_2

Rerun the login shell (exec $SHELL -l) or restart your shell.

Link the libclntsh.dylib. in ~/instantclient_11_2 to a versionless lib file name:

ln -s libclntsh.dylib.11.1 libclntsh.dylib

Your ruby-oci8 should install now.

Fun with the Ruby & (unary ampersand) and String#to_proc

In trying to grasp the unary & operator in ruby, I ran across [Stupid Ruby Tricks: String#to_proc].

Therefore, I decided I had to further twist the language features to my own will.

Like many things in Ruby with duck-typing, the following syntax:

('a'..'z').map &:upcase

…makes use of the fact that Symbol has a #to_proc method, which means that the following call is an explicit version of the above call.

('a'..'z').map &:upcase.to_proc

If you wanted to allow for a shorthand notation for map, then you could add #to_proc to String, as follows:

class String
  def to_proc
    eval "Proc.new { |*args| args.first#{self} }"

This allows for the following call:

[1,2,3].map &'*2'                     # [2, 4, 6]

…which is equivalent to:

[1,2,3].map { |*args| args.first*2 } # note: no space

…But what if I wanted to specify a library function to have the Array members passed to? (Certainly, this is venturing deeper into solving a problem that doesn’t exist.) Also, what if I want to pass an Array for the arguments?

class String
  def to_proc
    # assume an initial word character is a library function, otherwise evaluate as before
    if /^w/ =~ self
      eval "Proc.new { |*args| #{self}(*args.first) }"
      eval "Proc.new { |*args| args.first#{self} }"

The above code will expand an embedded Array into an argument list:

[[2,2],[9,3],[64,4]].map &'Math.log'  # [1.0, 2.0, 3.0]
[1.0, 2.0, 4.0].map &'Math.sqrt'      # [1.0, 1.4142135623730951, 1.7320508075688772]
[1,2,3].map &'*2'                     # [2, 4, 6]
[[2,2],[9,3],[64,4], 4].map &'Math.log' # [1.0, 2.0, 3.0, 1.3862943611198906]

Experimenting with Rails’ String Manipulation from ActiveSupport::CoreExtensions::String::Inflections

This is my experimentation with the string manipulation methods used [largely internally] in Rails. This is largely a repeat of the documentation here, but I did dig into pluralize/singularize and a couple of other examples more in depth. Look at [path to active_support gem version]/lib/active_support/inflections.rb for the full list of plural, singular, irregular, and uncountable regular expressions and words.

require 'active_support/core_ext/string/inflections'
# camelcase, camelize 
"separate_words".camelize # SeparateWords
"active_record".camelcase(:lower) # separateWords

# classify - converts to first upper CamelCase, last word made singular
"this_and_thats".classify # ThisAndThat

# constantize - looks for declared constant with the name specified
"Integer".constantize # returns constant [class] Integer
"integer".constantize # results in NameError: wrong constant name integer
"Boo".constantize # results in NameError: uninitialized constnat Boo (assuming it isn't declared)

# dasherize - underscores to dashes
"a_b_c".dasherize # a-b-c

# demodulize - removes module namespace from module in string.
ModuleA::SubModule::Module".demodulize # Module

# foreign_key - create name of a foreign key to class name, optional parameter to separate class from id
"Integer".foreign_key # integer_id
"Integer".foreign_key(false) # integerid

# humanize - capitalizes first word, down cases rest, turns underscores into space.
"McLean".humanize # Mclean
"Apple_Banana".humanize # Apple banana

# parameterize - replaces special characters in a string, default separator is '-'
#   - whitespace is a special character here, underscore is not
#   - consecutive special characters become a single separator
#   - converts to lowercase
"Apple@!Banana".parameterize # "apple banana"

# pluralize 
"raman".pluralize # "ramen" -- apparently, "man" --> "men" even on a made-up word
"jazz".pluralize # "jazzs" 
"box".pluralize # "boxes"
"beep".pluralize # "beeps"
"moose".pluralize # "mooses"
"rhombus".pluralize # "rhombuses"
"octopus".pluralize # "octopi"

# singularize
"ramen".singularize # "raman"
"jazzes".singularize # "jazze"
"boxen".singularize # "boxen" -- yeah, not a real word again

# tableize - snake_case string and pluralize last word
"RedHeadedStepchild".tableize # "red_headed_stepchildren"

# titlecase / titleize - capitalize all words, 
"two_three-four.five:six".titleize # "Two Three Four.Five:Six"

# underscore - snake_case words (opposite of CamelCase) and covert :: to /
"Number::OneTwo::Integer".underscore # "number/one_two/integer"