XML Handler Bypassing ParseError Handling Rack Middleware on Rails 6

Posted: April 7, 2021 | Author: | Filed under: ruby | Tags: , , , | Comments Off on XML Handler Bypassing ParseError Handling Rack Middleware on Rails 6

In a project using actionpack-xml_parser, and implementing a modified version of Catching Invalid JSON Parse Errors with Rack Middleware to also handle XML errors, an upgrade to Rails 6 broke the handling of the ParseError. Interestingly enough, JSON was being handled as expected.

TL;DR: The short answer to this is that config/initializers/wrap_parameters.rb only had json mentioned as a format. This can be remedied in the initializer or on a per controller basis. The entire app fix was just adding :xml to the initializer:

ActiveSupport.on_load(:action_controller) do
  wrap_parameters format: [:json, :xml]
end

In looking through the stack traces/call stack for where the parse error occurs (ActionDispatch::Http::Parameters#parse_formatted_parameters) using the caller array in the gem source and backtrace in the rescue, I was able to catch that ActionController::Metal::ParamsWrapper#_wrapper_enabled? was called for JSON, but not for XML. Ultimately, I was able to track down the difference to ActionController::Metal::ParamsWrapper#_wrapper_formats?, which was only returning [:json], which led to the wrap_parameters functionality in Rails.

Whitespace is Significant

Posted: July 6, 2017 | Author: | Filed under: programming | Tags: , , , , , | Comments Off on Whitespace is Significant

While I was in my first couple years of college, Python was gaining traction as a hot new programming language. I had been programming in C since high school, but was curious about new languages. I took an interest in learning more about Python. A classmate told me that whitespace (at least indentation) was significant in Python. I changed my mind.

The Holy Wars of Indentation Style

// From https://en.wikipedia.org/wiki/Indent_style
// K & R
while (x == y) {
something();
somethingelse();
}
// 1TBS
if (x < 0) {
puts("Negative");
} else {
nonnegative(x);
}
// Stroustup
if (x < 0) {
puts("Negative");
negative(x);
}
else {
puts("Non-negative");
nonnegative(x);
}
// Allman
while (x == y)
{
something();
somethingelse();
}
// GNU
while (x == y)
{
something();
somethingelse();
}
// Whitesmiths
while (x == y)
{
something();
somethingelse();
}
// Horstmann
while (x == y)
{ something();
somethingelse();
}
// Pico
while (x == y)
{ something();
somethingelse(); }
// Ratliff
while (x == y) {
something();
somethingelse();
}
// LISP
while (x == y) {
something();
somethingelse(); }

view raw
indent.c
hosted with ❤ by GitHub

It’s interesting that would’ve been so turned off by a language that placed significance on whitespace and indentation, when I had fantasized about holy wars revolving around C indentation style (Horstmann is the only way!) and tabs vs. spaces (tabs, tab width 4!). I wretched at anything in the GNU C coding style (okay, I still do…), but the funny thing was, I never was able to program in a consistent style: There were other co-workers who had a slightly different preferred style. I was also learning myself, so relative importance of information in code changed, and my opinions on bracing style evolved.

Non-C Languages that Use Curly Braces for Things

It was a combination of Java, JavaScript, Perl, and Ruby that altered my perception of “proper” bracing style (BUT NOT SPACES OVER TABS!). A new line after a conditional or function declaration/call before the brace either took up unnecessary amounts of space, or changed the meaning of the code altogether (e.g., blocks in ruby, for which adding a newline = syntax error). And so, I evolved into declaring that 1TBS style was the only proper way to write C (WITH TABS!)

Ruby

Ruby has a generally accepted coding style that is enforced by no one except for code reviewers and maybe draconian RuboCop build configurations. I’d link to the coding style, if I had any certainty that the one I grabbed was 99% correct. Even RubyMine doesn’t format code to the same standards as RuboCop.  (Continuation indent and spacing around braces were two settings I had to tweak.)

Small inconsistencies aside, Ruby code has ended up being the most consistent—visually—to support and add code to. I’ve introduced tighter RuboCop configurations on newer projects to try to adhere more closely to generally accepted coding style for Ruby.

Ruby also introduced me to an acceptance of significant whitespace. While C code often gets sloppy with spacing around punctuation markers in code, Ruby developers take more care in proper whitespace around the conditionals, braces, etc. Some C/C++ projects like Mozilla do similar standards. However, Ruby is the first language that I’ve seen a community-wide interest in code that presents in the same manner. Of course, most of this code isn’t syntactically significant, but it is of human significance–and humans have their own mental compilers that optimize on standard patterns.

Still, this isn’t the largest impact Ruby has had on my opinion of significant whitespace. Through Ruby, I’ve been exposed to YAML, HAML, and Slim. (I have dibs on forming a band of Rubyists with that name!) All three languages are unforgiving about inconsistent indentation. They take out explicit expression of boundaries and create visually implicit boundaries via hierarchies of indents. You don’t need to look for an end or closing brace to know that the context has moved up a level. The indentation tells you where that happens.

(And yes, because of the generally accepted coding style in Ruby, I use 2 spaces instead of tabs now.)

Golang

And yet, Ruby coding style is informal. You can code in improper style and still commit your code and/or run it. There’s something about having the freedom to write improper style and being able to be a good “citizen” on your own that is comforting.

Go does not do this. Writing Go in any editor configured to code in it (I’ve been using the vim-go plugin) automatically formats via gofmt.  It’s pretty unsettling. I longed for the ability to become dictator over coding styles, and then was okay when Ruby dictated one, but gave us the autonomy to do the right thing. This auto-formatting? This is tyranny!

But, perhaps, it’s time. We can mentally chunk pieces into larger concepts for storage, if code that does the same thing looks the code. Instead of having to remember every brace as an individual piece, we can put “for loop that iterates and prints” as a single chunk.

A stray brace isn’t going to permanently prevent distillation into larger chunks; it will just take a higher cognitive load to get there. This in turn will require more experience with the language before a programmer can make that leap. Being able to break things into large chunks helps us discover the what instead of dwelling on the how.

Hooking in an LDAP Test Server to Cucumber Tests

Posted: May 22, 2015 | Author: | Filed under: daily learning, ruby | Tags: , , , , | Comments Off on Hooking in an LDAP Test Server to Cucumber Tests

I managed to get a custom Devise strategy with LDAP working, but had no clear way of automating tests. I wanted to validate if I still had to keep the password fresh in the database, and needed to be able to write scenarios around that in case someone attempted to refactor out the code.

After trying to incorporate the spec strategy used in the development devise_ldap_authenticatable and failing, I found a ruby wrapper of ApacheDS called ladle that looked like it would serve my purposes.

I included in gem in my test group in my Gemfile:

  gem 'ladle'

At the top of my features/env.rb file for configuring cucumber, I turned off admin binding (wanted the connection as simple as possible):

::Devise.ldap_use_admin_to_bind = false

I then created an @ldap tag for my LDAP-dependent features than would start and stop the LDAP server in those instances. (Again, in my features/env.rb… probably need to clean that up.)

Around('@ldap') do |scenario, block|
  $ladle ||= Ladle::Server.new(
    :ldif => "spec/ldap/test_users.ldif",
    :domain => "dc=example,dc=org",
    :quiet => true
  )
  $ladle.start
  block.call
  $ladle.stop
end

I then created an the spec/ldap/test_users.ldif (from following the example in the ladle project).

version: 1

dn: ou=people,dc=example,dc=org
objectClass: top
objectClass: organizationalUnit
ou: people

dn: uid=eadmin,ou=people,dc=example,dc=org
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
cn: Example Admin
sn: Admin
givenName: Example
mail: eadmin@example.com
uid: eadmin
# Password is "b44b44bl@cksh33p!"
userpassword: {SHA}Aedq5WHQSxglvJSfpX0kgdGRdHk=

I generated the password with:

  slappasswd -h {SHA} -s b44b44bl@cksh33p!

One stupid mistake that I did in the process was kicking off two Ladle servers (with slightly different parameters). In one iteration, I couldn’t bind to the user. Another, the server using the test file failed to start. Be aware that Ladle will run happily with default parameters, but that they won’t be much use to you.

If you want to test your configuration file:

require 'net/ldap'
require 'ladle'

$ladle ||= Ladle::Server.new(
  :ldif => "spec/ldap/test_users.ldif",
  :domain => "dc=example,dc=org",
  :quiet => true
)
$ladle.start

ldap = Net::LDAP.new(host: 'localhost',
    :port => 3897,
)
filter = Net::LDAP::Filter.eq('mail', 'eadmin@example.com') # or ('uid', 'eadmin') 

ldap.search(:base => 'ou=people,dc=example,dc=org', :filter => filter) do |entry|
  ldap.auth(entry.dn, 'b44b44bl@cksh33p!') # or whatever your password is

  entry.each do |attribute, values|
    puts "   #{attribute}:"
    values.each do |value|
      puts "      --->#{value}"
    end
  end
end

Ruby Keyword Argument Implementation Differences Between 2.1 and 2.2

Posted: January 22, 2015 | Author: | Filed under: daily learning | Tags: | Comments Off on Ruby Keyword Argument Implementation Differences Between 2.1 and 2.2

I’m currently reading through Ruby Under a Microscope: An Illustrated Guide to Ruby Internals, and when I tried to test the “hidden hash” in ruby 2.2, I got different results from the book.

In Ruby 2.2, it appears that the default arguments are used as one expects, without regard to the Hash monkey patch:

### CODE ###
class Hash
  def key?(val)
    false
  end
end

def blah(a: 2, b: 5)
  puts a + b
end
blah
blah(a:3)
blah(b:3)
blah(a:2000, b:2000)

### DISASSEMBLY ###
== disasm: @>==========
0000 trace            1                                               (   1)
0002 putspecialobject 3
0004 putnil
0005 defineclass      :Hash, , 0
0009 pop
0010 trace            1                                               (   7)
0012 putspecialobject 1
0014 putspecialobject 2
0016 putobject        :blah
0018 putiseq          blah
0020 opt_send_without_block 
0022 pop
0023 trace            1                                               (  10)
0025 putself
0026 opt_send_without_block 
0028 pop
0029 trace            1                                               (  11)
0031 putself
0032 putobject        3
0034 opt_send_without_block 
0036 pop
0037 trace            1                                               (  12)
0039 putself
0040 putobject        3
0042 opt_send_without_block 
0044 pop
0045 trace            1                                               (  13)
0047 putself
0048 putobject        2000
0050 putobject        2000
0052 opt_send_without_block 
0054 leave
== disasm: @>========
0000 trace            2                                               (   1)
0002 trace            1                                               (   2)
0004 putspecialobject 1
0006 putspecialobject 2
0008 putobject        :key?
0010 putiseq          key?
0012 opt_send_without_block 
0014 trace            4                                               (   5)
0016 leave                                                            (   2)
== disasm: >================
local table (size: 2, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 2] val
0000 trace            8                                               (   2)
0002 trace            1                                               (   3)
0004 putobject        false
0006 trace            16                                              (   4)
0008 leave                                                            (   3)
== disasm: >================
local table (size: 4, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: 2@0, kwrest: -1])
[ 4] a          [ 3] b          [ 2] ?
0000 trace            8                                               (   7)
0002 trace            1                                               (   8)
0004 putself
0005 getlocal_OP__WC__0 4
0007 getlocal_OP__WC__0 3
0009 opt_plus         
0011 opt_send_without_block 
0013 trace            16                                              (   9)
0015 leave                                                            (   8)

### RESULTS ###
7
8
5
4000

If I reset back to ruby 2.1, I get the unconditional usage of the default arguments:

### CODE ###
class Hash
  def key?(val)
    false
  end
end

def blah(a: 2, b: 5)
  puts a + b
end
blah
blah(a:3)
blah(b:3)
blah(a:2000, b:2000)

### DISASSEMBLY ###
== disasm: @>==========
0000 trace            1                                               (   1)
0002 putspecialobject 3
0004 putnil
0005 defineclass      :Hash, , 0
0009 pop
0010 trace            1                                               (   7)
0012 putspecialobject 1
0014 putspecialobject 2
0016 putobject        :blah
0018 putiseq          blah
0020 opt_send_simple  
0022 pop
0023 trace            1                                               (  10)
0025 putself
0026 opt_send_simple  
0028 pop
0029 trace            1                                               (  11)
0031 putself
0032 putspecialobject 1
0034 putobject        [:a, 3]
0036 opt_send_simple  
0038 opt_send_simple  
0040 pop
0041 trace            1                                               (  12)
0043 putself
0044 putspecialobject 1
0046 putobject        [:b, 3]
0048 opt_send_simple  
0050 opt_send_simple  
0052 pop
0053 trace            1                                               (  13)
0055 putself
0056 putspecialobject 1
0058 putobject        [:a, 2000, :b, 2000]
0060 opt_send_simple  
0062 opt_send_simple  
0064 leave
== disasm: @>========
0000 trace            2                                               (   1)
0002 trace            1                                               (   2)
0004 putspecialobject 1
0006 putspecialobject 2
0008 putobject        :key?
0010 putiseq          key?
0012 opt_send_simple  
0014 trace            4                                               (   5)
0016 leave                                                            (   2)
== disasm: >================
local table (size: 2, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, keyword: 0@3] s1)
[ 2] val
0000 trace            8                                               (   2)
0002 trace            1                                               (   3)
0004 putobject        false
0006 trace            16                                              (   4)
0008 leave                                                            (   3)
== disasm: >================
local table (size: 4, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, keyword: 2@2] s0)
[ 4] a          [ 3] b          [ 2] ?
0000 getlocal_OP__WC__0 2                                             (   7)
0002 dup
0003 putobject        :a
0005 opt_send_simple  
0007 branchunless     18
0009 dup
0010 putobject        :a
0012 opt_send_simple  
0014 setlocal_OP__WC__0 4
0016 jump             22
0018 putobject        2
0020 setlocal_OP__WC__0 4
0022 dup
0023 putobject        :b
0025 opt_send_simple  
0027 branchunless     38
0029 dup
0030 putobject        :b
0032 opt_send_simple  
0034 setlocal_OP__WC__0 3
0036 jump             42
0038 putobject        5
0040 setlocal_OP__WC__0 3
0042 pop
0043 trace            8
0045 trace            1                                               (   8)
0047 putself
0048 getlocal_OP__WC__0 4
0050 getlocal_OP__WC__0 3
0052 opt_plus         
0054 opt_send_simple  
0056 trace            16                                              (   9)
0058 leave                                                            (   8)

### RESULTS ###
7
7
7
7

Programming Style: Language Specific Domains

Posted: December 27, 2014 | Author: | Filed under: programming | Tags: , , , | Comments Off on Programming Style: Language Specific Domains

My Beginner and C Days

The visual presentation of the code has mattered to me, from the first days I wrote C code in Microsoft editor. In those days, it was a Microsoft C 5.0 IDE on either DOS or OS/2.

#include "stdio.h"
main()
{

	int rows,columns,ROW, COLUMN;
	scanf("%d %d",&ROW,&COLUMN);
	rows = 0;
	columns = 0;
	while(columns++ < COLUMN)
		printf("\xdb");
	printf("\n");
	rows += 1;
	while(rows++ < (ROW - 2)){
		columns = 0;
		printf("\xdb");
		while(columns++ < (COLUMN - 2)){
			printf(" ");
			}
		printf("\xdb\n");
		}
	columns = 0;
	while(columns++ < COLUMN) {
		printf("\xdb");
		}


}

Ok, not *that* far back, but I did have strong opinions about C from the early days of my being paid to program in it:

  • TABS, not spaces
  • 3 (in Microsoft land) spaces per tab (eventually switched to 4)
  • always include braces for if clauses
  • generally no parentheses for return values
  • ternary operator GOOD

That made the code look like this (except that tabs are tabs in pre blocks).


int compare(const void *a, const void *b)
{
	return(strcmp(*(char **)a, *(char **)b));
}

int checkDictionary(char *word)
{
	if(!dictionary)
	{
		loadDictionary("WORD.LST");
	}
	if(bsearch((void *)&word, (void*)dictionary, wordCount, sizeof(char *), compare))
	{
		return 1;
	}
	return 0;
}

But I remember so much variation in coding style preferences, as evidenced by the Indent Style article on Wikipedia. Everyone had their own preferred style, and everyone had their own preference for how much everyone else should adhere to a common style. On one team I worked, they eventually developed code reviews which were nothing more than berating people for not adhering to an arbitrary style guide: an in-house version Hungarian Notation, exactly one return per function, etc.

Java and Perl days

In Java land, my preference for camelCasing was reaffirmed. Working with Perl and Java, I eventually started putting the opening brace on the same line as if statements and function declarations.

if(iAmAChangedProgrammer) {
   hugWithBraces();
}

I think the brace thing was a result of a block that evaluated differently in awk or perl if it didn't hug the clause. I don't remember, but from that point on, I was convinced of that way being the only way.

Ruby

Coming into ruby has been interesting. Multiple returns (which never bothered me) are common. Implicit returns are the norm as well. There's a style guide which, for the most part, experienced rubyists adhere to (even if there are slight variations in practice). I find some disagreement with some of the style recommendations (I think I have found legit use for and over &&, etc...), but I'm happy to fall in line with the style recommendations and help enforce them.

Some of the styles have obvious purpose, but some seem rather arbitrary. Why are guard clauses so accepted in Ruby when C was so scared of them, etc...

Smalltalk

In reading Smalltalk Best Practice Patterns, it all came together.

As part of the style guide near the end of the book, the author talks about using the line-wrapping of the browser window to naturally wrap argument lists instead of taking up vertical space with them. That's when it dawned on me how much preferences in code styles also come from the languages that influenced the current language.

Ruby's heritage from Smalltalk means that it carried forward that aversion to deep nesting that performs poorly in small code browser windows (even in Squeak and Pharo, the code windows can get pretty tiny.)

C coders may have come from GOSUB type languages where there truly was only one return (unless an IF clause buried another) and there could be multiple entry points. Multiple entry and exit point aversion came from languages in which you could GOSUB to anywhere before a RETURN statement, and a nested RETURN could be harder to detect than one on the main logic level.

Conclusion

Early language coding styles likely came from specific pain points experienced by a prior guru and then arbitrarily passed along because "We've always done it that way." At some point, tasks and readable code become hindered by following rules that we don't have a rationale behind.

Community style guides hopefully have the benefit of thousands of years of cumulative experience in challenges and best practices collaborating on equal footing, but that assumes that we're not just agreeing with our technical elders because that's the way it's always been done.

Try to understand the problem domain of coding in a specific language as it relates to coding style. Maybe your coding style thinks its still programming in COBOL or BASIC.

Mac OS X Mavericks, rbenv, and ruby-build “Missing the OpenSSL lib?”

Posted: September 19, 2014 | Author: | Filed under: daily learning | Tags: , , , , | Comments Off on Mac OS X Mavericks, rbenv, and ruby-build “Missing the OpenSSL lib?”

openssl version: 1.0.1i, rbenv version: 0.4.0, ruby-build as plugin in the ~/.rbenv/plugins directory.

The version of openssl on a Mavericks machine got out of sync with rbenv and ruby-build for some reason. For every attempt at installing a ruby version through rbenv attempted, we got the following message:

“The Ruby openssl extension was not compiled. Missing the OpenSSL lib?”

Tried every permutation of trying to fix one thing at a time, but ultimately ended up doing the following:

~/.rbenv/plugins/ruby-build directory
brew unlink openssl rbenv ruby-build
brew uninstall openssl rbenv ruby-build
brew update
brew install openssl
brew link openssl --force
brew install rbenv
eval "$(rbenv init -)"
brew install ruby-build

Then we were able to install whatever ruby version we wanted.

I’m sure some of the above process was overkill and unnecessary, but the steps in total did the job.

Refinements in Ruby: in map: super: no superclass method

Posted: August 14, 2014 | Author: | Filed under: daily learning | Tags: , , | Comments Off on Refinements in Ruby: in map: super: no superclass method

I was trying out refinements to see if they would help clean up some parsing code. I liked defining a couple of methods for the String class to respond to, but really didn’t want them as permanent monkey patches on String.

And so, I had a pipe mapping module with refinements:

module PipeMapping
  refine String do
    def only_lines_with_pipes
      self.lines.select { |s| s =~ /\|/ }
    end
    def pipes_to_array
      self.split('|').map(&:strip)
    end
  end
end

I added the following to my class to allow it to load itself:

  using PipeMapping
  def self.initalize_mappings
    data_to_load.only_lines_with_pipes.map(&:pipes_to_array).each do |mapping|
      # do stuff
    end
  end

I got the following output:

...in `map': super: no superclass method `pipes_to_array' for "A|B|C\n":String (NoMethodError)

The problem is that Symbol#to_proc will end up operating outside of the scope of the refinement using invocation, and will, therefore, have no knowledge of the method.

A quick change to a block works:

  using PipeMapping
  def self.initalize_mappings
    data_to_load.only_lines_with_pipes.map{|s| s.pipes_to_array}.each do |mapping|
      # do stuff
    end
  end

I’m still trying to decide if this is a satisfactory trade-off.

Serving a file from your Rails app from another server protected by basic authentication

Posted: May 28, 2014 | Author: | Filed under: daily learning | Tags: , , | Comments Off on Serving a file from your Rails app from another server protected by basic authentication

Interesting problem: Retrieve a document from one web service protected by basic auth (or some authentication mechanism) and serve it via a public link on another website.

It ended up boiling down to this code in the controller (reduced to be in the same method instead of factored out as it was.)

  def get_download
    response = Net::HTTP.start('localhost', '3010', {} ) do |http|
      request = Net::HTTP::Get.new('http://localhost:3010/my_document.pdf')
      request.basic_auth 'username', 's3cretpassword'
      http.request(request) do |response|
        send_data(response.body, filename: 'set_filename.pdf')
      end
    end
  end

Nesting resource routes with alternate scopes and keys + strong parameters.

Posted: May 15, 2014 | Author: | Filed under: daily learning | Tags: , , , , , | Comments Off on Nesting resource routes with alternate scopes and keys + strong parameters.

DISCLAIMER:
This is a contrived example translated from a case where it actually made way more sense to break the familiar nested resource pattern. Also, I haven’t double-checked how Rails’ pluralization rules would affect the naming of any given Species.

Say you have a pair of models:

class Species
  # id
  # latin_name_with_underscores
end

class Animal
  # name
  # species_id
end

Normally, you’d nest resource routes in your routes.rb file something like:

resources :species do
  resources :animal
end

And your routes would look something like:

/species/:species_id/animals
/species/:species_id/animals/:animal_id

But… what if you wanted different nesting, resource ids, and route names?

namespace :latin_species do
  scope '/:latin_name' do
    resources :animals
  end
end

Now you have:

/latin_species/:latin_name/animals
/latin_species/:latin_name/animals/:id

In your controller app/latin_species/animals_controller.rb, you can now do the following to set the appropriate parent and child:

class LatinSpecies::AnimalsController < ApplicationController
  before_action :set_species_and_animal
#
#
#
  def set_species_and_animal
    @species = Species.find_by(latin_name_with_underscores: params[:latin_name])
    @animal = Animal.find(params[:id]) if pararms[:id]
  end
end

The convenience of Ruby 2.1 def returning a symbol instead of nil

Posted: May 12, 2014 | Author: | Filed under: daily learning | Tags: , | Comments Off on The convenience of Ruby 2.1 def returning a symbol instead of nil

Prior to 2.1, if you defined a method, you received nil as a return value:

irb(main):001:0> def fun
irb(main):002:1> end
=> nil

With 2.1, the method definition returns a symbol for the method name:

irb(main):001:0> def fun
irb(main):002:1> end
=> :fun

One impact of this is that the following pattern for declaring a single private method

  def fun
  end
  private :fun

can become:

  private def fun
  end

Now to decide if I like how that looks.

← Older Entries