Rails’ and Ruby’s Hash transform_values

Rails: 4.2.1-5.2.3 and Ruby >= 2.5.5 have a transform_values method on Hash that allows you to pass a block to the method and transform the values of the key-value map in the hash based on the block contents. Essentially, it’s map but for the Hash values only, and with no weird Hash/array-element syntax. From the linked API documents above:

h = { a: 1, b: 2, c: 3 }
h.transform_values {|v| v * v + 1 }  #=> { a: 2, b: 5, c: 10 }
h.transform_values(&:to_s)           #=> { a: "1", b: "2", c: "3" }
h.transform_values.with_index {|v, i| "#{v}.#{i}" }
                                     #=> { a: "1.0", b: "2.1", c: "3.2" }

NoMethodError undefined method `shared_examples_for’ for main:Object for bundle gem rspec

If you create a gem stub using bundle gem thing and select rspec as your test suite, you may get an error similar to the following:

❯ bundle exec rspec

An error occurred while loading ./spec/thing_spec.rb.
  shared_examples_for 'saying hello' do
    puts "hi"

  undefined method `shared_examples_for' for main:Object
# ./spec/thing_spec.rb:1:in `<top (required)>'
No examples found.

whenever using describe, shared_examples, and shared_examples_for, etc…

After a lot of diving into rspec source code to verify where shared_examples_for was defined (rspec-core so…) I noticed the following code in the stubbed spec_helper.rb:

  # Disable RSpec exposing methods globally on `Module` and `main`

If you comment out config.disable_monkey_patching!, then those methods will be included at a top level.

Ruby: Enumerable grep, grep_v

UPDATE: After I wrote this, I started finding myself doing a lot of caller.grep(/(project_directory|suspected_gem)/) to aid in debugging obscure interactions with internal gems and projects.

In looking at a pull request and noticing some awkward “first” and “last” iteration detection which also required each_with_index, I started looking into what would be a cleaner way, and my first step was trying to figure out if there was an enumerable context.

Ultimately I landed on Enumerable#grep and Enumerable#grep_v, which somewhat perplexed me. It’s not really a “grep” unless your collection’s values respond to that:

irb(main):056:0> 1000.times.map(&:to_s).grep(/00/)
=> ["100", "200", "300", "400", "500", "600", "700", "800", "900"]

Maybe you want to look for Classes in the ObjectSpace… The argument to #grep is compared against an implicit element for each iteration with the === operator. So you could list everything in the ObjectSpace that’s a Class:

ObjectSpace.each_object.grep(Class) # too long to include here

One situation that I thought of that might be especially useful is Dir globbing:

irb(main):064:0> Dir['*'].grep(/(yarn|json)/)
=> ["package-lock.json", "package.json", "yarn.lock"]

If you were trying to the Ruby REPL as a shell, you could even:

irb(main):005:0> Dir['*/'].grep_v(%r{/packs/}).grep(/(.js$|.erb$|.rb$|.json$|.lock$)/)
=> ["app/controllers/application_controller.rb", "app/controllers/posts_controller.rb", "app/helpers/application_helper.rb", "app/helpers/posts_helper.rb", "app/javascript/src/jets/crud.js", "app/jobs/application_job.rb", "app/models/application_item.rb", "app/models/application_record.rb", "app/models/post.rb", "app/views/layouts/application.html.erb", "app/views/posts/edit.html.erb", "app/views/posts/index.html.erb", "app/views/posts/new.html.erb", "app/views/posts/show.html.erb", "app/views/posts/_form.html.erb", "babel.config.js", "config/application.rb", "config/environments/development.rb", "config/environments/production.rb", "config/environments/test.rb", "config/routes.rb", "config/webpack/development.js", "config/webpack/environment.js", "config/webpack/production.js", "config/webpack/test.js", "db/migrate/20210610023540_create_posts.rb", "db/schema.rb", "Gemfile.lock", "postcss.config.js", "spec/controllers/posts_controller_spec.rb", "spec/fixtures/payloads/posts-index.json", "spec/fixtures/payloads/posts-show.json", "spec/spec_helper.rb"]

And, as with .each, .map, etc… you can pass a block interact with each element. Your return from each iteration will map back to the output.

# array containing the contents of all the files matching:
Dir['*/'].grep_v(%r{/packs/}).grep(/(.js$|.erb$|.rb$|.json$|.lock$)/) { |f| File.open(f).read }

I’m not sure if I’ve seen any code that would be made cleaner by grep (unless for utility scripting), and there’s always the risk of lowering maintainability of the code by using features no one else uses, but it’s nice to know that Ruby always has something more even after using it for many years.

DKIM for Cloudflare and Fastmail

After creating the CNAME fm#._domainkey records in Cloudflare DNS Management, I was still getting “DKIM is not configured”. Initially I thought it was a TTL issue on the DNS records…

After playing around with dig fm1._domainkey.DOMAINNAME TYPE CNAME I went back to check the settings on the CNAME records:

Proxied CNAME records!

I went in to edit those to be DNS Only

Now my Fastmail DKIM is correctly configured

Ruby on Jets Webpacker errors and Invalid Configuration Object

The problem

tl;dr to the solution that worked for me

I was trying to get Ruby on Jets up and running and ran into webpacker errors including “CLI for webpack must be installed” (and others) all the way to “Invalid configuration object”.

On creating a new app with jets (3.0.8) with npm 7.11.2, node 16.1.0, and yarn 1.22.5 and creating a basic blog CRUD app like follows:

 jets new blog_app --database=postgresql
 cd blog_app
 jets generate scaffold post title:string post:text
 jets db:create
 jets db:migrate
 jets server

I get an error with browsing to localhost:8888/posts:

ActionView::Template::Error at /posts
Webpacker can't find application.js in /mnt/c/Users/twill/projects/jets/blog_app/public/packs/manifest.json. Possible causes: 1. You want to set webpacker.yml value of compile to true for your environment unless you are using the `webpack -w` or the webpack-dev-server. 2. webpack has not yet re-run to reflect updates. 3. You have misconfigured Webpacker's config/webpacker.yml file. 4. Your webpack configuration is not creating a manifest. Your manifest contains: { }

On the back end, I could see:

[Webpacker] Compilation failed:
warning package.json: No license field
CLI for webpack must be installed.
webpack-cli (https://github.com/webpack/webpack-cli)
We will use "npm" to install the CLI via "npm install -D webpack-cli".
Do you want to install 'webpack-cli' (yes/no):

Adding dependencies one by one

I added webpack-cli with yarn add webpack-cli and:

[webpack-cli] Failed to load '/mnt/c/Users/twill/projects/jets/blog_app/config/webpack/development.js' config
[webpack-cli] Error: Cannot find module '@rails/webpacker'

Ok, add yarn add @rails/webpacker and run jets server and reload the page:

[Webpacker] Compilation failed:
 warning package.json: No license field
 [webpack-cli] Invalid configuration object. Webpack has been initialized using a configuration object that does not match the API schema.
 configuration.node should be one of these: false | object { __dirname?, __filename?, global? } -> Include polyfills or mocks for various node stuff. Details: configuration.node has an unknown property 'dgram'. These properties are valid:
 object { __dirname?, __filename?, global? }
 -> Options object for node compatibility features.
 configuration.node has an unknown property 'fs'. These properties are valid:
 object { __dirname?, __filename?, global? }
 -> Options object for node compatibility features.
 configuration.node has an unknown property 'net'. These properties are valid:
 object { __dirname?, __filename?, global? }
 -> Options object for node compatibility features.
 configuration.node has an unknown property 'tls'. These properties are valid:
 object { __dirname?, __filename?, global? }
 -> Options object for node compatibility features.
 configuration.node has an unknown property 'child_process'. These properties are valid:
 object { __dirname?, __filename?, global? }
 -> Options object for node compatibility features.  

That led me eventually to Webpack 5: configuration.node has an unknown property ‘dram’. There properties are valid: issue comment on GitHub, with the mention of rails webpacker and that “Rails’ webpacker 5.x.x is only compatible with webpack 4.x.x”

The Solution

After lots of teardowns and tweaks to the project to troubleshoot, I finally ended up with a sequence that produces a scaffold for posts that renders with webpack:

jets new blog_app --database=postgresql
cd blog_app
jets generate scaffold post title:string post:text

# jets db:drop # used while experimenting with yard packages below
jets db:create
jets db:migrate

# this is the key line that makes the currently installed @rails/webpacker work
yarn add webpack@4 
yarn add webpack-cli
yarn add @rails/webpacker
jets server

Automatically Moving Files Between S3 Buckets with Lambda (part 1)

I have an S3 bucket that I want to attach to an application’s upload area, but I want to move them out of the bucket accessible to the application after they’ve been uploaded. Eventually, I want to have this be after a small delay, but initially I wanted to test out the concept itself.

Step 1: Have source and destination buckets in S3

Create buckets for source and destination. The ACLs on both of the buckets are the same (non-public) in my case.

Step 2: Create a Lambda Execution Role

  • Go to IAM > Roles > Create Role
  • Choose Lambda as a Use Case
  • Next: Permissions
  • Search for S3 and check AmazonS3FullAccess
AmazonS3FullAccess selection
  • Search for “lambdabasic” and check AWSLambdaBasicExecutionRole (for CloudWatch logs)
AWSLambdaBasicExecutionRole selection
  • Click [Next: Tags] > [Next: Review] and give your role a name and verify that the S3 and Lambda policies are added:
Verify policies and name role
  • Click [Create Role]

Step 3: Prep the Lambda Code

  • Clone https://github.com/stringsn88keys/move_file_on_upload
  • Be sure to have the correct ruby version (2.7.0 at the time of writing) installed
  • Change into move_file_on_upload folder
  • bundle install locally
  • bundle config set --local deployment 'true' to vendor the gems for the AWS Lambda
  • zip ../move_file_on_upload.zip ** to package the zip

Step 4: Create the Lambda

  • Go to AWS Lambda in the AWS Console and click [Create Function]
  • Name the function, set Ruby 2.7 as the runtime, and use the role you created
Function name, Runtime, and Permissions
  • [Create function]

Step 5: Add S3 Trigger

  • Click [+ Add Trigger]
  • Search and select S3
  • Fill in your source bucket and select all object create events
  • If you get this error (“configurations overlap”), select your bucket in S3, click the Properties tab, and you’ll see an Event Notifications that’s been orphaned by a previous config (be sure to delete the dependent Lambda as well if it exists):
Configurations overlap error

Step 6: Upload your code

  • Go back to the [Code] tab for your lambda and select the [Upload from] dropdown and select zip file.
Upload .zip file
  • Go to the [Configuration] section and add a FROM_BUCKET and TO_BUCKET environment variable for your Lambda to know what to process

Step 7: Monitor and test

  • You can test the Lambda execution via the Test dropdown
  • S3 put is one of the available test templates
  • Click “Create” after selecting S3 Put and you’ll be able to watch the event get executed.
  • Go to CloudWatch -> Logs -> Log Groups and you should see a log group for /aws/lambda/your_function_goes_here
  • If all else is successful, you should see “the specified key does not exist”

Step 8: Test it live

  • Create a folder in your source bucket.
  • Go into that folder
  • Upload a file.
  • The file should pretty quickly be copies to the destination bucket in the same folder structure and removed from its original location.
  • The top level folder under the bucket should remain.