Posted on

Go Concurrency Benchmarking on a Raspberry Pi

In hindsight, no one claimed that a Raspberry Pi of any model was a powerhouse machine. However, I had thoughts about how to benchmark computing power after leaving Dreamhost and noticing that even tar and gzipping my WordPress installs performed like an ancient machine despite the processor statistics. That made me think about a raw benchmarking metric that could deployed anywhere. I was also learning Go and trying to understand Go concurrency. I ended up with a "concurrency" of 2 because the lower power systems didn't get any more out of an additional goroutine.

The program requires 13,135,013 sha3 hashes and conversions to base 64 to come up with a string that starts with "TEST".

My work workstation's output:

[Specs: Late 2013 Retina MacBook Pro 15", 2.3 GHz Core i7 - (I7-4850HQ)]

  • Single thread: 16 seconds
  • Concurrency of two: 10 seconds
  • Compared with PassMark:
    • 9066
    • Single thread 1964

A Late 2011 MacBook Pro 13", 2.4 GHz Core i5 (I5-2435M):

  • Single thread: 34 seconds
  • Concurrency of two: 19 seconds
  • Compared with PassMark:
    • 3269
    • Single Thread 1347

A Core i3-5010U 2.10 GHz NUC (Windows 10 Pro)

  • Single thread: 27 seconds
  • Concurrency of two: 17 seconds
  • Compared with PassMark:
    • 3060
    • Single Thread 1171

A Mid 2009 MacBook Pro 13", 2.26 GHz Core 2 Duo (P7550)

  • Single thread: 57 seconds (Originally was 83 on go1.3.3, but better results with 1.8.3)
  • Concurrency of two: 32 seconds (Originally was 80 on go1.3.3)
  • Compared with PassMark:
    • 1521
    • Single Thread 892

A ThinkPad 11e with A4-6210 APU (1.8 GHz - Windows 10 Home)

  • Single thread: 135 seconds
  • Concurrency of two: 65 seconds
  • Compared with PassMark:
    • 2143
    • Single Thread 697

Raspberry Pi 3B

  • Single thread: 1265 seconds
  • Concurrency of two: heat warnings!

For the purposes of this "performance" benchmark, there is definitely a non-linear relationship between a canned benchmark score and the score in a brute force operation that has a heavy calculation component. I'm also isolating only one aspect of system performance. There is a SATA SSD on the newest machine, a SATA HDD on the 2011 and 2009 machines, and an SD card on the Raspberry Pi. The RAM is different in every machine as well. Still, it was a fun experiment that motivated me to spend a little extra time in Go and learning Go Concurrency.

Posted on

Whitespace is Significant

While I was in my first couple years of college, Python was gaining traction as a hot new programming language. I had been programming in C since high school, but was curious about new languages. I took an interest in learning more about Python. A classmate told me that whitespace (at least indentation) was significant in Python. I changed my mind.

The Holy Wars of Indentation Style

It's interesting that would've been so turned off by a language that placed significance on whitespace and indentation, when I had fantasized about holy wars revolving around C indentation style (Horstmann is the only way!) and tabs vs. spaces (tabs, tab width 4!). I wretched at anything in the GNU C coding style (okay, I still do...), but the funny thing was, I never was able to program in a consistent style: There were other co-workers who had a slightly different preferred style. I was also learning myself, so relative importance of information in code changed, and my opinions on bracing style evolved.

Non-C Languages that Use Curly Braces for Things

It was a combination of Java, JavaScript, Perl, and Ruby that altered my perception of "proper" bracing style (BUT NOT SPACES OVER TABS!). A new line after a conditional or function declaration/call before the brace either took up unnecessary amounts of space, or changed the meaning of the code altogether (e.g., blocks in ruby, for which adding a newline = syntax error). And so, I evolved into declaring that 1TBS style was the only proper way to write C (WITH TABS!)

Ruby

Ruby has a generally accepted coding style that is enforced by no one except for code reviewers and maybe draconian RuboCop build configurations. I'd link to the coding style, if I had any certainty that the one I grabbed was 99% correct. Even RubyMine doesn't format code to the same standards as RuboCop.  (Continuation indent and spacing around braces were two settings I had to tweak.)

Small inconsistencies aside, Ruby code has ended up being the most consistent—visually—to support and add code to. I've introduced tighter RuboCop configurations on newer projects to try to adhere more closely to generally accepted coding style for Ruby.

Ruby also introduced me to an acceptance of significant whitespace. While C code often gets sloppy with spacing around punctuation markers in code, Ruby developers take more care in proper whitespace around the conditionals, braces, etc. Some C/C++ projects like Mozilla do similar standards. However, Ruby is the first language that I've seen a community-wide interest in code that presents in the same manner. Of course, most of this code isn't syntactically significant, but it is of human significance--and humans have their own mental compilers that optimize on standard patterns.

Still, this isn't the largest impact Ruby has had on my opinion of significant whitespace. Through Ruby, I've been exposed to YAML, HAML, and Slim. (I have dibs on forming a band of Rubyists with that name!) All three languages are unforgiving about inconsistent indentation. They take out explicit expression of boundaries and create visually implicit boundaries via hierarchies of indents. You don't need to look for an end or closing brace to know that the context has moved up a level. The indentation tells you where that happens.

(And yes, because of the generally accepted coding style in Ruby, I use 2 spaces instead of tabs now.)

Golang

And yet, Ruby coding style is informal. You can code in improper style and still commit your code and/or run it. There's something about having the freedom to write improper style and being able to be a good "citizen" on your own that is comforting.

Go does not do this. Writing Go in any editor configured to code in it (I've been using the vim-go plugin) automatically formats via gofmt.  It's pretty unsettling. I longed for the ability to become dictator over coding styles, and then was okay when Ruby dictated one, but gave us the autonomy to do the right thing. This auto-formatting? This is tyranny!

But, perhaps, it's time. We can mentally chunk pieces into larger concepts for storage, if code that does the same thing looks the code. Instead of having to remember every brace as an individual piece, we can put "for loop that iterates and prints" as a single chunk.

A stray brace isn't going to permanently prevent distillation into larger chunks; it will just take a higher cognitive load to get there. This in turn will require more experience with the language before a programmer can make that leap. Being able to break things into large chunks helps us discover the what instead of dwelling on the how.

Posted on

Migrating WordPress Install From Dreamhost to Linode

Migrating hosts has always been a tedious process for me, and I've often stayed on horrible hosting providers just because I've dreaded it. Linode (referral link) and similar providers at least have a wealth of information on how to set up your new hosts, but I still found myself jumping around a lot to remember how to do certain things. Hopefully, this post helps put (most) of the information in one place.

Experience level required: You have logged into your own Linux box and have done *something* useful.

Grabbing your content from Dreamhost

Login using the shell username associated with your domain name that you're moving. Assume the domain name is example.com.

Grab the WordPress install complete with content using tar.

I'm using the domain name as part of the tar file name to keep things straight, but the name before the .tar file can be whatever works for you. (Note: you can be more surgical and just grab the content and have a fresh WordPress install, but then you'll be chasing after even more loose ends.)

tar pcvf example.com.tar example.com

Dump your mysql database.

 

Open your wp-config.php for you domain and note your DB_NAME, DB_USER, DB_PASSWORD, and DB_HOST.

Run the following mysqldump command line:

  • DB_NAME as the first argument
  • DB_HOST after the --host argument
  • DB_USER as the -u argument.
  • The -p argument will cause mysqldump to prompt you for a password.
  • Redirect output (> filename) to whatever file you want, but again, I'm using something like "domain.name.sql" for consistency and clarity in my move.

Transfer your files down.

scp username@dreamhost_host_name:~/example.com.tar .
scp username@dreamhost_host_name:~/example.com.sql .

Sign Up for Linode

I'd appreciate you signing up through my referral link for Linode . I chose the Linode 2048 Standard Instance , Deployed an Ubuntu 16.04 LTS image to it, and started it up.

From there, you need to set up the LAMP stack. Note: these instructions will install PHP 7, and I found at least one plugin (Sociable?) that used deprecated calls in php and caused me a few 500 errors.

Add non-root user

Be sure to also follow Securing Your Server recommendations for automatic updates and such, but for our purposes:

adduser example_user
adduser example_user sudo

Upload Your Site to Linode

scp example.com.tar example_user@linode_instance_ip:~
scp example.com.sql example_user@linode_instance_ip:~

Enable .htaccess if you use any rewrites

/etc/apache2/apache2.conf (AllowOverride)  It's possible you *don't* need this if you have WordPress, but I haven't had an install that didn't.

Enable apache mods

sudo a2enmod ssl rewrite mpm_prefork

Add other PHP and related mods that your plugins may need

sudo apt-get install php7.0-mbstring

sudo apt-get install php7.0-gd sudo apt-get install php7.0-curl php7.0-json php7.0-cgi sudo apt-get install php7.0 php-pear libapache2-mod-php7.0 sudo apt-get install php7.0-mysql

Set up your virtual host

/etc/apache2/sites-available/example.com.conf

Create subdirs and set permissions properly

sudo mkdir /var/www/html/example.com

sudo mkdir /var/www/html/example.com/{logs,public_html} sudo chown -R www-data:www-data /var/www/html/example.com

Untar your content

Because of the way Dreamhost has their domain subdirectories set up, you'll end up with two layers of example.com folders. (You could have also tarred from the folder, etc... but I just decided to keep this pattern.)

cd /var/www/html/example.com
sudo -u www-data tar xvf ~/example.com.tar .
cd example.com # from the tar file
sudo -u www-data mv .htaccess ../public_html
sudo -u www-data mv * ../public_html
ls -a # this should be empty now.
cd ..
pwd # be sure you're in /var/www/html/example.com
rm -rf example.com

Setup and Load Your DB

cd to the location of your sql dump.

CREATE DATABASE wp_example_com;
CREATE USER 'examplecom' IDENTIFIED BY '!@(87P@ss';
GRANT ALL PRIVILEGES ON wp_example_com.* TO 'examplecom';
USE wp_example_com;
SOURCE example.com.sql;

Change your wp-config.php for the DB setup

Your DB host is now localhost (unless you created a new DNS entry for your DB host)

Enable your site

sudo a2ensite example.com

Add a DNS zone in Linode and change your DNS servers to point to Linode.

Add SSL

For more details start at "Install Let's Encrypt to Create SSL Certificates"

I had to experiment a bit with the SSL configuration to get Let's Encrypt to generate the keys and certs.

Lastly... I had to add renewal of the SSL certs to my crontab (sudo crontab -e)

 

Posted on

Example of a Sincere Attempt at Security Questions That Still Goes Badly

Synchrony Bank has the following security questions when setting up an account. They look well thought out, at least. The only problem? All of the answers are required to be 6 characters or longer. There are plenty of proper names or other valid answers that don't meet that requirement.

2 of the answers I can readily recall from this list don't meet the requirement. Half of them I don't have an answer for, and at least one of the answers is public information.

This one is bad... seven of my answers here are 3-5 characters, two don't have an answer for me.

Again, 4 out of 10 are 4-5 characters, 1 doesn't have an answer, and 1 of them I wouldn't spell the same way twice. Also: Ford? Dodge? Kia? Honda? Lexus? GMC? AMC?

Beyond all this, programmers need to read Falsehoods Programmers Believe About Names. ...and anyone answering security questions just needs to store the answers to the security passwords as yet another generated password using something like https://1password.com/ (although I had to limit the symbols that my own password manager would use to generate because slashes weren't allowed.)

Addition nuisance:

I *just* registered in this same exact tab 20 minutes ago.

Posted on

Lua Script to Start, Unhide, and/or Auto-Arrange

I'm starting to try using HammerSpoon, which is driven by Lua, for automatically arranging my applications depending on application scenarios.

I have 3 screens, including a MacBook Pro at Retina native resolution and a Seiki 4K display. I grabbed the names of the monitors from Display Menu's menu bar menu.

I initially made the mistake of looking for "Outlook", "OneNote", and "Chrome", but all three applications require their respective company's names to be included in the application name.

Posted on

Hammerspoon Lua Script - Split Windows 3x3

I've been using using Spectacle pretty heavily, but with a larger screen, I always felt like quadrants were a poor use of real estate, so I began to crack open the source code, which was pretty challenging to navigate because it's a mix of JavaScript and Objective-C reminiscent of user interface programming from ages past. So, I turned next to Hammerspoon.

Hammerspoon uses Lua as a programming interface. I was able to automate moving windows around a 3x3 grid on the current screen with 18 lines of code.

Posted on

HTML5 support and the latest browsers

I got an email update from Microsoft on their upgrading of Bing Rewards to "Microsoft Rewards", which includes using Microsoft Edge as another way to earn rewards. My first thought was that that wouldn't be too bad for web developers who use multiple browsers, anyway. But, of course, Edge is quite a bit more advanced than the Internet Explorer versions that people are still using.

So I was curious how each browser stacked up, and went to html5test to check on current compatibility.

Browser versions:

  • Chrome Version 51.0.2704.106 (64-bit)
  • Firefox 48.0
  • Safari Version 9.1.2 (10601.7.7)
  • Edge (Okay, I haven't figured out how to find version info on Edge yet)

HTML5Test overall scores (all features in categories are given pretty much equal weight)

  • Chrome: 492/555
  • Firefox: 461/555
  • Edge: 433/555
  • Safar: 370/555

While Firefox looks like number 2 overall, the form elements features in HTML5.

  • Chrome: 64/65, only missing Directory Upload support (no browser listed has it)
  • Edge: 58/65, missing an output element, input type=color.
  • Firefox: 44/65, but missing all the date type inputs, including type=date itself.
  • Safai 39/65<, with similar gaps + datalist element./li>

I also decided to run all four against Acid3:

  • Safari: 100/100, mostly smooth
  • Edge: 100/100, but pregnant pauses and artifacts in the intermediate stages.
  • Firefox: 99/100, completely smooth until it stopped.
  • Chrome: 97 - 99/100. One artifact in the upper left corner, inconsistent results.

Conclusion:

There probably won't be much benefit in using Edge as far as uncovering problems traditionally associated with IE, but it is yet another browser that behaves "differently", yet not nearly as frustrating as old Microsoft browsers.

Posted on

RailsSettings vs parallel_tests

RailsSetting stubbing problem

I have an area of functionality that is controlled by a Settings class that is a subclass of RailsSettings::CachedSettings (rails-settings-cached gem) and changing that setting for tests was interfering with the parallel_tests gem when I ran my rspec tests.

I tried namespacing the cache, making TTL 0, invalidating the cache, monkeypatching the lookup... Every convoluted solution possible... to try and get tests to play nicely together.

The humbling thing about programming is how simple actual solutions are compared the things the brain comes up with.

The solution? Wrap the access to the Setting in a layer of abstraction (which I had partially done already.)

class ThatFunctionality
  def self.enabled?
    Setting['that_functionality.enabled']
  end
  def self.enabled=(value)
    Setting['that_functionality.enabled']=value
  end

And then in tests:

before do
  allow(ThatFunctionality).to receive(:enabled?).and_return(true) # or false
end

Trying again to stub RailsSettings

The class method approach is explicit, but once a large set of settings is needed, it can get unwieldy. If you haven't found a better strategy for wrapping settings values, you can go back to stubbing the :[] method. The caveat is that you need to save the original Settings behavior for any settings that you want to leave unchanged. By stubbing the :[] first to passthrough to .and_call_original, you can add additional stubs on top.

Posted on

Using an Arduino to keep a Seiki 4K Display On

I love my Seiki 4k Display for writing code on my Mac. The one downside is that it times out and goes to sleep after 2 or 4 hours of inactivity (from the remote, not the screen). I decided to fix this problem in a somewhat complicated way: With an Arduino Uno that I already had, and an IR control kit.

I used the IR receiver to find the Seiki "Volume Up" and "Volume Down" codes, which appear to be structured like NEC codes. Those are hardcoded because I haven't really bothered to refine the setup or code any more. The IRBlaster library uses digital out 3 for the IR sends, and I used digital 8, 9, and 10 for red, yellow, and green indicators to give some indication of where in the refresh cycle things are. The code as-is sends the volume up and volume down signals every hour or so, and the indicators start with a slow blink on red, a slightly faster blink on yellow, and a fast blink before remote send on green.

IR Blaster set up (with IR receiver partially wired in still)
IR Blaster set up (with IR receiver partially wired in still)
#include <IRremote.h>
 
IRsend irsend;
int led = 13;
int red = 8;
int yellow = 9;
int green = 10;
 
void setup()
{
  Serial.begin(9600);
  pinMode(led, OUTPUT);
  pinMode(red, OUTPUT);
  pinMode(yellow, OUTPUT);
 
  pinMode(green, OUTPUT);
}
 
 
int choice(int i) {
  if(i > 1300) {
    return red;
  } else if (i > 900) {
    return yellow;
  }
  return green;
}
 
void ledOn(int i) {
  digitalWrite(choice(i), HIGH);
}
void ledOff(int i) {
  digitalWrite(choice(i), LOW);
}
 
void loop() {
  while(true) {
    irsend.sendNEC(1086232679, 32); // seiki volume up
    delay(40);
    irsend.sendNEC(1086206159, 32); // seiki volume down
    for(int i=1800; i >=0; i--) {
      ledOn(i);
      delay(i);
      ledOff(i);
      delay(i);
    }
  }
}
Posted on

2015 in Review

Monthly Summary

January

  • Started work on a solo project at work that pretty much gave me lots of experience building something from end to end.
  • Working from home 2/3rds of the time to work out any kinks for a possible move to Florida.
  • Thank yous going out to people who helped fund Emily's medical mission to Kenya.

February

  • Emily is gone for 10 days on a trip to Africa.
  • I have to dress the kids by myself exactly one day of her absence.
  • ...because the kids were off school due to snow the entire following week.

March

  • The small solo project goes into "User Experience auditing" mode and I get help actually meeting the demands.
  • The kids were off school due to snow (for entire week?) again.

April

  • Painting, final prep for selling the house.
  • Spent a week with mostly no cell signal in St George Island

May

  • Broke news to parents officially that we were moving.
  • Sold the house then listed it. (At least that was the order the offer versus official listing happened in.)

June

  • Packed up the house, closed on the house, vacated the house, and flew off for the Philippines in a span of 5 days (with our 12th wedding anniversary in between.)
  • Last times accompanying at various churches in Louisville.
  • Went to the Daluyon resort on the island of Palawan in The Philippines.

July

  • Celebrated my grandmother's 100th birthday with the rest of the family in The Philippines.
  • Realized the madness that is flying to/from and traveling in a foreign country with very picky eaters as kids.
  • Flew back to the US, tried to adjust to jetlag in 2 days, drove our belongings not packed in a trailer to Florida in two cars (one with a broken A/C).
  • Scrambled to find a rental house that accepted a big dog in the school district we wanted, signed a lease, moved in (in a week from arriving in Florida)
  • Officially began my 100% remote work life.

August

  • Sister-in-law brought our dogs down.
  • Kids started at their new school.
  • I wilted on runs in the Florida sun and heat.
  • Transferred our licenses and cars to Florida officially.
  • Went to beaches

September

  • Emily waited around on callbacks from job applications.
  • Joined in with the Cathedral music program.
  • Went to beaches

October

  • Actually saw a decent amount of the area on foot--running.
  • Not so much with the beaches.
  • Emily started a new job!
  • Olivia made All County Chorus

November

  • Failed at Monkey (intentionally bailed at 11.5 miles, may have missed my bail out point a couple times.)
  • Drove back to Louisville, had good times, but way more hectic than expected.

December

  • Apparently the A/C can mostly not run in December in Florida.
  • Went back to Louisville for Christmas.

Media

Music

  • Quite a bit of Hamilton the Musical.
  • Surprisingly less music listened to at home office than on site. Strange.

Books

  • 72 Books. Getting into Hamilton the Musical led me Chernow's biography of Hamilton. Excellent book, very dense read.
  • Childhood's End by Arthur C. Clarke and the Space Odyssey series were also exceptional (though I was a bit disappointed in 3001).
  • In following the post-apocalyptic/YA-ish series reads of past years, I got into the Extinction Point series by Paul Antony Jones
  • ...less impressive was the Defiance trilogy (CJ Redwine), probably due to the believability of an alien race doing astounding things versus ordinary humans accomplishing futuristic feats in a world thrown back to the Dark Ages.