Updates from July, 2017 Toggle Comment Threads | Keyboard Shortcuts

  • ThomasPowell 5:06 pm on July 22, 2017 Permalink
    Tags: go concurrency, ,   

    Go Concurrency Benchmarking on a Raspberry Pi 

    In hindsight, no one claimed that a Raspberry Pi of any model was a powerhouse machine. However, I had thoughts about how to benchmark computing power after leaving Dreamhost and noticing that even tar and gzipping my WordPress installs performed like an ancient machine despite the processor statistics. That made me think about a raw benchmarking metric that could deployed anywhere. I was also learning Go and trying to understand Go concurrency. I ended up with a “concurrency” of 2 because the lower power systems didn’t get any more out of an additional goroutine.


    package main
    import (
    "encoding/base64"
    "fmt"
    "golang.org/x/crypto/sha3"
    "strings"
    "time"
    )
    func main() {
    var before string
    var after [64]byte
    var base64Encoding string
    var result chan string = make(chan string)
    var resultString string
    var addOnStart int64
    start := time.Now()
    addOn := 0
    concurrency := 2
    sem := make(chan bool, concurrency)
    for {
    before = fmt.Sprintf("Message%d", addOn)
    after = sha3.Sum512([]byte(before))
    base64Encoding = base64.StdEncoding.EncodeToString(after[:])
    if strings.HasPrefix(base64Encoding, "TEST") {
    fmt.Printf("%d seconds\n", int64(time.Since(start)/time.Second))
    break
    }
    addOn++
    }
    fmt.Printf("%d: %s\n", addOn, base64Encoding)
    addOnStart = 0
    start = time.Now()
    SEARCHY:
    for {
    sem <- true
    go scan1000000(addOnStart, result, sem)
    select {
    case resultString, _ = <-result:
    break SEARCHY
    default:
    }
    addOnStart++
    }
    fmt.Printf("%d seconds\n", int64(time.Since(start)/time.Second))
    fmt.Print(resultString)
    }
    func scan1000000(addOnStart int64, result chan string, sem chan bool) {
    var before string
    var after [64]byte
    var base64Encoding string
    defer func() { <-sem }()
    for i := addOnStart * 1000000; i < (addOnStart+1)*1000000; i++ {
    before = fmt.Sprintf("Message%d", i)
    after = sha3.Sum512([]byte(before))
    base64Encoding = base64.StdEncoding.EncodeToString(after[:])
    if strings.HasPrefix(base64Encoding, "TEST") {
    result <- fmt.Sprintf("%d: %s\n", i, base64Encoding)
    break
    }
    }
    }

    view raw

    gosha3.go

    hosted with ❤ by GitHub

    The program requires 13,135,013 sha3 hashes and conversions to base 64 to come up with a string that starts with “TEST”.

    My work workstation’s output:

    [Specs: Late 2013 Retina MacBook Pro 15″, 2.3 GHz Core i7 – (I7-4850HQ)]

    • Single thread: 16 seconds
    • Concurrency of two: 10 seconds
    • Compared with PassMark:
      • 9066
      • Single thread 1964

    A Late 2011 MacBook Pro 13″, 2.4 GHz Core i5 (I5-2435M):

    • Single thread: 34 seconds
    • Concurrency of two: 19 seconds
    • Compared with PassMark:
      • 3269
      • Single Thread 1347

    A Core i3-5010U 2.10 GHz NUC (Windows 10 Pro)

    • Single thread: 27 seconds
    • Concurrency of two: 17 seconds
    • Compared with PassMark:
      • 3060
      • Single Thread 1171

    A Mid 2009 MacBook Pro 13″, 2.26 GHz Core 2 Duo (P7550)

    • Single thread: 57 seconds (Originally was 83 on go1.3.3, but better results with 1.8.3)
    • Concurrency of two: 32 seconds (Originally was 80 on go1.3.3)
    • Compared with PassMark:
      • 1521
      • Single Thread 892

    A ThinkPad 11e with A4-6210 APU (1.8 GHz – Windows 10 Home)

    • Single thread: 135 seconds
    • Concurrency of two: 65 seconds
    • Compared with PassMark:
      • 2143
      • Single Thread 697

    Raspberry Pi 3B

    • Single thread: 1265 seconds
    • Concurrency of two: heat warnings!

    For the purposes of this “performance” benchmark, there is definitely a non-linear relationship between a canned benchmark score and the score in a brute force operation that has a heavy calculation component. I’m also isolating only one aspect of system performance. There is a SATA SSD on the newest machine, a SATA HDD on the 2011 and 2009 machines, and an SD card on the Raspberry Pi. The RAM is different in every machine as well. Still, it was a fun experiment that motivated me to spend a little extra time in Go and learning Go Concurrency.

     
  • ThomasPowell 12:08 pm on July 6, 2017 Permalink
    Tags: , indentation, , spaces, tabs, whitespace   

    Whitespace is Significant 

    While I was in my first couple years of college, Python was gaining traction as a hot new programming language. I had been programming in C since high school, but was curious about new languages. I took an interest in learning more about Python. A classmate told me that whitespace (at least indentation) was significant in Python. I changed my mind.

    The Holy Wars of Indentation Style


    // From https://en.wikipedia.org/wiki/Indent_style
    // K & R
    while (x == y) {
    something();
    somethingelse();
    }
    // 1TBS
    if (x < 0) {
    puts("Negative");
    } else {
    nonnegative(x);
    }
    // Stroustup
    if (x < 0) {
    puts("Negative");
    negative(x);
    }
    else {
    puts("Non-negative");
    nonnegative(x);
    }
    // Allman
    while (x == y)
    {
    something();
    somethingelse();
    }
    // GNU
    while (x == y)
    {
    something();
    somethingelse();
    }
    // Whitesmiths
    while (x == y)
    {
    something();
    somethingelse();
    }
    // Horstmann
    while (x == y)
    { something();
    somethingelse();
    }
    // Pico
    while (x == y)
    { something();
    somethingelse(); }
    // Ratliff
    while (x == y) {
    something();
    somethingelse();
    }
    // LISP
    while (x == y) {
    something();
    somethingelse(); }

    view raw

    indent.c

    hosted with ❤ by GitHub

    It’s interesting that would’ve been so turned off by a language that placed significance on whitespace and indentation, when I had fantasized about holy wars revolving around C indentation style (Horstmann is the only way!) and tabs vs. spaces (tabs, tab width 4!). I wretched at anything in the GNU C coding style (okay, I still do…), but the funny thing was, I never was able to program in a consistent style: There were other co-workers who had a slightly different preferred style. I was also learning myself, so relative importance of information in code changed, and my opinions on bracing style evolved.

    Non-C Languages that Use Curly Braces for Things

    It was a combination of Java, JavaScript, Perl, and Ruby that altered my perception of “proper” bracing style (BUT NOT SPACES OVER TABS!). A new line after a conditional or function declaration/call before the brace either took up unnecessary amounts of space, or changed the meaning of the code altogether (e.g., blocks in ruby, for which adding a newline = syntax error). And so, I evolved into declaring that 1TBS style was the only proper way to write C (WITH TABS!)

    Ruby

    Ruby has a generally accepted coding style that is enforced by no one except for code reviewers and maybe draconian RuboCop build configurations. I’d link to the coding style, if I had any certainty that the one I grabbed was 99% correct. Even RubyMine doesn’t format code to the same standards as RuboCop.  (Continuation indent and spacing around braces were two settings I had to tweak.)

    Small inconsistencies aside, Ruby code has ended up being the most consistent—visually—to support and add code to. I’ve introduced tighter RuboCop configurations on newer projects to try to adhere more closely to generally accepted coding style for Ruby.

    Ruby also introduced me to an acceptance of significant whitespace. While C code often gets sloppy with spacing around punctuation markers in code, Ruby developers take more care in proper whitespace around the conditionals, braces, etc. Some C/C++ projects like Mozilla do similar standards. However, Ruby is the first language that I’ve seen a community-wide interest in code that presents in the same manner. Of course, most of this code isn’t syntactically significant, but it is of human significance–and humans have their own mental compilers that optimize on standard patterns.

    Still, this isn’t the largest impact Ruby has had on my opinion of significant whitespace. Through Ruby, I’ve been exposed to YAML, HAML, and Slim. (I have dibs on forming a band of Rubyists with that name!) All three languages are unforgiving about inconsistent indentation. They take out explicit expression of boundaries and create visually implicit boundaries via hierarchies of indents. You don’t need to look for an end or closing brace to know that the context has moved up a level. The indentation tells you where that happens.

    (And yes, because of the generally accepted coding style in Ruby, I use 2 spaces instead of tabs now.)

    Golang

    And yet, Ruby coding style is informal. You can code in improper style and still commit your code and/or run it. There’s something about having the freedom to write improper style and being able to be a good “citizen” on your own that is comforting.

    Go does not do this. Writing Go in any editor configured to code in it (I’ve been using the vim-go plugin) automatically formats via gofmt.  It’s pretty unsettling. I longed for the ability to become dictator over coding styles, and then was okay when Ruby dictated one, but gave us the autonomy to do the right thing. This auto-formatting? This is tyranny!

    But, perhaps, it’s time. We can mentally chunk pieces into larger concepts for storage, if code that does the same thing looks the code. Instead of having to remember every brace as an individual piece, we can put “for loop that iterates and prints” as a single chunk.

    A stray brace isn’t going to permanently prevent distillation into larger chunks; it will just take a higher cognitive load to get there. This in turn will require more experience with the language before a programmer can make that leap. Being able to break things into large chunks helps us discover the what instead of dwelling on the how.

     
  • ThomasPowell 5:22 pm on July 3, 2017 Permalink
    Tags: dreamhost, linode, migrating wordpress,   

    Migrating WordPress From DreamHost to Linode Easily 

    Avoiding Migrating WordPress

    Migrating WordPress has always been a tedious process for me, and I’ve often stayed on horrible hosting providers just because I’ve dreaded it. Linode (referral link) and similar providers at least have a wealth of information on how to set up your new hosts, but I still found myself jumping around a lot to remember how to do certain things. Hopefully, this post helps put (most) of the information in one place if you want to migrate WordPress to new a host manually without a WordPress migration plugin.

    Experience level required: You have logged into your own Linux box and have done *something* useful.

    Grabbing your content from DreamHost

    Login using the shell username associated with your domain name that you’re moving. Assume the domain name is example.com.

    Grab the WordPress install complete with content using tar.

    I’m using the domain name as part of the tar file name to keep things straight, but the name before the .tar file can be whatever works for you. (Note: you can be more surgical and just grab the content and have a fresh WordPress install, but then you’ll be chasing after even more loose ends.)

    tar pcvf example.com.tar example.com

    Dump your mysql database.

    Open your wp-config.php for you domain and note your DB_NAME, DB_USER, DB_PASSWORD, and DB_HOST.


    // ** MySQL settings – You can get this info from your web host ** //
    /** The name of the database for WordPress */
    define('DB_NAME', 'wp_example_com');
    /** MySQL database username */
    define('DB_USER', 'examplecom');
    /** MySQL database password */
    define('DB_PASSWORD', '[email protected]!h12');
    /** MySQL hostname */
    define('DB_HOST', 'mysql.db.example.com');

    Run the following mysqldump command line:

    • DB_NAME as the first argument
    • DB_HOST after the –host argument
    • DB_USER as the -u argument.
    • The -p argument will cause mysqldump to prompt you for a password.
    • Redirect output (> filename) to whatever file you want, but again, I’m using something like “domain.name.sql” for consistency and clarity in my move.


    mysqldump wp_example_com –host mysql.db.example.com -u examplecom -p > example.com.sql

    Transfer your files down.

    scp [email protected]_host_name:~/example.com.tar .
    scp [email protected]_host_name:~/example.com.sql .

    Sign Up for Linode

    I’d appreciate you signing up through my referral link for Linode . I chose the Linode 2048 Standard Instance , Deployed an Ubuntu 16.04 LTS image to it, and started it up.

    From there, you need to set up the LAMP stack. Note: these instructions will install PHP 7, and I found at least one plugin (Sociable?) that used deprecated calls in php and caused me a few 500 errors.

    Add non-root user

    Be sure to also follow Securing Your Server recommendations for automatic updates and such, but for our purposes:

    adduser example_user
    adduser example_user sudo

    Upload Your Site to Linode

    scp example.com.tar [email protected]_instance_ip:~
    scp example.com.sql [email protected]_instance_ip:~

    Enable .htaccess if you use any rewrites

    /etc/apache2/apache2.conf (AllowOverride)  It’s possible you *don’t* need this if you have WordPress, but I haven’t had an install that didn’t.


    <Directory /var/www/html/example.com/public_html/>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
    </Directory>

    view raw

    apache2.conf

    hosted with ❤ by GitHub

    Enable apache mods

    sudo a2enmod ssl rewrite mpm_prefork

    Add other PHP and related mods that your plugins may need

    sudo apt-get install php7.0-mbstring

    sudo apt-get install php7.0-gd sudo apt-get install php7.0-curl php7.0-json php7.0-cgi sudo apt-get install php7.0 php-pear libapache2-mod-php7.0 sudo apt-get install php7.0-mysql

    Set up your virtual host

    /etc/apache2/sites-available/example.com.conf


    <Directory /var/www/html/example.com/public_html>
    Require all granted
    </Directory>
    <VirtualHost *:80>
    ServerName example.com
    ServerAlias http://www.example.com
    ServerAdmin [email protected]
    DocumentRoot /var/www/html/example.com/public_html
    ErrorLog /var/www/html/example.com/logs/error.log
    CustomLog /var/www/html/example.com/logs/access.log combined
    </VirtualHost>

    Create subdirs and set permissions properly

    sudo mkdir /var/www/html/example.com

    sudo mkdir /var/www/html/example.com/{logs,public_html} sudo chown -R www-data:www-data /var/www/html/example.com

    Untar your content

    Because of the way Dreamhost has their domain subdirectories set up, you’ll end up with two layers of example.com folders. (You could have also tarred from the folder, etc… but I just decided to keep this pattern.)

    cd /var/www/html/example.com
    sudo -u www-data tar xvf ~/example.com.tar .
    cd example.com # from the tar file
    sudo -u www-data mv .htaccess ../public_html
    sudo -u www-data mv * ../public_html
    ls -a # this should be empty now.
    cd ..
    pwd # be sure you're in /var/www/html/example.com
    rm -rf example.com

    Setup and Load Your DB

    cd to the location of your sql dump.

    CREATE DATABASE wp_example_com;
    CREATE USER 'examplecom' IDENTIFIED BY '[email protected]([email protected]';
    GRANT ALL PRIVILEGES ON wp_example_com.* TO 'examplecom';
    USE wp_example_com;
    SOURCE example.com.sql;

    Change your wp-config.php for the DB setup

    Your DB host is now localhost (unless you created a new DNS entry for your DB host)


    // ** MySQL settings – You can get this info from your web host ** //
    /** The name of the database for WordPress */
    define('DB_NAME', 'wp_example_com');
    /** MySQL database username */
    define('DB_USER', 'examplecom');
    /** MySQL database password */
    define('DB_PASSWORD', '[email protected]([email protected]');
    /** MySQL hostname */
    define('DB_HOST', 'localhost');

    Enable your site

    sudo a2ensite example.com

    Add a DNS zone in Linode and change your DNS servers to point to Linode.

    Add SSL

    For more details start at “Install Let’s Encrypt to Create SSL Certificates


    <Directory /var/www/html/example.com/public_html>
    Require all granted
    </Directory>
    <VirtualHost *:80>
    ServerName example.com
    ServerAlias http://www.example.com
    Redirect / https://example.com/
    </VirtualHost>
    <VirtualHost *:443>
    SSLEngine On
    # comment these three lines out
    # sudo apache restart
    # run letsencrypt
    # sudo apache restart (reload seemed to get in a weird state)
    SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
    ServerName example.com
    ServerAlias http://www.example.com
    ServerAdmin [email protected]
    DocumentRoot /var/www/html/example.com/public_html
    ErrorLog /var/www/html/example.com/logs/error.log
    CustomLog /var/www/html/example.com/logs/access.log combined
    </VirtualHost>

    I had to experiment a bit with the SSL configuration to get Let’s Encrypt to generate the keys and certs.


    #first time setup only
    sudo apt-get install git
    sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt
    # I had to restart apache with a stubbed *.443 section, do this, and then
    # add the SSL* directives in.
    sudo -H ./letsencrypt-auto certonly –standalone -d example.com

    view raw

    letsencrypt.sh

    hosted with ❤ by GitHub

    Lastly… I had to add renewal of the SSL certs to my crontab (sudo crontab -e)


    0 0 1 * * /opt/letsencrypt/letsencrypt-auto certonly –quiet –standalone –renew-by-default -d example.com >> /var/log/letsencrypt/letsencrypt-auto-update.log

    view raw

    crontab

    hosted with ❤ by GitHub

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel
%d bloggers like this: