March 22, 2013

Still Loving The Postal Service

10 years ago the band The Postal Service released their only full length album, Give Up. Back then they did a live studio recording at KCRW on their Morning Becomes Eclectic show. I had the video on my bookmarks for a long time but changes in the website some years back meant no more video. A few times I went trying to dig and find it, but no luck.

This morning I found a recently uploaded version of the video on YouTube:

Thanks to Miftah Bravenda for uploading.

In 2013 The Postal Service is touring, coming to Boston, and yes I have a ticket.

Posted by mike in Music at 9:59 AM

September 1, 2011

Andriod to iPod/iPhone Dock Adapter Cable

I've gotten a Zune connected to an iPod/iPhone dock in the past, but what seems more compelling these days with the rise of Android is to make an adapter or a cable to get a micro USB device to an iPhone, iPod, or iPad docking station.

Important caveat: it's technically impossible to provide the same kind of device integration that an iPod/iPhone car or speaker system provides; audio, video, track and playlist control, viewing track information on the car system, etc. Those are things that Apple's 30-pin docking adapter all have built in, which is why it is so compelling and revolutionary. A micro USB connector can only really charge and sync a device, it doesn't not have the connections to do all that fancy integration.

I wish there wasn't this caveat. However, if you're willing to accept the compromise that Android integration cannot match what Apple provides, but want to at least get something out of that iPod/iPhone speaker or other docking system, there's a way.

Apple's dock connector (the 30-pin plug on the bottom of the iPhone, iPod, and iPad) provides many functions. Most Android phones include two jacks; a micro USB connector for charging and syncing, and a standard 1/8" (2.5mm) audio jack for standard headphones. The pins for these two connectors on Android phones can be isolated on the 30-pin connector and provided for

CableJive's dockBoss+ adapter cable does exactly this, it lets you plug a smart little adapter onto your iPod or iPhone speaker dock or car system, and provides a micro USB and audio cable so you can charge your Android phone and listen to audio, both through your docking system.

It's not the ultimate solution, but until Apple decides to open up their proprietary communication system (never going to happen), getting micro USB and audio to connect an Android phone to your iPhone, iPod, or iPad dock at least keeps your phone charged and the music playing.

Posted by mike in Recreation at 5:11 PM

July 9, 2011

Ruby on Rails 3, Octopus, and Sharding with Groups

The Ruby on Rails app I'm building from the ground up needs to be able to handle *lots* of data. From the get-go I've known I'd need to put some kind of data partitioning to make this thing scale, but just finally tackled it this week.

Rails doesn't come with a built-in ability to connect to multiple databases, but there are lots of folks who have tackled it. Ryan Tomayko's post Rails and Scaling with Multiple Databases (written way back in 2007) is a good indicator that there are folks doing serious business with rails and multiple databases, and has some good links to other sources.

I was looking at writing something myself, but in digging around came across Octopus a few times and realized it does exactly what we need, data sharding with the ability to programmatically select databases in controllers and models, as well as good migration handling.

For the application we'll have a master database which centralizes account, user and session information with a pointer to the appropriate data shard. Once a user request has been authenticated, the actual application data gets pulled off of one of the shards they've been assigned to. The master database has a simple schema for user, account, and session information, the schema on the shards is significantly more complex, but the same on each shard.

Octopus documentation is there, but not extensive. Through some trial and error I found that the following shard.yml configuration file was what I needed (the Octopus documentation doesn't spell out how to use shard groups in a rails environment, but it makes sense once you get it working). The master shard for the environment is defined in database.yml.

octopus:
  environments:
    - development
    - test
    - production
  development:
        data_shards:
            data_1:
                adapter: postgresql
                database: dev_data_1
                username: something
                password: cryptic
                template: template0
            data_2:
                adapter: postgresql
                database: dev_data_2
                username: something
                password: cryptic
                template: template0
...

It's obviously paramount that the models know where to connect for their data. When a user authenticates I invoke ActiveRecord::Base.establish_connection() to connect to the shard so it's the default data source everywhere in the application for that request. Requests that need to interact with the master database are handled in the controller:

  around_filter :select_shard
  def select_shard(&block)
    Octopus.using(:master, &block)
  end

The other important piece of this is migrations. Octopus works well for our needs here. When a migration is written it either needs to update a single master database or be applied to all data shards. This is where shard groups come in. To apply a migration to all shards we specify the group with using_group() method, and the migration then runs for each shard in the specified group.

class CreateInventoryEvents < ActiveRecord::Migration
  using_group(:data_shards)
  ...
end

Pretty slick, happy with Octopus from a functionality perspective. Not sure about scalability and the overhead to move requests between database connections, will dig more into that next.

Posted by mike in Technology at 7:27 AM

May 19, 2011

Developer Happiness at Etsy

Listening to Chad Dickerson (CTO at Etsy) speaking about optimizing for developer happiness at RailsConf 2011.

Etsy releases software 25 times a day with 75 engineers. They emphasize constant progress with a radical decentralization. It shouldn't work, but they are built on trust. There is one button that folks can press (after some fanfare) that pushes code up to production

They've had developers push code on their first day, they've had board members deploy code.

Chad contrasts industrial revolution assembly line times with how Etsy works. Etsy highly prioritizes letting the developers be human, having lunches together, having offices that are full of creative, fun stuff. Dogs have deployed code by guiding a paw to pressing the button.

Releases are done with a self-managed push queue, developers put their names in a queue kept track of in an IRC topic. When the first person in line pushes and is clear he/she notifies the next in line.

Remember, your team is your community. Help people finish things and see their progress.

Optimize your teams for the happiness of people.

Posted by mike in Technology at 3:03 PM

Glenn Vanderburg on Building Software

Fantastic closing keynote by Glenn Vanderburg at RailsConf 2011 about software engineering.

Great ideas about software engineering and how it compares to classical engineering disciplines (civil, structural, industrial, mechanical, electrical, traffic, etc). Compares the definition and process of structural engineering to software engineering.

Is software engineering or craftsmanship? In software, the documentation that is required by other disciplines is actually the artifact that gets built in software engineering. Classical engineers diagram, plan, document, model and then take those and build the artifact. Software engineers diagram, plan, document, and model as a part of building the actual artifact. Also interesting, software is changing the way classical engineering works, a lot of software being used in the planning process to understand how the actual artifact

We are designers and builders. Software has to be the solution and describe the solution.

Programs must be written for people to read and only incindentally for machines to execute. Harold Abelson and Gerald Jay Sussman

We are engineers and craftsmen.

Posted by mike in Technology at 2:43 PM

Software Version Numbers

In the RailsConf 2011 RubyGems presentation this afternoon, Nick Quaranto spent some time talking about versioning software, showing the versioning of TeX as kind of crazy where they use PI, adding a new digit of PI on the end of the software version number for each release.

Nick suggests that Semantic Versioning is the best methodology. Read more at http://semver.org/

The idea is:

1.2.3 (major.minor.patch)

major versions - for major releases where there may be some backward incompatible changes
minor versions - for minor changes, no API or other changes that make it incompatible with previous versions
patch version - for fixes, improving stability

Posted by mike in Technology at 1:40 PM

Long Evening At Camden Yards

If the value of a baseball ticket is measured in hours spent in the park, I definitely got my money's worth last night. Baltimore Orioles vs New York Yankees, 15 innings, Orioles lose it in the 15th after having many chances to put it away earlier in the game.

I walked into the park at 6:30pm, left just under 6 hours later at 12:10am. Although I went by myself, I made a bunch of friends with the Orioles fans in the Picnic Perch (food isn't fancy, but a nice thing to have all-you-can-eat at your fingertips).

It was supposed to rain, and we saw a little lightning in the distance. It misted up a little, but not a drop of rain.

Posted by mike in Recreation at 9:46 AM

LivingSocial's Aaron Batalion says "Be Nervous"

Great keynote this morning at RailsConf 2011 from LivingSocial's Aaron Batalion.

Favorite thought: be nervouse or your aren't trying hard enough.

Aaron says the right way to go at a new idea is to make decisions that make you nervous, all the time. This means you are pushing forward.

Advice that makes me feel better about some of the crazy stuff I've faced as the leader at a growing company. And excited to try some new stuff.

Posted by mike in Technology at 8:29 AM

May 18, 2011

Rails 3 Gemfile: Installing PostgreSQL on OS X

After two months of building my Rails app working on remote servers, I'm sick of always having to SSH in and use a server-side text editor to work. Not to mention that any time I move the laptop I've got to break and then re-establish all the terminal windows I had open.

So I embarked on setting things up to run my Rails app right on my MacBook Pro. Yea, the thing that 99% of the rest of the Rails community has been doing for years.

I figured everything except PostgreSQL is either already on the laptop or is nicely grouped together in the my Rails application in git. So I installed Postgres from their OS X Postgres install page.

Checked my git repo out on OS X, ran the "bundle install" and got:
Installing pg (0.11.0) with native extensions /Library/Ruby/Site/1.8/rubygems/installer.rb:533:in `build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb
checking for pg_config... yes
Using config values from /Library/PostgreSQL/8.3/bin/pg_config
checking for libpq-fe.h... yes
checking for libpq/libpq-fs.h... yes
checking for PQconnectdb() in -lpq... no
checking for PQconnectdb() in -llibpq... no
checking for PQconnectdb() in -lms/libpq... no
Can't find the PostgreSQL client library (libpq)
*** extconf.rb failed ***

Obviously a failure. The Gemfile includes "gem 'pg'", which works fine up on the Linux servers but apparently doesn't do so well on OS X.

A lot of digging and reading, followed by a lot more digging and reading.

The first solution...when compiling pg, the gem needs to have an environment variable set:
env ARCHFLAGS="-arch i386" bundle install --path /Users/mike/inventory/vendor

No problem running the bundler, but when you actually try to use the database:
bash$ rake
rake aborted!
dlopen(/Users/mike/inventory/vendor/ruby/1.8/gems/pg-0.11.0/lib/pg_ext.bundle, 9): no suitable image found. Did find:
/Users/mike/inventory/vendor/ruby/1.8/gems/pg-0.11.0/lib/pg_ext.bundle: mach-o, but wrong architecture - /Users/mike/inventory/vendor/ruby/1.8/gems/pg-0.11.0/lib/pg_ext.bundle
/Users/mike/inventory/Rakefile:4

Tons more reading and digging, trying lots of suggestions.

Finally I stumbled into something that made a lot of sense. A post on the rails forum suggests that Ruby and PostgreSQL are compiled for different architectures and the gem pg connector isn't going to work unless Ruby and Postgres are on the same page.

I got an updated Ruby using MacPorts, same error. I got an updated PostgreSQL from MacPorts and attempted to run "bundle install" as standard.

Failed.

Then I went back to including the architecture-specific command:
env ARCHFLAGS="-arch x86_64" bundle install --path /Users/mike/inventory/vendor

Works, and I'm able to run "rake db" commands.

Everything else looks like it's in place, am running the Rails app in my browser at localhost:3000 and have TextMate loaded up with my project files.

Much, much, much better.

Posted by mike in Technology at 3:45 PM

RailsConf 2011: Deploying with Bundler

Listening to Andre Arko at RailsConf 2011 talking about Bundler.

Bundler exists to make running your app consistent, repeatable, and guaranteed. Bundler doesn't let you use a gem unless it's in the Gemfile. Gemfile.lock keeps track of which version of the gem is running.

To publish the app. A very simple way is to tell Heroku or EngineYard to do it, they build right from your Gemfile and deploy.

To do it yourself. Make sure gems are installed within the app, not as the root user: bundle install --path app/vendor

Bundler provides a --frozen mode that makes sure that all of the Gemfile entries match up with the Gemfile.lock. This is a good thing to have in place on production.

For production: bundle install --deployment (turns on --path and --frozen)

For bundling to send to a server that has no outbound internet access: bundle path (not advised unless the only option, because requires downloading a ton of binary data from version control)

Posted by mike in Technology at 1:59 PM