« June 2007 | Main | August 2007 »

July 28, 2007

Pink Martini at the Oregon Zoo

Friday night, after all of the excitement of the 2007 O'Reilly Open Source Convention was behind us, Heidi, Saul (18 months) and I went up to the Oregon Zoo to catch a concert. Originally we were going to head out of town Friday afternoon but Heidi learned that Pink Martini was playing an outdoor show at the Zoo so we extended by one day.

The show was awesome. We didn't have great seats (general admission on the lawn) but the sound was awesome from where we set out our blanket and it actually worked pretty good to be in a spot further back where we could get in/out to take walks around the area with Saul when he needed to move.

Heidi is a bigger Pink Martini fan than I am, but I so enjoyed the live music and came away with a deeper appreciation for their music, especially the diversity of languages and styles. The weather was perfect, and Saul was a good boy, which made the evening as enjoyable as we had hoped.

Posted by mike at 11:53 PM

Wedding on Georges Island

On July 14th (just slightly behind here), Holli and Joe got married on Georges Island in the Boston Harbor. It was something really special.

Friday night was a pre-wedding dinner with family and close friends at a nice pizza place near the south end where Holli and Joe live. The food was great, but more enjoyable than that was being with Holli and Joe and all of their family and friends for an informal evening dinner. Some of the folks I had met before, and most of the others I had heard about. The evening was filled with good conversation, meeting new folks, hearing stories from the past, and giving tributes to Holli and Joe. We left feeling so enriched by the goodness of the evening.

Saturday took us out to Georges Island for the big day. The weather was perfect. We arrived before noon, walked around a bit, had bagged lunches from a local caterer (taxied in a boat out to the island that morning), met more folks, continued conversations etc. At 2 we went into the fort and witnessed the ceremony, which was me saying a few words about Holli and Joe, marriage, and everyone who had come to participate in the wedding.

Yes, if you look closely at the photo you'll see that I'm standing between the wedding couple. Holli and Joe had been married by a justice of the peace earlier, but had me conduct a public ceremony where they exchanged vows, rings, and made it official in their minds and hearts.

A more detailed write up is on Holli's blog, and more photos.

Posted by mike at 11:47 PM

July 27, 2007

Hack the Real World with Open Source and Microcontrollers

Brian Jepson (MAKE Magazine) is talking about hacking at OSCON 2007.

Brian takes a look at open source hardware, talking about things that make hardware more hackable like mechanical diagrams, parts lists, layout diagrams etc.

The Make controller ($120) has a sketchup model, schematics, open source firmware and APIs/tools. Doesn't have a parts list or layout diagram.

The Arduino ($30) is more open source by Brian's checklist of "open source-ness". It includes mechanical diagrams, schematics and circuit diagrams, parts list, layout diagrams, open source firmware and open source APIs/Tools.

A microcontroller is a computer on a chip. Has a processor, some memory, analog and digital I/O. Example input is light or temp sensor, tilt sensor or RFID. Examples of output might be LED, motors, audio/video.

Brian gives a tour of some really interesting sensors. Alchohol sensor, force sensing, others.

Posted by mike at 11:18 AM

James Larsson: Pimp My Garbage (hardware hacking)

James Larsson gives the final keynote of Friday morning at OSCON 2007.

If you ever get a chance to hear this guy speak on hardware hacking, do not pass it up. Awesome and funny tricks James did with old junk (Operation game that hemorrages, dot-matrix printer turned into hunting practice, leather-fetish pong with whip punisher for missing the ball, and several others).

Posted by mike at 9:57 AM

Open or Closed? The Future of Search

Listening to Jimmy Whales (founder of the Wikipedia) at OSCON 2007 talking about Wikia. Wikia is a completely new organization. Not a sister or subsidiary of Wikipedia.

Building a wedbsite today is so much different than it was before open source. Prior to open source there was a huge financial investment (millions of dollars). Since open source it is much easier to get up and running.

Before Wikipedia there was one way to get encyclopedic information, the printed encyclopedia. Now this encyclopedic information is readily available to everyone, in many lancuages. Wikipedia is much more comprehensive than the printed encyclopedia (Jimmy admits not always as accurate).

Like building the web and the encyclopedia, search needs a shift. Currently there is a handful of proprietary players (Google, Yahoo!, msn, Ask). When you search today, the engine makes an editorial judgement on what is important. The general public doesn't know what drives the results.

The idea with wikia is to change the balance of power. Create search tools, algorithms, and crawlers that are public.

Jeremie Miller (founder of Jabber) is the lead on the project.

The driving principles:

There are things that humans do better and things that machines do better and right now there is too heavy reliance on computers.

Science progresses better when research is freely available. Currently students doing research in search don't have access to the latest technology. One of the reasons that proprietary search companies site for keeping search algorithms secret is that it prevents spammers. This security through obscurity isn't a great idea.

Wikia announced today that it has acquired Grub. It is a distributed crawler project. Something that can be done by vast numbers of end-user machine when there are free CPU cycles and bandwidth.

Jimmy gives a good example of the wikipedia/wikia way of thinking. The example is that you are desigining a restaurant. You know it will have knives for cooking, but people are bad when they have knives. They might stab something os someone. So with this in mind you design the restaurant so that people who eat there must sit in a cage. This makes a bad society. We shouldn't design things thinking about all of the bad things people will do.

Posted by mike at 9:24 AM

Open Source Second Life

First keynote of Friday morning at OSCON 2007 is Philip Rosedale (Founder/CEO Linden Labs) talking about Second Life. The Second Life client is now open source, the server is not. They are working toward open sourcing the server, but want to make sure that there aren't ways to do serious damage in Second Life from having the source or running a server.

Demonstrates a new feature of Second Life, now has audio so as you're walking through the world a person who has a small audio indicator over the person. Allows you to talk freely using your voice. At Linden they have all of their meetings within Second Life.

Philip talks a lot about their move to open source and how they could really use help with moving things forward. In Second Life (on the presentation slides) Philip walks over to a large flat screen and uses that to show his presentation slides. At the same time he's got a co-worker (in San Francisco) standing next to him who can make comments about the slides.

Posted by mike at 8:50 AM

July 26, 2007

Perl Lightning Talks

Listening to Perl lightning talks at OSCON 2007. The idea is that anyonce can get up and speak for 5 minutes. Talks are timed with a bell sounding at 4 minutes.

Vani: 5-minute version of her YUI CSS talk.

Michael: How to make a cup of tea.

Michael: Perl 5 now has a Wiki. Please use it to put in information about Perl, your company. [Perl demographics exercise] There is a Perl survey, please fill it out.

Ask: qpsmtpd - Perl-based qmail-smtp replacement

Andy Lester: ack - a awk/grep tool for huge amounts of data. New site Perl 101. Google code is a great site for managing projects. Use it over sourceforge.

Andy Lester: Religious speech about Perl not being a scripting language. We don't write scripts, we write programs. Don't call use a scripting languages.

[woman in Debian shirt]: Open source contributors are actually volunteers. Open source needs more volunteer coordination. The work will move forward better if there are tasks to do instead of telling people to pick something they'd like.

Eric Wilhelm: working on Tap::Parser for multi-cpu machines.

Eric Wilhelm: Module::Build is getting an update to reduce the number of steps to installing new modules. A new config configure_require which can be set to always check for upgrades on a few modules before updating.

Julian Cash: talks about having data in Flickr, Blogger etc and how that is a problem. Shows the concept for MoveMyData.org.

Tim Bunce: talks about a having a common DBI for Parrot and Perl 6. Is going to borrow the JDBC API, documentation etc to be the DBI for the underlying DBD. This is done with Inline::Java, which takes a set of Java classes and generates a set of Perl classes.

Tim Bunce: demonstrates use of MyNetIO, a module that lets you put one line of code into your Perl program and get network performance data.

Tim Bunce: Demonstrates Gofer, a scalable stateless proxy for DBI.

Michael Potter: Wants to open source messaging definition, inviting folks to join in a conversation on messenging standards.

Jon Rockaway: Catalyst in 5 minutes (kind of patterned after the rails screencasts, except you don't see the application working.)

Jon Rockaway: Angerwhale

John Peacock?: SVN::Notify::Config

[guy with guitar, Pudge?]: sings Barenaked Ladies "Life in a Nutshell" rewritten as "Perl in a Nutshell"

Posted by mike at 4:40 PM

Iptables Attack Visualization

Listening to Michael Rash (site) speak about visualizing iptables log data at OSCON 2007.

The toolset for doing this is psad-2.0.8, Gnuplot, and afterglow.

Sometimes you can do this by hand just by looking at numbers. Works OK if you have small sets of data or have an aggregation command. Much easier to find trends if you plot the data out.

Michael goes through a brief intro to TCP/IP, UDP, ICMP packet information.

Michael is using a dataset from honeynet (part of the honeynet challenge) which includes a variety of packet behaviors including portscans (packets to many ports from a single source address), port sweeps (packets to a single port from many hosts), slammer worm (404-byte request to port 1434), Nachi worm (sends a 92-byte ICMP echo request first), outbound connections, etc.

The psad utility allows you to run all kinds of options to plot data over certain periods of time, include/exlude certain ports, specify packet sizes etc to catch all kinds of snapshots. Allows regex matching to further granulate what you are looking to plot. A wide range of graphs are available from psad. You can do graphs or show links between addresses and ports (gets complex, but nice for smaller numbers of addresses). Gnuplot allows for 3-D rendering of graphs that you can rotate to see more clearly what is going on.

Posted by mike at 10:46 AM

Brand it: How to Ignore Marketing and Become Irrelevant in Two Easy Steps

Last keynote of Thursday morning at OSCON 2007 is Steve Yegge from Google talking about branding and marketing.

[Steve is operating without slides...technical dificulties]

Which compiler is faster TurboXXX or QuickXXX. Obviously it is the one branded as Turbo, regardless of which one actually runs faster.

A brand is really just a pointer to a concept, a proper noun. Steve worked at Amazon for 10 years. Jeff Bezos worked hard to brand it as a book store. Even though it has done more than books for a long time now, the brand still means books to most folks.

National Biscuit Company is much less compelling than Nabisco. GTE's brand was in the toilet a few years back and went to a marketing company to improve the brand and were told it could be done over the course of a generation. They changed their name to Verizon instead of trying to build their brand name back up.

JavaScript has a branding problem. The name evokes emotions of problems with browsers. If there ever was a programming language that needed a new name it is JavaScript.

Open source licensing also has a branding problem. GPL is the clearest "brand" there, but feeling is not clear if that is the right license to use. Open source name is also a branding problem. Not a clear concept that comes to mind when thinking about open source.

Solutions...educate customers, send polite letters to the people using the term. What we really need is for open source to mean something, a narrow focus definition.

Posted by mike at 10:01 AM

Open Source at Microsoft

Third morning keynote on Thursday at OSCON 2007 is Bill Hilf from Microsoft talking about Open Source at Microsoft.

[Nate gives a good introduction saying that we might not agree with everything we hear, but this is about learning and seeing new things.]

I think Bill is brave for being here. He says he's been called almost anything, and builds some credibility with the audience by talking about programming. He admits to being a C and Perl programmer from the early days.

[Side note: Bill Gates has a philosophy of called software darwinism that is rampant at Microsoft, it is a fight for the best software to survive.]

Bill (Hilf) talks about all of the participation from Microsoft in various communities. When Bill first started at Microsoft he had a senior engineer ask if it was OK to look at a Linux website.

At one point they had PHP folks up to meet with Windows folks and by the middle of the meeting it was clear that PHP on Windows was really, really bad. The good news was that in a day they had increased performance by 200%, the bad news was they started to wonder what else really stinks on Windows.

Windows is working with the OSI right now to get their shared source license reviewed.

Posted by mike at 9:22 AM

Robin Hanson: Overcoming Bias

Second keynote of Thursday morning at OSCON 2007 is Robin Hanson (Overcoming Bias blog).

We have congnitive biases which cause errors between what is and what we think is. We read our newspapers, textbooks and trust the truth. Even though we recognize that "yes, we have biases," it is much worse that we think.

On average a person's opinion move to what they hear from other people more than we think.

Folks in this room are passionate about things (open source, vegetarian). What happens when someone else doesn't agree with our passions? We think there are reasons, people were raised differently or badly. But even after we explain our position that other person still doesn't understand our passion.

An interesting fact, when you have to bet on something you are more honest. Betting markets are good for encouraging being unbiased.

Bottom line is that we should put more effort into working on overcoming our biases.

Posted by mike at 9:02 AM

Ben Fry: Visualizing Data

First keynote of Thursday morning at OSCON 2007 is Ben Fry looking at a slew of different examples of visualing data through static, 2-D animation, and 3-D animation.

Some pretty incredible stuff. Just going to enjoy this. Will try to find a link later.

Update: Link to Ben's site. Definitely worth taking some time to loook over the projects.

Posted by mike at 8:45 AM

July 25, 2007

Making the Hard Stuff Fun and Easy with YUI

Listening to Vani Raja speak at OSCON 2007 about using the YUI CSS (slides).

YUI CSS is predefined styles that offer over 750 layouts, tested. Can do it all with div elements and id or class in elements.

Vani does a demonstration of each of these. The grid stuff is pretty cool, with some very minimal styling you can have nicely formatted pages.

Vani makes a good point about using semantic naming of styles so there is meaning in the names, not just a description of what the style does.

If you use float but you don't want the next item to float use clear: left;.

Using position: absolute; goes relative to the next parent up with a known position, which is almost always the body. For relative; the position is determined by the location of the element.

Recommendations: TextMate, Transmit, CSSEdit, Coda, Free Ruler, DigialColer Meter, CSSViewer, Opera9/Mini, Dolor Sit Amet (OS X Widget for generating lorum ipsum).

Highly recommends the Web Developer Handbook.

Posted by mike at 2:34 PM

Testing and Debugging the Web Tier

Muness Alrubaie (ThoughtWorks) is speaking at OSCON 2007 on testing and debugging in with the web.
[Room is packed, folks on the floor and lined up in the back of the room]

Talking about debugging tools for web browsers and testing tools for web developers.

In Firefox you should create a special profile to keep the environment to be isolated. Testing environment might also be customized, managed differently, and shared. First thing Muness does is to set up a new user profile in Firefox so he can keep a clean environment for testing. There is a command line for launching the profile manager as well as launching Firefox with a specific profile.. Profiles can be stored separately, even stored in CVS.

Web developer toolbar is a good tool for Firefox.

Firebug (and Firebug lite) is another very valuable tool.

A few JS commands to know when using Firebug:


If you put an array or object in the console functions it will show them as a JSON object.

Strategies for working with IE? Microsoft Script Debugger, came with Office 2000. Someone else suggests the IE dev toolbar?

Unit Testing
For unit testing JsUnit provides unit testing for JavaScript.

[Muness goes into detail about creating and running tests using JsUnit]

Crosscheck is another tool for JavaScript unit testing.

UI Testing
Use selenium to simulate user interaction with the browser. Selenium IDE lets you record your keystrokes.

Posted by mike at 1:39 PM

Who Gets to Decide What Open Source Means?

Listening ot a panel entitled "Who Gets to Decide What Open Source Means?" at OSCON 2007.

Panelists are Chris DiBona (Google), Brian Behlendorf (CollabNet), John Roberts (SugarCRM), Ross Mayfield (Social Text), and Michael Tiemann (RedHat, Inc and president of OSI). The panel is moderated by Danese Cooper (Intel and Open Source Initiative (OSI)).

The OSI is a collaborative conversation with anyone. People can bring licenses to OSI and get approval to determine if a license is open source.

John starts to talk about how this panel came about because SugarCRM was asking questions. They have built an enterprise software package and would like to see an attribution clause in an open source software license.

Chris didn't want the attribution clause in open source licenses, was vocal about it. An attribution clause won't get approved by the OSI, so any license with an attribution clause doesn't seem right.

Russ used a Mozilla-based license with some added clauses. Didn't send to OSI for a long time (just getting to the end of the process after having submitting 9 months ago). Social Text also likes the idea of attribution, not for everything but for if the software gets forked. Russ (and Social Text) backed off from calling themselves open source until they had gotten OSI approval for their license.

Brian talks about one of the original BSD licenses developed at UC Berkeley which says you have to include information about the source of the code. They moved away from the attribution clause because it just wasn't possible to do correct attribution, especially if someone like RedHat was shipping hundreds of open source products under one umbrella.

Attribution is important on another level, where people are contributing because they want to build their name.

Michael talks about how things that happen in Red Hat tie into his position the OSI. Sometimes percieved as a conflict, but the joint work actually works out very well because he's seeing how the theoretical mixes with the real world. The definition on Wikipedia is one of the most hotly debated topic, and has to be regularly monitored to keep clean.

Posted by mike at 11:42 AM

Josh Berkus: Performance Whack-a-Mole

At OSCON 2007 listening to Josh Berkus talk about finding database performance issues in a web application. I see that Brian and Peter are here. I've been to many-a-MySQL tuning session, will be interesting to see how it differs looking at it from Postgres.

[prior to starting Josh is playing a Flash-based game (projected on the screen) called Mole where you try to whack a mole on the head]

Josh was involved in the recent benchmarks published comparing Oracle and PostgreSQL.

Josh looks at a stack with hardware, OS, database, middleware and application and talks about trying to figure out where in the stack. Each layer down the stack should be capable of handling.

Rule 1 of whack-a-mole is that the problem is actually not always in the database itself. Rule 2 is that 10% of the issues will cause 90% of the performance degradation. Not worth focusing on the 90%. Rule 3 is that you only see one problem at a time. One performance problem will mask others. Rule 4 is that different applications require looking at different parts of the system.

Database uses differ. There are three general classes of applications; the web app (CPU-intense, focus on caching, pooling, connections, online transaction processing (CPU or I/O bound, focus on locks, cache, transactions, write speed, logging), and data warehousing (I/O or RAM bound).

Prior to doing any performance work you need to go through each layer of the stack and gather information. Sometimes this process is more important than the actual adjustments will make. Gather all kinds of details about the hardware, operating system, database, middleware, and application.

In the end, there was very little database-specific discussion.

Josh didn't do a lot of work with Solaris before going to Sun. Didn't recommended it at all because he was a Linux and FreeBSD guy.

Posted by mike at 10:45 AM

Tim O'Reilly interviews Mark Shuttleworth

Listening to a keynote presentation at OSCON 2007 where Tim O'Reilly is interviewing Mark Shuttleworth.

Tim asks why is Ubuntu winning.

Mark responds that they are trying to build something that works for everyone. The software is buing built in a very modular way so that it can be flexible with changing hardware.

Mark uses Ubuntu on the laptop and kubuntu on his home machine.

The rationale behind the Ubuntu rapid release cycle is to play to our strengths, collaboration which leads to innovation. They don't have to play to a traditional release cycle that is driven by features.

Ubuntu started as a 20-person team, is now 110 folks. They built Launchpad as a way to keep track of the conversations about all of the different applications and tools being developed. It is part of the business process for Ubuntu, but they want to make it available for others to use and keep the data available to export.

Since Linux is getting more history people know what they want Ubuntu attempts to priorities.

Ubuntu focuses on internationalization, starts with Mark growing up in Capetown, South Africa but is also a driving force because there are brilliant people everywhere. In the west things are often driven by business, else where it is driven by passion. Ubuntu tries to cater to both sides.

Software is the intimate part of your computing experience. The quality of that experience is important. The decision to be involved with the desktop instead of server-based software is influenced by this. Mark suggests that the future might swing back to the desktop, but with the internet cloud providing rich data.

Posted by mike at 10:00 AM

Intel: Outfitting for Multi-core Parallelism

Listining to James Reinders from Intel talking about some of the work they are doing on threading to get the most of your hardward.

A fun drama ensues, the marketing guy starts the talk about the new threading tools and shows that there is a new book that goes along with the software package and you can get both for $299. Another Intel guy jumps on stage and takes control of the presentation saying something to the effect that at an open source conference you can't sell stuff. End result is that the slide is changed to show that the threading work is now open source (site).

Intel has done a lot on the Linux kernel, but want to be more involved in open source. A slide shows a list of the Intel open source contributions including graphics, drivers, power management etc.

The presention is somewhat technical, and definitely focused on folks interested in threading technology.

Posted by mike at 9:19 AM

Tim O'Reilly: The O'Reilly Radar

First keynote of OSCON 2007 (Wednesday morning) is Tim O'Reilly. He essentially

Open Source in the Web 2.0 era. Tim talks about the four freedoms of the Free Software Foundation. Just because you have the software doesn't mean you can run it. If we had Google's source code we couldn't run it without huge amounts of data, servers, and the business processes. That makes the software less free.

With the current state of the web sometimes improving the data is actually more important that improving the software.

Does congress need a version control system? Tim mentions the YouTube presidential debates and how it enabled more "real" questions.

Tim shows this interesting chart with different open source efforts (Linux, Amazon reviews, etc) and how many contributors and "modules" developed.

Open source is a new web 2.0 business model. Cites StumbleUpon, OpenAds as businesses built on open data.

The technology (memcached,Hadoop,Intel threading) is also moving forward.

Posted by mike at 8:52 AM

Blog Software (MovableType) Updated - Not Smooth

It hasn't been smooth, but I've been able to get through the upgrade to the MovableType 4 Beta. I had to make one source code change to get over a bug and am still having a major issue with the styles (hoping to fix shortly here).

Update: Stylesheet has been reverted to the original. The new MT has some cool styles that are tempting, but appear to not work well with entry format from previous versions.

Posted by mike at 8:00 AM

July 24, 2007

Improving the Blog Authoring Experience with MovableType 4

Every time I have a concentrated period of writing here on this weblog (ok, I guess it's also a blog), I am reminded just how cumbersome it is to create good posts using the current stable version (3.3x) of MovableType. Writing text isn't so much the issue, although the text entry area is very small as if designed for single paragraph posts.

The thing that is more frustrating is enriching posts with links, organizing information (bold, italics, lists, etc), and including photos. Invariably after having to struggle through a lengthy post I end up on the MovableType site looking for help in making this process easier.

For a long time it didn't seem like there was going to be much relief, but it appears that the next version of MovableType (4) is a huge step forward in making a better UI for authors. This screencast will give you an idea of what is coming.

I'm so tempted to upgrade now...

It also appears that with version 4, MovableType will split into an enterprise version and a open source/community version. From the open source page, it appears that the difference is a delayed release of the open source version, minus a slew of enterprise features.

Posted by mike at 10:28 AM

Alex Russell: Learning Ajax

Getting ready to listen to Alex Russell give a tutorial entitled "Learning Ajax" (slides) at OSCON 2007. Alex is the lead developer of the Dojo toolkit.

[I've been doing/using Ajax for some time, but never had any kind of training so am looking forward to a good overview and hopefully will pick up some new things.]

OK, we're underway.

The talk is about applications, not pages or sites. Web applications aren't necessarily better, the technology isn't actually that great for building applications compared to OS-built programs. But there are some tools to make today's web much better.

For a period of time a few years ago the web kind of stalled, HTML and CSS got hacked up over and over to try to make things better but nothing new. This period allowed other browsers to catch up to IE which had all kinds of development dumped into it. [Side note: IE team was dispanded after IE 6, evidence that the browser had stalled].

The fix to the stagnated browser was to have something that could enable incremental data change, let users explore data more easily, present data change better and provide better layer markup and behavior.

When Ajax was first coined it was something different from what we know Ajax as today. Ajax was originally all about improving the user experience and interaction design, not about the technology itself. Alex still likes this view and prefers to spend trainings like this talking about the less technical pieces and how the UI design happens.

Ajax was made possible technically with XMLHTTPRequest, introduced by Microsoft in 1999. Mozilla implemented in 2002. Allowed us to avoid the "click of death" using both GET and POST. There is a same-origin policy (can't go to another site to make requests) and not a standardized API, but XHR is the best thing that has happened to the web since the late 90s. XHR request was designed to get XML, but quickly figured out that the requests could be anything text that can be pulled off the wire.

[looking at a getXmlHttp() function and using it to grab a URL]

[Alex pauses and tells everyone to install Firebug before we continue (shows off the Net tab to look at all network traffic from the web page) - IE 7 has something similar in Microsoft Script Editor which came with Office 2000/2003 - Drosera is good for Safari, available in nightly builds as part of the webkit]

Alex plugs virtualization software (Parallels/VMWare/VirtualPC) as a must for a web developer so you can quickly get at pages through different browsers.

Extra XHR methods to know about:


When using Ajax techniques you can get data in a few formats, XML, JSON, JavaScript, or plain text. XML is large and you have to work around encoding (make sure you send the mimetype "text/xml"). JSON is much smaller on the wire and fast to parse, but transformation is slow and there is no XSLT. JavaScript is handy, small on the wire, and can be easily executed using eval(). Sending JavaScript can be dangerous.

[Alex digs into an example of using XML and XSLT]

Ajax Dos:

Ajax Don'ts:

Gotchas with XHR:

Alex spends a little time looking through the currently available Ajax toolkits. Bottom line is at this point there's no reason to write something new.

Ajax makes actions atomic.

Four important interaction design principles (from Jeff Veen):

[Alex gives a demo of live search with a built-in keystroke delay detecter to make sure it's not searching on every change, but is smart to only do the search when the user pauses]

...in progress...

Posted by mike at 8:26 AM

July 23, 2007

Building Scalable Internet Architectures

Monday afternoon tutorial at OSCON 2007 is Theo Schlossnagle (CTO of OmniTI) speaking on Scalable Internet Architectures.

This session is about how to big really big stuff. Theo talks fast and confesses that things will move fast...will catch interesting points (to me).

Scalability is not performance. To Theo, scalability is:

How well a solution to some problem will work when the size of the problem increases.

Performance is:

Performance is the capabilities of a machine or product when observed under particular conditions.

Solutions for High Uptime

Scalability also means that maintenance of the site isn't overbearing.

Procedures make for better scalability. Having code reviews, using version control, and upgrade strategies.

Production fundamentals

If you're developing stuff faster than you can check the code properly then you're going to end up with a problem down the road with bugs and poorly architected code.

Three Simple Rules

  1. optimize where it counts
  2. complexiy has costs
  3. use the right tool

Clustered Image Serving Example
Goal is to serve static images at 120 MBs throughput with 24x7 uptime across three geographically distributed sites.

Rather than have multiple dedicated HA/LB servers sitting on top of your systems as described in the "white paper", have all of your servers exposed and run the HA/LB as a process on each machine.

The solution is wackamole (Theo is an author), a peer-based HA solution. Uses commodity hardware, each machine being responsible for a set of IP addresses, but each machine capable of taking the IP address of the other machines if they fail.

Theo takes a tour of spread, which is a pre-requisite for wackamole. Wackamole works internally by sending messages around that servers are or are not part of the group. Externally, it works by responding to ARP requests from user machines.

Use clever DNS to get people to the set of IP addresses. First is server-side, have the application dynamically buld pages to point to a specific site for images based on login or other credentials. Second is proximity-base DNS. Not good to rely on internet DNS mechanism. Third solution is to use DNS on a shared IP (anycast and BGP).

Tiered Architecture
Theo doesn't like tiered systems. They are expensive technically and financially. Good way to scale up, makes it hard to scale down. Often locks you into a certain architecture and difficult to troubleshoot. Database replication is hard, but needed if you're trying to break the system into functions.

If you don't want to tier, how do you replicate databases?

Theo walks through an introduction to database replication with some of the pros and cons of different solutions.

Posted by mike at 1:31 PM

Mike Fitzgerald: Learning Ruby

Sitting in the Monday morning session of OSCON 2007 listening to Mike Fitzgerald's Learning Ruby tutorial.

Mike recommends the following books:
Ruby in a Nutshell (Matz)
Programming Ruby (Dave Thomas)
Ruby Cookbook (Lucas Carlson, Leonard Richardson)
Learning Ruby (Mike Fitzgerald)
Mike also refers to the ruby-doc site (generated by rdoc) as a great place for documentation about classes and methods.

Mike gives kudos to Matz, the creator and founder of Ruby.

Matz wanted to create a language since high school. Ruby development started in 1993, was first released 1995 (same year as Java). Didn't get attention in the west until around 2000, 2003 really into mainstream because of Ruby on Rails. Ruby was named by the person who wrote irb, the Ruby runtime environment who's birthstone was ruby.

What is Ruby? Object oriented. Interpreted, not compiled. Written in C, allows have C extension. Takes ideas from Lisp, Perl, Smalltalk etc.

Numbers are objects, not primitives. Most operators are methods. No declarations or static types (dynamic typing). Ruby does "duck" typing, treats variables. Ruby doesn't have ++ or --, but does allow abbreviated assignment. Can use underscores in numbers, which get ignored by the interpreter (1_000_000). Ranges can be inclusive (1..10) or exclusive (1...10).

Installing Ruby is pretty easy in most environments, if it isn't there already.

Mike now goes into a series of basic commands, looking at simple syntax examples running in irb.

For writing scripts #!/usr/bin/env ruby. Within a script use the backticks (`) or use the system method to execute a system call from within the script.

This line of code is interesting:

3.times { print "Go!" }

The 3 is an object (number) that has methods.

Variables start with a lowercase letter or _. Instance variables are @name, class variables are @@name, global variables ar $name (Matz discourages use of globals). Variables can be assigned in parallel (x, y = 37, 63).

Mike gives some interesting examples of very entrenched Java programmers that have switched wholesale. Switching a large Java application to RoR took about 10% of the time.

Ruby uses symbols as placeholders for strings and objects, makes it more efficient because there is only one copy of the data.

Expression substitution is done inside brackets prepended with #:

name = "guy"
puts "Hello #{name}"
puts "First argument #{ ARGV[0] }"

Different syntax for using a printf (string formatting) function:

hi = "Hello, %s"
puts hi % "people"

Works this way because % is actually a method that substitues the argument into the string.

Method definition looks like:

def hello
puts "Hi"

Can create aliases for methods as well:

alias hi hello

The hello method is in the ojbect main.

Ruby (return "Hello") automatically provides a method return value. Unless explicitly specified, the value returned from a method is the last evaluated expression.

In Ruby you do not overload methods, but you can override them. Ruby allows variable number of arguments which make overloading unnecessary. There is no typing of the arguments, but you could verify that the arguments belong to the correct class.

The block is very powerful in Ruby:

["Hi","Mike"].each { |e| print e }

Mike goes into detail about conditionals in Ruby (if, then, else, elsif, unless, begin, end, break, while, until, case, when). Can use colon or blocks to separate. Ruby allows you to put the conditional at the end of the statement (like Perl). Very nice for readability. In an array context, the >> pushes object onto an array.

Creating classes in Ruby is fairly straightforward. Each class will have an instantiation method run when object.new is defined.

Modules hold method definitions. You cannot instantiate modules, but you can include modules (referred to as mixin). Methods are implemented in modules but not available until the module is included.

Posted by mike at 8:35 AM

OSCON 2007 Underway

Yes, it is. I am here. Not speaking this year which makes the conference (much) more enjoyable.

Already bumped into a few familiar faces while registering and getting breakfast. Good to catch up with folks I haven't seen for awhile.

Posted by mike at 8:25 AM

The Spectacular Summer of 2007

Summer of 2007 has been spectacular. Unfortunately a little too much going on to be actively writing about it...but I'm trying to go through the backlog (starting now).

The main attractions this summer so far are (list might grow, but will eventually turn into links to more detailed entries):

  1. Trip to New York City
  2. Wedding on Georges Island
  3. Backpacking in Idaho
  4. O'Reilly Open Source Convention 2007

Posted by mike at 8:04 AM

July 5, 2007

Totally Worth It (iPhone)

I've only been using this recently purchased iPhone for a few minues now and am (as it appears to be for most users) very, very impressed. Even after watching the keynote in January, all of the build up since, watching the tutorial movies, reading all of the reviews, etc, etc, etc, it is still impressive and hard to believe how easy, intuitive, and natural this device is. What took so long? It seems so obvious using this phone how much the UI on handheld devices up to this point really hasn't been that good.

Many folks I know will scoff at spending so much money for a phone, and like the iPod, it's hard to explain why it is worth the expense. I only really worried about my splurge on the drive home from the Apple store. As soon as I got it open, registered, and started playing with all of that quickly turned into a feeling of satisfaction.

Maybe as time goes on there will be things that creep up that will bring and end to the honeymoon, but for now I'm happy with the decision and loving the thought of leaving my BlackBerry in the past.

Update: A few more hours have passed and I've gotten deeper into using the phone by syncing my data and media over and then actually using the phone during more routine daily activities. Even more conviced of its value.

So far this has all been over my home wifi, interested to see how things change with the experience when using the cell phone bandwidh.

Posted by mike at 4:57 PM

July 3, 2007

My Next Computer: Brushed Aluminum iMac

As a follow-up to my post about my home computing needs, there's a 90% chance that one of these machines will be on my desk shortly after they are announced.

I've looked carefully at the options. The Mac Pro is so much more expensive and I don't think the computing power and expansion justify the expense. There was a day when I'd go with that system because it is separate from the monitor and has a lot of extra room for scaling up down the road, but for more than twice the cost of an iMac it doesn't seem right.

The Mac mini is a step down from a laptop as far as power and capacity so wouldn't be much of a boost. Also a bad move compared to the iMace.

Hopefully these new iMacs will be priced similarly to the current iMac lineup and a good bump in processor speeds, memory slots, disk space etc. A faster DVD burner would be good too.

Posted by mike at 9:02 AM

July 2, 2007

Why I need two home computers

About a month ago I realized something fairly significant in my need/preference for personal and professional computing. The realization is that a lone laptop just isn't enough for me to keep my stuff together. I've been operating on a single laptop for many years, it has taken awhile for me to figure out why this doesn't really work for me.

I think the bottom line is one thing...disk space on the laptop is not keeping pace with today's computing needs.

When I bought my MacBook Pro a year ago the largest hard drive was 100 gigs. There are now larger ones available, but even the largest at 200 gigs this would still be an issue.

The problem right now is that with 100G I don't have room for anything more than applications, email, and documents.

Over the past year I've gradually removed the "other" things on my computer off to an external drive to clear space for day-to-day work. First to go were home movies (these consume a huge amount of space). Second to go was my iPhoto library. Next to go is my iTunes library. At this point I have so little space I can't even load up a home movie to burn a copy. I really hate having an external hard drive with stuff I want to use. Either I have to transfer it onto the laptop (slow) or use it directly off the external hard drive (also slow).

So I've decided that I could greatly improve my situation by adding a desktop computer that has a ton of storage, more processing power, and a huge screen. This computer will hold my iTunes, iPhoto, movies, and maybe a backup of some of my laptop files. It will be my working machine while in my office, using the underlying Unix tools for work.

My laptop will continue to operate as my primary mobile machine, have email and all of my personal and work files/documents that I need for on the go. The idea is that I can grab my laptop at any moment and have all work and personal stuff to keep going somewhere else. But while I'm in my office I can enjoy a larger screen, faster processing, unlimited disk space, and active access to my entire digital life.

I hope that for times when I'm on the road my laptop will have space to temporarily hold photos, music or movies until I get back home and can sync them over to the base station.

I guess this plan is still in progress, there might still need to be some tweaking over the next few months on what goes where.

Posted by mike at 10:29 AM