« June 2003 | Main | August 2003 »

July 25, 2003

Shakespeare on the Common (Hollywood's Formulas)

Went to see Macbeth performed on the Boston Common tonight. The weather was perfect for an outdoor show, evening was cool. Something about watching Shakespeare in the open air under the stars just seems right.

It's been awhile since I've read or seen something by Shakespeare, invariably at some point I end up thinking about how unoriginal Hollywood can be.

Someday I'd like to see a book or paper along the lines of "Guidebook to Hollwood Movie Formulas," which will cover both general formulas for entire movies as well as formulas for specific plot developments. Example overall formula would be something like "man/woman is dicovered asleep/unconcious and spends two hours disovering and getting out of a horribly scary/complex/thrilling past" (Bourne Identity, The Long Kiss Goodnight). Example specific plot development is something like "misunderstanding between man/woman sends them separate ways followed by a montage of actors in deep reflection with full-volume, heart-tugging music, convincing actors to rush back to each other."

This thread gets at the idea.

Posted by mike at 11:35 PM

July 24, 2003

Linux Discussion with Tufts USG

Today I made the trip from Boston to Medford for a meeting with folks at Tufts USG (University System Group). The primary purpose was to explore existing options for a new service, the meeting ended up on a delightful tangent.

I've long wanted to broach the subject of Linux with the USG folks, but didn't have a sense of the climate. Most of what I've heard and seen indicates there could be little tolerance for options outside of Sun.

To my delight, as our conversation turned to the possibility of running the new service on Linux, the USG director indicated that there was some existing support, and that movement was being made toward offering support similar to USG support for Solaris. Existing Linux support includes tapping into the nagios alert system, backups with legato, and cfengine maintenance of core config files. The primary service not offered for Linux is machine builds, currently done over a private network with Jumpstart. Sounds like in the next year USG will make Linux builds available.

On my most recent visit to the data center I had seen a cluster of Dell machines, come to find out they are USG-maintained Linux boxes. Upon some further questioning the USG director indicated they were leaning to Gentoo as the distribution of choice, simply for complete control over what is going on the machines (as opposed to other distributations which install everything by default). That got some of our folks excited . . . 2 of 3 developers swear by Gentoo, and I anticipate soon enough it will be 3 of 3.

I was envigorated by this conversation. On the way back my manager asked "Why are you so hot for Lnux?" Will try to tone it down a little, want to make sure we weigh all options and make the right choice.

Posted by mike at 11:05 AM

July 23, 2003

Industry Thoughts on Solaris/Sun vs Linux

I keep stumbling into articles, weblog entries, conversations where people are hinting/claiming/predicting that at some not-so-distant point Sun will be in such trouble that people using their hardware/OS will be in a bad predicament. Thought I'd take a stab at gathering some of those sources, which play an important part in our Linux plan.

Wired recently published McNealy's Last Stand, an article that ruffled a few feathers. In the article sources suggest uncertainty about where Sun will be in 5 years.

A Linuxworld opinion on the subject. News about Linux 2.6 kernel takes a jab in the last sentence, suggesting that with the new kernel features there is even less reason to stick with other Unix systems.

Newsfactor discusses problems with Sun's strategy in Solaris vs. Linux.

Have to pause on this entry . . . most likely will continue to build over time.

Posted by mike at 10:36 AM

July 21, 2003

Getting More Than Zero Nines

Since the inception of our project, we have never cared to keep track of downtime, and translate that into percentage of uptime.

So one could either say based on our lack of records either we've never had uptime, or we've never had downtime. I'm determined to change our attention to uptime, if nothing else to put a little pressure on myself (and a few team members) when considering making one of those mid-day, hope-it-works changes to production. So I started a small log in which I record any downtime, planned or unplanned.

My first entry was after 14 minutes of downtime for replacing a bettery in our hardware RAID array. I guess at that point if someone would have asked what our uptime statistics were I would have to had said 0.001% uptime.

It's now been 42 days and we've experienced an additional 45 seconds of downtime for Apache restarts (loading Perl libraries). I guess that works out to around 99.999%.

Have seen this a few places, most recently Jim documented the numbers on the nines.

Somehow any number of nines doesn't make me feel any better. There's a lot more to an application than being boolean ) or 1. What about the users who are experiencing bugs in the application, to them the system might as well be down.

Posted by mike at 4:57 PM

Seems Like a Good Choice for UPS

Last weekend we had a power-outage in the office (while no-one was here). Not sure how long it lasted, but it's raised a few questions around here, and prompted a search into getting a UPS.

Our primary, mission-critical machines are in a data center, but it seems that some of the development machines sitting under various desks in the office are referenced more regularly than we thought, and having them down is noticeable to people other than ourselves. And yes, we do care about people other than ourselves. Getting the *prefect* UPS isn't really a priority, just want to quickly poke around and get something that will do the job.

Didn't take much looking around to put together an idea of what we need. Stumbled into a nice UPS overview which provided good information both on UPS on Linux as well as how to calculate needs. The power-usage section estimated each machine taking around 60W.

Will be putting three or four boxes on the UPS, and want 30+ minutes to avoid short power interruptions and have time to shut down if the power doesn't come back. Decided to narrow my range to a UPS that will offer 300W for ~30 minutes.

I looked at APC, TrippLite, Belkin and Best Power, ended up choosing the APC Smart-UPS 1000VA. APC appeared to be a good choice. Good history, highly recommended and provides auto-shutdown software for Linux OS.

The poor life of a UPS, it's one of those things you put in place and then forget about. Most likely will protect us from numerous outages without us ever noticing. Guess it can only look forward to the time 2 or 3 years down the road when it's time to replace the battery.

Posted by mike at 3:55 PM

July 15, 2003

Microsoft Responds to Complaints about OSCON Lunch

I got the following email from Microsoft regarding my weblog complaint about Microsoft paying for OSCON lunch:

i noted your objections to the Microsoft-sponsered lunches at O'Reilly's conference. thought I'd quickly try to explain.

last year, i went to OScon02, and the prices of lunches were covered by O'Reilly as part of admission. nice meals.

so when when i noted that OScon03 didn't include lunches in the price of admission (cost-cutting at O'Reilly), and noted that the Microsoft marketing people wanted to do something like a LinuxWorld booth at OScon, i said that it wouldn't be appropriate, but why not help out with the lunches. that is how it got started.

the "free as in lunch" sign was my idea of a joke, too. unfortunately, they wouldn't let us serve beer at lunch, which was the point of the joke of the sign. also, the way the marketing people did the sign, the quotes around "free" only goofed up things a bit, so that some people didn't get the free-as-in-(beer,freedom) reference, but thought it was some kind of "no such thing as a free lunch" insult, which was *absolutely* note the case. and they even managed to mess up the basic smiley emoticon, forgot the -.

hope this helps clarify things a bit.

Was nice to get some clarification on how this all went down, still sorting out what it all means. I took a photo of the poster referred to in the email. Thanks to the Microsoft employee for taking the time to explain.

Posted by mike at 9:16 AM

A CERT Advisory I can Enjoy

CERT advisories can mean a chunk of work, depending on the particular vulnerablity. More often than not they are irrelevant given that my desktop is OS X, Andy's OS is Gentoo Linux, and we're running minimal installs of Solaris or Red Hat Linux on our other machines. Yesterday's CERT advisory is yet another entertaining read.

A buffer overflow vulnerability exists in a shared HTML conversion library used by Internet Explorer (IE) and other Windows applications. By enticing a victim to view an HTML document using IE, an attacker could execute arbitrary code with the victim's privileges or cause IE to crash.

A quick chuckle and I'm back to work.

Posted by mike at 8:37 AM

July 14, 2003

Back Home and Not Quite Into the Swing of Things

Took the all-nighter flight back to Boston on Friday night. It's kind of crazy, but I'd rather be home and really out of it than to spend all day Saturday travelling (took the same flight back from OSCON last year as well).

Still having a hard time getting back to east-coast time, getting to the office by 7am is quite a challenge considering that's 4am west-coast time.

But it feels good to be back, and cracking away at projects with a little better understanding of Perl, MySQL, project development etc.

Posted by mike at 9:27 PM

July 11, 2003


Attended the keynote presentation this morning by George Dyson about Von Neumann at the IAS (Institute for Advanced Studies). Fascinating look into some of the origins computing.

Dyson went to IAS and dug through documents and retrieved a collection of diagrams, photos and memos about the process and design of the original computer. The presentation intertwines all of them into a humorous and insightful look at engineering and programming in mid 1900's.

A few thoughts:

Used tubes to record the binary data, grid of spots stored in each tube and then could be checked, provided they could keep them in focus and no one walked by with a static sweater.

Back then the hardware was sloppy and the question was how to build reliable programs on sloppy hardware, today the hardware is good and the software is the sloppy element in unreliable programs.

Phil Windley's write-up gets some stuff I missed.

Spent the morning in Jeremy's MySQL talks. Both good, got a slew of great tips on tuning and benchmarking our MySQL database. I wish I had two or three weeks that I could dedicate to fiddling with our MySQL install. The only way that would happen is if MySQL started to fail, nothing like downtime to prioritize tuning. The demo on super-smack was cool, seems fairly straightforward to configure and run, am excited to use it.

Posted by mike at 12:30 PM

July 10, 2003


Today was different from all days, nervousness about my presentation set in and made it hard to enjoy or concentrate. I spent the day in the XML track seeing tutorials on Shorthand XML, Tips for Learning XSLT and 5 Things we do Wrong with XML.

Then I was on. The response to the presentation was better than I thought. The attendees were more reactive than I anticipated, and found myself relaxing a bit as I got into it. Thanks to everyone who was there for making a friendly environment. I had a few problems with using dual display and the PDF disappearing, but nothing major. I think for the future I'll avoid doing demos, just include everything on the slides (which can be found here). I think I could have been clearer on the different items stored in TUSK and how the presentation was primarily about the documents, not images etc.

Was energized by the number and quality of questions, they demonstrated interest and enthusiasm in how we've tackled this problem. Afterward a small group gathered up front and continued discussion. Was glad to meet Phil Windley (a friend of Pete's) and Jeremy Zawodny. Kind of nice to meet folks face-to-face after reading entry after entry in weblogs.

Felt a surge of energy leaving the room, not sure if it was relief of it being over or because it went well. Jumped on the Max and rode out to Gresham. Walked around a bit, trying to get a feel for greater Portland in case we ever want to move from Boston.

Posted by mike at 11:27 PM

Web Page Caching at OSCON

Somewhere along the path from my laptop here at OSCON to the kruckenberg server there is a machine caching web pages. Discovered this this morning after I created a weblog entry and saved it. No matter how many times I reloaded my site I couldn't get the new entry to show up. Tried a trick I've used in the past to get around the cache and found that the new entry was indeed there, but requests for my index page were not getting to the kruckenberg machine.

http://mike.kruckenberg.com/ - loads a page cached from yesterday
http://mike.kruckenberg.com/?1234 - loads the new page

Someday I'd love to get a detailed tour of caching practices.

I should be grateful that I actually have a network connection, something I couldn't get all day yesterday.

The caching is not happening from my hotel room via the high-speed connection the hotel provides.

Posted by mike at 1:00 PM

July 9, 2003


This morning started up with keynotes from Tim O'Reilly, interesting thoughts about a paradigm shift in how we think about open source. One point he made that stuck in my mind was this conversation where a Linux opponent comments how bad the user interface is on Linux. Typically a Linux advocate will respond with "have you seen the latest Gnome or GIMP?." Missing from that is discussion of web-based user interaces like Google, Amazon, etc. Are really good interfaces to Linux.

Today was the first day of sessions. Started out learning about Authentication in Apache 2.1 by Justin Erenkrantz, a fairly straightforward session about syntax and a view into some pieces of code.

Stayed with the Apache track for the next two sessions, Migrating to Apache 2.0 (which seemed close to something I saw last year) by Rich Bowen. At lunch went down to the vendor booths and did the typical tour getting my card stamped to get my two free books. Had good conversations with Pogo Linux folks, Helix, Amazon and GNOME folks. Am particularly interested in how people are deciding which vendor to buy into for Linux hardware and support.

*Did not* eat the Microsoft lunch.

After lunch I got back into the Apache track with Stas Beckman's presentation on mod_perl 2.0. Wish I could attend the session by Ticketmaster which talks about hiring Stas (or open source developers in general). Switched to the XML track to see what kind of stuff is going on with XML and Microsoft Office. Last presentation of the day was on Subversion, very exciting presentation about a replacement for CVS. Way too many people in the room, and so many questions to ask. Can't wait to make a plan to start using it.

Spent the evening practicing my presentation and taking a dip in the pool. I have worked on the presentation quite a bit, and massaged it over several weeks. I'm confident that I can deliver it, but am unsure how relevant or interesting it will be to folks. I guess I have to trust the OReilly staff who chose the presentation that they know what is of interest to the attendees, either way it will be over soon enough.

Posted by mike at 11:51 PM

In a Pissy Mood about OSCON and Open Source

I so rarely get in a bad mood that I figure I'd better enjoy this moment.

1) How am I supposed to feel confidence in Open Source software movements when the "State" addresses are some of the most boring, dry and unenthusiastic presentations I've ever heard? We need a Steve Jobs, or Damian Conway to give these kinds of addresses. If I ever had interest in fiddling with python it's gone now that I've learned it's founder is taking a new job and seems to lack confidence in future releases (I see Guido has changed his session to "Q & A about Python's Future".

2) What is Microsoft doing buying OSCON attendees lunch? I don't know what went on behind the scenes to get Microsoft involved in financing the lunches for Wednesday and Thursday (the only meals the conference is providing), but I don't feel good about eating it. Maybe I'm just not up to speed on Microsoft's directions, or there new plan to get people addicted to Server 2003 through secret microchips in the food.

3) OSCON attendees, OSCON badges are not required to get into restaurants and stores in Portland. Maybe I don't have the right kind of pride, or am not nerdy enough, but when I head out of the hotel I put my badge in my pocket. It does give me a good chuckle to see someone roaming around downtown with one. On Monday spotted one up by the steel bridge . . . a long way from home, cowboy.

4) Why can't they open the conference hall sooner that 15 minutes before the keynotes begin?

5) I think my grandma can climb stairs faster than the elevators here at the Marriot.

6) Is there anyone here who *hasn't* shaken their fist violently at the wireless network?

On the flip side (now that I've gotten that crap out):
1) I like that OSCON isn't doing meals, it means that 1500 of us are dumping our cash into local business and don't have to eat what a caterer planned. Of course, it can be inconveniennt to find something to eat first thing in the morning. Maybe having muffins, bagels and juice (from Microsoft) would be a good comprimise.

2) My presentation skills might suck too, glad I'm only speaking to a small portion of the people here.

Posted by mike at 11:30 AM


A technically exciting day, attended Damian Conway's Advanced Object Oriented Perl tutorial (full day). Found Damian to be both technically incredible at Perl, but also a dynamic, enjoyable speaker. Made the subject matter and learning more fun than I thought imaginable. I came out with some excellent methods for tackling problems with Perl as well as greater pride in the language.

Besides Larry Wall, the "State Addresses" weren't so great. The remainder of the state addresses (python,PHP,MySQL,Apache and Linux) were varying degrees of boring/dry/flat. This is a stark contrast from last year's Internet Game Show, which made for a fun evening. Found a lot of enjoyment watching the IRC channel.

Estimate for this year's conference are around 1500. Last year attracted ~1300.

Hanging out in the lobby, using the wireless network, waiting for the elevator demand to die down, can't stand waiting for 10 minutes and then find the elevator had already filled up one floor down.

Posted by mike at 1:38 AM

Using IRC

I *finally* pulled my act together and put an IRC client on my machine. Have heard many people's comments on using IRC in either the workplace or in the development community.

It was enlightening during the evenings "State Addresses" to watch the IRC channel and make comments.

Now I need to get pointers to one or two useful channels that I can watch from time to time.

Posted by mike at 1:19 AM

July 8, 2003

Concat Drives to Expand Disk Space

Have been needing some extra disk space to store our history of MySQL backups. Each night I do a hot backup of our MySQL databases on our RAID array. After the backup I move yesterday's backup off to another disk, and store a handful of day's worth of backups.

So my backup history disk was getting too full, setting off our alert system. I had a few older 9G drives in the machine and discovered it's pretty easy to concat those drives onto an existing drive. Added an additional 18G to the disk, which should give a little breathing room. At some point (when the drive concat is too small) will most likely replace the drives.

Posted by mike at 2:29 PM

Disabling Rendezveus Service

Apparently the Mac OS X users are causing some problems here at OSCON. We've been asked to issue this command to shut off wireless network sharing:

sudo /System/Library/StartupItems/mDNSResponder/mDNSResponder stop

Even though I have not configured my mac to share the wireless connection the script indicated that it was able to stop the responder:

Stopping Apple Multicast DNS Responder

Should I slap that in my startup, or just remember to issue the command when I power up?

Posted by mike at 1:41 PM

July 7, 2003


Day 1 of OSCON 2003 is drawing to a close. The day started with a great run around the riverwalk loop (up the west side, over the steel bridge, down the east side).

I attended What Works in Software Development, which was excellent. Took away some good notes for needed improvements in our development at Tufts.

I tracked down the Snow White House crepe truck at lunch. I couldn't decide between a few things so I decided to get one and go back tomorrow afternoon. A huge crepe for $5.

I attended Jabber Boot Camp in the afternoon. Also an excellent presentation, somewhat frustrating how often we've discussed moving to Jabber (from AOL) but setting it up never gets prioritized.

I raced out of the conference and headed up to Washington Park to spend some time in the Rose Garden. I don't care much for flowers, but the view was nice and it was peaceful. The amphitheater is a nice place to sit and reflect. The bus driver and I had a lengthy conversation about Portland, comparing Boston. I got of the Max and headed to the Pearl District to see if I could grab something to eat at the Pearl Bakery but it was closed. Will have to go back another day. Walked back to the hotel along the river.

I have added more photos to the gallery.

Posted by mike at 11:22 PM

1/3 Requests on Kruckenberg are for default.ida

Trying to put some new photos in the gallery today and noticed that the pages were loading slow so I started poking at the Apache error log. Discovered 100% of the error log entries were failed requests for default.ida (with the usual extended string of chars). A quick grep of the Apache access file reveals that around 1/3 of our traffic is for default.ida.

A grep|sort|uniq reveals that most of the IP addresses are unique. For a sample time period we got 388 requests from 288 unique IP addresses. 13 is the highest number of requests from any unique IP address.

Too bad we're not looking for venture money and can flaunt those extra "hits" in front of some investors who haven't made it out of the 90s (still can't believe some of the "here's-how-we-will-boost-our-hits" conversations I've been privy to back in those days).

Just to get that junk out of the error logs I created a dummy default.ida file. Seems like the simplest solution. Maybe someday will stick something in httpd.conf to remove the problem altogether.

Posted by mike at 10:38 PM

July 6, 2003

OSCON Gets Started

Arrived in Portland, OR today for the O'Reilly Open Source Convention. Took a few photos on a nice walk around town before registration opened up.

Am pleased with the hotel, got the $9.95/day high-speed internet access in my room, was pretty sure the wireless network from the conference wouldn't reach to the 14th floor.

The weather is perfect, looking forward to a great conference. Will be posting more pictures as the week progresses.

Posted by mike at 8:34 PM

July 5, 2003

Perfect Day for Sailing

Today we met up with some friends in Swampscott and had a wonderful afternoon sailing out into the ocean/harbor. Not sure about the boat details, has a cabin which sleeps 4, has a stove and a deck which seats 6 or 8.

Was a perfect day for sailing, approaching 100 degrees (with heat index) in Boston and before we were 50 yards offshore it was much cooler. We sailed for around 3 hours and enjoyed seeing some of the islands in the Boston harbor. The wind was fairly strong and required we only use the jib, never even uncovered the mainsail.

Posted by mike at 8:03 PM

July 4, 2003

5K Race to Declare Independance

This morning we travelled to Natick, MA and ran in the Rizzo Run for Liberty. A 5K race that winds through the semi-rural residential neighborhood and ends in Natick Center.

Overall it was a great way to start the day. The temperature was still somewhat cool (race started at 7:30am) and the green trees, nice houses and cheers from the people of Natick made it memorable. We ran around 10 minute miles, nothing incredible, but an improvement from the last race I ran (back in 1997).

As we pulled away Heidi and I wondered if this might become a tradition.

Posted by mike at 10:22 AM

July 3, 2003

Building Packages for Perl Modules (closure)

A few weeks back I wrote about building packages from Perl modules, want to finish that up.

As I started to go through the process of downloading, untarring, configing, compiling and installing each Perl module or package I longed for an easier method so I reconsidered Pete's recommendation of using the CPAN module's shell. The shell makes it easy to lump all stages of the install into one.

Once the install was complete it became fairly straightforward to create the package using my alternate package building method.

Posted by mike at 8:57 PM

July 1, 2003

Kinetic Water Ram Arrives (and is awesome)

My kinetic water ram arrived today. The simple explanation is that it's built to force out clogs in drains, but it deserves a better explanation than that.

The water ram sends a burst of compressed air into a column of water (presumably starting from your sink or tub all the way down to the clogged area), creating a fast moving wave that breaks through almost anything (an illustration on the site shows breaking a toothbrush). 98% of the blast travels through the water, only 2% goes into the pipes. Don't let the photo fool you, this thing is heavy duty (21" long, industrial grade steel) and is made for the professional plumber.

The arrival of this tool brings me much needed comfort. Our house is old, the tub drain pipe is small, has lots of corners, a drum trap and clogs every month or so. To clean the drain means opening the drum trap from below in the kitchen getting disgusting drain water running/dripping everywhere (yea, it's nasty). The alternative is to call a plummer who charges $140 for a 30 minute visit, which we did once (where I first saw a water ram).

The kinetic water ram made this process easy and clean. I filled the tub until it was backed up, pumped the ram to 10 lbs, stuck the ram into the drain and pulled the trigger, sending a kinetic shock down the drain and clearing out the problem.

Posted by mike at 10:59 PM