« June 2006 | Main | August 2006 »

July 28, 2006

Debugging Perl in Apache

Years ago I worked on a Apache/Perl-based web where Apache/mod_perl had been compiled by a former contractor. It had this cool feature that when Apache was starting if there was a compilation problem the startup would abort and print the Perl error on the terminal. Made it easy to see and resolve issues quickly.

Later on when I had to rebuild our Apache from source we lost that feature (no documentation on the build process). I spent a little time trying to find the solution, but never did.

I must not have stumbled into the mod_perl Debugging document, which has a lot of information on setting up debugging in various levels of detail and locations in your Apache/Perl environment.

Side note: I've really gotten to appreciate the Perl debugger over the past few months. Yes, adding print statements is a hard habit to break, but in most instances the debugger is easier, doesn't mess up the code, and adds some additional value (like being able to move down a few lines of code and test values again).

Posted by mike at 12:45 PM

The Pain of Upgrading to Apache 2 for Perl (mod_perl) Folks

I've spent the past few days working toward upgrading our webservers from Apache 1.3.x to Apache 2 (2.0.58 at the moment). As I mentioned earlier, I've done an Apache upgrade before (and have a long history with Apache and mod_perl), but never in instances where mod_perl was used to bolt Perl onto Apache (maybe infuse is a better term). I've read various conversion documents, and even attended a tutorial or two on making the switch but haven't seen much in the way of upgrading when working with mod_perl.

The bottom line is that after spending two days working toward getting our Perl code to play nicely with Apache 2 I've gotten most of the page loads working but we've decided that it's not worth any more attention (too many outstanding issues and uncertainties). Bummer, I'd like to tame this beast at some point. I was upgrading so we could use mod_deflate for streaming compressed data, but we can do that with Apache 1.3 and Apache::Dynagzip so are going to do that for now (the benefit of mod_deflate is not worth the effort and risk of Apache 2). This mod_perl compression FAQ is excellent for an overview of the choices.

So what makes moving from Apache 1.3 to 2 with mod_perl a difficult transition?

Complexity of Application

I need to note that the instances where upgrading to Apache 2 seemed unsurmountable have been very large applications, with hundreds of Perl modules and hundreds of thousands of lines of code. These aren't small perl scripts running in Apache, these are major applications that are tied closely to the various Apache request and response stages. Makes a big difference.

New Modules

If you use any of the mod_perl-installed Apache modules you have to replace them with Apache2, ModPerl or APR modules. Unfortunately it's not as easy as grepping through the code and replacing the class name. In many cases there's a new object or method to get what you're looking for. In some instances the arguments and return value from a method has changed which may require reworking some of your code.

For example, $r = Apache->request() is replaced with $r = Apache2::RequestUtil::request(). $r->args used to be able to return an array but now will return a string only. This one isn't an issue for us, but if you were processing $r->args as an array you'd have to rework your code to be happy getting a string. You get the idea.

Moving things over is definitely doable, just a matter of taking the time. Just don't be tricked into thinking that you can easily swap out your Apache 1 and seamlessly move up to Apache 2 with your Perl code not noticing a thing. It will notice in mostly obvious ways (and perhaps in subtle ways too).

Preloading Perl

I never got to the bottom of this, but we have a series of .pl files that get loaded into Apache when it starts. We use the example provided on page 637 of the Apache modules book, which is updated for Apache 2 in the documentation. The obvious was to convert Apache::RegistryLoader to ModPerl::RegistryLoader and replace the Apache::server_root_relative with it's new counterpart.

Once I had everything in our code changed to work with Apache 2, and had changed our preload.pl script to follow the docs, I started Apache. The modules wouldn't load though, it would attempt to run them but fail on each one. I traced things down to the run method in ModPerl::RegistryCooker and now suspect the failure had something to do with the loader not being able to change directories, something that Apache/mod_perl 1 could handle.

Alas I didn't get to figure it out because the bottom line of getting compression out the door for a customer didn't warrant all the uncertainty of this move. Hope to come back to it when there are other compelling reasons to upgrade.

The Unknown

I got a chance to peruse our application a bit running on Apache 2 and there were noticeable issues to deal with. The most obvious was JavaScript being printed to the screen where it wasn't before (we use CGI for most of our HTML generation). I also noticed pieces of data printed as object names (ARRAY...).

So even though I got close my guess is that we'd need a good long period of testing to give us time to address the unknown issues that would come up.


To the credit of the Apache-Perl folks, there's good documentation on the mapping between the old Apache stuff to Apache2 or ModPerl. Less impressive are the several places in the documentation that read something like this note in the ModPerl::Registry doc:

XXX: STOPPED here. Below is the old Apache::Registry document which I haven't worked through yet.

Which means reading or using anything below that line is probably going to lead to confusion and frustration.

Moving On

So I'm off onto other things now, but definitely a lot wiser for having really attempted and gotten close to completing a first pass at the upgrade. Not only did this give me a chance to see Apache 2 configured to do Perl it gave me a chance to dig down into the source of some of the Apache2 and ModPerl classes and see what's under the hood.

Posted by mike at 12:21 PM

July 25, 2006

Streaming Compressed Data with Apache Webserver

I'm fiddling with streaming compressed data from a set of Perl-driven web pages. I started working in our current webserver (Apache 1.3) where the clear choice seems to be Apache::Dynagzip.

This setup might be unique since we use Apache::Registry to get innocent CGI scripts to run in Apache with mod_perl, but you get the idea:

PerlModule Apache::Filter
PerlModule Apache::Dynagzip
<Files *.pl>
SetHandler perl-script
PerlSetVar Filter On
PerlSendHeader Off
PerlSetupEnv On
PerlHandler Apache::RegistryFilter Apache::Dynagzip

The only problem with this approach is that when you turn off Perl's ability to send header information, it cripples scripts that might need to push out a redirect or error.

Looking at the Apache 2 approach using mod_deflate . . . could it be that there's finally a reason to leave Apache 1.3 behind? I've been using Apache 2 for years but never have attempted an upgrade when mod_perl was involved.

Posted by mike at 3:58 PM

July 24, 2006

OSCON 2006 Underway

Not attending OSCON this year . . . the program looks excellent. Will be watching the official and other coverage.

Posted by mike at 1:52 PM

Ironic Problem on Planet MySQL

Trying to dig into what's been happening in MySQL-world over the last week and bumped into something that I've seen many times elsewhere, but did not expect to see on a site run by MySQL AB/Inc:

That is, of course, the error indicating that MySQL has reached it's connection limit and won't allow the page I requested to connect to the database to get the necessary information to display page 2.

Rumors are that there's an internal server infrastructure redesign underway. Maybe that includes the Planet/Forge server.

Posted by mike at 12:39 PM

July 23, 2006

Back from a Week in the Finger Lakes of NY

Got back Saturday evening from a week-long trip with Pete and family to the Finger Lakes area of upstate New York.

It was a great trip. We spent a lot of time around Honeoye Lake, where we had rented a lake house for the week, but ventured out on a few days to hit surrounding attractions. The finger lakes are suprisingly warm because of how shallow they are (which also makes them quite weedy). We swam in 82-degree water at Canandaigua Lake one day. It was hard to get us kids out of the water every day, but especially difficult that day.

The photo was taken at Stony Brook State Park which has a long, winding river that is fairly shallow with a smooth rock bed. Periodically on the walk up the river bed we'd come across falls with pools of water below for swimming (water was suprisingly warm). Some of the falls were small and had carved a smooth rock shute good for sliding down, others were very wide like the one shown here. The day spent hiking and swimming was definitely one of the highlights of the trip.

Now, to get caught up on things.

Posted by mike at 10:00 PM

July 14, 2006

Jim Starkey: Introducing Falcon (the video and the podcast)

This past Monday the Boston MySQL meetup featured Jim Starkey, a database guru who's hard at work coding a new database storage engine (code named Falcon) for MySQL.

This is a great presentation, Jim throws in a lot of database history and insight into working with MySQL. I left the camera on for the first bit of the Q/A. Also very good stuff.

I put in some extra video production effort this month. The video includes superimposed slides for better readability. Sheeri bought a new mic last month which captures the audience feedback better than in the past, seems to work well.

You can get the Quicktime here or here. Google video coming shortly.

Oh, and for folks who want it as a podcast (audio only), you can get that here.

As usual, thanks to Martin at kbglob for donating bandwidth.

Posted by mike at 8:07 AM

July 13, 2006

Book Sale

If you haven't picked up a copy of Pro MySQL, Barnes and Noble has a great deal on a selection of books called "Web Coding Tools." Between now and August 9th you can get 40% off these books.

That makes Pro MySQL $29.99 ($26.99 for members), not a bad price. You can order online if a physical B & N isn't an option. If there's a store nearby and you can get a photo of the special display these books are in there may be an iPod nano in it for you.

I took a trip to the downtown Boston Barnes and Noble yesterday to see for myself but they've closed that location. Bummer.

Posted by mike at 11:22 PM

July 11, 2006

History of Database Blobs

I learned something very important last night listening to Jim Starkey at the Boston MySQL meetup. It wasn't that Jim created the blob datatype on a scratch of paper while stuck in Colorado waiting out a snowstorm that had closed the Boston airport. It also wasn't that when he created the blob it was hard to sell it to the folks at DEC, and even after adoption they gave it a more sophisticated sounding name, "segmented strings".

No, the important thing I learned is that when Jim created the blob datatype it wasn't an acronym, it was a reference to the large jelly-like alien from the 1958 classic, The Blob. The BLOb (Binary Large Object) acronym was created later, much to Jim's dismay.

The history is nicely captured in these email responses from Jim and Ann Harrison (Jim's wife, also a database theorist/architect).

Posted by mike at 7:57 AM

July 10, 2006

Boston MySQL Meetup Tonight: Jim Starkey

If you don't know anything about Jim Starkey read this, this and this. He's coming to speak tonight at the Boston MySQL meetup.

Looks like it's going to be well attended. I'll have the video posted as soon as humanly possible.

Update: video and podcast now available

Posted by mike at 5:42 PM

Thunderbird Mail and Universal Binary on Mac

Despite what the Thunderbird mail client release notes say, the current version of Thunderbird running on my Intel mac does not seem to be running in Intel mode.

Reading further down the notes and it appears that in order to get it to run the Intel binary you have to completely remove the previous version of Thunderbird (drag the Application icon to the trash).

Ah, that's better. I guess something about just dropping a new version on top of the existing Application (aka upgrading) makes the PowerPC-ness stick.

Posted by mike at 3:50 PM

Life is Good in New Hampshire

I guess we just can't get enough of the White Mountain region of New Hampshire. We drove up Friday after work and spent two days at the lake house, getting back late Sunday. This time we took Heidi's sister and her boyfriend. Most of us would rather have stayed awhile longer, but we have another vacation starting next weekend and need to get things in order to be gone for a week. Of course work kind of gets in the way too.

This trip was much different than the last one, or the one before. It's summer and the lakes are warm (in March they were still frozen over). While at the lake house we made three trips for swimming at the beach (a 10-minute walk). The weather couldn't have been any better. It was cool in the shade but nice and hot out on the sandy beach. The lake has a picture-perfect beach, there's sand out to about 5' of water and then out in the deeper part of the lake a series of floating docks. With the sun shining you could see the bottom of the lake pretty well, mostly covered with rocks and grass. Must have been around 8' deep near the closest dock. Of course a trip out to the dock isn't complete without swimming underneath and coming up under the planks and spooking yourself with weeds touching your feet.

The highlight of our time at the beach (for me) was a challenge to swim out to a dock about half the distance of going across the lake. If you look closely in the picture you can see two black dots below the distant dock in the upper right, that's the two men on our way across the lake. We did a mix of crawl and breast stroke. Took about 20 minutes round trip, much faster than I would have expected.

We also had some pretty serious spades playing on both nights--boys vs girls. The first night the boys won hands down. The second night was much closer and looking like a victory for the girls. The last hand was amazing. The girls were 3 points from 300 (the number we set for winning) and we were back 137. We bid a blind 7 (worth 140, but you have to bid to win 7 hands without looking at the cards), stopped the girls from reaching their bid for 6 and won the game on a few well played cards. I've played cards for a long time but never felt like there was as much strategy as there was during that hand.

All in all it was a great weekend, each time we go there are serious conversations about if/when we will have our own lake house. It is never easy to come back home.

Posted by mike at 8:49 AM

July 5, 2006

5th Hole at Leo J Martin Golf Course

Turns out the usual suspects for golfing were all free on July 4th, so we headed to Leo J at 5am to catch a round of 18 before taking a few weeks off for various summer trips.

I took this photo on the 5th hole, a shorter hole off the end of the driving range that has a really cool sloping green. Another instance where my first stroke put me on the green for a shot at birdie (which I never actually pull off).

It was a record day for me, sliced three strokes off my all-time best for 18. Unfortunately that still puts me somewhere between bogey and double-bogey golf so it's not much more than a personal victory.

Posted by mike at 10:38 PM