July 28, 2005
Tufts OpenCourseWare Goes Live
Awhile back I wrote an entry about attending a meeting about OpenCourseWare at MIT and hinted at us being involved.
Indeed, Tufts University has been in the process of it's own site with open courses; Tufts OpenCourseWare. We had a soft go-live on June 20th where we pushed the first seven courses out to the Akamai network (or rather, they pulled it from our server).
My involvement? The site is build by an extract from the Tufts University Sciences Knowledgebase (TUSK), the system I work on at Tufts. Combining TUSK with the extract tool has a very weblog feel to it. The content is managed in our interactive management system and then every so often we publish to our origin server where Akamai can pick up the latest.
The whole project was done by a variety of departments at Tufts, and on almost no budget. The latest news is we were just awarded a small grant from the Hewlett Foundation to keep the work going and get another set of courses out there.
I haven't seen the usage reports yet, but have heard rumors about worldwide traffic and some great feedback. Exciting.
Posted by mike at 11:34 PM
MySQL 5.0.10 beta Fixes Significant Trigger Issues
Delighted to see that the release of MySQL 5.0.10 brings the following changes to triggers in MySQL (from the release notes):
- An extension to the SHOW command has been added: SHOW TRIGGERS
can be used to view a listing of triggers.
- The INFORMATION_SCHEMA database now includes a TRIGGERS table.
- It is no longer necessary to issue an explicit LOCK TABLES for any
tables accessed by a trigger prior to executing any statements that
might invoke the trigger.
Some of this was already in the code repository when the book, Pro MySQL was in it's final stages. Still, the wording in the chapter said things like "coming soon" or "this bug is reported and will likely be fixed by the time you read this."
I'm not sure I'm currently able of making a MySQL post withouth saying how whatever it is relates to the book. My apologies, that probably get's annoying.
Posted by mike at 11:03 PM
July 26, 2005
Getting to Know Sendmail
To date, I've always been fortunate enough to have a pre-installed and working version of Sendmail already running on a machine, or have had access to an expert who can do some Sendmail magic and just make it work.
This time around with the kruckenberg server rebuild, Pete spent a great deal of time getting a new machine set up (RAID 5 across 4 10,000 RPM 36G drives) and Gentoo bootstrapped and installed. After spending an entire Saturday working on that he was ready to have someone else pitch in.
In my mind, the most important (dare I say critical?) thing was having mail back up so people weren't getting bounces and the other family members could function. The four main tasks were:
- Sendmail processing incoming mail
- Sendmail processing outbound mail (local and relay)
- secure imap
- Web-based mail client over SSL (Squirrel Mail)
The truth is that I've done some tweaking with Sendmail but never where I was starting from scratch. I had the files from /etc/mail on the old server, which aleviated a good deal of the setup work. I ran into a few glitches where the new version of Sendmail didn't like what was in the old sendmail.cf which led to some changes in sendmail.mc. Nothing major there.
Once I thought I had Sendmail up and I watched /var/log/messages and saw lots of mail coming in but none of it was getting to /ver/spool/mail/mike. I scratched my head for a long time on this one until I realized that it was the local mail delivery through procmail that was the problem. /etc/procmailrc was configured to deliver to /home/.maildir. I changed it to DEFAULT=/var/spool/mail/$LOGNAME and immediately started seeing the mail going onto the mbox files.
Outbound messages worked just fine without any additional adjustments or complications.
Getting secure IMAP (or IMAPS) set up took me a long time, not because it was difficult but because I didn't heed the obvious. In theory all you need to do is change one line in /etc/xinetd.d/imaps to disable = no. After a lot of playing with the configuration options I decided to replace the pre-installed (on Gentoo Linux) .pem file with one I generated myself. This solved the IMAPS SSL connection failures immediately. It made me wonder why there is a pre-installed imapd.pem in /etc/ssl/certs/. If one hadn't been there I would have generated one myself and had IMAPS up and running much sooner.
With mail going in and out and IAMPS running, the next step was Apache, SSL and the web-based mail client. All fairly straightforward, I hadn't set up Squirrel Mail before but there's really not much more than untarring, putting the code in the right place, adding a few directives to the Apache config and changing a few items in the Squirrel Mail config file.
Mail delivered succesfully for several days and then last Friday I noticed that after 10:30 I wasn't seeing any mail. By 6pm, after taking some time to look around I noticed that mqueue showed ~1000 messages and many of them were on hold waiting for domain lookup. Lo and behold there was an issue with the nameservers. I changed a few of the nameserver addresses in /etc/resolv.conf, made sure we were getting lookups by issueing few server pings, restarted Sendmail and then watched as the mail queue started emptying.
Mail has been working fine ever since. Getting everything up and running, and then troubleshooting issues has been a nice experience. While frustrating at times, it feels good to have a better understanding of a major part of the system that previously has been a mystery.
At some point I'd like to get up to speed on bind. I've done DNS administration through a GUI, but haven't really looked under the hood at what's going on.
Posted by mike at 7:48 AM
July 25, 2005
Official Publication Date of Pro MySQL
Today is the official publication date of Pro MySQL, the book Jay Pipes and I slaved over for the past 6 months. It is very exciting to finally have it out there. We worked more than I ever could have imagined it would take.
I hope the book at least sells enough to make it worth all of the work, but more than that I hope it is useful to users of MySQL interested in some of the more advanced database uses.
I haven't seen my copy yet, it's apparently in the mail. Hopefully will arrive in the mail today.
Update: Added a photo, the cover changed toward the end of the publication process. Also, I found out that my copies of the book are sitting at my old house (were shipped to the old address). Will have to run over and grab them tonight.
Posted by mike at 12:26 AM
July 24, 2005
Pro MySQL forward by Brian Aker
Have known about this for a few weeks but thought it better to wait. Back at the MySQL conference I chatted briefly with Brian Aker (MySQL AB Director of Architect) about one of the chapters in Pro MySQL. He suggested I have Apress send along a copy of the book for him to look over and possibly review.
The folks at Apress thought it a good idea to see if Brian would be willing to write a forward for the book. I wasn't privy to all of the back and forth, but it in the end Apress fed most of the chapters to Brian who read and agreed to write a forward for the book. Of course we were pretty nervous in anticipation of Brian's comments, was an exciting day for Jay and me reading Brian's comments, who was quite complimentary.
Thanks Brian, your nice words in the forward mean a lot.
Posted by mike at 12:08 AM
July 23, 2005
One Week of Downtime . . . Priceless
Last week we underwent a complete server rebuild, which resulted in this weblog being offline until this morning. Sorry to anyone who got the server can't be found message.
I really had hoped to have the weblog back sooner, but having a healthy sendmail and a web-based email client for family members to use was the higher priority. With those complete, I finally got to copying over the database and code for MovableType. In the interest of time I didn't even look into upgrading MT.
You might ask why we didn't plan better and bring the new machine up before taking the old one down. The answer is that the old one got hacked (again) and we really didn't want to leave it online. We've been planning a hardware upgrade for some time and I guess having a machine hacked was the nudge we needed to get moving.
The new machine(s) that we're using are a significant improvement over what we were using, hopefully this results in better response time for page loads.
Posted by mike at 10:17 AM
July 15, 2005
Evening Listening to Podsafe Music
A few weeks ago when I upgraded to iTunes 4.9 I poked around at the podcasting features and I guess I subscribed to a few. A few months back I regularly listened to the Daily Source Code. I guess I subscribed because I found today's episode on my iPod on the way home. It was all about podsafe music and the new podsafe music network that's starting. It's kind of like the legal music sites from before iTunes, the ones that collected music from less-known artist.
The podsafe music is like those services, with the addition of the obvious agreement that the artists will allow music to be used in a podcast. Registration is required to download music, but you can sample without registering. There's some pretty good stuff up there.
The thing I like the most is that I'm not in the mindset of trying to find music I know. The old legal mp3 services (pre-iTunes) marketed themselves as a legal Napster or Gnutella but when you went there to look for music, you couldn't find what you went for.
The podsafe music network is focused on artists who let their music free, which is usually not big name musicians. Listening is a refreshing experience, a chance to check out some great bands that you've never heard.
The trick, in my opinion, will be how the music is organized. There will eventually be thousands of artists and tunes to sift through. The genre groupings, rating system, tools to find new songs/artists, and interconnections between songs played on podcasts and the artists will hopefully help in giving users a way to find what they are looking for.
Posted by mike at 11:49 PM
Today we said goodby to Bruce, a TUSK (Tufts University Sciences Knowledgebase) developer. Bruce is starting law school at Boston University in the fall and is taking a month to do some traveling in Russia.
Bruce will be missed. He added a lot of technical expertise to our team. More than that, he had a way of asking the right questions and a great sense of how to approach a problem from the right angle to get to the heart of the issue. I will miss having him around to bounce ideas off, and will no doubt be in meetings where I will think "if only Bruce were here to shed some light on this conversation."
One thing that I've become more aware of while working with Bruce is that not doing things the right way the first time always ends in huge costs down the road. Not that I like doing things wrong, or incomplete, I just don't always see the larger picture and am not as good at anticipating future outcomes from current decisions.
In my mind, honoring Bruce and his time here at TUSK means being more committed to technical professionalism and going that extra length to make a project shine. I hope that will someday be said about my work here.
Posted by mike at 5:28 PM
July 11, 2005
Finding a New Programming Language
I'm looking into changing to a new language for our web development. We're using Perl for everything; Perl modules for data access and business logic and Mason (Embperl in old code) for templating. As we've attempted to find Perl programmers to fill open positions, we have a hard time finding developers with solid Perl experience, or developers who are interested in working in Perl. This has prompted a lot of discusson about leaving Perl for a language that will give us a larger pool of people to work with.
The question is almost more about the longevity of a language than about it's technical merits, but we need to look at both. There's also an element of interoperability in that we'd like to be exchanging data with other schools and potentially bolt our system together with others. That suggests we'll need to look at language adoption trends etc.
I've started digging around a bit to see what other languages might be a good fit for our application. We've got 400+ Perl modules, and around 650 presentation pages (including Mason, Embperl, XSLT and CSS stylesheets). 104,648 lines of presentation code and 99,770 lines of object code. Java has been suggested many times over many years, and I suspect that upper management will push us in that direction unless we find something more compelling.
I ran across an article from the Journal of Defense Software Engineering and liked a few paragraphs that talked about system languages and scripting languages:
System programming languages (C, Ada, Java, C++, etc.) are designed to develop applications from scratch with the help of a few class libraries. Scripting languages assume the existence of the necessary components and quickly and easily join those components together to form a larger application. System programming languages have high overhead in terms of their structure (try writing a Hello World! program in Java). Scripting languages can do quite a lot with just a few lines. A single line of scripting code may execute hundreds of machine code instructions where a system language may only execute tens of machine code instructions.
Scripting languages will never replace system languages, as scripting languages are not very good at programming complex algorithms and complex data structures, or for manipulating large data sets. However, scripting languages have their own strengths, including easily connecting pre-existing components, robustly manipulating a variety of data types from a variety of sources, rapidly developing GUIs, straightforward text manipulation, and creating and executing code on the fly. Scripting languages are the duct tape of the programming world.
Scripting languages are still very young compared to system languages. In all likelihood, many more evolutionary improvements will be made to them to make their strengths even greater. We predict that the easy work of complex algorithms, elaborate data structures, and brute force processing of large data sets will continue to be accomplished by system programming languages. More and more reusable components and services will be constructed using systems programming languages. However, the more difficult part of programming, that of developing a robust, easy-to-use application that is easily extended and modified as requirements change and the operational environment varies, will become more and more the job of scripting languages. Both types of languages will continue to evolve, but toward their strengths.
So what kind of language do we need? In addition to finding something that will facilitate organization of our code base, we need the following functionality:
- HTML building and delivery (duh)
- XML parsing/building
- XSLT processing
- Excel file building
- Database connectivity
- Database abstraction
- Image Processing (reseze, annotate, overlay)
- LDAP (secure)
- HTTP communication
- System calls
More to come . . .
Posted by mike at 3:09 PM
July 8, 2005
Golfing in Emmett, ID
It's easy to convince Heidi's dad (father-in-law) to do at least one or two rounds of golf when in Idaho. This afternoon (temperatures climed to over 100) we hit the Gem County Golf Course.
To me, the best golfing is on courses described as "very forgiving" or "good for beginners" which I think describes the Emmett course pretty well. There aren't many hazards (only a few ponds and less sand traps) and the fairway's are wide. The price, $11 for 9 holes and $20 for 18, is hard to beat.
I golf about once a year and wouldn't classify myself much higher than a beginner. If I can golf double-bogey (two shots over on each hole) I'm pretty happy.
Today, dad and I really did put on a clinic (for beginners). Although neither of us ever chalked up an eagle (were close once or twice), there were many holes that we got par and only a few that went out of control (7 or 8 strokes). All said, at the end of 9 holes, I was 3 under my goal of double-bogey. It's not really much to be proud of, but better than I had done in several years. On the fourth hole (par 4) I had the rare treat of a nice shot off the tee and then a 3-iron shot that put me on the green. Two shots to the green is pretty rare for me on a longer hole.
Dad and I did 9 holes on the Emmett course, and then another two rounds on a par 3 course dad has set up on their 7 acre piece of farmland (which they don't farm).
Posted by mike at 11:58 PM
July 7, 2005
At Lake Coeur d'Alene for a Few Days
Relaxing at friend's (John and Roberta) cabin on Lake Coeur d'Alene (Windy Bay) for a few days. We haven't been to this lake since we moved to Boston in 1998, so it's been a real treat to be back. The cabin we used to come to had electricity but no running water or phone. It was being cared for by friends, but was on tribal land which is now in the process of new development. The cabin we're in for the days we're here is not far down the road and is owned by distant relatives. It's a more modern cabin with flush toilets, showers, phone, hot tub etc.
The cabin sits a few dozen feet up off the lake, the view is really nice. 25 yards down a path and you are at the dock. We've done a variety of things including taking a spin in the paddle boat, swimming in the lake, soaking in the hot tub, playing games, eating good food, going on hikes and spending a lot of time napping. Evenings are spent playing games or just sitting around and talking.
It's going to be hard to leave this place.
Posted by mike at 8:26 AM
July 4, 2005
Spokane, WA for a Few Days
Staying with friends in Spokane, Washington for a few days of pure vacation (no book, no work) for Independance day (July 4th). On Tuesday we head to Lake Coeur d' Alene in Idaho for a few days of pure relaxation and fun.
It is so nice to have nothing to do. On the flight out I watched Miss Congeniality 2 and enjoyed every minute I had to waste on that movie. It's been awhile since I was on a plane and didn't have some major deadline looming that demanded attention.
Posted by mike at 2:35 AM