« June 2004 | Main | August 2004 »

July 30, 2004

Damian Conway: Sufficiently Advanced Technologies

Damian has this talk where he encourages engineers to design Perl modules so they seem like magic, with simple interfaces that just do what you need done without having to figure them out. Some demonstration of what he means by looking at things like use strict and IO::All.

The bottom line is to examine your modules and determine if the subs are necessary and intuitive and by default do what you need without having to send in configuration information. Modules should attempt to understand the context, environment being called in and make decisions without the caller having to specify.

Some thought-provoking ideas that makes me think about how we could improve our modules.

Posted by mike at 12:30 PM

Novell's Move to Open Source

David Patrick, from Novell, is talking about making open source software the mainstream at OSCON 2004, day 5.

This last year Novell discontinued it's license with Microsoft. 90% of the 6,000 employees are using OpenOffice. 2000 have changed to a Linux desktop, and remainder will be on Linux desktop by end of year. Novell is putting together a desktop installation which will be used in the organization.

Proprietary software companies continue to lose licenses and raise maintenance fees.

Myths:

Posted by mike at 10:40 AM

David Rumsey: How Online Digital Libraries Provide Access to Culture

OSCON Day 5's first keynote is David Rumsey presenting an incredible collection of maps and a tool built to enable new ways to interact with them.

David suggests that having access to historical maps and images reveals the source code of cultural systems. Fair use, interoperability, content aggregation and cross site search capability seem to align with the ideals of open source.

10,000 maps in the repository, several terrabytes of images, largest being ~2G. Have a java client that lets you do all kinds of interesting things with images, load multiple at a time, zoom in, see relative sizes. Images have great deal of meta-data. One feature is to allow people to create a narrative with the images. Example is looking at a large-scale map of Mexico, zoom in on a top view of a volcano, open up a sketch of the volcano and then open up a small map which shows elevation from the sea.

David opens up a map made by Lewis and Clark, then overlays the current state boundaries and GIS plots of each of the camps. You can click on any of the camp-site dots and get GIS coordinates and information about the stay.

David opens up four maps of a small section of SanFrancisco, in a grid where you're looking at ~50 year time spans and talks about how difficult it would be to physically do something similar in a library. Looks at a small (10-block) chunk of LA and fades in different periods of time to get an idea of how the shoreline was developed over the past 100 years.

Are doing similar things with texts, gravestones, artwork (takes a look at an old Shakespeare book, comparing four different versions).

The application can search across repositories, and also use a local repository so you can put your own images.

All images are in OCLC with a URL to the repository. All images are Googled as well.

David saves the best for last, they took an old map of Yosemite, which has shading to show depth. They took an elevation map and merged the two creating a 3-D representation of the old map. The result is the power to do a fly-through or have an interactive, rotating image to zoom around.

Posted by mike at 9:48 AM

Notes on RSS Scaling Problem

I dropped by the RSS Scaling Problem BoF tonight. Even though I'm in a different ball game than Yahoo! or LiveJournal was interested in hearing the discussion. A bit of thinking about it has turned up a few thoughts from the ideas from the meeting.

Seems to me there are several different use cases that might demand different solutions. You've got the user who will launch the aggregator every time they want to read vs. the user who opens it up once and leaves it open all day. You also have the company who provides a weblog or feed service that gets millions of repeated requests for unchanged feeds and can afford to implement some large-scale solution vs the individual on a DSL modem who can afford nothing.

There were a few ideas in the meeting. A few stand out:

  1. respect the ttl tag if it's in the feed
  2. when a client requests a feed the HTTP connection is kept open and entries are fed down the wire as they are updated
  3. Create a DNS-like lookup where you can lookup the feed and get a response with the latest update time for that feed - perhaps be able to ask for last modified times of several feeds at once
  4. peer-to-peer feeds
  5. use jabber or chat-like protocol so feeds are sent when available

Maybe there can be some kind of combination:

The more I think about it the more I like the ttl idea, where if you're a techy and have your own server and know how to control the ttl you can, but if you're using a service it sets it for you based on your previous behavior (and lets you specify in preferences). The longest you'd have to wait if you change the ttl is until the end of your current ttl. Is it unrealistic to think that the client designers could respect the ttl?

Posted by mike at 1:32 AM

Looking forward to Solaris 10

Went to the Solaris 10 and Open Source BOF tonight. Four kernel engineers did a good job of fielding questions (sometimes a little hostile) about everything from kernel features to business and licensing issues with open source.

After a slew of questions they did demos for Zones (something like UML), DTrace (much improved truss-like tool), new system config (/etc/init.d has dependancy definitions), predictive self healing and ZFS (new file system). The conversation got much friendlier after the demos, and in some ways felt like a conversation you might have with a person from a company who used to be your competition but now are in partnership with. The guys were clearly excited about these projects and the attendees seemed to get into it.

The beta of Solaris 10 is available now.

As an annoyed-we-don't-use-Linux-at-work employee I got some joy at thinking of Solaris being improved by someone not employed by Sun. I wonder . . . some Sun enthusiasts have told me "I would never use Linux because you can't trust code built by a community." So what are you going to do now . . . switch to Longhorn?

Posted by mike at 12:09 AM

July 29, 2004

One reason to not eat dinner in a Portland park . . .

You look up just as you're chomping into your first bite and a dog is doing it's business right in front of you. Or perhaps find a park that isn't a regular spot for dog walkers.

Posted by mike at 8:30 PM

Building the Amazon.com Platform

Listening to David Ballenber, Senior Principal Engineer at Amazon.com talking about
building the Amazon.com platform at OSCON 2004. The slides are broken into a few categories, just
taking down some notable statistics.

About

Development

Operations

David makes this interesting point; building an order fullfilment center is a lot like writing an
operating system. You've got input, output, management of resources, processing queues etc.

Posted by mike at 3:26 PM

Managing 1,000 TicketMaster Servers In Your Spare Time

Listening to Sean Lynch from Ticketmaster talk about their process in managing 1,000+ servers.

Background

All systems should be active

Utilize a Flixible System Organization

Automate Everything

Posted by mike at 11:29 AM

Replace NFS with HTTP

Yesterday, during Brad Fitzpatrick's persentation on LiveJournal's backend I saw something I don't want to forget. Brad's presentation was a rapid-fire tour of the LiveJournal backend, demonstrating how they've dealt with having as much as 5 million dynamic-page views per day. Stuck in my mind is the bit about sharing files between servers. LJ attempted to use NFS, but found that there were memory leaks and under heavy load the files were getting corrupted.

Their solution was to switch to HTTP. The common HTTP methods are HEAD, GET and POST. I've seen the PUT and DELETE methods in documentation, but haven't actually seen them in use. LJ enabled these methods and is using HTTP to retrieve as well as manage the shared files. Interesting idea, might come in handy some day.

Posted by mike at 10:59 AM

Continuing Importance of Community Development

Second keynote of OSCON 2004 day 4 is Bdale Garbee from HP (Linux CTO). Bdale points out that a large percentage of open source developers start using or working on open source prujects in their free time. Interesting to think that the Wright brothers (airplane) and Robert Goddard (rocketry) both worked on their projects without funding in their spare time. Bdale was involved in an amateur sattelite project, OSCAR-7 put in orbit in 1974 and is still operational. OSCAR 40 took 4.5 million dollars donated in small chunks ($30) over 10 years.

Speaks about the Debian distribution as a good example of the efforts of community, used a social contract, guidelines document and constitution to help make decisions, clarify roles and assist with conflicts.

Posted by mike at 10:29 AM

Q & A with the Dysons: Freeman & George

OSCON Day 4 starts with a keynote where Freeman and George are on the stage answering questions from Tim O'Reilly. Ester was supposed to be here but got stuck in Texas.

Freeman talks about his current interest in biotechnology, and domestication of technology where people become "married" to their computers. Hobbyist using the tools of genetic engineering to in growing flowers, raising reptiles, etc. If one person wants to grow the most prickly cactus how to we unsure it's kept within control.

George, who last year talked about history of technology, talks about dropping out of school to go build kayaks. Schools try to divide people between those who are good with their minds and good with their hands.

In the past the American right of passage was to take apart a carbeurator, very happy to see a movement back to people hacking hardware.

Freeman's book Weapons and Hope talks about issues or risk and safety with nuclear weapons. We're living in a society that's risk adverse, but do all kinds of daring stuff.

Session gets opened to the audience for questions.

Posted by mike at 9:48 AM

July 28, 2004

Using MySQL for Binary Data

T-minus 10 minutes until my OSCON 2004 presentation, Using MySQL for Binary Data. Unfortunately my multi-tasking abilities aren't honed anough to allow me to present and write a weblog entry at the same time so the slides will have to do.

Posted by mike at 2:30 PM

Building More Secure OSS with OSS

OSCON 2004 Day 3 starts with a presentation from John Viega, about securing open source software.

Looks like the general overview is common misconceptions, understand context for security issues,
security in development and a look at useful and free open source software.

Risks are across all languages, even Java. The Java "designed with security in mind" concept was primarily for applets, but doesn't apply to a high percentage of Java code. Average 5 problems per 1000 lines in C/C++, 1 problem per 1000 lines in Java.

Can't eliminate all security risk.

Customers don't really know what what to ask for in secure software, even though there is a bit of fear based on what layperson's sees in news.

The market evolution of software security is in the early adopter phase, with financial and government mandating secure software but many other markets are mandating. Secure software products are fairly new, just a handful and less that two years old.

Even though the source can be looked at and improved in most large open source projects there aren't credible people scritiinizing the code for security issues.

Look at IBM's deveopment process diagram examining the complex set of phases, no security built into process. SecureSoftware is working with IBM to add practices to ensure security is a consideration throughout the process. Microsoft has a Threat Modeling Tool, we need an open source equivalent.

John walks through a case study looking at use of OpenSSL libraries incorrectly in an application. Need to do more than just check for a valid certificate, or that it's signed by a trusted source. Need to check explicitly, which the OpenSSL libraries don't do by default. Need to check DNS address of certificate compared to DNS of server. It should happen by default, but it doesn't.

Secure Coding Cookbook has code snippets.

John points to a protocol, authentication project at Stanford, SRP: Secure Remote Passwords. It's a good compromise between usability and security. It is patented, so can't use it in commercial application.

RATS can look at PHP, Perl, Python, C++ for looking through code and comparing code to knowledgebase to find instances of insecure code.

Posted by mike at 11:45 AM

Aha Moment: Restoring Movable Type Category Assignments lost in Upgrade

This morning I got back all the Movable Type category assignments I had made for entries that were lost when I migrated from BDB to MySQL in 2.661, and then dumped into MySQL tables for MT 3.0.

The aha moment happened while shaving: When I moved my entries from BDB to MySQL all the entry_category_id fields were null, which I chalked up to conversion problems. A bit later, after I had forgotten about the lost category assignments I was trying to manually (at the MySQL shell) change a category and discovered this table mt_placement. MT used to keep categories in the mt_entry table, but at some point they started putting them in mt_placement so a person could assign multiple categories. Wasn't until this morning that I connected my recent fiddling with the lost category assignments from a month ago.

Sure enough, the mt_placement table on the 2.661 database has all the assignments. Since I had done a number of manual reassigning entries to categories I had to make sure that anything I dumped from the 2.661 database into the 3.0 database wouldn't overwrite or duplicate records. A quick query in the 3.0 mt_placement database gave me a list of entry_ids to exclude. I ran these three commands and restored 351 category assignments that were lost:

In 2.661 database on old server:
select placement_entry_id, placement_blog_id, placement_category_id, placement_is_primary into outfile '/tmp/placement.tsv' from mt_placement where placement_blog_id in (<blog ids>) and placement_entry_id not in (<ids with existing category entry in new weblog>);

scp /tmp/placement.tsv new.server:/tmp/.

In 3.0 database on new server:
load data infile '/tmp/placement.tsv' into table mt_placement (placement_entry_id, placement_blog_id, placement_category_id, placement_is_primary);

As a person who knows MySQL, having MT entry data in MySQL makes solving these kinds of problems so much easier.

Posted by mike at 9:06 AM

July 27, 2004

Paul Graham on Great Hackers

Listening to Paul Graham at OSCON 2004. He's reciting what seems to be a selection from Hacker's and Painters, which I haven't read, but listening to Paul's speech makes me anxious to get a copy. Fresh, direct ideas about current trends in programming, hacker habits, work environements etc.

Paul says that Dean Kamen wrote Segway's instructions in one day . . . in python. Another reason why I need to free up some time to start playing with python.

Update: Paul posted an essay derived from his OSCON keynote.

Posted by mike at 9:21 PM

Pixies Concert on DVD from Fellow OSCON Attendee

I like to take a blank DVD and CD on trips just in case . . . who knows what might come up.

Sitting in a tutorial and got to talking to this guy (name withheld) behind me about Portland concerts and found that he'd recently seen the Pixies in Davis, CA and had a two-disc CD set of from the show. He offered to let me take a copy, but we couldn't get networked together. I ran up and grabbed my blank DVD and within 10 minutes I had burned a copy of the show.

Listening now, thanks for the tunes.

Posted by mike at 6:20 PM

MySQL Performance Workshop

Jeremy Zawodny is doing a tutorial about tuning MySQL at OSCON 2004. Kind of a cool approach to the tutorial, Jeremy puts up a list of Topics and is asking for audience input on how to proceed. I saw got quite a bit of information on tuning at MySQL 2004, but I'm still convinced there's a part of our system somewhere that needs tweaked. I haven't noticed any performance issues on our installation, but want to be educated about how to solve them should they arise.

Jeremy starts with a slew of questions to get familiar with the audience and then gets into the "how MySQL is used at Yahoo." 500-1000 installations, mostly run on FreeBSD 4.x on 32bit Intel (commodity hardware), anywhere from 2-30G of data. Lots of replication, load balancing. Alteon and Foundry load balancers used. Leaned in Alteon direction because you could have it execute a script which requested a custom php page from the database server which would run some checks and return a more sensible health check compared to a TCP/IP check on port 3306.

What things might affect performance

Things that don't matter

Schemas, Indexes, Table Type

Review of storage engines and features (MyISAM, HEAP, BDB, InnoDB, NDB, Archive). Archive table type is a new type in 4.1.

Queries

Memory

Disks and I/O

Application Design Issues

Query Optimization

What tools does Jeremy use to find problems?

First thing is to determine where the bottleneck is. Start with using vmstat, sar, top, taskmanager to find CPU, memory, disk etc. Sometimes getting I/O can be difficult, iostat is a good start. Once beyond the OS level uses mytop to dig into MySQL.

Replication

Is hard to imagine replication being a burden on the master, reading the binlog is easy because it's typically in memory, so the only real threat for replication is network bottleneck. Important to realize that the slaves have to execute the same queries as the master.

Posted by mike at 2:48 PM

Will Google Make Me Change How I think about Myself?

I have very few friends who would ever call me Michael, yet I tend to use that name on anything more formal than communication with friends. Every presentation, school paper, online registration, etc I use Michael.

Although most people know that Michael and Mike are interchangeable, Google doesn't and I find myself wondering if I might use Mike in more formal settings to unify my identity. Besides my weblog, a search on Google for "Mike Kruckenberg" and "Michael Kruckenberg" return completely different results. I wonder when someone is looking me up if their understanding of who I am varies depending on what search term they use.

Is it worth it? Should Google be driving a redefinition of how I identify myself?

Posted by mike at 2:32 PM

MT Upgrade

I've never done a "true" MT upgrade, until today. When 2.661 came out Pete did the upgraded, and moving to 3.0 wasn't as much an upgrade as it was installing a new instance and manually moving the MySQL data.

The completes process to upgrade from 3.0D to 3.01D:

cp -r /tmp/MT-3.01D-upgrade-en_us/* /home/mike/web/mt/.

Good for future reference, barely worth a weblog entry.

Posted by mike at 1:00 PM

July 26, 2004

SCO vs IBM Moot Court

Attended the OSCON evening entertainment tonight, a mock court where SCO and IBM sides were represented. The real lawyers for both sides were consulted heavily, but couldn't attend a public event like this and make statements per the magistrate's direction. Didn't get the names of the stand-in lawyers, but apparently they couldn't find any lawyer except the SCO lawyers themselves that actually agreed with the SCO side. The lawyer representing SCO had to be a little sarcastic, was nice of him to present the other side even if he didn't agree with it. Was interesting to hear both sides, much of which we've heard before.

One of the questions asked afterward was "what can we do moving forward as we code to avoid our work coming under litigation." The lawyers suggested that active documentation about what and how you are writing the code is important. Darn it, Damian was also pushing on good documentation this morning, for different reason of course. I agree documentation is good, why is it so hard to get motivated to document code?

I remember last year a panel suggested to never agree to sign an NDA before looking at some other code, because just seeing another piece of code might influence future works from the developer.

Posted by mike at 10:49 PM

Rare Treat: An Afternoon Movie

I go to the movies so rarely, before seeing Farenheit 9/11 last week the last movie I'd gone to in the theater was Austin Powers: Goldmember (2002). The combination of having some free time in the evening, having recently seen a trailer and that movies before 6 are $6 in Portland, I went and caught The Bourne Supremacy at 5:30. I liked the movie, but at least half of my enjoyment was being in the theater in the late afternoon. On my way out I noted what other films were playing in case a return trip might be in order later in the week.

Movie got out at 7:15, just in time to make it back for the OSCON SCO vs IBM Moot Court.

Now I'm thinking, my sister-in-law works a few blocks from my office and there is a nice movie complex between our buildings. We've often joked about going to see a film on our "lunch break," maybe it's time to make good on that.

Posted by mike at 9:59 PM

Time Management for System Administrators

Afternoon tutorial is Time Management for System Administrators: Getting it all Done and Not Going (More) Crazy! by Thomas Limoncelli.

Tom is doing something similar to what I imagine happens in a Franklin Covey seminar, but with a sysadmin twist, looking specifically at how time management should happen. A few years ago I listened to the Franklin training tapes, so this is a bit of review.

General Principles

Task Management

In this room there is a growing hostile environment between the PDA vs paper planner folks. I think what's important to remember is that you have to find what works best for your style, and accept that other people will do the same.

Posted by mike at 3:12 PM

Best Practice Perl

First presentation of OSCON 2004 is Damian Conway, giving a tutorial titled "Best Practice Perl." There is a good handout of the slides that details things, but Damian goes into futher detail on each slide, want to catch some of that. Not sure I'll catch everything Damian is saying, but at least the highlights.

Damian Conway not coming off the mountain to give us the 10 Commandments of perl (shows picture of Charlton Heston). There is not one best way to do most things, it changes between people. Rather than "the" best practice, these are "a" best practice. Learn to avoid IPS (Intuitive Programmer Syndrome), coding to feel good, as opposed to doing what's best. This presentation is a start, best practices need to be agreed apon by all team members, who must stick to it. Giving up IPS is difficult, like giving up smoking . . . . crack.

One of the overlying goals is to make code instantly recognizable. We shouldn't write from a writers point of view but a reader and comprehension point of view. Best practices are sometimes writing good code that works, but often can also be rules for coding that it will make it easy to find something stupid we've done.

General Thoughts

Layout Style

Layout style (K&R, DSB, GNU, Slash) doesn't matter, all are equally false. Agree on how to space etc.

Identifyers

Have a consistent style of naming identifiers.

Variables

Control Structures

Documentation

Halftime break - Damian shows a slide show of "friendly animals," a presentation about spiders, snakes, bugs etc in Australia that are all deadly - quite funny - a song by Scared Weird Little Guys about dangers of Australia

Built in Functions

Subroutines

I/O

Regex

Errors

Posted by mike at 9:45 AM

July 25, 2004

The Traditional Conference Music Purchase

I have a little conference tradition.

At OSCON 2002, in San Diego, I wandered around for a few hours the day before the conference started. I stumbled into a small record shop with a nice selection of rock/alternative/indie and bought a CD, which I ended up listening to during any free time in my room. Any time after the conference that I poped in the CD I would think back to San Diego.

Since that time I've tried to do something similar and get out on the first or second day and grab some new tunes to listen to at the conference (typically in the night when I'm working on the laptop). For me, buying just one CD is difficult, but it actually works good to buy two or three in case one isn't so great.

I'm not sure if it's a "way to remember the trip" as much is it's an excuse to buy some music.

Posted by mike at 9:44 PM

Welcome to OSCON 2004

Arrived in Portland today, weather is quite nice. Got a room on the 4th floor with a balcony this year (note the nice view of Mt. Hood)! Maybe I can figure out how to get up and down without using the elevator, which I recall from last year can get bogged down with the hundreds of people trying to get to and from their rooms.

Going up to get registered soon and then over to Music Millennium for my traditional conference music purchase. Probably be good to get some decent food too, so far have only eaten airline pretzels and a few pieces of candy that the family snuck into my bag last night after I had packed. Looking back at last year's preconference notes for places a friend recommended.

Hmm . . . seems like I haven't written about my traditional conference music purchase, will have to do that later.

Update: Posted a note about the tradition.

Posted by mike at 5:26 PM

July 24, 2004

Off to OSCON

Getting up at 5:30am tomorrow to catch a flight to Portland Oregon, via San Fancisco. Hoping to get in some extended leisure reading during the trip.

Still a few pieces of the presentation to work out, having practiced it a couple of times in the past two days. Sometimes I just can't seem to find the optimal way to say something, and I know if I can't say it right during practice there is no possible way I'll say it when I'm nervously delivering the real deal.

Weather looks to be good in Portland for the next week, 85-90 and mostly sunny (as if I believe that anyone can really tell what the weather will be like 6 days from now).

Posted by mike at 8:03 PM

July 23, 2004

My MySQL Certification Guide Review

I recently read and wrote a review for the MySQL Certification Guide, from MySQL press. I am a little suprised at how "glowing" the review sounds today, having been awhile since I wrote it. For the future I'll remember to not start writing a review at midnight, after a long day. Or if I do, make sure I set it down for a day or so and revisit to ensure it's got the right tone. That said, I do think it is a good book and recommend it, especially if you're wanting to get practice exercising your MySQL knowledge in preparation for the exam.

The reviews won't be linked into the site until reviewers get a chance to once-over their comments.

I wonder who else wrote reviews . . .

Posted by mike at 3:53 PM

An Objective for OSCON 2004

We're close to signing an agreement with another university who is going to start contributing code to our project. Exciting, but scary. There's some pressure (from folks here) to come back from OSCON with some ideas on licensing and management of code contributions. I'll certainly be rubbing shoulders with the right people (if I can muster courage to ask questions), and these events look like something worth checking out.

Posted by mike at 12:00 PM

July 21, 2004

Impressed at Speed of MySQL's 'load data infile'

We store images in a database. Over the past week I've been doing some load testing to get performance statistics for my OSCON presentation, using the binary_data table as a source for getting information to drive the load tests. After getting into some of the tests I thought it would be good to have an idea of the image size distribution, average image size etc. Unfortunately every query against the table is slow because we're dealing with records as big as 5Mb, and using length(binary_data) to determine image size for each record doesn't help speed things up.

I decided to dump the pieces of information I needed to a textfile for processing elsewhere. Tried OpenOffice and Excel, neither of which can handle such a large set of numbers (Excel's limit is 65536 records). Then it hit me that a separate table in MySQL was probably the best way to store this data.

I used MySQL's load data infile, fully expecting it to take a bit of time churning. With indexes on it took 8.99 seconds. Curiosity forced me to drop the table and create it without indexes . . . 1.29 seconds to load 513,549 records. Queries that were taking over 10 minutes are now in the fractions of seconds.

For the curious:
+-------------+-------------+---------------+
| image_count | total_bytes | average_bytes |
+-------------+-------------+---------------+
| 513549        |11817103519|       23010.66 |
+-------------+-------------+---------------+

Posted by mike at 5:46 AM

July 19, 2004

When will I Settle on A Planner?

I'm ready to have figured out what kind of planner I like best. At the end of 2003 I shelved the Palm and went to a paper-based calendar/notebook. I am much happier with the physical notebook with pages filled with my handwriting. Yet I'm not convinced it's just the right thing . . . and I want to know what the right thing is so I can move on.

Around the time I made the switch back to paper I was turned on to Moleskine notebooks, made in Italy. The idea is good quality lined paper, leather cover, elastic band to keep it closed and a great deal of history. Van Gogh, Hemingway, Matisse, Picasso all used Moleskine notebooks. I figured starting in 2005 I'd become entrenched in the Moleskine way.

I had run out of room in the notes section of my 2004 planner, and someone had requested a series of meetings from Jan - June 2005 so I figured it's time to hop on the Moleskine site. I ordered large (5x7) 2005 one-day-per-page diary/planner. I do believe I'm closer to finding "the" planner, but am now trying to make one final decision. The store shipped me the wrong size planner, a smaller 3x5" one-day-per-page instead of the 5x7." A quick email and the larger size one is now in my hand with a note to "keep the smaller one as a gift from us." Problem is that I'm torn between the smaller, easier to carry one vs the larger, more room to write. Gosh, if I could only make up my mind.

Why does it take so long to figure out something that seems so small? Is it because it's a one-year commitment and I don't want to get locked into something that just isn't right? Part of me hopes that if I keep active with writing notes, schedules, ideas etc in the planner that some day I'll have a bookshelf full of notebooks/planners that will represent my life's work. If I can find the right size sooner than later, that future shelf will have notebooks of uniform size, shape and quality.

Posted by mike at 9:25 PM

July 15, 2004

4 Days of Data Modelling and Writing SQL

Monday I started on a project to rewrite how our system thinks about a course. I wrote out a few thoughts we'd gathered in the past few years about what's wrong with our course data and then dove into documentation from several other projects who have been working on this issue; IMS Global Learning Consortium, MIT's Open Knowledge Initiative and OpenCourseWare, Internet2 MACE group's eduCourse and courseID and the Sakai project. Tufts University has been involved in all of these in one way or another. I guess we see value in using existing knowledge both because it makes our job easier and so that at some future point we may more easily share our courses having used something that's hopefully moving towards a standard.

After a day of review I came up with a primitive sketch, and on Tuesday a few of us met for 2 hours hashing a number of issues out. Since then I've been writing documentation, which includes SQL statements to build the database structure.

There's still a ways to go on the documentation, and no doubt after review there will be some changes, but I'm getting antsy to get on to the next phase. Not so much because I'm looking forward to the next phase, but because I'm exhausted. Having to keep my head around the terminology, tables and relationships is tiring. The next phase is writing conversion tools to migrate data from the existing structure to the new tables, should be lighter work. After that is writing some code which will actually present the data.

Posted by mike at 3:22 PM

July 13, 2004

Interview with Department of Defense Investigator

A previous co-worker is in the process of getting into an internship in Washington and has to provide a slew of references to get interviewed for security clearance. I met with an investigator from the Department of Defense in which I was asked to summarize the relationship with the co-worker and then answered a set of 30 or so questions (read to me from a form). I haven't been through this process before, but gather from the slew of information online that a lot of people have.

A few questions that I thought stood out:
Investigator: Describe in a few words the individual's allegance to the United States.
Me: Um, Um, Ah . . . we didn't really have any political conversations but I didn't ever hear anything negative.

Investogator: Has this person ever gained unauthorized access to one of your servers or systems?
Me: No. (thinking . . . but we sure had a lot of conversations about the different ways one *could* compromise a system)

Investigator: Does this person associate with any radical groups?
Me: What would be an example?
Investigator: Perhaps one that wears white hoods . . . there used to be a long list that started with the A and went down, the intials KKK would be something to note.
Me: No.

The investigator said that for this internship position he would do about 8 interviews. I thought that was a good number, but others here think that's pretty minimal compared to other stories.

Found this FAQ for people who are in the process of gaining clearance.

I get a little uneasy when government starts asking questions, was glad there was nothing in the interview that increased my suspicion about what they are up to in Washington. He did say that he didn't have any inside information on the War on Iraq. Bummer.

Posted by mike at 1:36 PM

July 9, 2004

A Photo I Took in London

I've had this photo I took in London on my desktop at work for a few weeks now and have had a number of people comment on it. I took it with the Canon PowerShot s400 on it's default settings with the flash disabled. The default, flash-disabled setting, keeps the shutter open until enough light has been gathered to make a meaningful photo.

The photo is taken while leaving Shakespeare's Globe, coming across the Millenium Bridge looking at St. Paul's cathedral.

Perhaps I should set up one of those nifty photo sections on the weblog and have some of my favorite photos rotated there. Will consider that next time I get time to play with MT.

Posted by mike at 8:41 AM

July 8, 2004

Network Outage in Boston (manhole fire burns fiber)

Yesterday evening around 6pm a fire in a manhole somewhere in Boston badly damaged the fiber feeding the gigabit internet connection to the Boston campus of Tufts University. I was at work when the connectivity dropped, left thinking it was some small interruption. Didn't think that when I got back to the office this morning that the outage would be wreaking havoc on everyone's work-day.

We had a nice lunch, a staff meeting and then everyone went home. Since our servers live in a datacenter in Somerville, MA the best bet for connectivity was to go home.

After 24-hours of downtime the word was that the appropriate people were still not able to get into the manhole due to fumes, heat and hazards with live electricity.

Just checking now it looks like NOC has figured out a way to get a T1 connection up via an alternate route. Bummer . . . I was hoping tomorrow would be a forced stay-at-home day.

July 9 Update: The temporary pipe to the office is being heavily guarded, only limited traffic allowed through, which doesn't seem to include ssh. Even if it did, the pipe for the entire campus is 1.4 Mb, about the same as my cable modem's downspeed. Looks like I'll be more productive working from home until the problem is resolved (technicians are now scheduled to get into the manhole this afternoon).

Posted by mike at 10:06 PM

July 7, 2004

Why do I Resist the Spreadsheet?

Just spent 30 minutes putting together a spreadsheet with details on 6 new machines coming in. The spreadsheet details their name, make, DNS, network interfaces and requested cabling speed, physical install location etc.

The spreadsheet is simple and clear, as opposed to the many hours I've spent on email and the phone over the past month explaining to various data center operators, installation folks, managers information about the new machines. I was trying to outline everything in a narrative, where a spreadsheet was clearly the better option (but didn't get created until someone demanded it). This isn't the first time . . . I tried to write a narrative to describe the list of servers, specifications and service levels when we renegotiated our service contract, and didn't go to the spreadsheet until someone demanded it.

Why do I resist the spreadsheet? Is it because I think the narrative is more creative, or allows me to craft the message just right? How do I get into the habit of using a spreadsheet when a narrative isn't best? I don't know the answers, but am hoping that I can start to curb my tendencies and use the most appropriate vehicle.

Posted by mike at 9:22 PM

July 3, 2004

The Yearly Trip to Burlington, Vermont

A through-the-window shot of the evening sky over Lake Camplain in Burlington, Vermont. The trip to Burlington has become a yearly tradition. In the past we've come up in late May, but decided this year to make it later in the summer to increase the chances of having warmer weather (it is warmer, but also much busier as it's Independance Day weekend).

We always stay at the Smart Suites hotel. They have good rates, large and clean rooms, indoor pool, delux continental breakfast and DSL (and our room looks out west over the lake to upstate New York). Today they didn't have our room ready on time. After I got them to knock $50 off the bill we swam in the pool for an hour. We were going to swim first thing anyhow. We also scored some Ben and Jerry's for the inconvenience.

Last year, I forgot my battery charger, didn't make the same mistake again.

I guess this is how family traditions are made, take a trip one year, like it, and start repeating it every year. Perhaps one other qualifying characteristic is that you have to be able to imagine your kids complaining about it. I can already hear the complaint: Why do we *have* to go to Burlington again this year, it is so boring and stupid.

Posted by mike at 7:12 PM

Anyone want my job (or one like it)?

It looks like we're hiring a new person, position description is being written this week to be posted shortly. The goal seems to be to find someone who can take the sysadmin and some my programming projects to free me up to work on other projects. Programming is primarily writing Mason and Perl modules in Perl (obviously) in an Apache/Perl/MySQL environment. Sysadmin is on a dozen Solaris boxes, building and updating packages, working with USG (University Systems Group) for patch updates, etc. I'm wondering if reading my weblog really gives a good sense of what I do . . . probably not.

Tufts is a great place to work. Pretty relaxed environment. The hardest thing to get used to is the flexibility in deadlines and the 35-hour work week.

I'll get the official position description posted once it's complete. We have a great team, but it has taken some time finding the right people. Last position we filled we looked through between 100 and 200 resumes and went through several rounds of interviews before we found a good fit. Where are the good Perl programmers?

Posted by mike at 6:24 PM

July 1, 2004

New iMacs in September

Looks like Apple has been forced into revealing a new line of iMacs.

Stumbled into this today as I started to look for a new machine for Heidi, who's becoming quite disgruntled about her PowerMac G3 300MHz (running OS X).

Apple has stopped taking orders for the current iMac as we begin the transition from the current iMac line to an all-new iMac line which will be announced and available in September. We planned to have our next generation iMac ready by the time the inventory of current iMacs runs out in the next few weeks, but our planning was obviously less than perfect. We apologize for any inconvenience to our customers.
Something to look forward to.

Posted by mike at 8:21 PM