« May 2004 | Main | July 2004 »

June 29, 2004

Hanging out with Pete in Colorado

I'm in Broomfield, Colorado. Nothing to write home about until Pete writes last night . . . "well I need to go, got to be up for a flight to Denver in the morning." Who bothers to check if business travel just happens to coordinate with another family members business travel?

After some flight and rental car shuffling Pete and I are on the road to Boulder to have dinner and hang out for a few hours. Found our way to Bart's CD Cellar, where I invariably end up having to leave with far fewer CDs than I want (based on the two times I've been there now).

Strange how that works.

Posted by mike at 7:40 PM

June 28, 2004

Hosting Mike Benson's Film Sites

In a strange turn of events, after having mentioned Mike's Evvy award, I'm ending up hosting Mike Benson's two sites on one of my machines.

I guess that means I can provide a reliable link to Manilla Sky, the award-winning short.

Other information about Mike and his films can be found at luckmonkey.com. Another site, moxiementary is in the works.

Posted by mike at 11:58 PM

Backward Compatibility with New Page Names

Although I like having weblog pages that aren't named from an ID in the MT database, I got to thinking about the old posts that are referenced out there and didn't want to leave them hanging.

The solution, change the weblog configuration to "old style" links and regenerate the site, then change it back to "new style" links. Since MT doesn't delete pages, the ID-named pages will remain in the archive directory alongside the new-named onces. For existing links I'll preserve the ID pages but moving forward will use named pages.

I actually generated two sets of ID-based pages, one with .html and another with .php extensions - at different times on the old weblog I experimented with weblog pages with embedded PHP. A look through the Apache error logs reveals that requests are coming in for pages with both extensions.

Posted by mike at 11:36 PM

Pete's Weblog Moved from Old Machine and Upgraded to MT 3.0

Went through the same MT-upgrade process for Pete tonight, as a favor while he looked into a few other server things.

Followed the same steps and went pretty smooth. Pete's was a little easier because he only had one weblog (I had two) and doesn't use categories (I had eleven).

I'm noticing something pretty cool about MT 3.0 (should probably read the feature's list); the URLs aren't just numbers, MT actually generates an HTML page with a name, using the first few words of the entry title.

Posted by mike at 10:55 PM

Convert MT Data from BDB to MySQL, Dump on Old and Restore on New Machine

Got access to the old server (Pete had blocked almost everything via ipchains). Thought I'd better take a crack at getting my old data over to the new machine before I had too many entries. To make things complicated we're blocking all web traffic and the entries on the old weblog are in BDB.

I decided I really didn't want to copy all the tables . . . just the entries and comments (including some spam that I hadn't caught).

On old server:
- Edit mt.cfg, adding config for MySQL database
- Run mt-db2sql.cgi via a wget request
- mysqldump from mt_entry and mt_comment where blog_id= (use full inserts because there are slight changes between 2.66 and 3.0)

On new machine
- Dump few entries I'd made in the new weblog
- Delete records from mt_entry (so old entries coming over from previous weblog could keep ids)
- Use new MT to set up blogs and categories
- Import mt_entry data
- Run a few mysql updates to change blog_id
- Manually change ids for the dumpfile I'd created of new entries and reimport

Either I didn't do something right, or there was a problem with the BDB conversion, none of hte old entries have categories on the new weblog. Any bets on whether I ever go back and update them? Probably not until after I get the energy to make the style tweaks.

Sometimes a fresh start can be good, but most often it's just a lot of work.

Posted by mike at 4:05 PM

Shibboleth Deployment

Keith Hazelton from University of Wisconsin-Madison is giving an overview of the deployment process for shib (a href="http://arch.doit.wisc.edu/keith/camp">slides).

Presentation focuses primarily on the shib identity provider, which is driven by apache, tomcat and some servlets. A WebISO needs to be in place.

Keith is showing examples of different ways to have Shibboleth pull addributes from LDAP, SQL database or files. Shib can export any attribute. eduCourse is coming and there is a URN identifier guideline available for course offering.

Posted by mike at 10:48 AM

Intro to Shibboleth

Michael Gettes from Duke is giving a Shibboleth overview. I heard Michael speak last year at CAMP briefly about shib.

Shibboleth preserves privacy, an attribute-based approach for authentication as opposed to identity. The service doesn't need to know who you are, but needs to know some things about you.

Installing Shibboleth can take between 3 hours and 3 years (depends on existing infrastructure).

Person tries to get a resource (web page). The ACS (assersion consumer service) says I don't know where you're from so redirects to WAYF (where are you from). The WAYF directs you to login at the identity provider, which creates a handle and forwards it to the ACS. The handle is passed to the attribute requestor (AR), which contacts the atribute authority (AA) which is typically on the identity providor and sends back a list of attributes. The attributes are then used in authorization to grant access.

Demo of shib-enabled Blackboard, give identity to home institution which passes a few attributes to Blackboard and Michael is signed in and perusing. Then he clicks on jstor and without a login or any further authentication he's on perusing JStor.

A few questions, which lead to the point that the WAYF is a completely separate thing from the trust fabric built by federations.

Michael goes into details of how Duke is using shibboleth.

There is hope that the higher ed certificate authority will be able to build a bridge to federal certificate authority and use eAuth using an inter-federation agreement.

Posted by mike at 9:18 AM

Welcome to CAMP Shibboleth

Kenneth Klingenstein (Internet2 Middleware Initiative Project Director) is giving an overview of the conference.

The tradition is to start with and IBM commercial where basketball team is composed of players named after systems (Linux, Firewall etc). Everyone gets tons of fan mail and Middleware gets one piece, and nobody wants his signature.

Shibboleth is moving overseas, folks here from Netherlands, UK, Australia.

No longer using target and origin to describe functions of shib, origin is now known as Shibboleth Identity Provider.

Ken goes into details about the purpose of shib, status, and where things are going.

Ken worked on NSFnet back in the day before the internet became big.

Lessons learned from NSFnet
- keep it simple and solve real problems
- make a marketplace
- stay low for as long as you can
- be prepared to travel

The hope is that shib will be something that fades into the background, with little development, somewhat like TCP/IP.

Posted by mike at 9:00 AM

Adding hot-plug disk drives (use format, Solaris Disk Suite and newfs)

I think I've written a few times about using SDS for software raid and disk concats. During last week's move of MySQL to a new server I threw in another 2 disks for our nightly MySQL backup. We do a hotcopy of the tables onto a separate set of disks from the actual data dir.

The disks are in an external multipack, which allows hot-swapping of disks. The disks were stuck in right before I left, no machine reboot is necessary, but if the disks don't show up in format then this command will do a refresh of the /dev aliases and make the disks show up:

devfsadm -c disk

After partitioning and labelling the disks the concat is easily created:

metainit d60 4 1 c1t1d0s0 1 c1t2d0s0 1 c1t4d0s0 1 c1t5d0s
newfs /dev/md/dsk/d60

With 4x36G disks we're now up to 140G of disk space for backups (we keep a few day's worth).

Posted by mike at 1:06 AM

Production Updated from MySQL 3.23.x to 4.0.20

One of the most nerve racking nights of my sysadmin career.

After many months of wanting to move up to the 4.0 branch we're finally there. Although it should have been a pretty simple process, it actually started on Thursday night at 10pm and completed around 5am Friday morning.

Our production MySQL was on an older box, which not only has all our production data, but also includes all user Unix shell accounts and home directories. The move involved going to a new machine, seamlessly moving shell accounts between machines and transferring the data which all sits on an A1000 hardware RAID array. We've had some bad bouts with the A1000, so I was prepared for the worse to happen and took 4 hours of extra downtime to make a complete copy of all the data before attempting to move everything to the new machine.

In the end it all went pretty smooth, but did take a long time to execute all the steps I had created in the few practice runs I made.

Posted by mike at 12:42 AM

Arrived at CAMP Shibboleth

Late, late . . . 2:30am EST, 12:30 in Broomfield, Colorado where things are set for CAMP Shibboleth 2004 to start tomorrow. Just a few things to get done before I may rest . . .

Posted by mike at 12:30 AM

June 24, 2004

Kruckenberg Hacked - Rebuilding with Movable Type 3.0

3 days ago kruckenberg.com got cracked/hacked, via an unpatched PHP vulnerability. It stinks, but there are good things. For some time we've been wanting to move to a new machine, which Pete built awhile back, but neither of us were actually doing much about getting stuff moved over. The idea is to move the web applications to the new machine and leave mail on the existing.

During the hack Pete shut down the webserver and adjusted the firewall to not allow web traffic. Perfect time to get the new machine up and running.

First order of business was getting MT up on the new machine. I had read the rage over MT 3.0 pricing but hadn't looked lately. Based on a year+ of writing a weblog I think the limited free version will meet my needs. It's free and will let one author (me) create up to three weblogs. In the past Pete and I had shared an installation of MT. Now either I'll maintain my own installation (not sure if Pete's going to stick with 2.66 or move up to 3.0) or pony up the $69 for a personal license.

Before I can do anything else I want to get the old BDB entries moved to MySQL and brought over to the new machine.

Posted by mike at 4:11 AM

June 18, 2004

Live from Poughkeepsie, NY

Dad (Don Kruckenberg) is attending a critical thinking seminar at Vassar College in Poughkeepsie, New York for two weeks. Decided to book a hotel and drive the 3.5 hours from Boston to hang out Friday and Saturday. Brought the two kids (Heidi's in DC for weekend), which has had it's moments, but generally has been quite fun.

Spotted a DQ on the way into town. Rarely come across one in greater Boston so must sieze the opportunity.

The Vassar campus is beautiful, took a tour of the library and couldn't help whipping out my camera for a few snapshots.

Posted by mike at 9:13 PM

June 17, 2004

Too many days without any connectivity . . .

Back from London. We used the in-hotel internet for 1 day, the remainder of the time there was completely unconnected (had even left my cell phone in the states). The in-hotel connecttion was pretty flaky, would be on for 10 minutes and then nothing would go through for 10 and then back on again. Not worth the trouble (esp at £15 a day).

For the most part not having a connection was good, we usually left the hotel by 8-9am and were rarely back before 11pm. Doesn't leave much time for the computer. On the last few nights I stayed up late putting together a slide show, some iMovie sequences and the London Trip DVD.

We stopped twice into a nearby internet cafe but was packed, ended up buying a £5 phone card which gave us ~100 minutes and tried to find times during the day or night when we could catch the kids at home and awake.

It is good to be back, and connected to a constant and reliable connection.

Posted by mike at 8:32 PM

June 10, 2004

Traceroute from London

30 minutes after midnight in London, which makes it 7:30pm for my body. Can't sleep so decided to check out the high-speed connection. It's kind of expensive (£15 each 24 hours) but we needed it for at least one 24 hour period.

Seemed like data transfer was a little flaky, so executed a traceroute from here to kruckenberg.com:

 1  vbn.inter-touch.net (  7.026 ms  2.52 ms  2.459 ms
2 ( 3.401 ms 3.138 ms 3.073 ms
3 ( 7.879 ms 7.176 ms 7.232 ms
4 ge-1-1-0.icr1.lon6.ins.cw.net ( 6.055 ms 6.124 ms 5.97 ms
5 so-6-3-0.icr1.lon2.adm.uk.cw.net ( 6.189 ms 6.158 ms 6.192 ms
6 ge-2-0-0.itr1.lon2.adm.uk.cw.net ( 7.101 ms 6.336 ms 6.115 ms
7 so-6-0-0-0-bcr1.lnd.cw.net ( 6.446 ms 6.438 ms 6.353 ms
8 bcr1-so-7-0-0.thamesside.cw.net ( 7.467 ms 6.563 ms 6.508 ms
9 dcr1.nyk.cw.net ( 76.967 ms 113.026 ms 80.606 ms
10 ( 76.938 ms 76.925 ms 77.202 ms
11 ge-0-3-0.bbr1.newyork1.level3.net ( 82.426 ms 76.882 ms 77.599 ms
12 so-0-0-0.mpls1.saltlakecity1.level3.net ( 142.671 ms 142.745 ms 143.036 ms
13 ge-6-0.hsa1.saltlakecity1.level3.net ( 142.384 ms 142.455 ms 142.48 ms
14 unknown.level3.net ( 143.008 ms 142.897 ms 142.89 ms
15 gw-uen-core3.uen.net ( 143.281 ms 143.08 ms 145.275 ms
16 * ( 148.376 ms 148.46 ms
17 minot.kruckenberg.com ( 148.22 ms 147.294 ms 147.394 ms

17 hops aint too bad. Was a little suprised that the ip assigned to the laptop was a public address.

Posted by mike at 7:41 PM

Arrived in London (saw sun set and sun rise on same flight)

We touched down at Gatwick around 8:30 this morning. Not a lot of sleep on the plane.

That fact that we took off around the time the sun set in Philadelphia, and were in the air when the sun rose in England suprised me. I guess the combination of being in the air for 6 hours and moving through 5 time zones means you can actually see the sun set and rise while on the same flight.

Spent the day walking down Charing Cross Rd, browsing through dozen's of small book and music shops. Found our way to Leicester and Trafalgar Squares and ended up at Picadelly Circus. Most of the travel was on foot except for a few rides on the tube and double-decker busses.

Posted by mike at 5:50 PM

June 8, 2004

Using XML to Dump and Restore Data

The past three days I've been working on a project to move a collection of documents from the Tufts instance of our software to University of Natal.

Why? University of California San Francisco has a huge collection of AIDS-related documents, and had funding to put a subset of the documents online in South Africa, at the University of Natal where the Tufts course/content management software is installed. The University of Natal was being updated, so the documents were stuck in our system with a promise that we would transfer them.

Originally I had thought I'd do a somewhat complicated mysqldump, but we decided that even though there was no chance of overlapping IDs (because our system is using IDs in the 100,000s and they're in the 100s), it would be better to do an ID independant dump/import with all the documents and relationships.

So I used XML to represent the relationships, essentially developing a recursive function that dug into each folder and output all the column values. In the end, I captured around 400 documents at varying levels up to 5 folders deep. The dump and import scripts took about the same amount of time to write, and after practiving the import a dozen times in a dev environment I copied the tarfile (xml tree and supporting PDFs, etc) to University of Natal and imported without issue. Was a little nervous that there might be issues with Perl libraries, they are running on Linux (we're Solaris), but it went off without a hitch.

Posted by mike at 8:41 PM

June 7, 2004

syslogd stops? is that allowed?

Over the weekend I had a machine kept complaining about disk space on /var and periodically would alert me that syslogd was stopping. We've had floods of logging to /var/log/syslog in the past when administrators are sending out mass email reminders about finishing end-of-year online evaluations. I rotated the logs once, then twice. Today I figured I'd better check and see what was up.

Come to find out the problem was a cron entry that didn't have stderr properly redirected to stdout and was attempting to send email locally (the machine doesn't allow), resulting in sendmail in a spiral of attempting to send messages, but having to stick them in the queue for another attempt. By the time I looked the queue was up to ~26,000 messages. Everytime sendmail would start through them a huge dump would go through syslogd and at some point became too much.

I fixed the output problem changing cron to include:
. . . 1>>sync.log 2>>error.log

Removed the messages in /var/spool/mqueue and we're back in business.

Posted by mike at 12:09 AM

June 2, 2004

Network Training

Today I'm attending a training course on using NOC tools at Tufts University for administration of DNS. In order to use the tools NOC requires a 1-day training and passing an exam.

I was reluctant to sit through the class, seems like DNS tools I've used in the past have been intuitive and taking a class would be overkill. However, I'm glad to be here. The course focuses primarily on learning network technology (history, TCP/IP Model, netmask/CDIR notation, duplex modes, packet headers, arp requests, classes, zones, etc) and tools (netstat, nslookup, etc). Intertwined with the networking information is how the technology is used at Tufts, and how to use the tools to initiate requests and make changes.

I probably should have taken a course like this a long time ago.

Posted by mike at 4:07 PM