« Post-Talk Thoughts about INFORMATION_SCHEMA | Main | 2007 MySQL Conference Comes to a Close »

April 26, 2007

Pasha Sadri: Yahoo! pipes

Closing keynote of the 2007 MySQL Conference is Pasha Sadri from Yahoo! talking about pipes.

Pasha starts with a story about looking for an apartment near a park (this happened to him a few years ago). It was very tedious to go looking on Craigslist every hour clicking on every apartment and looking to see if the apartment was near a park.

Made a 50-line Perl script to merge database between Craigslist and Yahoo! local to get data sets. This turned into Yahoo! pipes which provides a visual editor to take data sources and create a set of controls over that data source and output the data in a certain way.

If you haven't seen pipes it is hard to describe, but very cool.

For design, pipes relied heavily on the idea of Unix pipes. The goal to have as many inputs as possible, and provide any output (RSS, badges, HTML, end user applications). There are a ton of processing tools, but they are also looking to allow end users to stick web services in as a processing step.

Pasha shows some cool end uses of pipes. One is a chat tool in SecondLife that uses pipes to translate the chat conversion real-time.

The conclusion includes a point about disposable applications that is interesting. The idea is that with pipes you can create an application to do something like finding apartments near a park and once you find the apartment you can throw it away.

Pipes uses MySQL as a database, squid for caching, PHP/Perl for serving/processing the pipes.

Posted by mike at April 26, 2007 6:47 PM