« Django: The Web Framework for Perfectionists with Deadlines | Main | MySQL Movie Magic »

April 27, 2006

Jim Starkey: Introducing Falcon (Storage Engine for MySQL)

Last session of the MySQL UC is Jim Starkey giving an introduction to the new Falcon storage engine. Jim is an icon in the database field, was the creator of MVCC and the BLOB data type. There's a can of Falcon beer for anyone who asks a good question.

What is Falcon

Falcon is NOT

Jim's been at this for a long time, there have been some changes since he wrote his first database at DEC:

What Jim has learned

Falcon is designed for the next 20 years. Jim is comfortable saying that what he's learned over the past 20 years in databases and has put into Falcon will be .

Goals of Falcon

Architebtural Overview

Incomplete in-memory database with backfill from disk that has two caches. The traditional LRU page cache for disk. A larger row cache with age group scavenging. Falcon is multi-version in memory and single version on disk. All transaction state is in memory with automatic overflow to disk. Data and indexes are single file plus log files. In the future Jim would like to create BLOB repositories where the data is stored off to the side. Hoping to provide multiple page spaces in the future.

Falcon uses Btree indexes with prefix compression. There is no data except the key in the index.

Uncommitted row data is staged in memory (can overflow to to scratch file). Indexes are updated immediately. On commit row data is copied to the serial log and written. Post commit dedicated thread copies row data from serial log pages to data pages. Page cache is periodically flushed to disk. BLOB data is scheduled for write at creation.

Data reliability is protected by "careful write" where writes are sequensed to the disk so it's always valid and consistent. There is a repair mechanism but Jim's hope is that noone will ever have to use it.

Falcon has a do/redu/undo log in the serial.

Secret Agenda

Jim's got a secret agenda of things he'd like to do in the database world, starting with MySQL:

  1. Jim wants to replace varchar with string, varchar is a throwback to punchcard technology
  2. Replace tiny, small, medium, and big integers with "number"
  3. Adopt a security model useful for the web
  4. Row-level security (filter sets)
  5. Teach the database world the merits free context search - why do you have to use a SELECT statements

Q/A

Posted by mike at April 27, 2006 5:19 PM