I recently had the opportunity to get re-acquainted with Redis while evaluating solutions for a project on the Product Innovation team here at SoftLayer. I’d actually played with it a couple of times before, but this time it “clicked.” Or my brain broke. Either way, I see a lot of potential for Redis now.
No one product is a perfect fit for all of your data storage needs, of course. There are such fundamental tradeoffs to be made in designing storage architectures that you should be immediately suspicious of any product that claims to fit every need.
The best solutions tend to be products that actually embrace these tradeoffs. Redis, for instance, has sacrificed a small amount of data durability in exchange for being awesome.
What is it?
Redis is a key/value store, but describing it that way is sort of like calling a helicopter a “vehicle.” It’s a technically correct description, but it leaves out some important stuff.
You can think of it like a sophisticated older brother of Memcached. It presents a flat keyspace, and you can set those keys to string values. Another feature of Memcached is the ability to perform remote atomic operations, like “incr” and “append.” These are really handy, because you have the ability to modify remote data without fetching, and you have an assurance that you’re the only one performing that operation at that instant.
Redis takes this concept of remote commands on data and goes completely nuts with it. The database is aware of data structures like hashes, lists and sets in addition to simple string values. You can sort, union, intersect, slice and dice to your heart’s content without fetching any data. Redis is a data structure server. You can treat it like remote memory, and this has an awesome immediate benefit for a programmer: your code and brain are already optimized for these data types.
But it’s not just about making storage simpler. It’s fast, too. Crazy fast. If you make intelligent use of its data structures, it’s possible to serve a lot of traffic from relatively modest hardware. Redis 2.4 can easily handle ~50k list appends a second on my notebook. With batching, it can append 2 million items to a list on a remote host in about 1.28 seconds.
It allows the remote, atomic and performant manipulation of data structures. It took me a little while to realize exactly how useful that is.
What’s wrong with it?
Nothing. Move along.
OK, it’s a little short on durability. Redis uses memory as its primary store and periodically flushes to disk. A common configuration is to do so every second.
That sounds pretty reasonable. If a server goes down, you could lose a second of data. Keep in mind, however, how many operations Redis can perform in a second. If you’re in a high-volume environment, that could be a lot of data. It’s not for your financial transactions.
It also supports relatively limited availability options. Currently, it only supports master/slave replication. Clustering support is planned for an upcoming release. It’s looking pretty powerful, but it will take some real-world testing to know its performance impact.
These challenges should be taken into consideration, and it’s probably clear if you’re in a situation where the current tradeoffs aren’t a good fit.
In my experience, a lot of developers seriously overestimate the consequences of their application losing small amounts of data. Also consider whether or not the chance of losing a second (or less) of data genuinely represents a bigger threat to your application than any other compromises you might have made.
More Information
You can check out the slightly aging docs or browse the impressively simple source. There are probably already bindings for your language of choice as well.
-Tim
- Lilah Brown's Planets, Part II (or, Season II preview) - November 8th, 2009 [November 8th, 2009]
- Snow White needs a bailout - November 8th, 2009 [November 8th, 2009]
- To the moon - November 8th, 2009 [November 8th, 2009]
- S/1 90482 (2005) needs your help - November 8th, 2009 [November 8th, 2009]
- We'll always have Regulus - November 8th, 2009 [November 8th, 2009]
- Orcus Porcus - November 8th, 2009 [November 8th, 2009]
- Kant's Crowded Universe - November 8th, 2009 [November 8th, 2009]
- Look up! - November 8th, 2009 [November 8th, 2009]
- Baby Pictures - November 8th, 2009 [November 8th, 2009]
- Encore: Yelping at Saints - November 8th, 2009 [November 8th, 2009]
- Godspeed - November 8th, 2009 [November 8th, 2009]
- Heavens above! - November 8th, 2009 [November 8th, 2009]
- Homeward bound - November 8th, 2009 [November 8th, 2009]
- Sony Pictures and the end of the world - November 8th, 2009 [November 8th, 2009]
- Thank you from the future - November 8th, 2009 [November 8th, 2009]
- Lunar dreams - November 8th, 2009 [November 8th, 2009]
- The first of the Pluto books! - November 8th, 2009 [November 8th, 2009]
- Don't try to blame it on Rio - November 8th, 2009 [November 8th, 2009]
- Rio roundup - November 8th, 2009 [November 8th, 2009]
- The long road to a Titan storm - November 8th, 2009 [November 8th, 2009]
- Planetary Placemats - November 8th, 2009 [November 8th, 2009]
- Fog! Titan! Titan Fog! (and a peer review experiment) - November 8th, 2009 [November 8th, 2009]
- Millard Canyon Memories - November 8th, 2009 [November 8th, 2009]
- The problem with science - November 8th, 2009 [November 8th, 2009]
- P.S. on the problem with science - November 8th, 2009 [November 8th, 2009]
- How Big is 10 TB? - November 8th, 2009 [November 8th, 2009]
- Showing You Your Servers - November 8th, 2009 [November 8th, 2009]
- Pick Your Partnership: Referral Partners, Resellers and Affiliates - November 8th, 2009 [November 8th, 2009]
- Server Form Factors: Towers v. Rack-Mounts - November 8th, 2009 [November 8th, 2009]
- Lights-Out in the Data Centers - November 8th, 2009 [November 8th, 2009]
- Disruptive Technologies: Virtualization and The Cloud - November 8th, 2009 [November 8th, 2009]
- Know Thy Backups – Part I - November 8th, 2009 [November 8th, 2009]
- Know Thy Backups – Part II - November 8th, 2009 [November 8th, 2009]
- Boo Bash 2009 – Desktop Costume Included! - November 8th, 2009 [November 8th, 2009]
- Why No One Will Talk About “Cloud Computing” in 10 Years - November 8th, 2009 [November 8th, 2009]
- The end of the fall - December 13th, 2009 [December 13th, 2009]
- We Love ‘Server Huggers’ - December 13th, 2009 [December 13th, 2009]
- All About the Cloud: An Interview with Dell’s Cloud Evangelist - December 13th, 2009 [December 13th, 2009]
- Happy Solstice - December 21st, 2009 [December 21st, 2009]
- A ghost of Christmas past - December 31st, 2009 [December 31st, 2009]
- Learning from a Blender - January 5th, 2010 [January 5th, 2010]
- Changing my world - January 6th, 2010 [January 6th, 2010]
- A Server. From Scratch. - January 7th, 2010 [January 7th, 2010]
- The Planet Sand Castle: Upgrade Your Sandbox - January 12th, 2010 [January 12th, 2010]
- Hosting for Haiti - January 20th, 2010 [January 20th, 2010]
- Redefining Value - January 26th, 2010 [January 26th, 2010]
- My Experience as a Newbie at The Planet - January 28th, 2010 [January 28th, 2010]
- Confessions of Another New Planeteer - February 1st, 2010 [February 1st, 2010]
- How I Learned to Stop Worrying and Love Permissions - February 11th, 2010 [February 11th, 2010]
- Where at The Planet is Rachel? - February 15th, 2010 [February 15th, 2010]
- The Planet Storage Cloud: FYI - February 19th, 2010 [February 19th, 2010]
- Meet us in March - February 25th, 2010 [February 25th, 2010]
- The Planet in “The Channel” - March 2nd, 2010 [March 2nd, 2010]
- The Planet Server Challenge - March 13th, 2010 [March 13th, 2010]
- The Definitive Guide to Finding The Planet at SXSW - March 13th, 2010 [March 13th, 2010]
- The SXSW Iron Geek Champion! - March 15th, 2010 [March 15th, 2010]
- Drinking from the Fire Hose - March 16th, 2010 [March 16th, 2010]
- The Fastest Hands at SXSW - March 17th, 2010 [March 17th, 2010]
- System.out.println(“Hello World!”); - March 22nd, 2010 [March 22nd, 2010]
- Westmere – Get it Here - March 23rd, 2010 [March 23rd, 2010]
- Orbit on Your iPhone: A Sign of Things to Come - March 24th, 2010 [March 24th, 2010]
- #ShowMeMyServer 2.0 - March 25th, 2010 [March 25th, 2010]
- Get to Know Your Visitors - March 30th, 2010 [March 30th, 2010]
- The Next Big Thing in Hosting: The Hostatulator - April 1st, 2010 [April 1st, 2010]
- Storage Cloud and the City - April 4th, 2010 [April 4th, 2010]
- American Heart – Why I Walk - April 7th, 2010 [April 7th, 2010]
- The Cake Shouldn’t Be a Lie - April 8th, 2010 [April 8th, 2010]
- April Showers Bring May Flowers - April 9th, 2010 [April 9th, 2010]
- First at The Planet: Nehalem EX 4-Socket Servers - April 15th, 2010 [April 15th, 2010]
- Intel Guest Blog: Xeon 5600 - April 16th, 2010 [April 16th, 2010]
- Inside the Office: A Birthday Surprise - April 18th, 2010 [April 18th, 2010]
- The Planet @ Cloud Expo East - April 19th, 2010 [April 19th, 2010]
- The Planet @ ad:tech SF - April 22nd, 2010 [April 22nd, 2010]
- ad:tech Server Challenge - April 22nd, 2010 [April 22nd, 2010]
- ad:tech Panel: Developing Communities Online - April 23rd, 2010 [April 23rd, 2010]
- The Planet @ Interop Las Vegas - April 27th, 2010 [April 27th, 2010]
- Overflowing With Value: 10TB is Back! - April 28th, 2010 [April 28th, 2010]
- The Cloud is NOT the Revolution - April 29th, 2010 [April 29th, 2010]
- The Importance of Orbit 2.0 - May 5th, 2010 [May 5th, 2010]
- The Planet @ Web 2.0 Expo - May 6th, 2010 [May 6th, 2010]