Downloaded 560 times
![redis-py The most popular python client library Andy McCurdy ( [email_address] ) Github: http://github.com/andymccurdy/redis-py $easy_install redis OR $pip install redis Optional $easy_install hiredis $pip install hiredis](https://image.slidesharecdn.com/redisnpythonpycon2011-110917034139-phpapp01/75/Redis-And-python-at-pycon_2011-15-2048.jpg)
![redis-py The most popular python client library Andy McCurdy ( [email_address] ) Github: http://github.com/andymccurdy/redis-py $easy_install redis OR $pip install redis Optional $easy_install hiredis $pip install hiredis](https://crownmelresort.com/image.slidesharecdn.com/redisnpythonpycon2011-110917034139-phpapp01/75/Redis-And-python-at-pycon_2011-15-2048.jpg)
More Related Content
Similar to Redis And python at pycon_2011
Redis And python at pycon_2011
- 1. PyCon India, 2011Sunil Arora Redis And Python
- 2. Raising Hands... Howmany of you have used Redis before ? How many of you have got a laptop ?
- 3. About Me Itweet at Sunil Arora / @_sunil_ I work at ShopSocially I blog at http://sunilarora.org
- 4. Today's talk Whatis Redis How it works and what you can do with it Real life use-cases Real life use-cases Some hand-on with Redis
- 5. Redis - BriefHistory Initially written to improve performance of Web Analytics product LLOOGG out of his startup Salvatore Sanfilippo @antirez http://antirez.com
- 6. Redis – BriefHistory Released in March 2009 (Open Source, BSD licensed) VMWare hired Salvatore in March, 2010 Then Pieter Noordhuis (key contributor) was hired
- 7. What is Redis? Its between lot of stuff, so difficult to categorize it precisely
- 8. What is Redis? I see Redis definitely more as a flexible tool that as a solution specialized to solve a specific problem: his mixed soul of cache, store, and messaging server shows this very well -Salvatore Sanfilippo Picture by herzogbr http://www.flickr.com/photos/herzogbr/2274372747/sizes/z/in/photostream/
- 9. Redis is... RemoteData Structure Server
- 10. Redis is... Supportsrich data types of computer science - Strings, Lists, Sets, Sorted Sets, Hashes... Rich sets of primitives (commands) to manipulate these types Predictive complexity measurements
- 11. A few fundamentalsWritten in C (no external dependency) Uses memory as main storage Uses disk for persistence Single Threaded Every command performs atomic execution
- 12. Performance Screamingly fastperformance ~50K read/write operations per seconds 100K+ read/write ops per second on a regular EC2 instance
- 13. Installation $ gitclone http://github.com/antirez/redis $ cd redis $ make $ ./src/redis-server .......... .......... $./src/redis-cli Redis> PING PONG
- 14. Python Libraries Redis-py- Python client library Txredisapi - An asynchronous Python client for the Redis database, based on Twisted. Redisco - ORM for redis along the lines Ohm for Ruby
- 15. redis-py The mostpopular python client library Andy McCurdy ( [email_address] ) Github: http://github.com/andymccurdy/redis-py $easy_install redis OR $pip install redis Optional $easy_install hiredis $pip install hiredis
- 16. Lets get started...$ cd redis $ ./src/redis-server ......... >>> from redis import Redis >>> redis_client = Redis() >>> redis_client.keys() >>> help(redis_client)
- 17. Redis Keys Notbinary safe. Should not contain space or newline character A few rules about keys: Too long keys are not a good idea Too short keys is also not a good idea “ object-type:id:field” can be a nice idea, i.e. “user:1001:name”
- 18. Operations on KeysKEYS EXISTS DEL EXPIRE OBJECT PERSIST RANDOMKEY RENAME TYPE TTL EXPIREAT MOVE
- 19. Lets play withkeys >>>redis_client.keys() >>>redis_client.exists('key') >>>redis_client.delete('key') >>>redis_client.type('key') >>>...... >>>......
- 20. Data Structures StringsLists Sets Sorted Sets Hashes
- 21. Strings SET GETMSET MGET SETEX GETSET SETNX INCR INCRBY DECR DECRBY
- 22. Strings – withredis client >>> redis_client.set('key', 'value') >>> redis_client.get('key') >>> redis_client.delete('key')
- 23. Fetch multiple keysat once mget/mset redis_client.mset({'key1': 'val1', 'key2': 'val2'}) redis_client.mget('key1', 'key2',......)
- 24. Expiration Set avalue with expire >>>redis_client.setex('key', 'value', 2) #key to expire in 2 secs >>>redis_client.expire('key', 2) >>>redis_client.get('key') >>>None
- 25. Expire Semantics Keywith expiry known as volatile keys Whenever a volatile key is modified, its expiry is reset - To avoid inconsistency in following cases Replication Append Only Log
- 26. Uses To storetransient states in your web application
- 27. Uses Who isonline?
- 28. Uses Redis asLRU cache (http://antirez.com/post/redis-as-LRU-cache.html)
- 29. Atomic Increments >>>help(redis_client.incr)>>>help(redis_client.decr) >>> >>> redis_client.incr('counter', 1) >>>1 >>> redis_client.incr('counter') >>>2 >>> redis_client.incr('counter') >>>3
- 30. Uses High Speedcounters (views/clicks/votes/likes..)
- 31. Uses API RateLimiting
- 32.
- 33. Lists Ordered listof binarysafe strings Doubly linked list Memory footprint optimized for smaller list O(1) insertion/deletion at both ends
- 34. Lists - operationsLPUSH RPUSH LSET LRANGE LPOP BLPOP BRPOP BRPOPLPUSH LINSERT RPOP RPOPLPUSH LPUSHX RPUSHX
- 35. Uses Web appsare full of lists :)
- 36.
- 37. Uses Real timemessage Queue Background Worker queues (Resque, Celery) https://github.com/blog/542-introducing-resque
- 38. Uses Social ActivityStreams or notifications
- 39. Sets An unorderedcollection of distinct byte strings Nothing different from the data type in python
- 40. Sets - OperationsSADD SCARD SREM SISMEMBER SMEMBERS SPOP SRANDMEMBER
- 41. Operations between SetsSMOVE SUNION SDIFF SINTER
- 42. Sets - OperationsSDIFFSTORE SINTERSTORE SUNIONSTORE
- 43. Sets - UsesPicking random items from a set using SRANDMEMBER Ex. Picking a random article from daily news Pick a random Ad for serving Pick a random option for A/B Note: Time complexity is O(1). Compare it with SQL's “order by Rand()”
- 44. Sets - UsesTo model Relations in social graph Ex. >>>redis_client.sadd('friends:john', 'jenny', 'maria') >>>redis_client.sadd('friends:ben', 'maria', 'kate') >>>redis_client.sinter('friends:john', 'friends:ben')
- 45. Relations (friend/followers) Commonfollowlist for A and B >>> sinter('users:A:follows', 'users:B:follows') Unique to B compared to C >>>sdiff('users:B:follows', 'users:C:follows') Mutual relationship (friend as well as follower) >>>sinter('users:B:followers', 'users:B:friends') Who does not follow me back ? >>>sdiff('users:B:friends', 'users:B:followers') Who am I not following back? >>>sdiff('users:B:followers', 'user:B:friends')
- 46. Which of myfriend's are online right now? http://www.lukemelia.com/blog/archives/2010/01/17/redis-in-practice-whos-online/ SINTER online_people my_friends
- 47. Which of myfriends are online right now? >>>RENAME online:fresh online:stale #every minute >>>SUNIONSTORE online:fresh online:stale >>>#friend jack connects >>>SADD online:fresh 'jack' >>>SUNIONSTORE online online:fresh online:stale >>>#who is online ? >>>SINTER online friends
- 48. Sorted Sets Orderedsets on the basis of score
- 49. Sorted Sets ZADDZCARD ZCOUNT ZINCRBY ZINTERSTORE ZRANGE ZRANGEBYSCORE ZRANK ZREM ZREMRANGEBYRANK ZREMRANGEBYSCORE ZREVRANGE ZREVRANGEBYSCORE ZSCORE ZUNIONSTORE
- 50. Sorted Sets –Use Case To build index on your dataset
- 51. Uses - RealtimeLeaderboards >>>zincrby('leaderboard', 'john', 2) >>>zincrby('leaderboard', 'jack', 5) >>>zincrby('leaderboard', 'kate', 1) >>>zincrby('leaderboard', 'kate', 10) .... >>>zrange('leaderboard', 0, -1, withscores = True)
- 52. Uses - RealtimeLeaderboards Most download resources Most popular articles on the website Weekly popular list Most downloaded stuff between date1 and date2 ..... .....
- 53. AutoComplete with SoredSets http://antirez.com/post/autocomplete-with-redis.html
- 54. Hashes Equivalent toPython dictionary or Ruby hash or Java hashmap Operations hmset users:1 {'username': 'jim', 'score': 23} hgetall users:1 Hincrby Useful for storing structured data hmset user:1:preferences {'flash_shown': 'yes', 'bgcolor': '#fff'} Expire user:1:preferences <duration> Hmgetall user:1:preferences #returns entire dict
- 55. Redis Transactions UsingMulti/Watch >>>p = redis_client.pipeline() >>>p.lpush('a', 1) >>>p.ltrim('a', 0, 100) >>>p.execute Limitation Since commands are queued, Can't read values No conditional execution If not high write contention scenario, WATCH can be used Use Redis Scripting to write your own primitive
- 56. Designing with RedisData layout design on basis of Query No Query Optimizer Manually build indexes
- 57. Durability Snapshotting modeBinary dump every x secs or y ops Append Only File (AOF) Every command is written to a file On restart/crash, commands replayed Fsync on every new command Fsync every second OS decide to Replication
- 58. Publish/Subscribe A simpleand efficient implementation of publish/subscribe messaging paradigm Client can subscribe/psubscribe to receive messages on channels (key patterns)
- 59. Publish/Subscribe PSUBSCRIBE PUBLISHPUNSUBSCRIBE SUBSCRIBE UNSUBSCRIBE
- 60. Uses Many toMany message passing
- 61.
- 62. References Redis Website( http://redis.io ) SimonWillison Tutorial ( http://simonwillison.net/static/2010/redis-tutorial/ ) Redis from ground up ( http://blog.mjrusso.com/2010/10/17/redis-from-the-ground-up.html#heading_toc_j_19 ) Redis under the hood ( http://pauladamsmith.com/articles/redis-under-the-hood.html ) Tumbler and Redis (http://engineering.tumblr.com/post/7819252942/staircar-redis-powered-notifications)
- 63. References Redis atdisqus (http://bretthoerner.com/2011/2/21/redis-at-disqus/) Redis at Craiglist http://blog.zawodny.com/2011/02/26/redis-sharding-at-craigslist/ Redis at Bump http://devblog.bu.mp/how-we-use-redis-at-bump Redis: AK 47 of postrelational database http://www.slideshare.net/karmi/redis-the-ak47-of-postrelational-databases
- 64.
- 65. Thanks Contact detailsTwitter: @_sunil_ Email: arora.sunil AT gmail.com