[转]基于Twemproxy的Redis集群方案

转:http://www.cnblogs.com/haoxinyue/p/redis.html

概述

由于单台redis服务器的内存管理能力有限,使用过大内存redis服务器的性能急剧下降,且服务器发生故障将直接影响大面积业务。为了获取更好的缓存性能及扩展型,我们将需要搭建redis集群来满足需求。因redis 3.0 beta支持的集群功能不适合生产环境的使用,所以我们采用twitter正在使用的twemproxy来搭建redis缓存服务器集群,目前用户包括Pinterest、Tumblr、Twitter、Vine、Kiip、Wuaki.tv、Wanelo、Kontera、Wikimedia、Bright、56.com、Snapchat、Digg、Gawkermedia、3scale.net等。

Twemproxy是memcached和redis协议的代理服务器,并能有效减少大量连接对redis服务器的性能影响,它提供的主要特性如下:

image

 

 

集群架构

image

 

安装Redis

有三台服务器,一台COS1安装twemproxy,另外两台COS2,COS3安装redis。

 

  1. 下载最新安装包:redis-2.8.9.tar.gz , tcl-8.5.7-6.el6.x86_64.rpm ,nutcracker-0.3.0.tar.gz
  2. image
  3. 安装必要组件rpm:
    [root@COS2 redis-2.8.9]# yum install gcc
    [root@COS2 src]# rpm -ivh tcl-8.5.7-6.el6.x86_64.rpm
  4. 安装Redis:
    复制代码
    [root@COS2 src]# tar xvf redis-2.8.9.tar.gz
    [root@COS2 src]# cd redis-2.8.9
    [root@COS2 redis-2.8.9]# make
    …
    Hint: To run 'make test' is a good idea ;)
    make[1]: Leaving directory `/usr/local/src/redis-2.8.9/src'
    
    [root@COS2 redis-2.8.9]# make test
    All tests passed without errors!
    
    Cleanup: may take some time... OK
    make[1]: Leaving directory `/usr/local/src/redis-2.8.9/src'
    [root@COS2 redis-2.8.9]# make install
    [root@COS2 redis-2.8.9]# cd /usr/local/bin/
    [root@COS2 bin]# ll
    total 13908
    -rwxr-xr-x. 1 root root 4170264 Apr 26 11:51 redis-benchmark
    -rwxr-xr-x. 1 root root   22185 Apr 26 11:51 redis-check-aof
    -rwxr-xr-x. 1 root root   45419 Apr 26 11:51 redis-check-dump
    -rwxr-xr-x. 1 root root 4263471 Apr 26 11:51 redis-cli
    -rwxr-xr-x. 1 root root 5726791 Apr 26 11:51 redis-server
    复制代码
  5. 编辑redis配置文件:
    复制代码
    [root@COS2 redis-2.8.9]# cp redis.conf /etc/
    [root@COS2 redis-2.8.9]# vim /etc/red
    redhat-release  redis.conf      
    [root@COS2 redis-2.8.9]# vim /etc/redis.conf
    
    把里面的
    daemonize no  修改成 daemonize yes
    复制代码
  6. 启动redis服务:
    [root@COS2 redis-2.8.9]# redis-server /etc/redis.conf
  7. 测试redis服务:
    [root@COS2 redis-2.8.9]# redis-cli 
    127.0.0.1:6379> set kin kin
    OK
    127.0.0.1:6379> get kin
  8. 同样的步骤安装其他redis服务器。

 

 

安装twemproxy

  1. 安装twemproxy:
    [root@COS1 src]# tar xvf nutcracker-0.3.0.tar.gz
    [root@COS1 nutcracker-0.3.0]# cd nutcracker-0.3.0
    [root@COS1 src]#./configure 
    [root@COS1 nutcracker-0.3.0]# make && make install
  2. 编辑配置文件:
    复制代码
    [root@COS1 conf]# cd /usr/local/src/nutcracker-0.3.0/conf
    [root@COS1 conf]# cp nutcracker.yml /etc/
    [root@COS1 conf]# vim /etc/nutcracker.yml
    alpha:
      listen: 0.0.0.0:22121
      hash: fnv1a_64
      distribution: ketama
      auto_eject_hosts: true
      redis: true
      server_retry_timeout: 2000
      server_failure_limit: 1
      servers: --两台redis服务器的地址和端口
       - 10.23.22.240:6379:1   
       - 10.23.22.241:6379:1
    复制代码
  3. 测试配置文件:
    [root@COS1 nutcracker-0.3.0]# nutcracker -t /etc/nutcracker.yml 
    nutcracker: configuration file 'conf/nutcracker.yml' syntax is ok
  4. 启动twemproxy:
    复制代码
    [root@COS1 nutcracker-0.3.0]# nutcracker  --help
    This is nutcracker-0.3.0
    
    Usage: nutcracker [-?hVdDt] [-v verbosity level] [-o output file]
                      [-c conf file] [-s stats port] [-a stats addr]
                      [-i stats interval] [-p pid file] [-m mbuf size]
    
    Options:
      -h, --help             : this help                           
      -V, --version          : show version and exit                 
      -t, --test-conf        : test configuration for syntax errors and exit 
      -d, --daemonize      : run as a daemon                    
      -D, --describe-stats   : print stats description and exit
      -v, --verbosity=N      : set logging level (default: 5, min: 0, max: 11)
      -o, --output=S         : set logging file (default: stderr)
      -c, --conf-file=S      : set configuration file (default: conf/nutcracker.yml) #配置
      -s, --stats-port=N     : set stats monitoring port (default: 22222)
      -a, --stats-addr=S     : set stats monitoring ip (default: 0.0.0.0)
      -i, --stats-interval=N : set stats aggregation interval in msec (default: 30000 msec)
      -p, --pid-file=S       : set pid file (default: off)
      -m, --mbuf-size=N      : set size of mbuf chunk in bytes (default: 16384 bytes)
    [root@COS1 nutcracker-0.3.0]# nutcracker -d -c /etc/nutcracker.yml
    [root@COS1 nutcracker-0.3.0]# ps -ef|grep nutcracker
    root     15358     1  0 02:40 ?        00:00:00 nutcracker -d -c /etc/nutcracker.yml
    复制代码
  5. 测试twemproxy:
    复制代码
    [root@COS1 ~]# redis-cli -p 22121
    127.0.0.1:22121> get kin
    "kin"
    127.0.0.1:22121> set kin king
    OK
    127.0.0.1:22121> get kin
    "king"
    复制代码

 

 

 

 

性能测试

这里使用redis自带的redis-benchmark进行简单的性能测试,测试结果如下:

  1. Set测试:
    1. 通过twemproxy测试:
      [root@COS1 src]# redis-benchmark -h 10.23.22.240 -p 22121 -c 100 -t set -d 100 -l –q
       
           
      SET: 38167.94 requests per second
    2. 直接对后端redis测试:
      [root@COS2 ~]# redis-benchmark -h 10.23.22.241 -p 6379 -c 100 -t set -d 100 -l –q
       
           
      SET: 53191.49 requests per second
  2. Get测试:
    1. 通过twemproxy测试:
      [root@COS1 src]# redis-benchmark -h 10.23.22.240 -p 22121 -c 100 -t get -d 100 -l -q
      GET: 37453.18 requests per second
    2. 直接对后端redis测试:
      [root@COS2 ~]# redis-benchmark -h 10.23.22.241 -p 6379 -c 100 -t get -d 100 -l -q
      GET: 62111.80 requests per second
  3. 查看键值分布:
    [root@COS2 ~]# redis-cli info|grep db0
    db0:keys=51483,expires=0,avg_ttl=0
    
    [root@COS3 ~]# redis-cli info|grep db0
    db0:keys=48525,expires=0,avg_ttl=0

测试结果:以基本的set get命令通过twemproxy性能有所下降;通过twemproxy分布基本平均。测试数据以业务测试为准。

 

 

---附 twemproxy 支持的redis命令:

https://raw.githubusercontent.com/twitter/twemproxy/master/notes/redis.md

 

## Redis Command Support

### Keys Command

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |        DEL        |    Yes     | DEL key [key …]                                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       DUMP        |    Yes     | DUMP key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      EXISTS       |    Yes     | EXISTS key                                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      EXPIRE       |    Yes     | EXPIRE key seconds                                                                                                  |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     EXPIREAT      |    Yes     | EXPIREAT key timestamp                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       KEYS        |    No      | KEYS pattern                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      MIGRATE      |    No      | MIGRATE host port key destination-db timeout                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       MOVE        |    No      | MOVE key db                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      OBJECT       |    No      | OBJECT subcommand [arguments [arguments …]]                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PERSIST      |    Yes     | PERSIST key                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PEXPIRE      |    Yes     | PEXPIRE key milliseconds                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     PEXPIREAT     |    Yes     | PEXPIREAT key milliseconds-timestamp                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PTTL         |    Yes     | PTTL key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     RANDOMKEY     |    No      | RANDOMKEY                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RENAME       |    No      | RENAME key newkey                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     RENAMENX      |    No      | RENAMENX key newkey                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RESTORE      |    Yes     | RESTORE key ttl serialized-value                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SORT         |    Yes     | SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination]   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       TTL         |    Yes     | TTL key                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      TYPE         |    Yes     | TYPE key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SCAN         |    No      | SCAN cursor [MATCH pattern] [COUNT count]                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Strings Command

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       APPEND      |    Yes     | APPEND key value                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      BITCOUNT     |    Yes     | BITCOUNT key [start] [end]                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      BITPOS       |    Yes     | BITPOS key bit [start] [end]                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       BITOP       |    No      | BITOP operation destkey key [key ...]                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       DECR        |    Yes     | DECR key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      DECRBY       |    Yes     | DECRBY key decrement                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       GET         |    Yes     | GET key                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      GETBIT       |    Yes     | GETBIT key offset                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     GETRANGE      |    Yes     | GETRANGE key start end                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      GETSET       |    Yes     | GETSET key value                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      INCR         |    Yes     | INCR key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      INCRBY       |    Yes     | INCRBY key increment                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     INCRBYFLOAT   |    Yes     | INCRBYFLOAT key increment                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      MGET         |    Yes     | MGET key [key ...]                                                                                                  |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      MSET         |    Yes*    | MSET key value [key value ...]                                                                                      | 
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      MSETNX       |    No      | MSETNX key value [key value ...]                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PSETEX       |    Yes     | PSETEX key milliseconds value                                                                                       |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SET          |    Yes     | SET key value [EX seconds] [PX milliseconds] [NX|XX]                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SETBIT       |    Yes     | SETBIT key offset value                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SETEX        |    Yes     | SETEX key seconds value                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SETNX        |    Yes     | SETNX key value                                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SETRANGE     |    Yes     | SETRANGE key offset value                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      STRLEN       |    Yes     | STRLEN key                                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* MSET support is not Atomic 

### Hashes

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       HDEL        |    Yes     | HDEL key field [field ...]                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HEXISTS      |    Yes     | HEXISTS key field                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       HGET        |    Yes     | HGET key field                                                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HGETALL      |    Yes     | HGETALL key                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HINCRBY      |    Yes     | HINCRBY key field increment                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    HINCRBYFLOAT   |    Yes     | HINCRBYFLOAT key field increment                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HKEYS        |    Yes     | HKEYS key                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HLEN         |    Yes     | HLEN key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HMGET        |    Yes     | HMGET key field [field ...]                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HMSET        |    Yes     | HMSET key field value [field value ...]                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HSET         |    Yes     | HSET key field value                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HSETNX       |    Yes     | HSETNX key field value                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HVALS        |    Yes     | HVALS key                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HSCAN        |    Yes     | HSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Lists

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       BLPOP       |    No      | BLPOP key [key ...] timeout                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       BRPOP       |    No      | BRPOP key [key ...] timeout                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     BRPOPLPUSH    |    No      | BRPOPLPUSH source destination timeout                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LINDEX       |    Yes     | LINDEX key index                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LINSERT      |    Yes     | LINSERT key BEFORE|AFTER pivot value                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LLEN         |    Yes     | LLEN key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LPOP         |    Yes     | LPOP key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LPUSH        |    Yes     | LPUSH key value [value ...]                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LPUSHX       |    Yes     | LPUSHX key value                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LRANGE       |    Yes     | LRANGE key start stop                                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LREM         |    Yes     | LREM key count value                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LSET         |    Yes     | LSET key index value                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LTRIM        |    Yes     | LTRIM key start stop                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RPOP         |    Yes     | RPOP key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     RPOPLPUSH     |    Yes*    | RPOPLPUSH source destination                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RPUSH        |    Yes     | RPUSH key value [value ...]                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RPUSHX       |    Yes     | RPUSHX key value                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* RPOPLPUSH support requires that source and destination keys hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for source and destination key. Twemproxy does no checking on its end to verify that source and destination key hash to the same server, and the RPOPLPUSH command is forwarded to the server that the source key hashes to

### Sets

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SADD         |    Yes     | SADD key member [member ...]                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SCARD        |    Yes     | SCARD key                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SDIFF        |    Yes*    | SDIFF key [key ...]                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SDIFFSTORE    |    Yes*    | SDIFFSTORE destination key [key ...]                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SINTER       |    Yes*    | SINTER key [key ...]                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SINTERSTORE    |    Yes*    | SINTERSTORE destination key [key ...]                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SISMEMBER     |    Yes     | SISMEMBER key member                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SMEMBERS      |    Yes     | SMEMBERS key                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SMOVE        |    Yes*    | SMOVE source destination member                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SPOP         |    Yes     | SPOP key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SRANDMEMBER    |    Yes     | SRANDMEMBER key [count]                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SREM         |    Yes     | SREM key member [member ...]                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SUNION        |    Yes*    | SUNION key [key ...]                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |   SUNIONSTORE     |    Yes*    | SUNIONSTORE destination key [key ...]                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SSCAN        |    Yes     | SSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* SIDFF, SDIFFSTORE, SINTER, SINTERSTORE, SMOVE, SUNION and SUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.


### Sorted Sets

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZADD         |    Yes     | ZADD key score member [score] [member]                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZCARD        |    Yes     | ZCARD key                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZCOUNT       |    Yes     | ZCOUNT key min max                                                                                                  |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZINCRBY      |    Yes     | ZINCRBY key increment member                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     ZINTERSTORE   |    Yes*    | ZINTERSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]                 |
    +------------------------------------------------------------------------------------------------------------------------------------------------------+
    |      ZLEXCOUNT    |    Yes     | ZLEXCOUNT key min max                                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZRANGE       |    Yes     | ZRANGE key start stop [WITHSCORES]                                                                                  |
    +------------------------------------------------------------------------------------------------------------------------------------------------------+
    |    ZRANGEBYLEX    |    Yes     | ZRANGEBYLEX key min max [LIMIT offset count]                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    ZRANGEBYSCORE  |    Yes     | ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT offset count]                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZRANK        |    Yes     | ZRANK key member                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       ZREM        |    Yes     | ZREM key member [member ...]                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |   ZREMRANGEBYLEX  |    Yes     | ZREMRANGEBYLEX key min max                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |   ZREMRANGEBYRANK |    Yes     | ZREMRANGEBYRANK key start stop                                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |  ZREMRANGEBYSCORE |    Yes     | ZREMRANGEBYSCORE key min max                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    ZREVRANGE      |    Yes     | ZREVRANGE key start stop [WITHSCORES]                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |  ZREVRANGEBYSCORE |    Yes     | ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     ZREVRANK      |    Yes     | ZREVRANK key member                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     ZSCORE        |    Yes     | ZSCORE key member                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    ZUNIONSTORE    |    Yes*    | ZUNIONSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZSCAN        |    Yes     | ZSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* ZINTERSTORE and ZUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.

### HyperLogLog

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       PFADD       |    Yes     | PFADD key element [element ...]                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PFCOUNT      |    Yes     | PFCOUNT key [key ...]                                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PFMERGE      |    Yes*    | PFMERGE destkey sourcekey [sourcekey ...]                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* PFMERGE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.


### Pub/Sub

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     PSUBSCRIBE    |    No      | PSUBSCRIBE pattern [pattern ...]                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     PUBLISH       |    No      | PUBLISH channel message                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    PUNSUBSCRIBE   |    No      | PUNSUBSCRIBE [pattern [pattern ...]]                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SUBSCRIBE     |    No      | SUBSCRIBE channel [channel ...]                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     UNSUBSCRIBE   |    No      | UNSUBSCRIBE [channel [channel ...]]                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Transactions

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      DISCARD      |    No      | DISCARD                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       EXEC        |    No      | EXEC                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       MULTI       |    No      | MULTI                                                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      UNWATCH      |    No      | UNWATCH                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       WATCH       |    No      | WATCH key [key ...]                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Scripting

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       EVAL        |    Yes*    | EVAL script numkeys key [key ...] arg [arg ...]                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     EVALSHA       |    Yes*    | EVALSHA sha1 numkeys key [key ...] arg [arg ...]                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SCRIPT EXISTS  |    No      | SCRIPT EXISTS script [script ...]                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SCRIPT FLUSH   |    No      | SCRIPT FLUSH                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SCRIPT KILL    |    No      | SCRIPT KILL                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SCRIPT LOAD    |    No      | SCRIPT LOAD script                                                                                                  |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

 * EVAL and EVALSHA support is limited to scripts that take at least 1 key. If multiple keys are used, all keys must hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for all keys. If you use more than 1 key, the proxy does no checking to verify that all keys hash to the same server, and the entire command is forwarded to the server that the first key hashes to

### Connection

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       AUTH        |    No      | AUTH password                                                                                                       |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       ECHO        |    No      | ECHO message                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       PING        |    No      | PING                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       QUIT        |    No      | QUIT                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SELECT       |    No      | SELECT index                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Server

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    BGREWRITEAOF   |    No      | BGREWRITEAOF                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      BGSAVE       |    No      | BGSAVE                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    CLIENT KILL    |    No      | CLIENT KILL ip:port                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    CLIENT LIST    |    No      | CLIENT LIST                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    CONFIG GET     |    No      | CONFIG GET parameter                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    CONFIG SET     |    No      | CONFIG SET parameter value                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |  CONFIG RESETSTAT |    No      | CONFIG RESETSTAT                                                                                                    |
    +-------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
    |     DBSIZE        |    No      | DBSIZE                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    DEBUG OBJECT   |    No      | DEBUG OBJECT key                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    DEBUG SEGFAULT |    No      | DEBUG SEGFAULT                                                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     FLUSHALL      |    No      | FLUSHALL                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     FLUSHDB       |    No      | FLUSHDB                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      INFO         |    No      | INFO                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     LASTSAVE      |    No      | LASTSAVE                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     MONITOR       |    No      | MONITOR                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SAVE         |    No      | SAVE                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SHUTDOWN      |    No      | SHUTDOWN [NOSAVE] [SAVE]                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SLAVEOF       |    No      | SLAVEOF host port                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SLOWLOG       |    No      | SLOWLOG subcommand [argument]                                                                                       |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SYNC         |    No      | SYNC                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      TIME         |    No      | TIME                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

## Note

- redis commands are not case sensitive
- only vectored commands 'MGET key [key ...]', 'MSET key value [key value ...]', 'DEL key [key ...]' needs to be fragmented

## Performance

### Setup

+ redis-server running on machine A.
+ nutcracker running on machine A as a local proxy to redis-server.
+ redis-benchmark running on machine B.
+ machine A != machine B.
+ nutcracker built with --enable-debug=no
+ nutcracker running with mbuf-size of 512 (-m 512)
+ redis-server built from redis 2.6 branch

### redis-benchmark against redis-server

    $ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 6379
    SET: 89285.71 requests per second
    GET: 92592.59 requests per second
    INCR: 89285.71 requests per second
    LPUSH: 90090.09 requests per second
    LPOP: 90090.09 requests per second
    SADD: 90090.09 requests per second
    SPOP: 93457.95 requests per second
    LPUSH (needed to benchmark LRANGE): 89285.71 requests per second
    LRANGE_100 (first 100 elements): 36496.35 requests per second
    LRANGE_300 (first 300 elements): 15748.03 requests per second
    LRANGE_500 (first 450 elements): 11135.86 requests per second
    LRANGE_600 (first 600 elements): 8650.52 requests per second

### redis-benchmark against nutcracker proxing redis-server

    $ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 22121
    SET: 85470.09 requests per second
    GET: 86956.52 requests per second
    INCR: 85470.09 requests per second
    LPUSH: 84745.77 requests per second
    LPOP: 86206.90 requests per second
    SADD: 84745.77 requests per second
    SPOP: 86956.52 requests per second
    LPUSH (needed to benchmark LRANGE): 84745.77 requests per second
    LRANGE_100 (first 100 elements): 29761.90 requests per second
    LRANGE_300 (first 300 elements): 12376.24 requests per second
    LRANGE_500 (first 450 elements): 8605.85 requests per second
    LRANGE_600 (first 600 elements): 6587.62 requests per second

## redis-auth feature

+ you can enable redis-auth for a pool with 'redis_auth':

        alpha:
          listen: 127.0.0.1:22121
          hash: fnv1a_64
          distribution: ketama
          redis: true
          redis_auth: testpass

+ notice:
    + *MUST* set all redis with a same passwd, and all twemproxy with the same passwd
    + Length of password should less than 256 

 

 

 

 

 

 

 

 

 

猜你喜欢

转载自breezylee.iteye.com/blog/2274348