docker-compose编排redis集群

redis常用架构

主从

一主多从 主从读写分离

daemonize no docker要求容器内至少有一条前台进程,故不能设置为守护进程

从节点设置主节点ip replicaof <masterip> <masterport>

version: '3'

services:
  master:
    image: redis
    container_name: redis-master
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-master-follower/master/conf:/etc/redis
      - /usr/local/etc/docker-compose/redis-master-follower/master/data:/data
      - /usr/local/etc/docker-compose/redis-master-follower/master/log:/log
    ports:
      - 6380:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.80


  follower:
    image: redis
    container_name: redis-follower
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-master-follower/follower/conf:/etc/redis
      - /usr/local/etc/docker-compose/redis-master-follower/follower/data:/data
      - /usr/local/etc/docker-compose/redis-master-follower/follower/log:/log
    ports:
      - 6381:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.81



networks:
  redisNet:
    external: true 

sentinel哨兵

保证高可用,解决主节点宕机后无法写入的问题 主节点挂掉后哨兵会发布sdown(主观宕机),当sdown数量达到预设值时即为odown(客观宕机) 一般配置成半数sdown即odown,所以sentinel架构最小单位应为1主2从3哨兵 哨兵启动成功后写入myid

设置哨兵监控的主服务器和odown数(sdown数累计达到odown即开启故障转移主从切换)
sentinel monitor mymaster 192.168.11.128 6379 2

x秒不回应即单sentinel判定为sdown
sentinel down-after-milliseconds

单次故障转移最长时间,超过则认定为故障转移失败
sentinel failover-timeout mymaster 10000 

当发生failover主备切换时最多可以有多少个slave同时对新的master同步,数字越小favilover越快.通常设为1
sentinel parallel-syncs <master-name> <numslaves>

哨兵启动后自动写入的唯一id,每个sentinel不一样,故不能用同一份配置文件
sentinel myid

这里哨兵容器应该等待redis容器先启动,使用depends_on控制

version: '3'

services:
  master:
    image: redis
    container_name: redis-master
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-sentinel/master/conf:/etc/redis
      - /usr/local/etc/docker-compose/redis-sentinel/master/data:/data
      - /usr/local/etc/docker-compose/redis-sentinel/master/log:/log
    ports:
      - 6380:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.80

  follower1:
    image: redis
    container_name: redis-follower1
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-sentinel/follower1/conf:/etc/redis
      - /usr/local/etc/docker-compose/redis-sentinel/follower1/data:/data
      - /usr/local/etc/docker-compose/redis-sentinel/follower1/log:/log
    ports:
      - 6381:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.81

  follower2:
    image: redis
    container_name: redis-follower2
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-sentinel/follower2/conf:/etc/redis
      - /usr/local/etc/docker-compose/redis-sentinel/follower2/data:/data
      - /usr/local/etc/docker-compose/redis-sentinel/follower2/log:/log
    ports:
      - 6382:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.82
      

  sentinel1:
    image: redis
    depends_on:
      - master
      - follower1
      - follower2
    container_name: redis-sentinel1
    command: /bin/bash -c "redis-sentinel /etc/redis/redis-sentinel.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel1:/etc/redis
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel1/data:/data
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel1/log:/log
    ports:
      - 26379:26379
    networks:
      redisNet:
        ipv4_address: 192.168.88.83

  sentinel2:
    image: redis
    depends_on:
      - master
      - follower1
      - follower2
    container_name: redis-sentinel2
    command: /bin/bash -c "redis-sentinel /etc/redis/redis-sentinel.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel2:/etc/redis
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel2/data:/data
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel2/log:/log
    ports:
      - 26380:26379
    networks:
      redisNet:
        ipv4_address: 192.168.88.84

  sentinel3:
    image: redis
    depends_on:
      - master
      - follower1
      - follower2
    container_name: redis-sentinel3
    command: /bin/bash -c "redis-sentinel /etc/redis/redis-sentinel.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel3:/etc/redis
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel3/data:/data
      - /usr/local/etc/docker-compose/redis-sentinel/sentinel/sentinel3/log:/log
    ports:
      - 26381:26379
    networks:
      redisNet:
        ipv4_address: 192.168.88.85


networks:
  redisNet:
    external: true  
      

redis-cluter

redis5.0之后可以直接使用redis-cli创建集群,不再需要ruby. redis会将slot平均分配给master.所有数据操作都会通过CRC16(key) mod 16384得到的值决定此次操作所在分区.

cluster  cluster-enabled yes 开启集群模式

cluster-node-timeout：连接超时时间

cluster-announce-ip：集群各节点IP地址(以下不配置也行,redis会自动搜索)

cluster-announce-port：集群节点映射端口

cluster-announce-bus-port：集群总线端口

cluster-require-full-coverage no　
redis cluster需要16384个slot都正常的时候才能对外提供服务，换句话说，只要任何一个slot异常那么整个cluster不对外提供服务。 因此生产环境一般为no

version: '3'

services:
  master1:
    image: redis
    container_name: redis-cluster1-master
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-cluster/cluster1/master:/etc/redis
      - /usr/local/etc/docker-compose/redis-cluster/cluster1/master:/data
      - /usr/local/etc/docker-compose/redis-cluster/cluster1/master:/log
    ports:
      - 6379:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.80

  follow1:
    image: redis
    container_name: redis-cluster1-follower
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-cluster/cluster1/follower:/etc/redis
      - /usr/local/etc/docker-compose/redis-cluster/cluster1/follower:/data
      - /usr/local/etc/docker-compose/redis-cluster/cluster1/follower:/log
    ports:
      - 6380:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.81

  master2:
    image: redis
    container_name: redis-cluster2-master
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-cluster/cluster2/master:/etc/redis
      - /usr/local/etc/docker-compose/redis-cluster/cluster2/master:/data
      - /usr/local/etc/docker-compose/redis-cluster/cluster2/master:/log
    ports:
      - 6381:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.82

  follow2:
    image: redis
    container_name: redis-cluster2-follower
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-cluster/cluster2/follower:/etc/redis
      - /usr/local/etc/docker-compose/redis-cluster/cluster2/follower:/data
      - /usr/local/etc/docker-compose/redis-cluster/cluster2/follower:/log
    ports:
      - 6382:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.83

  master3:
    image: redis
    container_name: redis-cluster3-master
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-cluster/cluster3/master:/etc/redis
      - /usr/local/etc/docker-compose/redis-cluster/cluster3/master:/data
      - /usr/local/etc/docker-compose/redis-cluster/cluster3/master:/log
    ports:
      - 6383:6379
    networks:
      redisNet:
        ipv4_address: 192.168.88.84

  follow3:
    image: redis
    container_name: redis-cluster3-follower
    command: /bin/bash -c "redis-server /etc/redis/redis.conf"
    volumes:
      - /usr/local/etc/docker-compose/redis-cluster/cluster3/follower:/etc/redis
      - /usr/local/etc/docker-compose/redis-cluster/cluster3/follower:/data
      - /usr/local/etc/docker-compose/redis-cluster/cluster3/follower:/log
    ports:
      - 6384:6379   
    networks:
      redisNet:
        ipv4_address: 192.168.88.85

networks:
  redisNet:
    external: true  
      

这里yml中的master/follower没有实际意义,具体的主从分配是redis决定的.

docker-compose成功启动六个redis容器之后,进入任意一个,配置redis-cluster集群.

编排容器
docker-compose up -d

进入容器
docker exec -it container_id -bin/bash

配置redis集群,每个master一个follower
redis-cli --cluster create
192.168.88.80:6379
192.168.88.81:6379
192.168.88.82:6379
192.168.88.83:6379
192.168.88.84:6379
192.168.88.85:6379
--cluster-replicas 1

如图,16384个slot平均分配给三台master.绑定关系为
83 replicaof 82
85 replicaof 81
84 replicaof 80