Skip to content

Redis Sentinel unable to start with tls and auth #1014

@stekole

Description

@stekole

When using TLS and auth on Redis sentinel, the envoyproxy/ratelimit tool errors with a panic when making the initial sentinel discovery call.

ratelimit-1       | time="2025-11-28T05:05:42Z" level=info msg="Stats initialized for stdout"
ratelimit-1       | time="2025-11-28T05:05:42Z" level=info msg="Stats flush interval: 10s"
ratelimit-1       | time="2025-11-28T05:05:42Z" level=info msg="Tracing disabled"
ratelimit-1       | time="2025-11-28T05:05:42Z" level=warning msg="connecting to redis on mymaster,redis-sentinel-1:26379 with pool size 10"
ratelimit-1       | time="2025-11-28T05:05:42Z" level=debug msg="Implicit pipelining enabled: false"
ratelimit-1       | panic: read tcp 172.22.0.8:44402->172.22.0.7:26379: read: connection reset by peer
ratelimit-1       | 
ratelimit-1       | goroutine 1 [running]:
ratelimit-1       | github.com/envoyproxy/ratelimit/src/redis.checkError(...)
ratelimit-1       | 	/ratelimit/src/redis/driver_impl.go:69
ratelimit-1       | github.com/envoyproxy/ratelimit/src/redis.NewClientImpl({0xccce30, 0x40004be300}, 0x1, {0x0, 0x0}, {0x400004205a, 0x3}, {0x400004203b, 0x8}, {0x400004403a, ...}, ...)
ratelimit-1       | 	/ratelimit/src/redis/driver_impl.go:141 +0x8a8
ratelimit-1       | github.com/envoyproxy/ratelimit/src/redis.NewRateLimiterCacheImplFromSettings({0x0, {0x8ec90e, 0x7}, 0x1f90, {0x900801, 0x7}, 0x17b6, {0x0, 0x0}, {0x8fcf3b, ...}, ...}, ...)
ratelimit-1       | 	/ratelimit/src/redis/cache_impl.go:25 +0x234
ratelimit-1       | github.com/envoyproxy/ratelimit/src/service_cmd/runner.createLimiter({_, _}, {0x0, {0x8ec90e, 0x7}, 0x1f90, {0x900801, 0x7}, 0x17b6, {0x0, ...}, ...}, ...)
ratelimit-1       | 	/ratelimit/src/service_cmd/runner/runner.go:94 +0x3cc
ratelimit-1       | github.com/envoyproxy/ratelimit/src/service_cmd/runner.(*Runner).Run(0x4000148b08)
ratelimit-1       | 	/ratelimit/src/service_cmd/runner/runner.go:158 +0x4c8
ratelimit-1       | main.main()
ratelimit-1       | 	/ratelimit/src/service_cmd/main.go:10 +0xbc

From my understanding of the redis client library used, NewSentinel should have both SentinelPoolFunc and SentinelConnFunc set to include a TLS/auth config. The missing SentinelConnFunc is causing configuration of TLS/auth/timeouts to only be passed to Redis data nodes, but not the initial connections to Sentinel nodes themselves.

I have documented a local steps to replicate the problem.

  1. create TLS certs
mkdir -p redis-tls
cd redis-tls

openssl req -x509 -nodes -newkey rsa:2048 \
  -keyout redis.key \
  -out redis.crt \
  -days 365 \
  -subj "/CN=redis"

openssl dhparam -out redis.dh 2048
  1. Create configs

Create file redis-tls/redis.conf:

port 0
tls-port 6379
tls-cert-file /tls/redis.crt
tls-key-file /tls/redis.key
tls-dh-params-file /tls/redis.dh
tls-ca-cert-file /tls/redis.crt
tls-auth-clients no
tls-replication yes

Create file redis-tls/sentinel.conf:

port 0
tls-port 26379
tls-cert-file /tls/redis.crt
tls-key-file /tls/redis.key
tls-dh-params-file /tls/redis.dh
tls-ca-cert-file /tls/redis.crt
tls-auth-clients no
tls-replication yes
sentinel resolve-hostnames yes
sentinel announce-hostnames yes
sentinel monitor mymaster redis-master 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 10000
  1. Start docker-compose (see attached)
docker-compose -f docker-compose-sentinel-tls.yml up --build

docker-compose-sentinel-tls.yml

The application will then error with the message above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions