Category: Databases
Managed Redis and Valkey: caching, queues and sessions without losing data
Your application's cache, job queues and sessions on a Redis or Valkey that does not run out of memory, fails over on its own if it goes down and knows which data it can lose and which it cannot.
- Where it runs
- In your account, in your name
- Backups
- With tested restores
- Updates
- Planned and documented
- Licence
- Valkey: BSD 3-Clause. Redis 8: RSALv2, SSPLv1 or AGPLv3
In short
Redis is an in-memory key-value database used as a cache, a queue and a session store; Valkey is its open source fork under the Linux Foundation. We install and run them in your cloud account: persistence, replicas with Sentinel or Cluster, memory limits, metrics and upgrades, all described as code in your repository.
What do we take care of?
Redis usually enters a project as «the cache» and ends up holding sessions, job queues, rate limits and locks. Each use needs a different configuration, and the trouble starts when they all share the same instance with the default settings.
Persistence
RDB, AOF or none, depending on what happens if the contents are lost.
Replicas
Asynchronous replication and failover with Sentinel or with cluster mode.
Memory
maxmemory, eviction policy and fragmentation under control.
Metrics
redis_exporter in Prometheus: memory, hit rate, latency and replication.
Access
Per-application ACLs, TLS and dangerous commands blocked.
As code
Configuration in Terraform or Ansible, in your repository and documented.
What architecture do we recommend?
The first decision is to separate what can be lost from what cannot. A pure cache goes on an instance with no persistence and evicting the least used keys. Sessions and queues go on another one, with AOF and the noeviction policy: if it fills up, Redis rejects writes and an alert fires, instead of silently deleting pending jobs.
| Situation | What we set up | Why |
|---|---|---|
| A cache that can be rebuilt | One node, no persistence, allkeys-lru |
The simplest; if it goes down, the application falls back to the database |
| Sessions or queues on a single data server | Primary, replica and three Sentinels, with AOF | Automatic failover and loss limited to one second |
| Data that does not fit on one node | Redis Cluster or Valkey in cluster mode | Spreads keys across several primaries with their replicas |
| Provider with a managed service | ElastiCache, Memorystore, Azure Managed Redis or another | Fewer moving parts; we tune, monitor and test |
Replication in Redis is asynchronous: an acknowledged write can be lost if the primary goes down right afterwards. If a particular operation cannot tolerate that, we use WAIT or also store it in the main database.
What problems do we come across most often?
- No maxmemory: Redis grows until the system kills it for lack of memory and everything is lost at once.
- Wrong eviction policy: a queue with
allkeys-lrudeletes jobs without warning when it fills up. - Keys with no expiry: the cache grows without limit because nobody gave the keys a time to live.
- Blocking commands: a
KEYS *in production or huge keys that stall the server for seconds. - Exposed to the internet: no password and on the default port, one of the most exploited mistakes.
- Sentinel on two machines: with no possible majority, failover does not happen or leaves two primaries.
How do we handle backups and upgrades?
Even if you use AOF, we keep periodic RDB snapshots off the server, encrypted, in another account or with another provider. The test consists of starting an isolated instance from that backup, checking the number of keys and reading a few control keys. The time it takes goes into the report.
To upgrade, we start with the replicas, do a controlled failover so that an already upgraded replica becomes the primary, and finish with the old primary. That way the interruption is limited to the seconds of the failover. Beforehand we rehearse it on a copy and review what changes in the new version, above all in commands and in the format of the persistence files.
What about the Redis licence?
In 2024 Redis moved from BSD to RSALv2 and SSPLv1, and Valkey was born from that change, with a BSD licence; Redis 8 added the AGPLv3 option, which is open source. The full story and when Valkey makes sense are in managed Valkey.
For most companies that use Redis inside their application, any of the options will do. We review it with you if you are going to offer Redis as a service to third parties or if your internal policy requires OSI-approved licences. We choose on practical grounds: what your managed provider offers, which modules you need and which version your client libraries support.
How do we migrate from a managed service to your account, or the other way round?
We start from an exported RDB backup or copy the keys live with a migration tool, depending on the size and what the application can tolerate. For a cache, it is often enough to point the application at the new instance and let it fill up on its own. For queues and sessions we plan the cutover for a quiet moment. The metrics end up in monitoring with Grafana and Prometheus, and if Redis sits alongside a relational database, we manage it with the same approach as managed PostgreSQL.
What about Memcached?
Memcached is only an in-memory cache: it stores nothing on disk, has no replication and the application itself spreads the data across nodes. For caching pages or query results it is simple and fast, and if your application already uses it there is no need to change it. We size it by memory, monitor the hit rate and evictions, and do not treat it as storage: if it restarts, it empties. When you need data to survive a restart, queues or structures such as lists and sets, we recommend Redis or Valkey. AWS and Google Cloud offer it as a managed service; everywhere else we set it up ourselves.
Where do we set it up?
In your account, with the provider you already use or the one that fits best. If the data has to stay in the EU, Hetzner and OVHcloud are the European options.
Providers we set it up on
- Hetzner
- AWS
- Google Cloud
- Azure
- DigitalOcean
- OVH
- Own server
| Provider | How we set it up | Things to bear in mind |
|---|---|---|
| Hetzner | Cloud servers with Redis or Valkey, Sentinel on three machines and a private network. RDB backups to Hetzner Object Storage, which is S3-compatible. | Hetzner offers no managed cache: we set up replication, failover and backups ourselves. |
| AWS | ElastiCache for Valkey or Redis OSS, or MemoryDB when Redis is the primary database and a cache is not enough. | We review node size, eviction policy, cross-zone replicas and that the backup restores. |
| Google Cloud | Memorystore for Valkey or Memorystore for Redis, with cluster mode if the data needs to be spread out. | Private access from your VPC; nothing exposed to the internet. |
| Azure | Azure Managed Redis, which replaces Azure Cache for Redis, or virtual machines with Valkey and Sentinel. | If you already use Azure Cache for Redis, we plan the migration Microsoft sets out before its retirement. |
| DigitalOcean | Managed Caching for Valkey, which has replaced its former managed Redis, or Droplets with Sentinel. | |
| OVH | Managed Valkey from OVHcloud Public Cloud, or instances with Valkey and Sentinel set up by us. | |
| Own server | Redis or Valkey on your servers or virtual machines, with Sentinel and RDB backups to external S3 storage. | Three machines for Sentinel; with two, a network split can leave two primaries at once. |
What is included
- Redis or Valkey in your cloud account or on your servers, or your provider's managed service if it is a better fit.
- Persistence chosen by use: none for a pure cache, RDB, AOF or both for queues and sessions.
- Replication and automatic failover with Sentinel, or Redis Cluster when the data does not fit on one node.
- maxmemory and eviction policy tuned for each instance, keeping cache apart from data that cannot be lost.
- Per-application ACL users, TLS and access only from the private network.
- Metrics with redis_exporter and alerts for memory, evictions, rejected connections and lagging replicas.
- RDB backups off the server and a test restore with the time measured.
- Rehearsed upgrades, one replica at a time, and the configuration in Terraform or Ansible in your repository.
Frequently asked questions
Should we choose Redis or Valkey?
For caching, queues and sessions both work the same: Valkey was born from Redis 7.2.4 and speaks the same protocol, so the client libraries work. Valkey has a BSD licence and is maintained by the Linux Foundation. Redis 8 can be used under AGPLv3, which is open source, or under RSALv2 or SSPLv1. If your provider offers one of the two as a managed service, we usually start from that one.
Can we lose data if Redis restarts?
It depends on persistence. Without it, everything is lost, which is fine for a cache. With RDB you lose what was written since the last snapshot; with AOF and writing to disk every second, one second at most. For queues and sessions we configure AOF and write down how much can be lost.
Sentinel or Redis Cluster?
Sentinel when the data fits in the memory of one server: a primary, replicas and automatic failover. Cluster when it does not fit or when a single node cannot handle the writes. Cluster spreads the keys out and requires multi-key operations to be in the same slot, so the application has to be ready for it.
Do you run it inside Kubernetes?
Yes, with an operator or a StatefulSet, persistent volumes and rules so that the replicas do not land on the same node. If the cache is critical and the provider has a managed service, we often recommend taking it out of the cluster.
Tell us about your case
In a free 20-minute call we look at where you are and where to start. No commitment.
Official documentation consulted
The technical details on this page are based on:
Reviewed on