SOLUTION · REPLICATE TO REDIS STREAMS

Real-time change replication to Redis Streams from your operational databases

Every committed insert, update, and delete from your systems of record is appended to a Redis stream as one entry, so consumer groups react to changes as they happen instead of polling the database.

Gluesync by MOLO17 captures changes from Oracle, SQL Server, PostgreSQL, MySQL, MongoDB, and other heterogeneous sources with a dedicated agent per database. The Redis Streams target agent appends each change with XADD through the Lettuce Java client, against a standalone server or a cluster, after a snapshot seeds each stream. Core Hub, the Gluesync control plane, runs every pipeline from one web UI and REST API.

Start a Gluesync trial Talk to us

WHO THIS IS FOR

Teams that want database changes delivered as a Redis event stream

Application engineers who already run Redis and want services to react to row changes through consumer groups, with an operation type, a key, and the row values on every entry
Platform leads who need a lightweight event backbone without a Kafka cluster, fed from Oracle, SQL Server, PostgreSQL, MySQL, or MongoDB through one Core Hub, backed by best-in-class enterprise support, rated 4.9/5 by customers
Architects who want streams that stay bounded, with MAXLEN or MINID trimming applied by the writer itself
Engineers replacing outbox tables, polling jobs, or custom publishers that exist only to put database changes on Redis, who want every source delivered by one product with snapshots and monitoring built in

THE PROBLEM

Event streams built on polling and outbox tables run behind the database

Most teams that want change events in Redis build them themselves: an outbox table that a job polls, a trigger, or a publisher inside the application. Events then trail the commit, each poll adds read load to production tables, and every source ends up with its own publisher and failure mode. Teams looking for Redis Streams CDC usually weigh the Debezium Server Redis sink, Kafka with a Redis connector, Redis Data Integration (RDI), or scripts they write and run themselves.

Gluesync addresses that with per-agent CDC into Redis Streams. A source agent reads each database's native change log, journal, or change stream; Core Hub routes the changes; the Redis Streams target agent appends them as stream entries as they arrive. PostgreSQL to Redis Streams, Oracle to Redis Streams, and MongoDB to Redis Streams share the same entry model, snapshot, and monitoring.

HOW IT WORKS

How Gluesync writes to Redis Streams

One XADD per change, pipelined through Lettuce

The agent is a target agent: it receives changes from any Gluesync source agent and appends each one to a stream with XADD. An insert, an update, and a delete each become one entry, and Redis assigns the entry ID. The agent only appends, so which entries stay is decided by the trimming you choose, not by the writer.

Pipelined batches: XADD commands are sent in pipelined batches on one connection and each batch is awaited before the next, so entries reach each stream in order. The batch size is configurable per agent and per entity.
Standalone or cluster: the agent connects to one server, or to a Redis Cluster when you list additional hosts, and a stream key lives on a single node. Redis Streams agent overview ↗
No DDL: a stream is created by its first XADD, so there is nothing to prepare in Redis before a pipeline starts.

Snapshot first, then change entries on the same stream

Seed, then stream: each entity is loaded from its full table as snapshot entries, then the same stream receives the change feed from the source agent in real time.
Snapshot flag: every entry carries a snapshot field set to true for snapshot rows, so a consumer can tell the initial load from live changes.
Truncate markers: a truncate at the source, or the deletion that precedes a snapshot, is appended as one marker entry with the truncate operation. Marker entries can be turned off.
Scheduled reseeds: the Chronos scheduler runs a snapshot, or a snapshot followed by CDC, on a cadence. See schedules and events.

Entry layout, metadata, and keys

Choose the entry shape that suits your consumers. Both layouts carry the same information, and every value is written as text or JSON.

Flat layout: the default. Metadata fields come first, then one field per row column, so a consumer reads columns directly with XREAD or XREADGROUP. Null columns are listed in one field, or written as a sentinel value you choose. The prefix on metadata field names is configurable.
JSON layout: one field holds a single JSON document with the operation, entity, key, timestamps, transaction, and the after and before images. Numbers and booleans stay JSON numbers and booleans.
Metadata on every entry: the operation (insert, update, delete, or truncate), the snapshot flag, the entity name, the row's document key and key columns, the time the entry was built, the source commit time when the source provides it, and the transaction identifier.
Before image: optionally, an update entry carries the old column values and a delete entry carries the row as the source sent it.
Types as text: dates and times are ISO-8601, decimals are plain text, never scientific notation, and binaries are written as hex or Base64, your choice.

Stream keys and trimming

Stream names: by default the key is built from an optional prefix, the entity's key namespace, and its collection name. A key template with {prefix}, {scope}, {collection}, and {entity} placeholders reshapes it for every entity, and a per-entity override names one stream exactly.
Trimming: choose no trimming, MAXLEN to keep the newest N entries per stream, or MINID to keep a time window measured in seconds. Trimming is approximate or exact, as you prefer, and runs as part of the XADD itself.
Consumers stay yours: the agent does not create consumer groups or read the stream. Your services create their own groups and acknowledge entries as they process them.

Setup: server address, authentication, and TLS

The agent needs network reach to the Redis server and an account that can run XADD. Every field is listed in the Redis Streams target setup guide ↗.

Create a Redis ACL user, or use a password, with permission to run XADD, PING, and INFO on the stream keys the pipeline writes. Authentication can also be switched off for a closed network.
In Core Hub, enter the host and the port, which defaults to 6379, plus the logical database index for a standalone server. For a cluster, add the other nodes as additional hosts.
Turn on TLS when your deployment requires it, and either trust the JVM's certificates, point to a trust store, or upload a certificate for the agent.
Choose the entry layout, the stream key pattern, and the trimming, then map each entity to a stream or let the key pattern name it.

Architecture around Core Hub

Lightweight agents sit close to each source, and Core Hub orchestrates them through its web UI and REST APIs. A pipeline groups the source agents, the Redis Streams agent, and the entities they replicate, and Core Hub and the agents deploy with Docker, Docker Compose, or Kubernetes, on-premises or in any cloud. See CDC streaming and snapshot tasks ↗.

Because Redis Streams is one target among many on the same Core Hub, a source table can feed a stream for event consumers and land in a database or warehouse in another pipeline, from the same capture agent and with the same monitoring.

Explore the general CDC streaming architecture →

WRITE OPTIONS

The Redis Streams target agent: one XADD per change

Redis Streams has one Gluesync target agent. Every source agent feeds it through Core Hub, so the choice that shapes the pipeline is the source, while the Redis side stays the same.

Agent	Write technique	Versions	Best for
Redis Streams agent ↗	Lettuce Java client appending one `XADD` entry per change in pipelined batches, flat or JSON entry layout, with optional `MAXLEN` or `MINID` trimming	Any Redis server with Streams support, standalone or cluster	Teams that want database changes as a Redis event stream read with consumer groups. Pick it when each entity should have its own stream, with snapshots loaded first and the same entry metadata across every source.

SOURCES AND TOPOLOGIES

Stream changes from the databases you already run

Any Gluesync source agent can feed Redis Streams, each with its own native capture technique. Open the integrations finder with Redis Streams pre-selected to see every source you can pair with it.

One Core Hub runs Oracle to Redis Streams, PostgreSQL to Redis Streams, and MongoDB to Redis Streams side by side, each pipeline with its own snapshot and monitoring. Gluesync keeps pace with your change volume at any scale, and MOLO17 Professional Services can plan stream naming and trimming with your team. To store rows as Redis hashes instead of events, see Replicate to Redis.

Oracle to Redis Streams from the redo logs through LogMiner or XStream: see Oracle CDC
SQL Server to Redis Streams through Change Data Capture or Change Tracking: see SQL Server CDC
PostgreSQL and MySQL to Redis Streams from the WAL and the binlog: see PostgreSQL CDC and MySQL CDC
MongoDB to Redis Streams from change streams: see MongoDB CDC

FAIR, HIGH-LEVEL COMPARISON

Where Gluesync fits among Redis Streams approaches

Approach	What buyers usually get	Where Gluesync fits
Debezium Server with the Redis sink	Open-source log-based capture that writes change events to Redis Streams; your team deploys, configures, and upgrades Debezium Server per source	A native capture agent per database under one Core Hub, with snapshots and monitoring built in; read the Debezium alternative comparison
Kafka with a Redis connector	Events travel through Kafka and a connector into Redis; your team runs the brokers, Connect, and the connector	Changes go straight from each source agent to the stream, with no Kafka cluster in the path. Kafka stays available as another target; see Replicate to Kafka
Redis Data Integration (RDI)	Redis's own pipeline for bringing database changes into Redis, designed around Redis as the destination	One control plane for every source and target, so the same capture also feeds databases, warehouses, and Kafka, with MOLO17 enterprise support
Outbox tables and polling jobs	Full control; your team owns the outbox schema, the polling job, the publisher code, and the read load each poll puts on the source	Log-based capture with no polling queries against production and no publisher code to maintain; read log-based CDC explained
Custom scripts and consumers	A script per source that reads changes and runs XADD; your team handles keys, restarts, initial loads, and monitoring	Snapshot seeding, continuous CDC, and monitoring from one console; see migrating to Gluesync

Redis Streams replication research and implementation detail

What does replicating to Redis Streams with Gluesync involve?

A source agent captures committed changes from your database through its native change mechanism, Core Hub routes them, and the Redis Streams agent appends each one to a stream with XADD. Each table is seeded with a snapshot first and then follows the change stream.

What does one stream entry contain?

The operation (insert, update, delete, or truncate), a snapshot flag, the entity name, the row's key, timestamps, the transaction identifier, and the row's column values. You can choose a flat layout with one field per column, or a single JSON field. An optional before image carries the old values for updates and deletes.

Do I have to create the streams first?

No. Redis creates a stream with its first XADD, so a pipeline can start against an empty server. The stream name comes from a prefix, the entity's namespace, and its collection, or from a template or per-entity override that you set.

How are deletes and truncates represented?

A delete is appended as one entry with the delete operation and the row's key columns, and a truncate is appended as one marker entry. The agent only appends, so earlier entries stay in the stream until your trimming policy removes them.

Can the streams be kept from growing without bound?

Yes. Pick MAXLEN to keep a fixed number of newest entries per stream, or MINID to keep a time window in seconds. Trimming can be approximate or exact and is applied by the XADD command itself.

How do consumers read the entries?

With the Redis commands you already use, such as XREAD and XREADGROUP. Your services create their own consumer groups and acknowledge entries on their side, so each application keeps its own position in the stream.

How does the agent connect and authenticate?

Through the Lettuce Java client, to one server or to a cluster when additional hosts are listed. Authentication is an ACL user and password, a password alone, or none for a closed network, and TLS can be enabled with the JVM's trust store, a trust store file, or an uploaded certificate.

Which sources can replicate to Redis Streams?

Any Gluesync source agent. Oracle, SQL Server, PostgreSQL, MySQL, and MongoDB can all feed Redis Streams, and each one uses its own native capture technique.

Which Redis servers work?

Any Redis server with Streams support, standalone or cluster.

REPLICATE TO A TARGET