New in Gluesync 2.3: the Redis Streams target agent

Gluesync 2.3 adds a new Redis Streams target agent. Every snapshot row and every CDC change the source hands over is appended to a Redis Stream with XADD, standalone or cluster, as flat fields or one JSON document per entry, with optional stream trimming. The existing Redis agent stays as it is; this one is a separate target built for event streaming.

Redis Streams is a target in Gluesync. Any supported source feeds it through a pipeline in Core Hub, and downstream consumers read the stream with the tools they already use. Streams don't need to exist beforehand: the first XADD creates them.

Docs: Redis Streams agent · Target setup · Changelog. Overview: Replicate to Redis Streams with Gluesync.

What the agent does

• Appends one XADD entry per insert, update and delete, and one marker entry per truncate or snapshot deletion
• Connects to standalone Redis or to a cluster through the Lettuce client
• Writes entries in a flat layout (default) or a JSON layout
• Can include the before image on updates and deletes
• Trims streams with MAXLEN or MINID when you ask for it
• Accepts all 16 Gluesync data types

The 2.3.0.2 release notes list it as a new target agent, released 19 October 2026. It is tested from Redis 7.4, Community Edition and Enterprise Edition.

Why a stream

A stream keeps every change in order. Each entry says what happened (insert, update, delete or truncate), to which entity and key, and when. Consumers replay, fan out, or reconstruct state at their own pace, and the agent only ever appends: old entries leave the stream only through the optional MAXLEN or MINID clause on XADD itself.

Operation	What is appended
Insert	op insert, the row columns
Update	op update, the row columns, plus the old values when Include before image is on
Delete	op delete, the key columns, plus the row the source sent when Include before image is on
Truncate	one marker, op truncate, when Emit truncate marker events is on (default)
Snapshot deletion	the same marker with snapshot true

A snapshot row keeps the operation the source gave it and sets snapshot to true. Redis assigns the entry id.

Flat or JSON entries

Flat layout (default): metadata fields first, each with the prefix _ by default, then one field per row column:

• _op, _snapshot, _entity, _ts_ms on every entry
• _key (the document id) and _keys (a JSON object of the key columns) on row entries
• _source_ts_ms and _transaction when the source provides them
• _before when the before image is on
• _nulls, a JSON array of the columns that were null, or a sentinel value per null column if you prefer

XADD customers * \
  _op insert \
  _snapshot false \
  _entity customers \
  _key customers::7 \
  _keys {"id":7} \
  _ts_ms 1700000000000 \
  id 7 \
  full_name Ada \
  _nulls ["age"]

JSON layout: one field, payload by default, holding one JSON document with op, snapshot, entity, key, ts_ms, source_ts_ms, transaction, keys, after and before. Nulls are JSON null, numbers stay numbers, and BIG_DECIMAL is a JSON string so no digit is lost.

Stream keys

The stream for each entity is, highest precedence first:

1. The entity Stream key override, used verbatim
2. A Stream key template with {prefix}, {scope}, {collection} and {entity}
3. Otherwise <streamKeyPrefix>:<scope>:<collection>, skipping blank parts

With no prefix and no scope, the stream key is the collection name. In the entity, Key namespace is the optional scope and Stream is the collection.

Delivery and throughput

• Writes are pipelined XADD commands on one connection: Pipeline batch size is 1000 per round trip by default, with a per-entity override.
• Batches go out in order and are awaited one after another, in cluster mode too, because one stream key is owned by one node.
• Delivery is at-least-once. Command timeouts and connection errors are retried in place; a batch that Redis partly accepted before a failure is appended again on retry.
• Command timeout is 60 seconds by default.

Trimming

Trim strategy is NONE by default. MAXLEN keeps the newest N entries per stream; MINID keeps entries newer than a retention in seconds. Approximate trimming (~) is on by default.

Data types

The agent accepts all 16 Gluesync data types. In the flat layout every value is text: numbers and booleans as decimal or true/false, dates and times as ISO-8601, BIG_DECIMAL as plain decimal text, binaries as lowercase hex (or Base64 with Binary encoding), arrays and maps as JSON text. In the JSON layout, booleans and finite numbers stay JSON booleans and numbers, and arrays and maps are JSON arrays and objects.

Connection setup

Prerequisites: a Redis server that accepts XADD on the stream keys, network reachability to the host (and every cluster seed), and permission to run XADD, PING and INFO server.

In the Control Plane, add Redis Streams as the target of a pipeline:

• Hostname / IP Address, Port 6379 by default
• Username for a Redis ACL user, or blank for password-only authentication, and Password
• Database index, 0 by default, on a standalone connection
• Additional hosts to connect in cluster mode
• Enable TLS, with a trust store from the JVM, a certificate path, or a certificate uploaded for the agent

The same connection goes through the Core Hub REST API:

curl --location --request PUT 'http://core-hub-ip-address/pipelines/{pipelineId}/agents/{agentId}/config/credentials' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
    "hostCredentials": {
        "connectionName": "myAgentNickName",
        "protocol": "Vendor SDK",
        "host": "host-address",
        "port": 6379,
        "username": "",
        "password": "",
        "databaseName": "0",
        "disableAuth": false,
        "enableTls": false,
        "trustServerCertificate": false,
        "certificatePath": null,
        "certificatePassword": null
    },
    "customHostCredentials": {
        "commandTimeoutSeconds": 60,
        "clientName": "gluesync-redis-streams"
    }
}'

Where it fits

Operational databases on one side, event-driven services on the other. PostgreSQL, Oracle, SQL Server, MySQL, MongoDB and the rest of the Gluesync catalog turn every committed change into a Redis Stream entry that microservices, caches and workers consume in order. The full list of sources and targets is on the Gluesync page.

Next steps

1. Read the Redis Streams target setup.
2. Deploy Gluesync 2.3.0.2 and add Redis Streams as the target of a pipeline in Core Hub.
3. Pick flat or JSON entries, set a stream key template, and read the stream with XREAD or a consumer group.

Want to see it on your data? Request a free trial or book a demo. Have questions about a specific setup? Just get in touch.