New in Gluesync 2.3: the Elasticsearch and OpenSearch target agent

Gluesync 2.3 adds a new Elasticsearch target agent, with OpenSearch as a flavor of the same agent. It writes snapshots and CDC changes into Elasticsearch and OpenSearch indexes through the _bulk REST API. Missing indexes are created with a mapping generated from the entity columns, and every operation on a row reaches the same document, so a replay writes it once.

Elasticsearch is a target in Gluesync. Any supported source feeds it through a pipeline in Core Hub: the snapshot first, then the source agent's CDC stream, both through the same POST /_bulk path. One HTTP JSON client serves both products.

Docs: Elasticsearch agent · Target setup · Changelog. Overview: Replicate to Elasticsearch with Gluesync.

What the agent does

• Writes snapshots and CDC changes as documents with POST /_bulk over the HTTP JSON API
• Serves Elasticsearch and OpenSearch with one agent and one client
• Creates missing indexes on first write, with an explicit mapping generated from the entity columns
• Uses the Gluesync document id as _id, so inserts, updates and deletes of a row all reach the same document
• Authenticates with BASIC (default), API_KEY or BEARER, with TLS that always validates the server certificate

The 2.3.0.2 release notes list it as a new target agent, released 19 October 2026.

Why one agent for Elasticsearch and OpenSearch

Both products share the HTTP JSON API the agent speaks: GET /, index creation, POST /_bulk, _delete_by_query and mapping reads. So Gluesync runs one write path for both.

The flavor is read from GET /. When version.distribution is opensearch, the agent treats the cluster as OpenSearch. You can pin it with Server flavor: AUTO (default), ELASTICSEARCH or OPENSEARCH. The flavor only names the product next to the version in the UI and the logs; the client stays the same.

The integration test suite runs against live clusters of both products, Elasticsearch 8.15.5 and OpenSearch 2.17.1, fed by a PostgreSQL WAL CDC source, covering snapshot, insert, update and delete.

How changes become documents

_id is the Gluesync document id of the row: <collection>::<key>, or <scope>::<collection>::<key> when the entity has a scope.

• Insert: index by default, which writes the document with that _id. Pick CREATE and a version conflict is reported as a duplicate key, so the Core Hub insert-conflict strategies apply.
• Update: a bulk update with doc and doc_as_upsert. A JSON null clears that field.
• Delete: a bulk delete by _id. A missing document counts as done.
• Truncate and snapshot deletion: _delete_by_query with match_all and conflicts=proceed by default, or SKIP to leave the documents. The index and its mapping always stay.

The confirmed transaction id advances only after the cluster acknowledges the bulk request. A _bulk request can accept some items and reject others, so after an interruption the agent replays from the last confirmed transaction id and writes the same document ids again. Same row, same document.

Index names and mappings

The index name comes from, in order:

1. The entity Index name override, when set
2. An Index name template with {prefix}, {scope} and {collection}
3. Otherwise <prefix>-<scope>-<collection>

Names are lowercased, and characters the cluster refuses become underscores.

Create index if missing is on by default, with 1 shard and 1 replica. Explicit mapping is on by default and generates one property per column. A text field also gets a keyword sub-field (ignore_above 256), so it can still be sorted and aggregated. Turn explicit mapping off and the cluster infers every field. An existing index is left alone, mapping included, and discovery reads its mapping back.

Per entity you can also set an ingest pipeline sent with the bulk request, and a bulk batch size.

Write settings

• Bulk batch: 1000 documents by default
• Bulk bytes: 5,242,880 by default. A request is split before it exceeds the limit
• Refresh: NONE by default. The confirmed transaction id follows the bulk acknowledgement, whatever the refresh policy
• Connect timeout 30 seconds and request timeout 120 seconds by default. A timeout is retried as a transient failure
• Decimals as strings: off by default, so decimals go out as JSON numbers in plain notation with every digit. On, they go out as strings that a keyword field keeps verbatim
• Document id field: copy the Gluesync id into a field of the document as well as _id
• Path prefix: for a cluster behind a reverse proxy, for example /es

Data types

Gluesync type	Default mapping
STRING	keyword (alternative text)
BOOLEAN	boolean
SHORT / INT / LONG	short / integer / long
FLOAT / DOUBLE	float / double
BIG_DECIMAL	double (alternatives scaled_float, keyword)
LOCAL_DATE	date
LOCAL_DATE_TIME / OFFSET_DATE_TIME	date (alternatives date_nanos, keyword)
LOCAL_TIME / OFFSET_TIME	keyword, as ISO time strings
BYTE_ARRAY	binary
ARRAY	dynamic (or keyword, object, nested)
MAP	object (or nested, flattened, dynamic)

Temporal values are ISO-8601 strings, which the default date format of both products parses. flattened is an Elasticsearch field type; OpenSearch calls its equivalent flat_object. The full table is in the target setup docs.

Connection setup

Prerequisites: permission to bulk-index, update and delete documents, to run _delete_by_query when snapshot deletion stays at its default, and to create an index when create-if-missing is on.

In the Control Plane, add Elasticsearch as the target of a pipeline:

• Hostname / IP address of one cluster node, Port 9200 by default
• Authentication: BASIC (username and password, default), API_KEY (id:api_key or an encoded key in the password field) or BEARER (the token in the password field)
• TLS: off by default. When on, the server certificate is always validated. Give the agent the cluster CA (http_ca.crt on Elasticsearch, root-ca.pem on OpenSearch) as a PKCS12, JKS or PEM file, or upload it through Core Hub
• Server flavor: AUTO, ELASTICSEARCH or OPENSEARCH

Turn TLS on for any cluster outside a private network. In the entity, schema is the optional index namespace and table is the index.

Where it fits

Search and analytics on live operational data. PostgreSQL, Oracle, SQL Server, MySQL, MongoDB and the rest of the Gluesync catalog land their snapshot and CDC changes in Elasticsearch or OpenSearch indexes as documents, one per row, kept current key by key. The full list of sources and targets is on the Gluesync page.

Next steps

1. Read the Elasticsearch target setup.
2. Deploy Gluesync 2.3.0.2 and add Elasticsearch as the target of a pipeline in Core Hub.
3. Point it at your Elasticsearch or OpenSearch cluster and watch the indexes follow the source.

Want to see it on your cluster? Request a free trial or book a demo. Questions on a concrete build: I am around.