← All articles

Article

Gluesync now speaks Apache CouchDB, as a source and as a target

Gluesync now speaks Apache CouchDB, as a source and as a target featured image

Gluesync 2.3.1.4 adds an Apache CouchDB agent: snapshot and CDC from the _changes feed, parallel bulk writes, CouchDB 3.3 to 3.5, single node or cluster.

Apache CouchDB holds a lot of JSON data that other systems want to use: reporting databases, search engines, analytics platforms, other CouchDB servers. Moving it reliably usually means custom scripts that poll, page and retry. With Gluesync 2.3.1.4, released on 2 November 2026, there is a better option: a new Apache CouchDB agent that works both as a source and as a target, on a single node or on a cluster.

Why a native CouchDB agent

The agent talks to CouchDB over its HTTP API with JSON bodies, with no vendor SDK. As a source it creates nothing in your databases, installs nothing on the server and needs no change to your applications: it reads the databases and follows their _changes feed. As a target it writes JSON documents with parallel _bulk_docs requests and handles revisions, conflicts and deletions for you.

One agent, two roles, the same pipeline model you already use in Gluesync Core Hub.

When to use it

  • CouchDB as a source: stream documents from CouchDB into any target Gluesync supports, such as PostgreSQL.
  • CouchDB as a target: land data from your relational or NoSQL sources into CouchDB as JSON documents, one entity per database.
  • CouchDB to CouchDB: a pipeline can have a CouchDB source and a CouchDB target. Use one direction: the agent has no recursion protection, so two-way sync between CouchDB databases would send each change back and forth.

Supported versions are CouchDB 3.3, 3.4 and 3.5. The agent checks the server when it connects and refuses anything that is not CouchDB 3.x, so CouchDB 2.x is not supported. Couchbase is a different product with its own Gluesync agents.

What it does

Snapshot and CDC

The source agent copies a database with a snapshot and then follows its changes feed. The snapshot does not page _all_docs, which is bound to one stream on the server and does not scale. Instead it takes document ids from the changes feed and reads the documents with many parallel _bulk_get requests.

Because the reads run in parallel, the snapshot scales with the number of concurrent requests instead of being tied to a single server stream.

An interrupted snapshot resumes from the feed position it had safely reached. CDC then polls the changes feed of every entity in long poll mode, from the position saved by Core Hub, and changes made while the snapshot ran are replayed from where the snapshot started. Delivery is at least once, and the target applies a change twice without harm. Inserts, updates and deletes are captured; a deletion reaches the target carrying only the document _id, so the entity key must be _id.

Data type mapping

CouchDB documents are JSON, and their fields are not declared anywhere. On the source side the agent infers fields from a sample of documents (Schema discovery sample size, default 1000, half from the start of _all_docs and half from its end). Integers become integer, other numbers become number, and single-kind fields become string, boolean, object or array. A field with mixed kinds, or only nulls, becomes json. Raise the sample size for databases with documents of different shapes.

On the target side every Gluesync type has a default JSON shape and documented alternatives. Two details worth knowing:

  • CouchDB stores numbers as IEEE doubles, so a BIG_DECIMAL is written as a string by default to keep every digit. You can choose number if about 15 significant digits are enough.
  • Date and date-time values (LOCAL_DATE, LOCAL_DATE_TIME, OFFSET_DATE_TIME) are written as ISO text by default, with epoch milliseconds as an option for those types. Time-only values (LOCAL_TIME, OFFSET_TIME) are written as ISO text, with no alternative.

Bulk-write throughput

The target writes each batch with parallel _bulk_docs requests (default 1000 documents per request, an 8 MiB size limit, 8 requests in flight, gzip compression on). It looks up current revisions, retries documents that changed under the write, writes deletions as tombstones and merges partial changes into the stored document.

The main lever is the number of shards (q) of the target database. When the agent creates a missing database it uses the Shards setting, 8 by default. The second lever is Write concurrency, the number of requests in flight, and gzip compression of request bodies helps too. To write faster, create target databases with more shards and raise Write concurrency together with them.

Truncate strategies

CouchDB has no truncate, so when Core Hub truncates an entity, for example with "TRUNCATE before snapshot", the agent follows its Truncate strategy:

  • DELETE_DOCUMENTS (default): pages _all_docs and deletes every document in bulk, keeping design documents.
  • RECREATE_DATABASE: saves the _security object, deletes and recreates the database with the same q and n, then restores _security. Design documents are lost, and it needs a server administrator.
  • SKIP: logs the request and leaves data as it is.

Security

The agent uses HTTP Basic authentication, or anonymous access if you disable authentication. With TLS on, the server certificate is always validated; there is no option to trust it blindly. For a private certificate authority, give the agent the CA certificate. Turn TLS on for any CouchDB that is not on a private network.

Good to know

Attachments and purges are not replicated: attachments are neither read nor written. The changes feed carries the current document only, so there is no before image. Top-level field names starting with an underscore (other than _id) are reserved by CouchDB, so rename technical fields such as _TRANSACTION_ID to something like TRANSACTION_ID.

How to get started

  1. Check the prerequisites. Core Hub needs network access to the CouchDB HTTP port (5984 by default) or your TLS port. A source user must be a member of every database to replicate, with read access. Listing databases via _all_dbs needs a server administrator in CouchDB 3.x; otherwise name the databases in the connection. A target user needs read and write access; creating missing databases, or using RECREATE_DATABASE, needs a server administrator.
  2. Add the agent in Core Hub. Pick the CouchDB agent (tag couchdb), enter the node address, port and credentials, enable TLS if needed, and list any additional cluster nodes so the agent can fail over in order when a node is down.
  3. Tune the target. Set Shards, Write concurrency and Truncate strategy for your workload.
  4. Run the snapshot, then let CDC follow.

Full details, including REST API examples:

Try it

The CouchDB agent ships with Gluesync 2.3.1.4, available now. Start a free trial at https://molo17.com and connect your first CouchDB database today, or read the docs to plan your pipeline.