Canonical: https://molo17.com/solutions/mongodb-cdc/ Markdown mirror: https://molo17.com/solutions/mongodb-cdc/index.md Title: MongoDB CDC and real-time replication with Gluesync | MOLO17 Description: MongoDB CDC with Change Streams from replica sets, sharded clusters, and Atlas. Replicate to warehouses, NoSQL, and streams in real time. Start a trial. [Solutions](/solutions/) MongoDB CDC SOLUTION · MONGODB CDC # MongoDB change data capture and real-time data replication **Change data capture from the MongoDB Change Streams API, delivered continuously, with no Kafka Connect cluster for your team to run.** Gluesync by MOLO17 captures changes from MongoDB through a dedicated source agent that reads the native Change Streams API on replica sets and sharded clusters, including MongoDB Atlas. The same agent family can also write to MongoDB as a target, and changes are delivered continuously to the databases, warehouses, lakes, and event streams your teams already run. Manage every pipeline from the Core Hub web UI, the Gluesync control plane. [Start a Gluesync trial](/get-gluesync/) [Talk to us](/contacts/) WHO THIS IS FOR ## MongoDB teams moving operational data into the rest of the estate - DBAs who need to see exactly which MongoDB privileges the runtime user holds, why `collMod` is required, and which topology and TLS settings the agent uses, before they approve a source connection - Data platform leads who feed warehouses, lakes, or event streams from MongoDB and want one control plane for sources and targets, not a set of connectors to assemble and maintain - Architects replacing a MongoDB change-stream script, a scheduled export, or a batch ETL job that reads whole collections on a timer - Engineers who prototyped the Debezium MongoDB connector or the MongoDB Kafka connectors and now own Kafka Connect, offsets, and on-call for a pipeline the business treats as infrastructure THE PROBLEM ## MongoDB data that only moves in batches goes stale, and the usual workarounds add moving parts Scheduled exports and collection scans leave analytics and downstream applications working from a copy that is already behind, and they add read load to clusters that serve live traffic. Buyers searching for MongoDB CDC or MongoDB replication usually compare a few options: the Debezium MongoDB connector or MongoDB Kafka connectors, which mean running Kafka and Kafka Connect; Atlas-native triggers, which stay inside Atlas; and ETL or ELT services, whose MongoDB modes vary by product. Each has a place, and each leaves a different set of operations with your team. Gluesync addresses that pattern with **per-agent CDC**. The MongoDB agent reads the native Change Streams API, the source agent sends changes through Core Hub, and target agents write to the destination you choose. The same pipeline model, snapshots, and Core Hub controls apply whichever target you pick. HOW IT WORKS ## How Gluesync does MongoDB CDC ### Change Streams, the native MongoDB change feed The agent uses the MongoDB Kotlin SDK and the built-in Change Streams API. The Change Streams API streams oplog-derived events, which the agent consumes continuously for low-latency replication. - **Snapshot first:** entities can run full snapshots against MongoDB collections for initial loads before CDC resumes. - **No bulk-load staging:** bulk-load staging tables apply to relational targets only; MongoDB applies snapshot and CDC flows directly. - **State in Core Hub:** state tracking lives in Core Hub metadata tables. [MongoDB Change Streams setup ↗](https://docs.molo17.com/gluesync/latest/agents/mongodb-change-data-capture-change-streams.html) ### What your DBA will be asked to grant The agent needs a user that can read the source and change collection properties, because it enables the change capture feature on collections that have it disabled. A custom role covers this. - The built-in `dbAdmin` role also allows `collMod` and is the other option. - That role is the minimum requirement and can be fine-tuned to your databases. - If the user authenticates against a database other than the target, set the Auth Database option. 1. Create a role named `GluesyncUserRole` with the `collMod` action on an empty resource, plus `readWriteAnyDatabase` on the `admin` database, using `db.runCommand({ createRole: ... })`. 2. Assign that role to the Gluesync user with `db.updateUser(...)` on the `admin` database. ### Topology, Atlas, and connectivity - **Replica sets and sharded clusters:** the source role reads Change Streams from replica sets or sharded clusters. Name the replica set in the Replica Set option, or use Direct Connection to reach a single server. - **Node discovery:** the agent uses automatic node discovery and supports load balancing across cluster nodes. - **MongoDB Atlas:** enable Use DNS Seed List and enter the cluster hostname, for example `cluster0.instancereferencename.mongodb.net`. With a `.pem` certificate, a username and password pair is not needed. - **TLS:** upload your certificates in the connection settings. The target TLS switch is off by default. ### Architecture around Core Hub Agents sit close to each cluster or database, and Core Hub is the control plane above them: web UI, REST APIs, and routing from the MongoDB source agent to its targets. A pipeline groups the source agent, its target agents, and the entities (collections) they replicate. Core Hub and agents deploy with Docker, Docker Compose, or Kubernetes. [Explore the general CDC streaming architecture →](/solutions/cdc-streaming/) CAPTURE OPTIONS ## The MongoDB source agent: one agent, one capture method MongoDB has one source agent, and it captures through Change Streams, so the decision is topology and privileges rather than which agent to pick. | Agent | Capture technique | Versions | Best for | | --- | --- | --- | --- | | [MongoDB agent ↗](https://docs.molo17.com/gluesync/latest/agents/mongodb-intro.html) | Change Streams API (oplog-derived change events), with snapshots for initial loads | MongoDB 3.6 and later; replica sets and sharded clusters; MongoDB Atlas through the DNS seed list and a .pem certificate | Any MongoDB estate where replica sets, sharded clusters, or Atlas are in place and the DBA can grant `collMod` to the runtime user. | TARGETS AND TOPOLOGIES ## Deliver MongoDB changes to the systems you already run The [integrations directory](/integrations/?source=MongoDB#integration-finder) lists each pairing. MongoDB can be a source, a target, or both, as each agent's documented source and target roles define. Gluesync keeps pace with your change volume at any scale. [MOLO17 Professional Services](/solutions/professional-services/) can size the deployment with your team. Target agents write in optimized batches, never row by row, and switch to native bulk load for both snapshots and CDC on targets such as [Snowflake](/solutions/replicate-to-snowflake/), [Google BigQuery](/solutions/replicate-to-bigquery/), [Amazon Redshift](/solutions/replicate-to-redshift/), [Microsoft SQL Server](/solutions/replicate-to-sql-server/), and [PostgreSQL](/solutions/replicate-to-postgresql/). - Offload read traffic from MongoDB to a relational or NoSQL store: see [database offload](/solutions/database-offload/) - Feed MongoDB changes to a cloud warehouse, data lake, or object store: see [warehouse sync](/solutions/warehouse-sync/) - Publish MongoDB changes to Apache Kafka or another event stream for downstream services - Run two-way sync with MongoDB: set Enable Recursion Protection on the agent for two-way pipelines FAIR, HIGH-LEVEL COMPARISON ## Where Gluesync fits among MongoDB CDC approaches | Approach | What buyers usually get | Where Gluesync fits | | --- | --- | --- | | Debezium MongoDB connector or MongoDB Kafka connectors | Change-stream capture on Kafka Connect; you run Kafka, Connect workers, offsets, sink connectors, and upgrades | Productized MongoDB agent with Core Hub operations and [best-in-class MOLO17 enterprise support (rated 4.9/5 by customers)](/support/#customer-ratings), and no Connect cluster required; read the [Debezium alternative](/solutions/debezium-alternative/) comparison | | Atlas-native triggers and functions | Built into Atlas and tied to the Atlas platform; logic runs inside the managed service | Works with self-managed and Atlas clusters, and delivers to targets outside Atlas under one Core Hub | | Commercial CDC suites | Broad source coverage and mature operations; MongoDB support and packaging vary by product generation | Agent-based CDC with MongoDB as source and target and a single control plane; see [migrating to Gluesync](/migrate-to-gluesync/) | | ETL and ELT services | Managed loads into warehouses; incremental or log-based MongoDB modes differ by connector | Continuous change delivery to warehouses, lakes, databases, and streams from the same MongoDB agent; see [warehouse sync](/solutions/warehouse-sync/) | | DIY change-stream scripts | Full control; your team builds snapshots, restarts, monitoring, and every target writer | Snapshots, Core Hub monitoring, and per-agent targets without writing or maintaining the pipeline code; see [CDC streaming](/solutions/cdc-streaming/) | RELATED CONTENT ## MongoDB CDC research and implementation detail - [Real-time replication from MongoDB to Couchbase: tutorial](/blog/real-time-data-replication-from-mongodb-to-couchbase-tutorial/) - [MongoDB to Couchbase replication with Gluesync](/blog/mongodb-to-couchbase-replication/) - [Batch ETL vs real-time data replication: how to choose](/blog/batch-etl-vs-real-time-data-replication/) - [G-Able: real-time telco data integration from MongoDB and SQL sources](/blog/aerospike-real-time-telco-data-integration-g-able-gluesync/) - [Debezium alternative: managed CDC versus Kafka + Debezium](/solutions/debezium-alternative/) - [CDC streaming without source overhead](/solutions/cdc-streaming/) - [MongoDB agent overview ↗](https://docs.molo17.com/gluesync/latest/agents/mongodb-intro.html) - [MongoDB CDC with Gluesync: Change Streams setup ↗](https://docs.molo17.com/gluesync/latest/agents/mongodb-change-data-capture-change-streams.html) FAQ ## MongoDB CDC questions What is MongoDB CDC with Gluesync? Change data capture from MongoDB through the built-in Change Streams API. A MongoDB source agent reads the change events and Core Hub delivers them to the targets you configure, through Gluesync agents. Which MongoDB versions does the agent support? MongoDB 3.6 and later is supported, with replica sets and sharded clusters as the sources for Change Streams. Does the source need a replica set? The source role reads Change Streams from replica sets or sharded clusters, so the source must be one of those topologies. Use the Replica Set option to name the set, or Direct Connection to reach a single server. What permissions does the DBA grant? The runtime user needs read and write access to the source database and the collMod privilege, either through the dbAdmin role or a custom role. The custom role example grants collMod with readWriteAnyDatabase on admin and can be fine-tuned. Why does the agent need collMod? The agent checks and enables the change capture feature on collections where it is disabled, using the collMod command. Does Gluesync work with MongoDB Atlas? Yes. MongoDB Atlas is supported: enable Use DNS Seed List, enter the cluster hostname, and connect with a .pem certificate. Can MongoDB be a target as well as a source? Yes. The MongoDB agent has a target role with its own settings, including TLS, Auth Database, Replica Set, and Direct Connection. Where should we start? Start a Gluesync trial with a non-production MongoDB replica set or Atlas cluster, then read the MongoDB agent overview and the Change Streams setup page. Talk to MOLO17 if you need help with the DBA prerequisites or the target design. CDC BY SOURCE DATABASE ## Other sources Gluesync captures from - [Cosmos DB CDC](/solutions/cosmos-db-cdc/) - [Couchbase CDC](/solutions/couchbase-cdc/) - [Db2 LUW CDC](/solutions/db2-luw-cdc/) - [DynamoDB CDC](/solutions/dynamodb-cdc/) - [IBM i CDC](/solutions/ibm-i-cdc/) - [Informix CDC](/solutions/informix-cdc/) - [MariaDB CDC](/solutions/mariadb-cdc/) - [MySQL CDC](/solutions/mysql-cdc/) - [Oracle CDC](/solutions/oracle-cdc/) - [PostgreSQL CDC](/solutions/postgresql-cdc/) - [SAP ASE (Sybase) CDC](/solutions/sap-ase-cdc/) - [SAP HANA CDC](/solutions/sap-hana-cdc/) - [ScyllaDB CDC](/solutions/scylladb-cdc/) - [SQL Server CDC](/solutions/sql-server-cdc/) - [YugabyteDB CDC](/solutions/yugabytedb-cdc/) ## Evaluate Gluesync with your MongoDB change streams Start a trial on your infrastructure, or talk to MOLO17 about the MongoDB topology, the privileges your DBA can approve, and the targets you need. [Start a Gluesync trial](/get-gluesync/) [Talk to MOLO17](/contacts/)