Canonical: https://molo17.com/solutions/replicate-to-gridgain/ Markdown mirror: https://molo17.com/solutions/replicate-to-gridgain/index.md Title: Replicate to GridGain in real time with Gluesync | MOLO17 Description: Real-time replication to GridGain from Oracle, SQL Server, PostgreSQL, MySQL, and MongoDB, in optimized thin client batches, snapshot then CDC. Start a trial. [Solutions](/solutions/) Replicate to GridGain SOLUTION · REPLICATE TO GRIDGAIN # Real-time data replication to GridGain from your operational databases **Committed changes from your systems of record reach GridGain continuously through its thin client, so the in-memory data your applications read stays current without a nightly reload.** Gluesync by MOLO17 captures changes from Oracle, SQL Server, PostgreSQL, MySQL, MongoDB, and other [heterogeneous sources](/integrations/?target=GridGain#integration-finder) with a dedicated agent per database. The GridGain target agent connects through the thin client driver bundled with Gluesync, seeds the target tables with snapshot batches, and then applies the change stream in real time. Core Hub, the Gluesync control plane, runs every pipeline from one web UI and REST API. [Start a Gluesync trial](/get-gluesync/) [Talk to us](/contacts/) WHO THIS IS FOR ## Teams that need the in-memory grid to reflect the system of record - Architects of in-memory compute platforms who need GridGain to follow the operational database without a reload window or a batch window - Data platform leads feeding GridGain from Oracle, SQL Server, PostgreSQL, MySQL, or MongoDB who want one Core Hub for every source, backed by [best-in-class enterprise support, rated 4.9/5 by customers](/support/#customer-ratings) - Application engineers who query GridGain tables and need the keys and column names to match the source, with Core Hub normalizing column names to upper case on save - Engineers replacing custom loaders, or a Kafka chain that exists only to feed GridGain, who want [every source](/integrations/?target=GridGain#integration-finder) delivered by one product with snapshots and monitoring built in THE PROBLEM ## GridGain data that loads by script is already behind Most in-memory grids are loaded by hand-written jobs: a loader queries the source, pushes rows through a client, and runs again on a schedule. Between runs the grid serves the state of the business as it was at the last load, every reload adds read load to the production database, and each source needs its own loader and failure handling. Teams looking for real-time GridGain ingestion or a GridGain CDC pipeline usually weigh Kafka Connect or Debezium with a Kafka sink, custom loader code, or scheduled ELT jobs that were built for warehouses. Gluesync addresses that with **per-agent CDC into GridGain**. A source agent reads each database's native change log, journal, or change stream; Core Hub routes the changes; the GridGain target agent applies them in real time. Oracle to GridGain, PostgreSQL to GridGain, or MongoDB to GridGain all share the same snapshot, monitoring, and pipeline model. HOW IT WORKS ## How Gluesync writes to GridGain ### The write path: a thin client connection The GridGain agent connects through the GridGain thin client driver, bundled with Gluesync, so the agent reaches the cluster as a lightweight client rather than joining it as a full node. It is a target agent: it receives changes from any Gluesync source agent and applies snapshot batches and live changes to the target tables through the thin client API. - **Security:** TLS secures the connection with the SSL certificate your cluster uses. ### Optimized batches, never row by row Gluesync never writes one row at a time to GridGain. Incoming changes are grouped into chunks and sent as batched writes through the thin client, so each round-trip to the grid carries many rows and the load on the cluster stays predictable. The chunk size is configurable on the target agent, so you can push throughput while seeding the grid and keep batches small once applications query it at full load. ### Snapshot first, then continuous CDC - **Seed:** snapshot batches load each entity's rows into GridGain before live changes start. - **Stream:** after the snapshot, inserts, updates, and deletes from the source agent's change mechanism are applied in real time. - **Resume:** an interrupted snapshot resumes from its last saved state when you start the entity again. - **Scheduled reloads:** the Chronos Scheduler runs snapshots on a cadence for tables you prefer to reload. See [schedules and events](/schedules-and-events/). ### Duplicate keys and identifier case - **On duplicate key:** set per entity to Upsert, Skip, or Fail. Upsert overwrites the stored row, Skip keeps it and raises a Notifications Hub warning that names the skipped keys, and Fail stops the entity so someone can look at it. - **Identifier case:** GridGain folds unquoted identifiers to upper case, so Core Hub normalizes target column names and filter clauses to `UPPERCASE` when an entity is saved, and the editor enforces that case as you type. ### What your GridGain administrator sets up The agent needs a GridGain user with read and write permission on the target tables and the cluster. Every connection field is listed in the [GridGain target setup guide ↗](https://docs.molo17.com/gluesync/latest/agents/gridgain-target.html). 1. Make sure the cluster is running and reachable from the agent host on the thin client port, 10800 by default. 2. Create a GridGain user with read and write permission on the target tables and the cluster. 3. When the cluster secures connections with TLS, provide the SSL certificate it uses. 4. In Core Hub, enter the hostname or IP address, the port, the cluster name, and the username and password. ### Architecture around Core Hub Lightweight agents sit close to each source, and Core Hub orchestrates them through its web UI and REST APIs. A pipeline groups a source agent, the GridGain agent, and the entities they replicate, so several sources can feed one GridGain cluster from one control plane. Core Hub and agents deploy with Docker, Docker Compose, or Kubernetes, on-premises or in any cloud. See [CDC streaming](/solutions/cdc-streaming/) for the wider design. [Explore the general CDC streaming architecture →](/solutions/cdc-streaming/) WRITE OPTIONS ## The GridGain target agent: one agent, one thin client path GridGain has one Gluesync target agent. Every entity from every source is written through the same thin client connection, so the choice you make is the pipeline and its sources. | Agent | Write technique | Versions | Best for | | --- | --- | --- | --- | | [GridGain agent ↗](https://docs.molo17.com/gluesync/latest/agents/gridgain-intro.html) | GridGain thin client driver; optimized batched writes for snapshot and CDC, applied to tables through the thin client API | GridGain 8.8 and later, over the thin client port (10800 by default) | Teams that serve operational data from GridGain and need it to follow the source as it changes, with Core Hub monitoring every feed. Pick it when several sources should land in one in-memory grid under one control plane. | SOURCES AND TOPOLOGIES ## Feed GridGain from the systems of record you already run Any Gluesync source agent can feed GridGain, each with its own native capture technique. Open the [integrations finder with GridGain pre-selected](/integrations/?target=GridGain#integration-finder) to see every source you can pair with it. One Core Hub runs several sources into the same GridGain cluster side by side, each pipeline with its own snapshot, monitoring, and duplicate-key policy. Gluesync keeps pace with your change volume at any scale. [MOLO17 Professional Services](/solutions/professional-services/) can plan the tables and the topology with your team. - Oracle to GridGain from the redo logs through LogMiner or XStream: see [Oracle CDC](/solutions/oracle-cdc/) - SQL Server to GridGain through Change Data Capture or Change Tracking: see [SQL Server CDC](/solutions/sql-server-cdc/) - PostgreSQL and MySQL to GridGain from the WAL and the binlog: see [PostgreSQL CDC](/solutions/postgresql-cdc/) and [MySQL CDC](/solutions/mysql-cdc/) - MongoDB to GridGain from Change Streams: see [MongoDB CDC](/solutions/mongodb-cdc/) FAIR, HIGH-LEVEL COMPARISON ## Where Gluesync fits among in-memory grid loading approaches | Approach | What buyers usually get | Where Gluesync fits | | --- | --- | --- | | Kafka Connect or Debezium with a Kafka sink | Open-source capture into Kafka topics, loaded by a sink. You run Kafka, Connect, offsets, and schemas, and you decide how events merge into current-state tables | Changes applied to current-state GridGain tables by key, with no Kafka cluster in the path; read the [Debezium alternative](/solutions/debezium-alternative/) comparison | | Custom loader code | Full control of the client code, retries, and backfills. Your team builds and maintains a loader for every source | Snapshot seeding, continuous CDC, and Core Hub monitoring without loader code to maintain; see [log-based CDC explained](/blog/real-time-replication-log-based-cdc/) | | Scheduled ELT jobs | Extracts on a schedule, run by a managed service or by your team. Freshness equals the schedule, and each run reads the source again | Changes read from the source log as they happen, so GridGain tracks the source between loads; read [batch ETL vs real-time replication](/blog/batch-etl-vs-real-time-data-replication/) | | Managed ELT services | Connector catalogs with scheduled syncs, mostly aimed at analytical destinations. In-memory grid coverage differs by product | Agents you deploy beside each source, with native capture and Core Hub monitoring for the whole pipeline | | Commercial replication platforms such as Qlik Replicate | Mature log-based replication across many targets. Coverage of each target differs by product and version | Agents that run on-premises or in any cloud with Docker, Docker Compose, or Kubernetes, writing to GridGain from one Core Hub; see [migrating to Gluesync](/migrate-to-gluesync/) | RELATED CONTENT ## GridGain replication reading and documentation - [GridGain target connector: high-performance data integration with Gluesync](/blog/gridgain-target-connector-high-performance-data-integration-with-gluesync/) - [Batch ETL vs real-time data replication: how to choose](/blog/batch-etl-vs-real-time-data-replication/) - [Log-based CDC explained](/blog/real-time-replication-log-based-cdc/) - [Database offload: read-optimized targets](/solutions/database-offload/) - [GridGain agent overview ↗](https://docs.molo17.com/gluesync/latest/agents/gridgain-intro.html) - [GridGain target setup guide ↗](https://docs.molo17.com/gluesync/latest/agents/gridgain-target.html) - [On duplicate key: Upsert, Skip, and Fail ↗](https://docs.molo17.com/gluesync/latest/core-hub/insert-conflict-strategy.html) - [Write strategies: optimized batches ↗](https://docs.molo17.com/gluesync/latest/core-hub/write-strategies.html) FAQ ## GridGain replication questions What does replicating to GridGain with Gluesync involve? A source agent captures committed changes from your database through its native change mechanism, Core Hub routes them, and the GridGain target agent applies them to GridGain tables continuously after a snapshot seeds each table. How does Gluesync load data into GridGain? The GridGain agent connects through the thin client driver and groups snapshot rows and live changes into optimized batches applied through the thin client API. Batch size is configurable. Which sources can replicate to GridGain? Any Gluesync source agent can feed GridGain, including Oracle, SQL Server, PostgreSQL, MySQL, MongoDB, and IBM Db2. The integrations finder shows every pairing for GridGain. How are duplicate keys handled in GridGain? Each entity sets On duplicate key to Upsert, Skip, or Fail. Upsert overwrites the stored row, Skip keeps the row already in GridGain and raises a warning that lists the skipped keys, and Fail stops the entity until the collision is resolved. What does the GridGain administrator need to set up? A GridGain user with read and write permission on the target tables and the cluster, a cluster reachable on the thin client port, 10800 by default, and the SSL certificate when connections are secured with TLS. Core Hub takes these credentials in its web UI or REST API. How are column names written in GridGain? Core Hub normalizes target column names and filter clauses to upper case when an entity is saved, and the editor enforces that case as you type, so queries against GridGain tables use the same names. Which GridGain versions does the agent support? GridGain 8.8 and later, reached through the thin client port, which defaults to 10800. REPLICATE TO A TARGET ## Other targets Gluesync delivers to - [Replicate to Aerospike](/solutions/replicate-to-aerospike/) - [Replicate to DynamoDB](/solutions/replicate-to-dynamodb/) - [Replicate to Redshift](/solutions/replicate-to-redshift/) - [Replicate to Amazon S3 & S3-compatible](/solutions/replicate-to-amazon-s3/) - [Replicate to Cassandra](/solutions/replicate-to-cassandra/) - [Replicate to Kafka](/solutions/replicate-to-kafka/) - [Replicate to Cosmos DB](/solutions/replicate-to-cosmos-db/) - [Replicate to Azure Data Lake](/solutions/replicate-to-azure-data-lake/) - [Replicate to ClickHouse](/solutions/replicate-to-clickhouse/) - [Replicate to CockroachDB](/solutions/replicate-to-cockroachdb/) - [Replicate to Couchbase](/solutions/replicate-to-couchbase/) - [Replicate to file stores](/solutions/replicate-to-file-stores/) - [Replicate to BigQuery](/solutions/replicate-to-bigquery/) - [Replicate to Google Cloud Storage](/solutions/replicate-to-google-cloud-storage/) - [Replicate to Google Pub/Sub](/solutions/replicate-to-google-pubsub/) - [Replicate to Db2](/solutions/replicate-to-db2/) - [Replicate to Informix](/solutions/replicate-to-informix/) - [Replicate to MariaDB](/solutions/replicate-to-mariadb/) - [Replicate to SQL Server](/solutions/replicate-to-sql-server/) - [Replicate to MongoDB](/solutions/replicate-to-mongodb/) - [Replicate to MySQL](/solutions/replicate-to-mysql/) - [Replicate to Oracle](/solutions/replicate-to-oracle/) - [Replicate to PostgreSQL](/solutions/replicate-to-postgresql/) - [Replicate to RavenDB](/solutions/replicate-to-ravendb/) - [Replicate to Redis](/solutions/replicate-to-redis/) - [Replicate to SAP ASE](/solutions/replicate-to-sap-ase/) - [Replicate to SAP HANA](/solutions/replicate-to-sap-hana/) - [Replicate to ScyllaDB](/solutions/replicate-to-scylladb/) - [Replicate to SingleStore](/solutions/replicate-to-singlestore/) - [Replicate to Snowflake](/solutions/replicate-to-snowflake/) - [Replicate to Solace PubSub+](/solutions/replicate-to-solace/) - [Replicate to Vertica](/solutions/replicate-to-vertica/) - [Replicate to YugabyteDB](/solutions/replicate-to-yugabytedb/) ## Evaluate Gluesync with your GridGain cluster Start a trial on your infrastructure, or talk to MOLO17 about your sources, the tables your applications query, and how GridGain sits next to your operational databases. [Start a Gluesync trial](/get-gluesync/) [Talk to MOLO17](/contacts/)