Real-time data replication to WarehousePG from your operational databases

• EDB customers running WarehousePG who need current-state tables fed from operational systems all day, keyed like the source, with duplicate keys resolved by a per-entity setting instead of a hand-written merge job
• Teams that moved from Greenplum to WarehousePG after Greenplum went closed source and want the same parallel loading they already know, without rebuilding their pipelines around a new tool
• WarehousePG DBAs who want loads that use the cluster the way it is designed: every primary segment pulling its share of the data in parallel, tables distributed by key, and append-optimized column storage with compression
• Platform leads who want one Core Hub for Oracle, SQL Server, PostgreSQL, MySQL, IBM i, and MongoDB instead of one loader per source, backed by best-in-class enterprise support, rated 4.9/5 by customers

The Greenplum agent, pointed at WarehousePG

WarehousePG keeps Greenplum's wire protocol, catalog, segments, external tables, and gpfdist, so the Gluesync Greenplum agent writes to it exactly as it writes to Greenplum: you create a pipeline, enter the coordinator's connection details, and the same two write paths, the same table creation, and the same type mapping apply. It connects through the bundled PostgreSQL JDBC driver on WarehousePG and Greenplum 6.x and 7.x alike, and applies the change stream from any Gluesync source agent after a snapshot seeds each table. Greenplum agent overview, covering WarehousePG ↗

• Same type mapping: values are written with the PostgreSQL target's type mapping, including enum types, arrays, and hstore.
• Clean reload on demand: the Core Hub option to truncate before a snapshot is honored on the target tables.
• Database connection only: the agent needs a connection to the coordinator and calls no management API, so a self-managed cluster and a cluster run by EDB tooling look the same to it.

Two write paths: optimized batches and parallel bulk loads

The write path is chosen per entity, so a small reference table and a billion-row fact table can each use the one that suits them.

• Batch path: optimized JDBC batches of INSERT and DELETE statements, with each upsert applied as an update of the existing keys followed by an insert of the rest.
• Bulk path: turned on per entity for the snapshot and for CDC. Rows are encoded as tab-delimited text in a spool file on Core Hub and loaded into a staging table in the GLUESYNC schema, then merged into the target table.

Every segment loads its share with gpfdist

If your team already runs gpfdist for file loads, this is the same mechanism, started and fed by Gluesync.

• gpfdist: the agent serves the file from an embedded gpfdist server and loads it through a temporary readable external table. Every primary segment pulls its share of the data in parallel, so the load grows with the number of segments.
• COPY: the file streams through COPY … FROM STDIN on the coordinator.
• Automatic choice: the default uses gpfdist when the segments can reach Core Hub and the load is large enough to benefit, and COPY otherwise. At setup the agent checks that every primary segment can reach the server.

Merged in one transaction, loaded straight when the key is enforced

• One transaction per flush: the staging table is merged into the target table in the same transaction as the load. If the load fails, the transaction rolls back and the batch is retried without writing any row twice.
• Co-located merges: the staging table is distributed by the key, so the merge joins rows that sit on the same segment as the target table's rows.
• Snapshot straight into the table: a snapshot of a table with an enforced unique key, such as a primary key, is loaded directly into the table with no staging step, so every row is written once.

Tables that look like WarehousePG tables

When a target table does not exist, Core Hub proposes a CREATE TABLE from the source definition that you can edit before it runs. The agent also writes any table in the database that the connection user can write to, keeping the distribution policy and storage it already has, which makes pointing Gluesync at tables you migrated from Greenplum straightforward.

• Distribution: tables are distributed by their primary key, and randomly when they have none. Edit the statement to distribute by another key.
• Storage: choose HEAP, AO_ROW, or AO_COLUMN append-optimized storage for the tables the agent creates, with ZSTD or ZLIB compression and a configurable compression level.
• System schemas stay out of the way: gp_toolkit and the other schemas the cluster keeps for its tools, plus external and foreign tables, are not listed in discovery.

Data types

The type coverage is the PostgreSQL target's, matched to the types WarehousePG has, and every column can be mapped to the alternatives Core Hub lists.

• Numbers and booleans: smallint, integer, bigint, the serial types, boolean, numeric and decimal, money, real, and double precision.
• Dates and times: date, time, timetz, timestamp, timestamptz, and interval.
• Text and documents: varchar, char, text, json, jsonb, xml, uuid, enum types, tsvector, and hstore when the extension is installed.
• Binary, network, geometric, and ranges: bytea and bit strings, inet, cidr, and macaddr, the geometric types, and the range types, plus arrays.

What your WarehousePG admin sets up

The agent does not need a superuser. It needs a role that can log in to the coordinator and create its helper schema and tables. Full statements and settings are in the Greenplum target setup guide, which covers WarehousePG ↗.

1. Create a login role for Gluesync and grant CONNECT and CREATE on the database, and USAGE and CREATE on the target schema. The CREATE on the database lets the agent create its GLUESYNC helper schema.
2. For the gpfdist load, add one role attribute, the same one you would use for any gpfdist reader: ALTER ROLE gluesync CREATEEXTTABLE (type='readable', protocol='gpfdist');
3. Allow the Core Hub host in the coordinator's pg_hba.conf and open the coordinator port, 5432 by default.
4. For the gpfdist load, open port 8580, the default, from every segment host to Core Hub, and set the advertised host to the name or address of the Core Hub machine that the segment hosts resolve.
5. In Core Hub, enter the coordinator's host, port, database, username, and password, and choose the storage type for the tables Gluesync creates.

TLS, Docker, and Kubernetes

• TLS for the connection: turn on TLS for the coordinator connection and either trust the server certificate or verify it against a trust store you provide.
• TLS for gpfdist: serve gpfdists from a PKCS12 keystore, with optional client certificates presented by the segments, signed by the same CA.
• Docker: publish the gpfdist port of the Core Hub container, 8580 by default, and set the advertised host to the Docker host address that the segments can reach.
• Kubernetes: expose the gpfdist port of the Core Hub pod with a Service or a host port and allow it in the network policies between the segment hosts and the pod.

Architecture around Core Hub

Source agents sit close to each database, and Core Hub orchestrates them through its web UI and REST APIs. A pipeline groups a source agent with the Greenplum agent and the tables it replicates, and pipeline state lives in Core Hub. Core Hub and agents deploy with Docker, Docker Compose, or Kubernetes. See CDC streaming for the wider pattern.

Explore the general CDC streaming architecture →

Any Gluesync source agent can feed WarehousePG. Open the integrations finder with WarehousePG pre-selected to see every source you can pair with it. Running Greenplum as well? The same agent writes to it too: see Replicate to Greenplum.

One Core Hub runs Oracle to WarehousePG, SQL Server to WarehousePG, and PostgreSQL to WarehousePG side by side, with the same snapshot, monitoring, and duplicate-key settings for each. Gluesync keeps pace with your change volume at any scale, and MOLO17 Professional Services can help plan distribution, storage, and the write path for each table with your team.

• Oracle to WarehousePG from the redo logs: see Oracle CDC
• SQL Server to WarehousePG through Change Data Capture or Change Tracking: see SQL Server CDC
• PostgreSQL and MySQL to WarehousePG from the WAL and the binlog: see PostgreSQL CDC and MySQL CDC
• IBM i (AS/400) and MongoDB to WarehousePG through the native journal APIs and Change Streams: see IBM i CDC and MongoDB CDC