SOLUTION · REPLICATE TO MICROSOFT FABRIC

Real-time data replication to Microsoft Fabric from your operational databases

Committed changes from your operational databases reach a Fabric mirrored database continuously, with inserts, updates, and deletes applied by key, instead of waiting for the next pipeline run.

Gluesync by MOLO17 captures changes from PostgreSQL, Oracle, SQL Server, MySQL, MongoDB, and other heterogeneous sources with a dedicated agent per database. The Fabric Open Mirroring agent is a partner application of Open Mirroring: it lands Parquet files in the landing zone of a mirrored database on OneLake, and Fabric turns them into Delta tables for the SQL analytics endpoint, semantic models, and Spark. Core Hub, the Gluesync control plane, runs snapshots and CDC for every pipeline from one web UI and REST API.

Start a Gluesync trial Talk to us

WHO THIS IS FOR

Teams that want Fabric to mirror the operational systems, not copies of them

Data engineers building on OneLake who want mirrored tables kept equal to the source, with every insert, update, and delete carried by a row marker and no Delta files to write themselves
Platform leads running Microsoft Fabric who want one Core Hub for every source database, backed by best-in-class enterprise support, rated 4.9/5 by customers
Architects in Microsoft-centric estates who need ingestion that signs in as a Microsoft Entra service principal, with access granted at the Fabric workspace and outbound HTTPS to OneLake as the only network path
Engineers replacing copy pipelines, scheduled extracts, or custom Parquet writers that exist only to fill Fabric, who want every source delivered by one product with snapshots and monitoring built in

THE PROBLEM

Fabric tables fed by scheduled loads are always behind the source

Most Fabric ingestion starts as a pipeline: a job queries each database on a timer, writes the result, and merges it into a table. Reports then trail the last run, each run adds read load to production, and every new source adds a pipeline, a schedule, and the credentials to reach it. Teams looking for real-time Fabric ingestion or a CDC feed into a mirrored database usually weigh Data Factory pipelines and copy jobs, managed ELT services such as Fivetran or Airbyte, a Debezium and Kafka Connect chain with a loader of their own, or scripts that write Parquet files.

Gluesync addresses that with per-agent CDC into Fabric Open Mirroring. A source agent reads each database's native change log or change stream, Core Hub routes the changes, and the Fabric agent writes them as Parquet files in the landing zone. PostgreSQL to Fabric and Oracle to Fabric follow the same model, with the same snapshot and operations.

HOW IT WORKS

How Gluesync writes to Microsoft Fabric

Parquet files in the landing zone, Delta tables built by Fabric

Open Mirroring is the way Microsoft Fabric lets any application replicate data into a mirrored database. The Fabric agent is a target agent: it receives the snapshot and the change stream from any Gluesync source agent and writes Parquet files, plus the small metadata files Open Mirroring expects, into the landing zone folder of the mirrored database. Fabric reads them and keeps the Delta tables up to date, so Gluesync writes no Delta files itself.

Row markers: change files end with an __rowMarker__ column that says insert, update, or delete, so the mirrored table stays equal to the source. An option writes inserts and updates as upserts, which makes a change replayed after a restart harmless.
Snapshots as plain loads: snapshot files carry no row marker, so Fabric loads them as inserts, as the Open Mirroring guidance recommends for the initial load.
Keys: the source primary key is written once per table as the key columns of the mirrored table. An update that changes the key is written as a delete of the old key and an insert of the new row.
Over HTTPS: Core Hub talks to OneLake through the Azure Data Lake Storage Gen2 API, so there is no JDBC driver and no connection pool. Fabric Open Mirroring agent overview ↗

Files published atomically, in order, with nothing lost on a restart

Fabric reads a file as soon as it appears under its final name and never accepts it changing, so the agent publishes each file in a way that cannot be seen half written.

Atomic publication: a file is uploaded under a temporary name that Open Mirroring ignores, closed with one flush, and renamed to its final 20-digit name without replacing an existing file.
Sequence kept across restarts: file names are strictly increasing, and the first write of a table after a start continues from the highest name already in the table folder, so Gluesync needs no local store for it.
Confirmed only when it is on OneLake: Core Hub acknowledges a change to the source once the file that holds it has been published, so a restart or a crash loses nothing.
Schema evolution: a column added to an entity appears in later files and Fabric adds it to the Delta table.

Large snapshots and batched change files

Large tables load as large files. A snapshot streams its pages into one Parquet file per table, closed and published when it reaches the configured number of rows or when the snapshot ends. Files of different tables upload in parallel, and each file is appended in parallel blocks.

Changes are collected per table and written as one file when the first of three limits is reached: a time window that defaults to one second, a row count, or a size in megabytes. Every file costs Fabric a Delta commit, so a longer window means fewer, larger files and less work for the replication, while a shorter one delivers changes sooner. A window of zero writes one file per commit.

Seed, then stream: the snapshot populates each table, then changes follow from the source agent as they arrive, in commit order per table.
Compression you choose: Parquet files use ZSTD by default, with SNAPPY, GZIP, and uncompressed output available, and Fabric reads all of them.
Scheduled reloads: the Chronos scheduler runs a snapshot, or a snapshot followed by CDC, on a cadence for tables you refresh on a timetable. See schedules and events.

Tables, folders, and column types

Each table lands in its own folder of the landing zone, in a <schema>.schema folder when the table has a schema, and the SQL analytics endpoint shows it under that schema. Values are written as typed Parquet columns, with microsecond timestamps.

Integers and floats are written as INT32, INT64, FLOAT, and DOUBLE, and decimals keep their source precision and scale as Parquet decimals.
Dates and times map to Parquet DATE, TIME_MICROS, and TIMESTAMP_MICROS, and offset values are written as the UTC instant.
Booleans, strings, and binaries keep their types; arrays and maps are written as JSON text, as Open Mirroring asks for complex values.

Setup in your Fabric workspace

The agent needs a mirrored database that uses Open Mirroring and a service principal with access to its workspace. Full steps are in the Fabric Open Mirroring target setup guide ↗.

Register an application in Microsoft Entra ID, create a client secret, and record the application (client) ID, the directory (tenant) ID, and the secret value.
Add the application to the Fabric workspace as a member with the Contributor role or higher, and enable the Fabric tenant setting that lets service principals use Fabric APIs for it.
Create a mirrored database that uses Open Mirroring, and copy the landing zone URL shown on its Home page.
In Core Hub, enter the client ID, the client secret, the tenant ID, and the landing zone URL as it is, or the workspace and mirrored database IDs.
Map each entity to a table, and Gluesync starts with a snapshot and then follows the changes.

Architecture around Core Hub

Source agents sit close to each database, and Core Hub orchestrates them through its web UI and REST APIs. A pipeline groups a source agent with the Fabric agent and the tables it replicates. On setup, Gluesync signs in to Entra, asks for a OneLake token, and checks that the landing zone exists, and a wrong secret or missing workspace access comes back as one error that names what to change. In Fabric, Monitor replication shows each table's status, the rows replicated, and the time of the last update. See CDC streaming for the wider pattern.

Explore the general CDC streaming architecture →

WRITE OPTIONS

The Fabric Open Mirroring target agent: Parquet change files with row markers

Microsoft Fabric has one Gluesync target agent. It writes snapshots and change files into the landing zone of a mirrored database, and Fabric maintains the Delta tables.

Agent	Write technique	Versions	Best for
Fabric Open Mirroring agent ↗	Parquet files with `__rowMarker__` in the Open Mirroring landing zone on OneLake, written over the Azure Data Lake Storage Gen2 API with atomic publication	Any Microsoft Fabric workspace with a mirrored database that uses Open Mirroring	Teams feeding Fabric from transactional databases who want inserts, updates, and deletes applied by key in a mirrored database, with Microsoft Entra service principal authentication under one Core Hub.

SOURCES AND TOPOLOGIES

Feed Fabric from the databases you already run

Any Gluesync source agent can feed Fabric, each with its own native capture technique. Open the integrations finder with Microsoft Fabric pre-selected to see every source you can pair with it.

One Core Hub runs PostgreSQL to Fabric, Oracle to Fabric, and SQL Server to Fabric side by side, with the same snapshot and write settings for each. Gluesync keeps pace with your change volume at any scale, and MOLO17 Professional Services can help with the layout of your mirrored databases.

PostgreSQL to Fabric from the write-ahead log: see PostgreSQL CDC
Oracle to Fabric from the redo logs: see Oracle CDC
SQL Server and MySQL to Fabric: see SQL Server CDC and MySQL CDC
MongoDB to Fabric from change streams: see MongoDB CDC

FAIR, HIGH-LEVEL COMPARISON

Where Gluesync fits among Fabric ingestion approaches

Approach	What buyers usually get	Where Gluesync fits
Fabric pipelines and copy jobs	Native orchestration inside Fabric, usually run on a schedule, with change detection designed into each pipeline	Log-based agents that write changes continuously into a mirrored database, with one Core Hub across every source; see CDC streaming
Fivetran, Airbyte, and other ELT tools	Managed or open-source connectors that load on a sync schedule; change capture varies by source connector	Log-based capture with a commercial agent per source and MOLO17 enterprise support, writing continuously rather than on a sync schedule
Debezium with Kafka Connect and a custom loader	Open-source log-based capture into Kafka; your team builds the step that writes Parquet files and runs keys, offsets, deletes, and restarts	A native capture agent per database that writes Open Mirroring files itself, with no Kafka cluster in the path; read the Debezium alternative comparison
Custom Parquet writers and scripts	Full control; your team owns the extract queries, the file naming, the row markers, the retries, and the load each run puts on the source database	Log-based capture with no polling queries against production, and file sequencing, atomic publication, and restarts handled by the agent; see migrating to Gluesync

Fabric replication research and implementation detail

What does replicating to Microsoft Fabric with Gluesync involve?

A source agent captures committed changes from your database through its native change mechanism, Core Hub routes them, and the Fabric Open Mirroring agent writes them as Parquet files into the landing zone of a mirrored database. Each table is seeded with a snapshot first and then follows the change stream, and Fabric keeps the Delta tables up to date.

How does Gluesync write data into Fabric?

Through Open Mirroring. The agent uploads Parquet files to the landing zone of the mirrored database on OneLake over HTTPS, using the Azure Data Lake Storage Gen2 API. Each change file carries a row marker for insert, update, or delete, and Fabric applies it to the Delta table.

Does Gluesync write Delta files?

No. Gluesync writes the Parquet files and metadata files that Open Mirroring expects, and Fabric builds and maintains the Delta tables of the mirrored database.

Which sources can replicate to Microsoft Fabric?

Any Gluesync source agent. PostgreSQL, Oracle, SQL Server, MySQL, and MongoDB can all feed Fabric, and each one uses its own native capture technique.

How does the agent authenticate to Fabric?

As a Microsoft Entra service principal with a client secret. The application is added to the Fabric workspace as a member with the Contributor role or higher, and the Fabric tenant setting for service principals is enabled for it.

Can a crash or restart lose or duplicate changes?

No changes are lost. Core Hub confirms a change to the source only when the file that holds it is on OneLake, and the changes of a file that was not yet published are applied again. Inserts and updates can carry the upsert marker, so applying them twice leaves the table as it was.

How quickly do changes reach Fabric?

Changes of a table are collected for one second by default, or until a row count or size is reached, and then published as one file. The window is configurable, down to one file per commit.

How do I see the state of the mirrored tables?

In Fabric, Monitor replication on the mirrored database shows each table's status, rows replicated, and last update. The SQL analytics endpoint of the mirrored database serves the tables, and the Gluesync logs report every upload.

REPLICATE TO A TARGET