Solution · Lakehouse ingestion

Land operational changes in the data lake

Write snapshots and continuous changes into Amazon S3, Azure Data Lake Storage Gen2, and Google Cloud Storage as Parquet, CSV, or JSON—without adding analytical load to production databases.

Explore data lake ingestion Talk to our team
SOURCE CHANGES → OBJECT STORE PostgreSQL · orders log reader I insert U update D delete no analytical queries on production batch what changed Gluesync Core Hub columnar writer Parquet CSV JSON format selected per entity write objects OBJECT STORE · BRONZE LAYER landed changes/orders/2026/09/… transaction type · table · year · month Amazon S3 ADLS Gen2 Cloud Storage
  1. 01 Read changes at the source
  2. 02 Write columnar batches
  3. 03 Organize objects on write

The challenge

The bronze layer ages faster than it loads

Lake ingestion has to move historical volume once and then keep landing changes, in a format downstream engines can read without another conversion step.

01 / Freshness

Extracts land once a night

Scheduled dumps leave the landing zone describing a version of the business that already moved on hours ago.

02 / Format

Raw files need another pass

Delimited exports force downstream jobs to convert, infer types, and re-partition before anything can be queried.

03 / Lineage

Files lose the change context

A flat row export does not record which transaction produced it, making audit, replay, and deduplication harder.

How it works

Snapshot once, then keep landing changes

Gluesync captures at the source, batches what changed, and writes columnar files into object storage through each target agent's native SDK.

  1. 01

    Capture

    Read changes at the source

    Source agents use transaction logs, journals, and native change streams, so the lake feed does not scan production tables.

  2. 02

    Shape

    Write columnar batches

    Changes are serialized to Apache Parquet by default, with CSV and JSON selectable per entity.

  3. 03

    Land

    Organize objects on write

    Files are written under a folder path built from transaction type, table name, year, month, and timestamp.

Gluesync capabilities

Object storage targets ready for downstream engines

Choose the object store, the file format, and the metadata each entity carries—from the same Core Hub that runs the rest of the estate.

01

Parquet by default

Store replicated entities in Apache Parquet, selected per entity, with CSV and JSON as alternatives.

  • Per-entity format selection
  • MOLO17 ParquetKt writer
  • Folder-to-table object layout

Parquet and CSV support

02

Cloud object-store agents

Write batches through each provider's native SDK instead of a generic file interface.

  • Amazon S3 and S3-compatible buckets
  • Azure Data Lake Storage Gen2
  • Google Cloud Storage

AWS S3 target ADLS Gen2 target

03

Change context on every record

Enrich landed records with transaction metadata read from the source log during both snapshot and CDC phases.

  • Transaction and operation metadata
  • Underscore-prefixed technical columns
  • Per-pipeline toggles

Technical fields

Keep the lake as current as the source

Start a trial or talk with MOLO17 about object-store targets, file formats, and expected change volume.

Request a free trial Talk to our team