Canonical: https://molo17.com/blog/how-to-build-on-premise-data-lake-gluesync-universal-file-store-agent/ Markdown mirror: https://molo17.com/blog/how-to-build-on-premise-data-lake-gluesync-universal-file-store-agent/index.md Title: How to Build an On-Premise Data Lake with Gluesync Description: Learn how to build a private cloud or on-premise data lake by streaming real-time database changes into FTP, SFTP, SMB and NFS targets using Gluesync. [← All articles](/blog/) Article # How to build an on-premise data lake using Gluesync Universal File Store Agent 3 Jul 2026 [Gluesync integrates](/blog/?category=gluesync-integrates) 2 min read Universal File Store Agent provides a direct way to stream real-time replicated data into file-based storage systems using standard network protocols.  As organizations adopt increasingly distributed data architectures, requirements around data sovereignty, compliance, and infrastructure control are becoming more stringent. In many industries, including finance, healthcare, and regulated enterprise environments, data must often remain within private infrastructure rather than being streamed exclusively to public cloud storage services. While cloud object stores are widely used for real-time data pipelines, many organizations require the ability to land streaming data directly into on-premise or privately managed file systems. With [**Gluesync 2.2.9**](https://molo17.com/blog/gluesync-2-2-9/), the **Universal File Store Agent** provides a direct way to stream real-time replicated data into file-based storage systems using standard network protocols. ## A unified file-based integration layer The Universal File Store Agent acts as a target connector that enables real-time data delivery from supported source systems into file storage endpoints. It supports a range of standard enterprise protocols, including: - FTP - FTPS - SFTP - WebDAV - WebDAVS - SMB/CIFS - NFS This allows teams to integrate with a wide variety of file-based storage systems, including network shares, on-premise storage arrays, and private cloud file services. ## Flexible output formats for downstream systems To support different consumption patterns, the agent can generate data in multiple formats: - Apache Parquet (default, optimized for analytics workloads) - Line-delimited JSON - CSV Each format is designed to support interoperability with downstream processing systems, data lakes, and analytics engines. ## Metadata-driven recovery with sidecar files File-based pipelines do not inherently preserve transactional boundaries. To support reliable recovery and traceability, the Universal File Store Agent generates **sidecar JSON metadata files** alongside each output file. These metadata files store replication-related information such as offsets and transactional context, enabling Gluesync to resume processing accurately in the event of interruptions or restarts. ## Designed for on-premise and private data lakes The Universal File Store Agent is particularly well suited for: - on-premise data lake architectures - network-attached storage (NAS) environments - private cloud file systems - hybrid file-based integration workflows By using standard protocols, it avoids dependency on proprietary object storage APIs and enables deployment in controlled infrastructure environments. ## Operational considerations When configuring file-based targets, the destination storage system must be accessible through a pre-configured network share or mount point, and appropriate read/write permissions must be in place for the Gluesync runtime environment. **Try Gluesync 2.2.9 today** The Universal File Store Agent in **Gluesync 2.2.9** extends real-time replication capabilities beyond traditional object storage systems, enabling organizations to deliver streaming data directly into file-based infrastructures using standard protocols and formats. This makes it possible to design flexible on-premise or hybrid data lake architectures without relying exclusively on public cloud storage services. For the complete list of raw JSON schemas, connection parameters, and Web UI input boxes, visit the official [**Gluesync Documentation Hub**](https://docs.molo17.com/gluesync/latest/release-notes/release-notes.html). Want to evaluate the Universal File Store Agent directly against your own on-premise network shares or private file targets? Kickstart a sub-45ms real-time data replication evaluation kit by [requesting a free 30-day trial](https://molo17.com/get-gluesync/). [← Older article Data masking and transformation at the edge: introducing Gluesync field functions](/blog/data-masking-transformation-edge-gluesync-field-functions/) [Newer article → How we achieve sub-45ms real-time replication with log-based CDC](/blog/real-time-replication-log-based-cdc/) ## Keep reading 1. [Mentioned in this article Gluesync 2.2.9: Query Studio, Universal File Store Agent, Filed Functions and more 2.2.9 introduces Query Studio, new Control Plane UI, native DB2 LUW log-based CDC, Field Functions and the iOS/Android mobile companion app. 30 Jun 2026](/blog/gluesync-2-2-9/) 2. [Gluesync integrates Amazon RedShift target agent: high-performance cloud data warehousing with Gluesync Stream real-time data into Amazon Redshift with Gluesync. Leverage AWS S3 staging and native COPY commands for high-throughput bulk loads and batch CDC ingestion. 5 May 2026](/blog/amazon-redshift-target-agent-high-performance-cloud-data-warehousing-with-gluesync/) 3. [Gluesync integrates SAP HANA agent for Gluesync: real-time CDC for SAP environments Gluesync now offers full real-time CDC for SAP HANA. Use our shadow table technique to capture changes as a source or leverage high-performance target capabilities. 4 May 2026](/blog/sap-hana-agent-for-gluesync-real-time-cdc-for-sap-environments/) [Back to all articles](/blog/) [More in Gluesync integrates →](/blog/?category=gluesync-integrates)