Gluesync SQL to NoSQL

Targeting AWS S3 Bucket

Core principles

Objects storage in Amazon Simple Storage Service (S3) is capable of providing a flexible, scalable and cost-effective solution for storing large amounts of data in an object format like JSON files.

Gluesync offers the support to store data coming from supported RDBMS into AWS S3 in JSON format using native AWS S3 SDK. Data could be modeled accordingly to your needs using the data modelling features available in Gluesync and stored in an S3 bucket where it has read & write permissions.

JSON stored in the S3 destination bucket will be stored using the best-practices suggested by Amazon AWS with keyspace support, this means that documents will be stored in folders grouped by source schema and table name, named by their primary key.

Not just documents storage

Gluesync does not only store your object in JSON format: with a specific configuration you can obtain more out from your S3 object storage when it comes to reading data through a query engine: you can specify the metadata key-value fields that you would like Gluesync to store as object metadata, this will boost your indexing and document retrieval capabilities when managing large data lakes.

Another useful option, especially if your bucket is intended for cold storage of objects, is the possibility to tell Gluesync to skip any document deletion action to preserve them inside the bucket while you are removing this data from your source database.

Architectural overview

Here in the following diagram represents an architectural overview of the environment you are going to have after having deployed Gluesync.

a diagram illustrating the architectural overview of Gluesync for AWS S3

Q&A

I am looking to store other file formats like CSV or XML files, is it supported? We’re open to support a wider amount of different use cases and that also means different file formats for your object storage needs. Please to not hesitate to reach out to us to let us know your need: we’re more than happy to accommodate your feature request.

I am looking to use an object storage service like AWS S3 but from another vendor/provider, is it supported? We’re working on extending the object storage support to other vendors/providers/transfer protocols and the support for object storage in OCI, Azure, Alibaba, Aruba and many others is planned for future product releases. Please reach out to us to let us know your specific needs so we can plan our future roadmap accordingly.

For a list of supported versions have a look at our list of supported databases/versions available here.

Having further questions? For more information regarding Gluesync please reach us via email by pressing here: Contact MOLO17.