Canonical: https://molo17.com/data-modeling/ Markdown mirror: https://molo17.com/data-modeling/index.md Title: Data modeling · Gluesync · MOLO17 Description: Turn SQL queries into flat or nested JSON documents, split documents into related SQL tables, and migrate Gluesync 1.x data models. Kept current by CDC. Gluesync Module · SQL and NoSQL modeling 2.3.2 # Data modeling Shape data between SQL and NoSQL without custom code. Write a SQL query and get flat or nested JSON documents, split one document into related SQL tables, and move your Gluesync 1.x data models over in one conversion. [See how it works](#how-it-works) [Talk to our team](/contacts/?topic=gluesync-platform#contact-form-section) Data modeling 1. 01 Write a query, preview the document 2. 02 Split one document into related tables 3. 03 Convert your Gluesync 1.x models Why Data modeling ## Data models should not live in custom code 01 / SQL to NoSQL ### Joins do not fit a document store An application wants one order document with its customer and lines inside. Building it means joining tables, watching every table in the join for changes and rewriting the right documents. 02 / NoSQL to SQL ### A document does not fit one table A document with arrays of cards and tokens belongs in several related tables. Splitting it by hand means keys, parent links and deletes handled in custom code. 03 / The platform outcome ### Describe the shape once Gluesync keeps documents and tables current from the same definition, and carries your Gluesync 1.x models over with one conversion. How it works ## From a definition to a live model in three steps 1. 01 Define ### A query, or a document Write the SQL query and add a level for each array, or choose the levels of the document to split. The wizard for Query entities walks the query, the key, the target mapping and the review. 2. 02 Check ### Describe, preview, validate Describe types the columns and returns the CDC verdict. Preview builds the first documents from the live source without writing them. Validate reports every issue at once. 3. 03 Run ### Snapshot, then CDC A snapshot loads the current data, then change data capture keeps every document and table in step with its source. Capabilities ## Query entities, document models and a path from 1.x Data modeling runs inside Core Hub and the agents you already use. SQL Server CDC is the source of Query entities, Couchbase is the source of document models, and every definition is checked before it runs. Query entities ### SQL query in, JSON documents out #### Flat or nested, kept current by CDC The source of an entity is a SQL query. Joins, filters and sub-selects over your SQL Server tables become one document per root key, written to Couchbase and other NoSQL targets. Add a level for each array of child rows and the document nests them. - Flat documents, or nested documents with an array for each child level - SQL Server CDC reads every table the query touches: a change to a joined table updates every document it reaches - Describe shows the typed columns, the proposed key and the CDC verdict before anything runs - Preview builds the first documents from the live source without writing them - A guided wizard takes the entity from the SQL query through target mapping to review and save [View Query entities docs ↗](https://docs.molo17.com/gluesync/latest/data-modeling/query-entities.html) Data modeling ### From joined tables to the document you need Write the SELECT once. A flat entity gives one document per root row. Add a level and each array becomes part of the same document, rebuilt whenever a table it reads changes. Document models ### One document, related tables #### NoSQL to SQL, one level per table A Couchbase document with nested arrays lands in SQL Server as related tables. Each level gets its own table, its own key and a link to its parent. A holder document with cards and tokens becomes three tables. - Cards\[\] and Cards\[\].Tokens\[\] become their own tables, with keys and parent links - Embedded objects flatten into columns of the parent row, or get a table of their own - A token added in the document appears as a new row in Tokens - A delete policy per level: apply, report only or off, with chain scope for child keys that do not carry the root key [View Document models docs ↗](https://docs.molo17.com/gluesync/latest/data-modeling/document-models.html) Data modeling ### Choose what a delete does, level by level Apply removes the rows. Report only keeps them and records what would have been deleted. Chain scope reaches child tables whose keys do not carry the root key. Each level sets its own policy. Gluesync 1.x migration ### Bring your data models to Gluesync 2 #### Converted in Gluesync Automator, checked by Core Hub Gluesync Automator turns the entities of a Gluesync 1.x configuration into Gluesync 2 entities on an existing pipeline, from its Import tab or the command line. It reports what became of every entity, and Core Hub checks every draft before anything is saved. - Upload the configuration, pick the pipeline and the Gluesync 1.x product, and convert: secret-looking keys are removed unread - The report lists every entity as flat, multi-level or document split, with the describe and validate result of Core Hub next to each draft - Choose the drafts to keep: nothing is saved without your confirmation - Migrated entities keep their behaviour: dates in custom formats, arrays and maps as JSON text, type discriminators and duplicate keys - Document ids are rendered as Gluesync 1 rendered them, so existing documents keep their ids - Then compare live documents with what the new entity writes, and cut over collection by collection [View migration docs ↗](https://docs.molo17.com/gluesync/latest/data-modeling/migrate-from-gluesync-1.html) [Get Gluesync Automator →](/gluesync-automator/) Data modeling ### Same behaviour, same output Dates in custom formats, arrays and maps as JSON text, type discriminators, duplicate keys and document ids come out of the migrated entity the way they came out of the 1.x one. FAQ ## Frequently asked questions What is a Query entity? An entity whose source is a SQL query instead of a table. Gluesync writes one document per root key, flat or with nested arrays of child rows, and keeps it current as the source tables change. Which source and targets do Query entities use? The source is SQL Server with change data capture. Documents are written to Couchbase and other NoSQL targets, as flat JSON or as multi-level JSON with an array per child level. How does a change in a joined table reach the documents? SQL Server CDC reads the change tables of every table the query reads. Gluesync finds the documents a change touches, reads them again from the source and writes them whole, so a change to a customer updates every order document that holds it. Can I check a query before saving the entity? Yes. Describe returns the typed columns, the proposed key, the tables read and the CDC verdict without running the statement. Preview shows the documents the query would write, with their ids, and validate reports every issue at once. Preview values are masked like Query Studio results. What does a document model do? It splits one Couchbase document into SQL Server tables, one level per table, each with its key and a link to its parent. Embedded objects become columns of the parent row or tables of their own, and arrays of scalars become rows. How are deletes handled in a document model? Each level has its own delete policy. Apply removes the rows, report only keeps them and records what would have been deleted, and off ignores deletes. Chain scope reaches child tables whose keys do not carry the root key. How do I migrate Gluesync 1.x data modeling? Open Gluesync Automator, or run its command line, upload the 1.x configuration, pick the pipeline and the Gluesync 1.x product, and convert. The report lists every entity and Core Hub checks each draft. You choose the drafts and confirm the save, and nothing is saved without that confirmation. Then compare live documents with what the new entity writes and cut over collection by collection. Where does the Gluesync 1.x conversion run? In Gluesync Automator, from the Import tab or the command line. Both run the same steps: convert the configuration, let Core Hub describe and validate every draft without writing anything, then save the drafts you choose once you confirm. Do migrated entities behave like the 1.x ones? Yes. They keep dates in custom formats, arrays and maps written as JSON text, type discriminators, duplicate keys with the last item winning, and Gluesync 1 document ids. Where do I configure data modeling? In the Control Plane, where a guided wizard walks Query entities from the SQL query to the review, and through the REST API and the MCP server. The conversion of Gluesync 1.x models runs in Gluesync Automator. Explore the platform ## Continue exploring Gluesync Data modeling builds on the pipelines, the SQL workbench and the schedules you already use. [### Data transformation Field functions, filters and UDFs that shape the values inside every document and row. Explore Data transformation →](/data-transformation/) [### Query Studio The SQL workbench where you try a query before it becomes an entity. Explore Query Studio →](/query-studio/) [### Schedules and events Refresh a snapshot-only Query entity on a Chronos schedule. Explore Schedules and events →](/schedules-and-events/) [ ### Gluesync Automator Converts your Gluesync 1.x data models into Gluesync 2 entities, in its UI or from the command line. Explore Gluesync Automator →](/gluesync-automator/) Resources ## Guides, roadmap and support Documentation ### Documentation Technical guides, API reference, deployment tutorials. [Read documentation ↗](https://docs.molo17.com) Roadmap ### Public Roadmap What we're building next. Submit feature requests. [View roadmap ↗](https://roadmap.molo17.com) Support ### Support Access technical support and operational assistance. [Get support →](/support/) Status ### Service Status Real-time platform availability and incident history. [Check status ↗](https://status.molo17.com) ## Model your data once, keep it current Request a free trial or talk with the MOLO17 team about SQL to NoSQL modeling and migrating your Gluesync 1.x models. [Request a free trial](/get-gluesync/) [Talk to our team](/contacts/?topic=gluesync-platform#contact-form-section) [Docs](https://docs.molo17.com/gluesync/latest/data-modeling/index.html)