Field skipping & field renaming from NoSQL

On-the-fly field skipping

Leaving the default configuration file as documented in the “Basic” section, that means leaving just the document type name, will let the replicator transfer and convert all the keys present in the specified JSON document. There are certain scenarios in which sometimes you would like to not sync certain keys from your JSON documents. Gluesync supports field skipping, declaring them directly inside the configuration file will tell the engine to skip the specified column name from the parent table. Here below you can find an example of the sourceEntities key declaration to support field skipping:

"sourceEntities": {
    "DocumentType1": {
        "KEY1",
        "KEY2",
        "KEY3"
    },
    "DocumentType2": {...}
}

Notes and considerations

Gluesync will not transfer and sync any other key present in the specified document other than the ones typed inside the document’s fields definition. You can add as many new fields your document has to put them in sync with SQL, just add them in the definition file and be sure you restart the service / container to reload the updated configuration file.

Your rows inside the SQL target database won’t update automatically as you add new fields in the config file: they will receive the new value as soon as its key gets updated in the NoSQL document. In case you need to have the newest fields available you may want to trigger that with an update query statement against the dataset in the specific bucket.

On-the-fly field renaming

It is likely common to find huge column names inside SQL tables and to avoid wasting space in the target NoSQL database Gluesync supports what is called on-the-fly field renaming.

To enable that feature per column name you can follow this example below.

"sourceEntities": {
    "DocumentType1": {
        "HUGECOLUMNNAME1": "NEW_COLUMN1",
        "VERYLARGECOLUMNNAME2": "NEW_COLUMN2",
        "LARGESTCOLUMNNAMEEVERSEENINASQLTABLE3": "NEW_COLUMN3"
    },
    "DocumentType2": {...}
}

Notes and considerations

Enabling on-the-fly field renaming will automatically enable field skipping, so for keys not present inside the documentType(s) definition the content won’t be replicated on the target SQL database. Field skipping and on-the-fly field renaming can be used simultaneously. Gluesync will of course translate the fields coming from the SQL database back to its original value [be sure they are configured in the proper section in the SQL to NoSQL module].