When talking about mobile apps, making available offline data to the users is a critical point. Enterprise apps are good candidates for such a need. Apps like ones meant to simplify the production process of a factory, ones meant to serve on-field resources, and more. They all need data available immediately for letting the user do the job with profit.
As explained in the previous posts, a stunning feature provided by Couchbase is the ability to sync data across multiple devices in a very simple way thanks to the Sync Gateway.

Video introduction about the contents of this article by Damiano Giusti

Couchbase Sync Gateway

Couchbase Sync Gateway is a data synchronization engine provided by Couchbase. It plays together with Couchbase Server and Couchbase Lite, letting the developer to sync user-specific data with ease. You can find more about Couchbase Sync Gateway reading its official documentation here.

The idea behind it is simple: when a new document is written, updated, or deleted in the instance of Couchbase Server, then the operation is immediately reflected into the app local database. On the contrary, when an operation takes place into the app’s local database the change is immediately pushed upstream to the server. This process is made possible by algorithms providing advanced synchronization and complex diffing between data. All based on websockets.

Now, let’s jump into how to integrate those features into an existing app which uses Couchbase Lite.

Quick installation

First, for trying by yourself those features, we are going to follow some steps for getting up and running a Couchbase Server instance. Also we are going to connect to it a Couchbase Sync Gateway instance. Let’s bring to life the DevOps guy that hides inside you! 

For this example, we are going to use Docker, due to its ease to use.

1. Setup with Docker Compose

First, move to an empty directory and create a new docker-compose.yml file. Also, create two directories: couchbase_demo and couchbase_config.

Here you have a working Docker Compose file. It will run an instance of Couchbase Server together with an instance of the Couchbase Sync Gateway. They will talk each other thanks to the network that Compose creates. Just copy-paste this snippet in the brand new docker-compose.yml file and go further with the next step.

version: "3.7"
services:
  couchbase-server:
    image: couchbase:community-6.0.0
    ports:
      - "8091:8091"
      - "8092:8092"
      - "8093:8093"
      - "8094:8094"
      - "11210:11210"
    volumes:
      - "./couchbase_demo:/opt/couchbase/var"
  couchbase-sync-gateway:
    image: couchbase/sync-gateway
    ports:
      - "4984:4984"
    volumes:
      - "./couchbase_config:/tmp/config"
    command: /tmp/config/config.json
    depends_on:
      - couchbase-server
    restart: "on-failure"
volumes:
  couchbase_demo:
  couchbase_config:

For the sake of this article, the Compose configuration needs two volumes for operating correctly. The first is named couchbase_demo and represents the directory in which the server will store its files. The latter is called couchbase_config, and will contain a single file, the config.json. The volumes point to the directories you just created.

2. Sync Gateway configuration file

The config.json file contains the configuration which will be read by the Sync Gateway. It’s used for determining which buckets to read from, which users use for the read/write operations and so on. Here I provide you a sample configuration, meant to be used with the server configuration we will do further. Just copy-paste this file into a new config.json file inside the couchbase_config directory. We will cover this later.

{
  "databases": {
    "beers": {
      "server": "http://couchbase-server:8091",
      "bucket": "beer-sample",
      "username": "Administrator", 
      "password": "password", 
      "enable_shared_bucket_access": true, 
      "import_docs": true,
      "num_index_replicas": 0, 
      "users": {
        "GUEST": { "disabled": false, "admin_channels": ["*"] }
      }
    }
  },
  "logging": {
    "console": {
      "log_level": "info",
      "log_keys": ["HTTP+", "CRUD", "Sync"]  
    }
  }
}

3. Start containers and setup server

Now all you need to do is just run the following command in the directory that contains the docker-compose.yml file:

$ docker-compose up -d

Then, connect to http://localhost:8091 and when ready the Couchbase Server panel will appear to you.
Next, proceed with the following steps:

  1. Click on “Setup New Cluster” inserting then couchbase-demo-cluster as name, Administrator as user (if not already set) and password as a password. So strong, isn’t it?
  2. Accept Terms & click on “Finish with defaults”.
  3. Yeah the server is up! Go to “Buckets” on the left and click on “load a sample bucket with data & indexes” on the placeholder text.
  4. Select “beer-sample” from the available samples list and click “Load sample data”.
  5. Done! On the “Buckets” menu you will see the “beer-sample” bucket that is being populated with sample data.

Eventually the Sync Gateway should have recognized the bucket we created thanks to the provided configuration file. Open your browser, and by navigating to http://localhost:4984/beers/ you should see a JSON sent as response. You did it!

Sync Gateway config.json configuration file

The config.json is a key file in the whole process you just did. With the properties it specifies you can define how the Sync Gateway is going to access to your data, how it exposes them as REST endpoints, how it replicates them to your mobile clients and so on. Let’s see what kind of configuration we applied using the file I provided to you:

  • databases: an object which contains for each key a definition of a bucket that will be managed by the Sync Gateway;
  • beers: a key that is the name of the endpoint that we are exposing. The corresponding object contains additional configuration;
  • server: the URL that points to the Couchbase Server. We specified the service name defined into the docker-compose.yml because is the network name that Compose assigns to the running container;
  • bucket: the bucket that is exposed aliased by the “beers” name;
  • username: the user that is used for accessing the bucket;
  • password: the password for the given user;
  • enable_shared_bucket_access: set to true for enabling the data sync with the mobile;
  • import_docs: works together with enable_shared_bucket_access enabling the Sync Gateway to process the documents import;
  • num_index_replicas: set the number of replicas that the database indexes should have. We set it to 0 because we don’t have other nodes;
  • users: in this object we enable the guest user;
  • logging: defines the kind of loggers enabled for the Sync Gateway;
  • console: allows the Sync Gateway to print the logs in the standard output;
  • log_level: the logging level applied to the logger.

Connecting the Mobile SDK

All we got until here is a running instance of Couchbase Server with a bucket of sample data regarding beers and breweries. Also, we set up an instance of the Couchbase Sync Gateway for replicating the sample data to our mobile clients.
Well, actually we only miss one step: the mobile client!
As we saw in the first post, integrating the Couchbase Lite is pretty simple. Now we will cover in a more specific way how to query the dataset that has been synced to the client.

A closer look to the database

The beer-sample bucket contains two kinds of documents: brewery and beer. You can find them by yourself by running the following N1QL query in the “Query” panel the Couchbase Server dashboard:

SELECT DISTINCT type
FROM `beer-sample`

Feel free to explore the dataset! N1QL offers the same syntax as the standard SQL does, so the learning curve is very low.
For example, let’s extract all the beers that are available in the database.

SELECT
    META(b).id,
    b.name, 
    b.description, 
    b.category, 
    b.style
FROM `beer-sample` AS b
WHERE b.type == "beer"

The META(<database_name>) function projects the metadata of the selected row, allowing to extract additional data such as the default document identifier. If no identifier has been set when creating a new document, Couchbase will assign it for you.

Syncing an Android app

Now setup new Android Studio project and add the Couchbase Lite SDK in your build.gradle file:

implementation 'com.couchbase.lite:couchbase-lite-android:2.6.0'

As we seen in the first post, start a Replicator for letting the sync process begin.

// Create a new database reference, and call it “beers.db”
val database = Database("beers.db", DatabaseConfiguration())
// Create a new replicator configuration, pointing to the 
// Sync Gateway bucket “beers”.
val config = ReplicatorConfiguration(
  database, URLEndpoint(URI.create("ws://10.0.2.2:4984/beers"))
)
// Ask for continuous replication. This will continuously sync 
// the database with the remote and vice-versa.
config.isContinuous = true
// Create a new replicator, start the replica 
// and keep a reference to it.
val replicator = Replicator(config).apply { start() }

The code above gives you a clue on how to configure the Replicator for syncing data to the running Sync Gateway instance. The replication URL points to your localhost and is meant to be run on an Android emulator. The specified IP address is the address that Android maps to 127.0.0.1 of the host machine.

Continuous replication within the Sync Gateway

Also, we set the replication to be continuous. A continuous replication, as specified in the comments, lets your client to be always updated when the data change on the remote bucket. On the other hand when the client application writes some data into the local database, such data will be automatically synced to the remote. By setting the flag to false, the sync will run just one time.

Sync Gateway replication directions

The “directions” of the data we mentioned have been given specific names by Couchbase. We speak about a push replication when the documents are sent from the client to the server. On the contrary, we discuss about pull replication when documents are just dowloaded from the remote. Eventually, the replication can happen in both the ways: the push-and-pull replication is the union of both, in which the data are transmitted in a bidirectional way. The latter is the default choice for the mobile SDK.

Channels

A more intriguing feature of the Sync Gateway is bring to you by the sync channels. A channel is sort of a tag that allows the gateway to correctly route data to the various clients. Basically, a channel is represented as a string inside a channels array inside the document. Each document tagged with a particular channel will be visible to the clients that subscribe to the gateway for it.

By default, the Sync Gateway uses the default sync function for routing each document into its proper channels. But you can also customize this behavior, by writing your own sync function and ship it inside the config.json file. This allows you to have a fine-grained control on how data are dispatched, applying particular business logics. You can discover more on this topic here.

After this quick overview, let’s see how to specify the channels in our Replicator object. Nothing more than:

config.channels = listOf("my-channel-1", "my-channel-2")

Pretty easy, isn’t it?

Listen to Sync Gateway replication progress

When the replication is in progress, you may want to notify your user about the data transfer status. The Couchbase Lite SDK provides you basically two ways for doing that. You can listen to the replication changes, or listening to the document transfer progress.

You can attach a change listener to the Replicator object, in this way:

replicator.addChangeListener { change ->
  ...
}

The change object of type ReplicatorChange gives you access to the undelying Replicator and to a Replicator.Status object. Such instance will let you check the activity level of the sync, that can assume the following meanings:

  • Stopped: the replication is finished or hit a fatal error.
  • Offline: the replicator is offline because the remote host is unreachable.
  • Connecting: the replicator is connecting to the remote host.
  • Idle: the replication is inactive; either waiting for changes or offline as the remote host is unreachable.
  • Busy: the replication is actively transferring data.

Also, you have access to the progress of a replication, represented by the Replicator.Progress class. The provided instance will give you the total count of the changes that are going to be synced together with the actual completed count. Eventually, you get also a CouchbaseLiteException instance when an error occurs, which is not thrown.

And now is your turn!

This article gave an overview about how to build up from scratch a Couchbase Server and a Sync Gateway instance. Also, now you’re given the tools to integrate a powerful synchronization engine into your mobile app! Keep us updated about your experiments using our social media!

Did you miss the other articles on Couchbase?
Click here to read the series and here to read the next article!