Offline Sync with Azure Mobile Apps and Apache Cordova

In the past, I’ve introduced you to a TodoList application built in Apache Cordova so that it is available for iOS, Android or any other platform that Apache Cordova supports. Recently, we released a new beta for the Azure Mobile Apps Cordova SDK that supports offline sync, which is a feature we didn’t have.

Underneath, the Cordova offline sync functionality uses SQLite – this means it isn’t an option at this point for HTML/JS applications. We’ll have to work out how to do this with IndexDB or something similar, but for now that isn’t an option without a lot of custom work.

Let’s take a look at the differences.

Step 1: New variables

Just like other clients, I need a local store reference and a sync context that is used to keep track of the operational aspects for synchronization:

    var client,        // Connection to the Azure Mobile App backend
        store,         // Sqlite store to use for offline data sync
        syncContext,   // Offline data sync context
        todoItemTable; // Reference to a table endpoint on backend

Step 2: Initialization

All the initialization is done in the onDeviceReady() method. I have to set up a model so that the SQLite database is set up to match what is on the server:

function onDeviceReady() {

    // Create the connection to the backend
    client = new WindowsAzure.MobileServiceClient('https://yoursite.azurewebsites.net');

    // Set up the SQLite database
    store = new WindowsAzure.MobileServiceSqliteStore();

    // Define the table schema
    store.defineTable({
        name: 'todoitem',
        columnDefinitions: {
            // sync interface
            id: 'string',
            deleted: 'boolean',
            version: 'string',
            // Now for the model
            text: 'string',
            complete: 'boolean
        }
    }).then(function () {
        // Initialize the sync context
        syncContext = client.getSyncContext();
        syncContext.pushHandler = {
            onConflict: function (serverRecord, clientRecord, pushError) {
                window.alert('TODO: onConflict');
            },
            onError: function(pushError) {
                window.alert('TODO: onError');
            }
        };
        return syncContext.initialize(store);
    }).then(function () {
        // I can now get a reference to the table
        todoItemTable = client.getSyncTable(tableName);

        refreshData();

        $('#add-item').submit(addItemHandler);
        $('#refresh').on('click', refreshData);
    });
}

There are three distinct areas here, separated by promises. The first promise defines the tables. If you are using multiple tables, you must ensure that all promises are complete before progressing to the next section. You can do this with Promise.all() as an example.

The second section initializes the sync context. You need to define two sections for the push handler – the conflict handler and the error handler. I’ll go into the details of a conflict handler at a later date, but this is definitely something you will want to spend some time thinking about. Do you want the last one in to be the winner, or the current client edition to be the winner, or do you want to prompt the user on conflicts? It’s all possible.

Once I have created a sync context, I can get a reference to the local SQLite database table, which is used instead of the getTable() call that it replaces. The rest of the code is identical – I refresh the data and add the event handlers.

Step 3: Adjusting the Refresh

In the past, refresh was just a query to the backend. Now I need to do something a bit different. When refresh is clicked, I want to do the push/pull cycle for synchronizing the data.

function refreshData() {
    updateSummaryMessage('Loading data from Azure');
    syncContext.push().then(function () {
        return syncContext.pull(new WindowsAzure.Query('todoitem'));
    }).then(function () {
        todoItemtable
            .where({ complete: false })
            .read()
            .then(createTodoItemList, handleError);
    });
}

Just like the initialization, the SDK uses promises to proceed asynchronously. First push (which resolves as a promise), then pull (which also resolves as a promise) and finally you do EXACTLY THE SAME THING AS BEFORE – you query the table, read the results and then build the todo list. Seriously – this bit really didn’t change.

That means you can add offline to your app without changing your existing code – just add the initialization and something to trigger the push/pull.

Wrap Up

This is still a beta, which means a work-in-progress. Feel free to try it out and give us feedback. You can file issues and ideas at our GitHub repository.

Cross-posted to the Azure App Service Team Blog.

Apache Cordova, Azure Mobile Apps and CORS

I am continuing my discovery of Apache Cordova to integrate it into Azure Mobile Apps. This post is about fixing a relatively minor issue. That issue is Cross-Origin Resource Sharing or CORS. One might think this is not an issue with Apache Cordova. After all, a mobile app should be able to access the remote data store with no problem. Where is the CORS issue? The issue occurs in two situations – when using cordova run browser and when using Ripple. In both cases, Apache Cordova starts a server on localhost to serve the pages. The client (the browser session) is pulling the pages from localhost, but retrieving the data from your Azure Mobile Apps service. This hits CORS.

The Server

I’ve got a simple server that is a copy of the basic app from the azure-mobile-apps samples directory. My server looks like this:

var express = require('express')(),
    morgan = require('morgan'),
    azureMobileApps = require('azure-mobile-apps');

express.use(morgan('combined'));
var app = new azureMobileApps();

app.tables.import('./tables');
app.tables.initialize().then(function () {
    express.use(app);
    express.listen(process.env.PORT || 3000);
});

I have the following code in the tables/TodoList.js:

var table = require('azure-mobile-apps').table();

table.dynamicSchema = true;

module.exports = table;

You can find the complete code sample at my GitHub repository. I’ve deployed this to an Azure App Service using continuous deployment, linking my App Service to the GitHub repository.

The Client

I’ve adjusted my src/lib/storage-manager.js file as follows:

import uuid from 'uuid';
/* global OData */

export default class Store {
    constructor() {
        console.info('Initializing Storage Manager');

        // We need to add the ZUMO-API-VERSION to the headers of the OData request
        this._defaultHttpClient = OData.defaultHttpClient;
        OData.defaultHttpClient = {
            request: (request, success, error) => {
                request.headers['ZUMO-API-VERSION'] = '2.0.0';
                this._defaultHttpClient.request(request, success, error);
            }
        };;

        this._service = 'https://ahall-todo-list.azurewebsites.net';
        this._store = `${this._service}/tables/TodoList`;
    }

    /**
     * Read some records based on the query.  The elements must match
     * the query
     * @method read
     * @param {object} query the things to match
     * @return {Promise} - resolve(items), reject(error)
     */
    read(query) {
        console.log('[storage-manager] read query=', query);

        var promise = new Promise((resolve, reject) => {
            /* odata.read(url, success(data,response), error(error), handler, httpClient, metadata); */

            var successFn = (data, response) => {
                console.info('[storage-manager] read data=', data);
                console.info('[storage-manager] read response=', response);
                resolve(data);
            };
            var errorFn = (error) => {
                console.error('[storage-manager] read error=', error);
                reject(error);
            };

            OData.read(this._store, successFn, errorFn);
        });
        return promise;
    }

I’m using DataJS to do the actual call. Note that I’m not going to complete the call in this blog post – I’m just going to get past CORS. I’ve added the package to my npm install:

npm install --save datajs

I’ve also got a Gulp rule to copy the datajs library into my www/lib directory. Check out the GitHub repository to see how that works. I need to include the datajs library into my HTML page as well – that’s just a script reference. Now, let’s run the app and check out the console:

cors-1

The Client Security Policy

There are a couple of places in Apache Cordova where you have to configure content security policies. The content security policy tells Apache Cordova where it can fetch data from. This is configured in two places – one of which is already done for you.

In config.xml, you will note the following line:

    <access origin="*" />

This tells Apache Cordova that the app should be able to access data from anywhere. But that isn’t the only place. In your index.html file – and any other HTML file you reference, there is a Content-Security-Policy tag:

        <meta http-equiv="Content-Security-Policy" content="default-src 'self' data: gap: https://ssl.gstatic.com 'unsafe-eval'; style-src 'self' 'unsafe-inline'; media-src *">

Let’s recode this into a friendlier setting:

  • default-src
    • ‘self’
    • data: = Allow loading of local base64 values
    • gap: = Allow JS -> native communication on iOS
    • https://ssl.gstatic.com = Allow Talkback on Android
  • style-src
    • ‘self’ but don’t allow inline styles
  • media-src
    • Anywhere

You can read more about the Content-Security-Policy – it’s very flexible. For now, I need to add my URL to the default-src:

        <meta http-equiv="Content-Security-Policy" content="default-src 'self' https://ahall-todo-list.azurewebsites.net data: gap: https://ssl.gstatic.com 'unsafe-eval'; style-src 'self' 'unsafe-inline'; media-src *">

Handling CORS in Azure App Service

Before this will work, you also need to tell the server to trust your local instance. Log onto https://portal.azure.com and open up your web site. Click on Settings and find the CORS setting:

cors-2

You can enter any URLs you want here. When I run cordova run browser, Apache Cordova will start a server at http://localhost:8000:

cors-3

Don’t forget to save your changes once you are done.

Wrapping Up

Once you have done these changes, you can try the project again. This time the pre-flight connection will succeed. The actual transfer will succeed but the data decoding will not work. That’s because I haven’t finished writing the code for utilizing data.js as yet. That, however, is a blog post for another time.

A lap around Azure Mobile Apps Node SDK Preview

Microsoft Azure App Service recently released a slew of mobile announcements – PowerApps being the big one. Of lesser note, the Azure Mobile Apps Node SDK hit a milestone and entered preview. It’s been a while since I lapped around the SDK, so let’s take a look at some of the breaking changes.

The SQL Azure data provider is now called mssql

There was one provider in the alpha version of the SDK – the sql driver that used mssql to access a SQL Azure instance or an on-premise SQL Server instance. However, there are plans for more providers and support was needed for multiple providers. If you are specifying a SQL Azure connection string via the Azure Portal, then nothing changes. For local development, however, it’s fairly common to use an azureMobile.js file to specify the data connection. Something like this:

module.exports = {
    logging: {
        level: 'silly'
    },
    data: {
        provider: 'mssql',
        server: 'localhost',
        database: 'sample',
        user: 'testuser',
        password: 'testpass'
    }
};

Note the highlighted line. That used to be just ‘sql’ – now it’s ‘mssql’. Again, this is all to prepare for multiple providers, so it’s a good move in my book.

You should initialize the database before listening

The common server.js file would just import tables and then start listening. This caused a significant delay on first access along with potential failures because the database was not set up properly. To combat this, a Promise structure was introduced that allowed you to defer listening for connections until after the database was initialized. You use it like this:

// Import the files from the tables directory to configure the /tables API
mobile.tables.import('./tables');

// Initialize the database before listening for incoming requests
// The tables.initialize() method does the initialization asynchronously
// and returns a Promise.
mobile.tables.initialize()
  .then(function () {
    app.use(mobile);    // Register the Azure Mobile Apps middleware
    app.listen(process.env.PORT || 3000);   // Listen for requests
  });

If a table has dynamic schema, then the initialize() call immediately resolves since the database is managed by the SDK. If you have set up static tables, then those tables are created prior to listening for connections, which means that your users get a timeout instead of a 500 server failure, which is probably a better experience.

You can seed tables with static data

There are cases when you want data to be “already there”. An example of this is in testing – you want to test the GET response and ensure specific records are there for you. Another example is if you want to have a read-only table for settings. To assist with this, you can add a seed property to the table definition. Something like this:

var table = require('azure-mobile-apps').table();
table.columns = {
	"text": "string",
	"complete": "boolean"
};
table.dynamicSchema = false;

// Seed data into the table
table.seed = [
	{ text: "Example 1", complete: true },
	{ text: "Example 2", complete: false }
];

module.exports = table;

The seed property is defined as an array of objects. Data seeding happens during the initialize() call, so you need to ensure you call initialize() if you want to seed data.

There is a new getIdentity() API

Azure App Service introduced a new app-specific authentication mechanism during the November 2015 update. The authentication gateway is now deprecated. Along with that change is a new getIdentity() call in the Node SDK that calls the authentication backend to retrieve the claims that you defined. You can use it like this in a table definition file:

table.access = 'authenticated';

table.read(function (context) {
    return context.user.getIdentity()
        .then(function (identity) {
            logger.info('table.read identity = ', identity);
            return context;
        })
        .then(function (context) {
            return context.execute();
        })
        .catch(function (error) {
            logger.error('Error in table.read: ', error);
        });
});

The getIdentity() call returns a Promise. Once the promise is resolved, the identity is available in the resolution. You can then adjust the context (just like the personal-table sample) with any information in the claims that you registered. Note that I’m chaining the promises together – when getIdentity() resolves, I get an identity. I then adjust the context and return the adjusted context, which is used asynchronously to execute the query and return the result of that.

You can disable access to table operations

Ever wanted to have a read-only table? How about a logging table that you can add to, but you can’t update/delete? All of those are possible by adjusting permissions on the table.

// Read-only table - turn off write operations
table.insert.access = 'disabled';
table.update.access = 'disabled';
table.delete.access = 'disabled';

You can also make it so that reads can be unauthenticated, but writes require authentication. The values for the access parameter are ‘anonymous’, ‘authenticated’ and ‘disabled’. Pick the right one for all operations and another one for an individual operation, if you like.

There are lots of samples

The team is dedicated to providing lots of sample variations to show off specific situations. You can still check out the canonical todo sample – that has been documented to a point where there is more documentation than code. However, there are lots of additional samples now.

CORS

CORS is a difficult subject right now as Azure App Service has just rolled out a change that delegates the handling of CORS to App Service. This means that you only care about CORS in your code when you are running the server locally. In this case, set skipVersionCheck to true in your azureMobile.js, like this:

module.exports = {
  skipVersionCheck: true,
  cors: {
    origins: [ '*' ]
  }
};

This will enable CORS for development purposes.

Need more information?

As yet another avenue for asking questions, you can log onto the Gitter channel and have a direct chat with the developers. That’s in addition to the Azure Forums, Stack Overflow and GitHub Issues. We aren’t starved of methods to discuss problems with the team!

Did I mention documentation? No? Well, here you are – the API documentation is published as well.