Updating Documents in DocumentDb

In my last few posts, I’ve been working on an Azure Mobile Apps replacement service. It will run in Azure Functions and use DocumentDb as a backing store. Neither of these requirements are possible in the Azure Mobile Apps server SDK today. Thus far, I’ve created a CRUD HTTP API, initialized the DocumentDb store and handled inserts. Today is all about fetching, but more importantly it is about replacing documents and handling conflict resolution.

The DocumentDb Driver

Before I get started with the code for the endpoint, I need to add some more functionality to my DocumentDb promisified driver. In the document.js file, I’ve added the following:

module.exports = {
    createDocument: function (client, collectionRef, docObject, callback) {
        client.createDocument(collectionRef._self, docObject, callback);
    },

    fetchDocument: function (client, collectionRef, docId, callback) {
        var querySpec = {
            query: 'SELECT * FROM root r WHERE r.id=@id',
            parameters: [{
                name: '@id',
                value: docId
            }]
        };

        client.queryDocuments(collectionRef._self, querySpec).current(callback);
    },

    readDocument: function (client, docLink, options, callback) {
        client.readDocument(docLink, options, callback);
    },

    replaceDocument: function(client, docLink, docObject, callback) {
        client.replaceDocument(docLink, docObject, callback);    
    }
};

My first attempt at reading a document used the readDocument() method. I would construct a docLink using the following:

var docLink = `${refs.table._self}${refs.table._docs}${docId}`;

However, this always resulted in a 400 Bad Request response from DocumentDb. The reason is likely that the _self link uses the shorted (and obfuscated) URI, whereas the Document Id I am using is a GUID and is not obfuscated. If you take a look at the response from DocumentDb, there is an id field and a _rid field. The _rid field is used in the document links. Thus, instead of using readDocument(), I’m using a queryDocuments() call on the driver to search for the Id. I’ve also promisified these calls in the normal manner using the Bluebird library.

Fetching a Record

The Azure Mobile Apps SDK allows me to GET /tables/todoitem/id – where id is the GUID. With the driver complete, I can do the following in the Azure Function table controller:

function getOneItem(req, res, id) {
    driver.fetchDocument(refs.client, refs.table, id)
    .then((document) => {
        if (typeof document === 'undefined')
            res.status(404).json({ 'error': 'Not Found' });
        else
            res.status(200).json(convertItem(document));
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

When doing this, I did notice that some semantics seem to have changed in the Azure Functions SDK. I can no longer use context.bindings.id and has to switch to using req.params.id. Aside from this small change in the router code, this code is relatively straight forward. I established the convertItem() and convertError() methods in my last article.

Replacing a Record

The more complex case is replacing a record. There is a little bit of logic around conflict resolution:

  • If there is an If-Match header, then ensure the version of the current record matches the If-Match header, otherwise return a 412 response.
  • If there is no If-match header, but the new record contains a version, return a 409 response.
  • Otherwise update the record

Because we want the version and updatedAt fields to be controlled as well, we need to ensure the new object does not contain those values when it is submitted to DocumentDb:

function replaceItem(req, res, id) {
    driver.fetchDocument(refs.client, refs.table, id)
    .then((document) => {
        if (typeof document === 'undefined') {
            res.status(404).json({ 'error': 'Not Found' });
            return;
        }

        var item = req.body, version = new Buffer(document._etag).toString('base64')
        if (item.id !== id) {
            res.status(400).json({ 'error': 'Id does not match' });
            return;
        }

        if (req.headers.hasOwnProperty('if-match') && req.header['if-match'] !== version) {
            res.status(412).json({ 'current': version, 'new': item.version, 'error': 'Version Mismatch' })
            return;
        }

        if (item.hasOwnProperty('version') && item.version !== version) {
            res.status(409).json({ 'current': version, 'new': item.version, 'error': 'Version Mismatch' });
            return;
        }

        // Delete the version and updatedAt fields from the doc before submitting
        delete item.updatedAt;
        delete item.version;
        driver.replaceDocument(refs.client, document._self, item)
        .then((updatedDocument) => {
            res.status(200).json(convertItem(updatedDocument));
            return;
        });
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

I’m using the same Base64 encoding for the etag in the current document to ensure I can do a proper match. I could get DocumentDb to do all this work for me – the options value in the driver replaceDocument() method allows me to specify an If-Match. However, to do that, I would need to still fetch the record (since I need the document link), so I may as well do the checks myself. This also keeps some load off the DocumentDb, which is helpful.

While this is almost there, there is one final item. If there is a conflict, the server version of the document should be returned. That means the 409 and 412 responses need to return convertItem(document) instead – a simple change.

Deleting a Record

Deleting a record does not delete a record. Azure Mobile Apps uses soft delete (whereby the deleted flag is set to true). This means that I need to use replaceDocument() again for deletions:

function deleteItem(req, res, id) {
    driver.fetchDocument(refs.client, refs.table, id)
    .then((document) => {
        if (typeof document === 'undefined') {
            res.status(404).json({ 'error': 'Not Found' });
            return;
        }

        var item = convertItem(document);
        delete item.updatedAt;
        delete item.version;
        item.deleted = true;
        driver.replaceDocument(refs.client, document._self, item)
        .then((updatedDocument) => {
            res.status(200).json(convertItem(updatedDocument));
            return;
        });
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

This brings up a point about the GetOneItem() method. It does not take into account the deleted flag. I need it to return 404 Not Found if the deleted flag is set:

function getOneItem(req, res, id) {
    driver.fetchDocument(refs.client, refs.table, id)
    .then((document) => {
        if (typeof document === 'undefined' || document.deleted === true)
            res.status(404).json({ 'error': 'Not Found' });
        else
            res.status(200).json(convertItem(document));
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

It’s a simple change, but important in getting the protocol right.

What’s left?

There is only one method I have not written yet, and it’s the biggest one of the set – the getAllItems() method. That’s because it deals with OData querying, which is no small task. I’ll be tackling that in my next article. Until then, get the current codebase at my GitHub repository.

Creating Documents in DocumentDB with Azure Functions HTTP API

Thus far in my story of implementing Azure Mobile Apps in a dynamic (consumption) plan of Azure Functions using DocumentDB, I’ve got the basic CRUD HTTP API stubbed out and the initialization of my DocumentDB collection done. It’s now time to work on the actual endpoints that my Azure Mobile Apps SDK will call. There are five methods to implement:

  • Insert
  • Update / Replace
  • Delete
  • Fetch a single record
  • Search

I’m going to do these in the order above. Before I do that, I need to take a look at what DocumentDB provides me. Azure Mobile Apps requires five fields to work properly:

  • id – a string (generally a GUID).
  • createdAt – the date the record was created, in ISO-8601 format.
  • updatedAt – the date the record was updated, in ISO-8601 format.
  • deleted – a boolean, if the record is deleted.
  • version – an opaque string for conflict resolution.

DocumentDB provides some of this for us:

  • id – a string (generally a GUID).
  • _ts – a POSIX / unix timestamp of the number of seconds since the epoch since the record was last updated.
  • _etag – a checksum / version identifier.

When we create a record, we need to convert the document that DocumentDB returns to us into the format that Azure Mobile Apps provides. I use the following routine:

/**
 * Given an item from DocumentDB, convert it into something that the service can used
 * @param {object} item the original item
 * @return {object} the new item
 */
function convertItem(item) {
    if (item.hasOwnProperty('_ts')) {
        item.updatedAt = moment.unix(item._ts).toISOString();
        delete item._ts;
    } else {
        throw new Error('Invalid item - no _ts field');
    }

    if (item.hasOwnProperty('_etag')) {
        item.version = new Buffer(item._etag).toString('base64');
        delete item._etag;
    } else {
        throw new Error('Invalid item - no _etag field');
    }

    // Delete all the known fields from documentdb
    if (item.hasOwnProperty('_rid')) delete item._rid;
    if (item.hasOwnProperty('_self')) delete item._self;
    if (item.hasOwnProperty('_attachments')) delete item._attachments;

    return item;
}

I’m using the moment library to do date/time manipulation. This is a very solid library and well worth learning about. In addition to the convertItem() method, I also need something to convert the error values that come back from DocumentDB. They are not nicely formed, so some massaging is in order:

/**
 * Convert a DocumentDB error into something intelligible
 * @param {Error} error the error object
 * @return {object} the intelligible error object
 */
function convertError(error) {
    var body = JSON.parse(error.body);
    if (body.hasOwnProperty("message")) {
        var msg = body.message.replace(/^Message:\s+/, '').split(/\r\n/);
        body.errors = JSON.parse(msg[0]).Errors;

        var addl = msg[1].split(/,\s*/);
        addl.forEach((t) => {
            var tt = t.split(/:\s*/);
            tt[0] = tt[0].replace(/\s/, '').toLowerCase();
            body[tt[0]] = tt[1];
        });

        delete body.message;
    }

    return body;
}

I had to work through the error object several times experimenting with the actual response to come up with this routine. This seems like the right code by experimentation. Whether it holds up during normal usage remains to be seen.

I’ve already written the createDocument() method in the DocumentDB driver:

module.exports = {
    createDocument: function (client, collectionRef, docObject, callback) {
        client.createDocument(collectionRef._self, docObject, callback);
    }
};

This is then promisifyed using the bluebird promise library. With this work done, my code for inserts becomes very simple:

function insertItem(req, res) {
    var item = req.body;

    item.createdAt = moment().toISOString();
    if (!item.hasOwnProperty('deleted')) item.deleted = false;

    driver.createDocument(refs.client, refs.table, item)
    .then((document) => {
        res.status(201).json(convertItem(document));
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

The item that we need to insert comes in on the body. We need to add the createdAt field and the deleted field (if it isn’t already set). Since this is an insert, we call createDocument() in the driver. If it succeeds, we return a 201 Created response with the new document (converted to the Azure Mobile Apps specification). If not, we return the error from DocumentDB together with the formatted object.

We can test inserts with Postman. For example, here is a successful insert:

insert-1

DocumentDB creates the id for me if it doesn’t exist. I convert the _ts and _etag fields to something more usable by the Azure Mobile Apps SDK on the way back to the client. If I copy the created object and push it again, I will get a conflict:

insert-2

Notice how DocumentDB does all the work for me? All I need to do is some adjustments on the output to get my insert operation working. I can use the Document Browser within the Azure Portal to look at the actual records.

In the next post, I’m going to move onto Update, Delete and Fetch all in one go.