Azure App Service Authentication in an ASP.NET Core Application

In my prior posts, I’ve covered:

This is a great start. Next on my list is authentication. How do I handle both a web-based authentication and a mobile-based authentication pattern within a single authentication pattern. ASP.NET Core provides a modular authentication system. (I’m sensing a theme here – everything is modular!) So, in this post, I’m going to cover the basics of authentication and then cover how Azure App Service Authentication works (although Chris Gillum does a much better job than I do and you should read his blog for all the Easy Auth articles) and then introduce an extension to the authentication library that implements Azure App Service Authentication.

Authentication Basics

To implement authentication in ASP.NET Core, you place the following in the ConfigureServices() method of your Startup.cs:

services.AddAuthentication();

Here, services is the IServiceCollection that is passed in as a parameter to the ConfigureServices() method. In addition, you need to add a UseXXX() method to bring in the extension method within the Configure() method. Here is an example:

app.UseJwtBearerAuthentication(new JwtBearerOptions
{
    Authority = Configuration["JWT:Authority"],
    Audience = Configuration["JWT:Audience"]
});

Once that is done, your MVC controllers or methods can be decorated with the usual [Authorize] decorator to require authentication. Finally, you need to add the Microsoft.AspNetCore.Authentication NuGet package to your project to bring in the authentication framework.

In my project, I’ve added the services.AddAuthentication() method to ConfigureServices() and added an [Authorize] tag to my /Home/Configuration controller method. This means that the configuration viewer that I used last time now needs authentication:

auth-failed

That 401 HTTP status code for Configuration loading is indicative of a failed authentication. 401 is “Authorization Failed”. This is completely expected because we have not configured an authentication provider yet.

How App Service Authentication Works

Working with Azure App Service Authentication is relatively easy. A JWT-based token is submitted either as a cookie or as the X-ZUMO-AUTH header. The information necessary to decode that token is provided in environment variables:

  • WEBSITE_AUTH_ENABLED is True if the Authentication system is loaded
  • WEBSITE_AUTH_SIGNING_KEY is the passcode used for signing the key
  • WEBSITE_AUTH_ALLOWED_AUDIENCES is the list of allowed audiences for the JWT

If WEBSITE_AUTH_ENABLED is set to True, decode the X-ZUMO-AUTH header to see if the user is valid. If the user is valid, then do a HTTP GET of {issuer}/.auth/me with the X-ZUMO-AUTH header passed through to get a JSON blob with the claims. If the token is expired or non-existent, then don’t authenticate the user.

This has an issue in that you have to do another HTTP call to get the claims. This is a small overhead and the team is working to fix this for “out of process” services. In process services, such as PHP and ASP.NET, have access to server variables. The JSON blob that is returned by calling the /.auth/me endpoint is presented as a server variable so it doesn’t need to be fetched. ASP.NET Core applications are “out of process” so we can’t use this mechanism.

Configuring the ASP.NET Core Application

In the Configure() method of the Startup.cs file, I need to do something like the following:

            app.UseAzureAppServiceAuthentication(new AzureAppServiceAuthenticationOptions
            {
                SigningKey = Configuration["AzureAppService:Auth:SigningKey"],
                AllowedAudiences = new[] { $"https://{Configuration["AzureAppService:Website:HOST_NAME"]}/" },
                AllowedIssuers = new[] { $"https://{Configuration["AzureAppService:Website:HOST_NAME"]}/" }
            });

This is just pseudo-code right now because neither the UserAzureAppServiceAuthentication() method nor the AzureAppServiceAuthenticationOptions class exist. Fortunately, there are many templates for a successful implementation of authentication. (Side note: I love open source) The closest one to mine is the JwtBearer authentication implementation. I’m not going to show off the full implementation – you can go check it out yourself. However, the important work is done in the AzureAppServiceAuthenticationHandler file.

The basic premise is this:

  1. If we don’t have an authentication source (a token), then return AuthenticateResult.Skip().
  2. If we have an authentication source, but it’s not valid, return AuthenticateResult.Fail().
  3. If we have a valid authentication source, decode it, create an AuthenticationTicket and then return AuthenticateResult.Success().

Detecting the authentication source means digging into the Request.Headers[] collection to see if there is an appropriate header. The version I have created supports both the X-ZUMO-AUTH and Authorization headers (for future compatibility):

            // Grab the X-ZUMO-AUTH token if it is available
            // If not, then try the Authorization Bearer token
            string token = Request.Headers["X-ZUMO-AUTH"];
            if (string.IsNullOrEmpty(token))
            {
                string authorization = Request.Headers["Authorization"];
                if (string.IsNullOrEmpty(authorization))
                {
                    return AuthenticateResult.Skip();
                }
                if (authorization.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase))
                {
                    token = authorization.Substring("Bearer ".Length).Trim();
                    if (string.IsNullOrEmpty(token))
                    {
                        return AuthenticateResult.Skip();
                    }
                }
            }
            Logger.LogDebug($"Obtained Authorization Token = {token}");

The next step is to validate the token and decode the result. If the service is running inside of Azure App Service, then the validation has been done for me and I only need to decode the token. If I am running locally, then I should validate the token. The signing key for the JWT is encoded in the WEBSITE_AUTH_SIGNING_KEY environment variable. Theoretically, the WEBSITE_AUTH_SIGNING_KEY can be hex encoded or base-64 encoded. It will be hex-encoded the majority of the time. Using the configuration provider from the last post, this appears as the AzureAppService:Auth:SigningKey configuration variables and I can place that into the options for the authentication provider during the Configure() method of Startup.cs.

So, what’s the code for validating and decoding the token? It looks like this:

            // Convert the signing key we have to something we can use
            var signingKeys = new List<SecurityKey>();
            // If the signingKey is the signature
            signingKeys.Add(new SymmetricSecurityKey(Encoding.UTF8.GetBytes(Options.SigningKey)));
            // If it's base-64 encoded
            try
            {
                signingKeys.Add(new SymmetricSecurityKey(Convert.FromBase64String(Options.SigningKey)));
            } catch (FormatException) { /* The key was not base 64 */ }
            // If it's hex encoded, then decode the hex and add it
            try
            {
                if (Options.SigningKey.Length % 2 == 0)
                {
                    signingKeys.Add(new SymmetricSecurityKey(
                        Enumerable.Range(0, Options.SigningKey.Length)
                                  .Where(x => x % 2 == 0)
                                  .Select(x => Convert.ToByte(Options.SigningKey.Substring(x, 2), 16))
                                  .ToArray()
                    ));
                }
            } catch (Exception) {  /* The key was not hex-encoded */ }

            // validation parameters
            var websiteAuthEnabled = Environment.GetEnvironmentVariable("WEBSITE_AUTH_ENABLED");
            var inAzureAppService = (websiteAuthEnabled != null && websiteAuthEnabled.Equals("True", StringComparison.OrdinalIgnoreCase));
            var tokenValidationParameters = new TokenValidationParameters
            {
                // The signature must have been created by the signing key
                ValidateIssuerSigningKey = !inAzureAppService,
                IssuerSigningKeys = signingKeys,

                // The Issuer (iss) claim must match
                ValidateIssuer = true,
                ValidIssuers = Options.AllowedIssuers,

                // The Audience (aud) claim must match
                ValidateAudience = true,
                ValidAudiences = Options.AllowedAudiences,

                // Validate the token expiry
                ValidateLifetime = true,

                // If you want to allow clock drift, set that here
                ClockSkew = TimeSpan.FromSeconds(60)
            };

            // validate the token we received
            var tokenHandler = new JwtSecurityTokenHandler();
            SecurityToken validatedToken;
            ClaimsPrincipal principal;
            try
            {
                principal = tokenHandler.ValidateToken(token, tokenValidationParameters, out validatedToken);
            }
            catch (Exception ex)
            {
                Logger.LogError(101, ex, "Cannot validate JWT");
                return AuthenticateResult.Fail(ex);
            }

This only gives us a subset of the claims though. We want to swap out the principal (in this case) with the results of the call to /.auth/me that gives us the actual claims:

            try
            {
                client.BaseAddress = new Uri(validatedToken.Issuer);
                client.DefaultRequestHeaders.Clear();
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                client.DefaultRequestHeaders.Add("X-ZUMO-AUTH", token);

                HttpResponseMessage response = await client.GetAsync("/.auth/me");
                if (response.IsSuccessStatusCode)
                {
                    var jsonContent = await response.Content.ReadAsStringAsync();
                    var userRecord = JsonConvert.DeserializeObject<List<AzureAppServiceClaims>>(jsonContent).First();

                    // Create a new ClaimsPrincipal based on the results of /.auth/me
                    List<Claim> claims = new List<Claim>();
                    foreach (var claim in userRecord.UserClaims)
                    {
                        claims.Add(new Claim(claim.Type, claim.Value));
                    }
                    claims.Add(new Claim("x-auth-provider-name", userRecord.ProviderName));
                    claims.Add(new Claim("x-auth-provider-token", userRecord.IdToken));
                    claims.Add(new Claim("x-user-id", userRecord.UserId));
                    var identity = new GenericIdentity(principal.Claims.Where(x => x.Type.Equals("stable_sid")).First().Value, Options.AuthenticationScheme);
                    identity.AddClaims(claims);
                    principal = new ClaimsPrincipal(identity);
                }
                else if (response.StatusCode == System.Net.HttpStatusCode.Unauthorized)
                {
                    return AuthenticateResult.Fail("/.auth/me says you are unauthorized");
                }
                else
                {
                    Logger.LogWarning($"/.auth/me returned status = {response.StatusCode} - skipping user claims population");
                }
            }
            catch (Exception ex)
            {
                Logger.LogWarning($"Unable to get /.auth/me user claims - skipping (ex = {ex.GetType().FullName}, msg = {ex.Message})");
            }

I can skip this phase if I want to by setting an option. The result of this code is a new identity that has all the claims and a “name” that is the same as the stable_sid value from the original token. If the /.auth/me endpoint says the token is bad, I am returning a failed authentication. Otherwise, I’m skipping the response.

I could use the UserId field from the user record. However, that is not guaranteed to be stable as users can and do change the email address on their social accounts. I may update this class in the future to use stable_sid by default but allow the developer to change it if they desire.

The final step is to create an AuthenticationTicket and return success:

            // Generate a new authentication ticket and return success
            var ticket = new AuthenticationTicket(
                principal, new AuthenticationProperties(),
                Options.AuthenticationScheme);

            return AuthenticateResult.Success(ticket);

You can check out the code on my GitHub Repository at tag p5.

Wrap Up

There is still a little bit to do. Right now, the authentication filter returns a 401 Unauthorized if you try to hit an unauthorized page. This isn’t useful for a web page, but is completely suitable for a web API. It is thus “good enough” for Azure Mobile Apps. If you are using this functionality in an MVC application, then it is likely you want to set up some sort of authorization redirect to a login provider.

In the next post, I’m going to start on the work for Azure Mobile Apps.

Configuring ASP.NET Core Applications in Azure App Service

The .NET Core framework has a nice robust configuration framework. You will normally see the following (or something akin to it) in the constructor of the Startup.cs file:

        public Startup(IHostingEnvironment env)
        {
            var builder = new ConfigurationBuilder()
                .SetBasePath(env.ContentRootPath)
                .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
                .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
                .AddEnvironmentVariables();
            Configuration = builder.Build();
        }

This version gets its settings from JSON files and then overrides these settings with things from environment variables. It seems ideal for Azure App Service because all the settings – from the app settings to the connection strings – are provided as environment variables. Unfortunately, they don’t necessarily get placed in the right place in the configuration. For example, Azure App Service has a Data Connections facility to define the various other resources that the app service uses. You can reach it from the menu of your App Service via the Data Connections menu. The common thing to do for Azure Mobile Apps is to define a connection to a SQL Azure instance called MS_TableConnectionString. However, when this is turned into an environment variable, the environment variable is called SQLAZURECONNSTR_MS_TableConnectionString, which is hardly obvious.

Fortunately, the ASP.NET Core configuration library is extensible. What I am going to do is develop an extension to the configuration library for handling these data connections. When developing locally, you will be able to add an appsettings.Development.json file with the following contents:

{
    "ConnectionStrings": {
        "MS_TableConnectionString": "my-connection-string"
    },
    "Data": {
        "MS_TableConnectionString": {
            "Type": "SQLAZURE",
            "ConnectionString": "my-connection-string"
        }
    }
}

When in production, this configuration is produced by the Azure App Service. When in development (and running locally), you will want to produce this JSON file yourself. Alternatively, you can adjust the launchsettings.json file to add the appropriate environment variable for local running.

To start, here is what our Startup.cs constructor will look like now:

        public Startup(IHostingEnvironment env)
        {
            var builder = new ConfigurationBuilder()
                .SetBasePath(env.ContentRootPath)
                .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
                .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
                .AddAzureAppServiceDataConnections()
                .AddEnvironmentVariables();
            Configuration = builder.Build();
        }

Configuration Builder Extensions

In order to wire our configuration provider into the configuration framework, I need to provide an extension method to the configuration builder. This is located in Extensions\AzureAppServiceConfigurationBuilderExtensions.cs:

using Microsoft.Extensions.Configuration;

namespace ExampleServer.Extensions
{
    public static class AzureAppServiceConfigurationBuilderExtensions
    {
        public static IConfigurationBuilder AddAzureAppServiceDataConnections(this IConfigurationBuilder builder)
        {
            return builder.Add(new AzureAppServiceDataConnectionsSource());
        }
    }
}

This is a fairly standard method of hooking extensions into extensible builder-type objects.

The Configuration Source and Provider

To actually provide the configuration source, I need to write two classes – a source, which is relatively simple, and a provider, which does most of the actual work. Fortunately, I can extend other classes within the configuration framework to ease the code I have to write. Let’s start with the easy one – the source. This is the object that is added to the configuration builder (located in Extensions\AzureAppServiceDataConnectionsSource.cs:

using Microsoft.Extensions.Configuration;
using System;

namespace ExampleServer.Extensions
{
    public class AzureAppServiceDataConnectionsSource : IConfigurationSource
    {
        public IConfigurationProvider Build(IConfigurationBuilder builder)
        {
            return new AzureAppServiceDataConnectionsProvider(Environment.GetEnvironmentVariables());
        }
    }
}

I’m passing in the current environment into my provider because I want to be able to mock an environment later on for testing purposes. The configuration source (which is linked into the configuration builder) returns a configuration provider. The configuration provider is where the work is done (located in Extensions\AzureAppServiceDataConnectionsProvider.cs):

using Microsoft.Extensions.Configuration;
using System.Collections;
using System.Text.RegularExpressions;

namespace ExampleServer.Extensions
{
    internal class AzureAppServiceDataConnectionsProvider : ConfigurationProvider
    {
        /// <summary>
        /// The environment (key-value pairs of strings) that we are using to generate the configuration
        /// </summary>
        private IDictionary environment;

        /// <summary>
        /// The regular expression used to match the key in the environment for Data Connections.
        /// </summary>
        private Regex MagicRegex = new Regex(@"^([A-Z]+)CONNSTR_(.+)$");

        public AzureAppServiceDataConnectionsProvider(IDictionary environment)
        {
            this.environment = environment;
        }

        /// <summary>
        /// Loads the appropriate settings into the configuration.  The Data object is provided for us
        /// by the ConfigurationProvider
        /// </summary>
        /// <seealso cref="Microsoft.Extensions.Configuration.ConfigurationProvider"/>
        public override void Load()
        {
            foreach (string key in environment.Keys)
            {
                Match m = MagicRegex.Match(key);
                if (m.Success)
                {
                    var conntype = m.Groups[1].Value;
                    var connname = m.Groups[2].Value;
                    var connstr = environment[key] as string;

                    Data[$"Data:{connname}:Type"] = conntype;
                    Data[$"Data:{connname}:ConnectionString"] = connstr;
                    Data[$"ConnectionStrings:{connname}"] = connstr;
                }
            }
        }
    }
}

The bulk of the work is done within the Load() method. This cycles through each environment variable. If it matches the pattern
we are looking for, it extracts the pieces of data we need and puts them in the Data object. The Data object is provided by the
ConfigurationProvider and all the other lifecycle methods are handled for me by the ConfigurationProvider class.

Integrating with Dependency Injection

I want to provide a singleton service to my application that provides access to the data configuration. This service will be injected
into any code I want via dependency injection. To do this, I need an interface:

using System.Collections.Generic;

namespace ExampleServer.Services
{
    public interface IDataConfiguration
    {
        IEnumerable<DataConnectionSettings> GetDataConnections();
    }
}

This refers to a model (and I produce a list of these models later):

namespace ExampleServer.Services
{
    public class DataConnectionSettings
    {
        public string Name { get; set; }
        public string Type { get; set; }
        public string ConnectionString { get; set; }
    }
}

I also need a concrete implementation of the IDataConfiguration interface to act as the service:

using Microsoft.Extensions.Configuration;
using System.Collections.Generic;

namespace ExampleServer.Services
{
    public class DataConfiguration : IDataConfiguration
    {
        List<DataConnectionSettings> providerSettings;

        public DataConfiguration(IConfiguration configuration)
        {
            providerSettings = new List<DataConnectionSettings>();
            foreach (var section in configuration.GetChildren())
            {
                providerSettings.Add(new DataConnectionSettings
                {
                    Name = section.Key,
                    Type = section["Type"],
                    ConnectionString = section["ConnectionString"]
                });
            }
        }

        public IEnumerable<DataConnectionSettings> GetDataConnections()
        {
            return providerSettings;
        }
    }
}

Now that I have defined the interface and concrete implementation of the service, I can wire it into the dependency injection system in the ConfigureServices() method within Startup.cs:

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddSingleton<IDataConfiguration>(new DataConfiguration(Configuration.GetSection("Data")));

            // Add framework services.
            services.AddMvc();
        }

I can now write my test page. The main work is in the HomeController.cs:

        public IActionResult DataSettings([FromServices] IDataConfiguration dataConfiguration)
        {
            ViewBag.Data = dataConfiguration.GetDataConnections();
            return View();
        }

The parameter to the DataSettings method will be provided from my service via dependency injection. Now all I have to do is write the view for it:

<h1>Data Settings</h1>

<div class="row">
    <table class="table table-striped">
        <tr>
            <th>Provider Name</th>
            <th>Type</th>
            <th>Connection String</th>
        </tr>
        <tbody>
            @foreach (var item in ViewBag.Data)
            {
                <tr>
                    <td>@item.Name</td>
                    <td>@item.Type</td>
                    <td>@item.ConnectionString</td>
                </tr>
            }
        </tbody>
    </table>
</div>

Publish this to your Azure App Service. Don’t forget to link a SQL Azure instance via the Data Connections menu. Then browse to https://yoursite.azurewebsites.net/Home/DataSettings – you should see your SQL Azure instance listed on the page.

Now for the good part…

ASP.NET Core configuration already does this for you. I didn’t find this out until AFTER I had written all this code. You can just add the .UseEnvironmentVariables() to your configuration builder and the rename comes along for free. Specifically:

  • ConfigurationStrings:MS_TableConnectionString will point to your connection string
  • ConfigurationStrings:MS_TableConnectionString_Provider will point to the provider class name

So you don’t really need to do anything at all. However, this is a great reference guide for me for producing the next configuration provider as the sample code is really easy to follow.

The code for this blog post is in my GitHub Repository at tag p3. In the next post, I’ll tackle authentication by linking Azure App Service Authentication with ASP.NET Core Identity.

Running ASP.NET Core applications in Azure App Service

One of the things I get asked about semi-regularly is when Azure Mobile Apps is going to support .NET Core. It’s a logical progression for most people and many ASP.NET developers are planning future web sites to run on ASP.NET Core. Also, the ASP.NET Core programming model makes a lot more sense (at least to me) than the older ASP.NET applications. Finally, we have an issue open on the subject. So, what is holding us back? Well, there are a bunch of things. Some have been solved already and some need a lot of work. In the coming weeks, I’m going to be writing about the various pieces that need to be in place before we can say “Azure Mobile Apps is there”.

Of course, if you want a mobile backend, you can always hop over to Visual Studio Mobile Center. This provides a mobile backend for you without having to write any code. (Full disclosure: I’m now a program manager on that team, so I may be slightly biased). However, if you are thinking ASP.NET Core, then you likely want to write the code.

Let’s get started with something that does exist. How does one run ASP.NET Core applications on Azure App Service? Well, there are two methods. The first involves uploading your application to Azure App Service via the Visual Studio Publish… dialog or via Continuous Integration from GitHub, Visual Studio Team Services or even Dropbox. It’s a relatively easy method and one I would recommend. There is a gotcha, which I’ll discuss below.

The second method uses a Docker container to house the code that is then deployed onto a Linux App Service. This is still in preview (as of writing), so I can’t recommend this for production workloads.

Create a New ASP.NET Core Application

Let’s say you opened up Visual Studio 2017 (RC right now) and created a brand new ASP.NET Core MVC application – the basis for my research here.

  • Open up Visual Studio 2017 RC.
  • Select File > New > Project…
  • Select the ASP.NET Core Web Application (.NET Core).
    • Fill in an appropriate name for the solution and project, just as normal.
    • Click OK to create the project.
  • Select ASP.NET Core 1.1 from the framework drop-down (it will say ASP.NET Core 1.0 initially)
  • Select Web Application in the ASP.NET Core 1.1 Templates selection.
  • Click OK.

I called my solution netcore-server and the project ExampleServer. At this point, Visual Studio will go off and create a project for you. You can see what it creates easily enough, but I’ve checked it into my GitHub repository at tag p0.

I’m not going to cover ASP.NET Core programming too much in this series. You can read the definitive guide on their documentation site, and I would recommend you start by understanding ASP.NET Core programming before getting into the changes here.

Go ahead and run the service (either as a Kestrel service or an IIS Express service – it works with both). This is just to make sure that you have a working site.

Add Logging to your App

Logging is one of those central things that is needed in any application. There are so many things you can’t do (including diagnose issues) if you don’t have appropriate logging. Fortunately, ASP.NET Core has logging built-in. Let’s add some to the Controllers\HomeController.cs file:

using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;

namespace ExampleServer.Controllers
{
    public class HomeController : Controller
    {
        private ILogger logger;

        public HomeController(ILoggerFactory loggerFactory)
        {
            logger = loggerFactory.CreateLogger(this.GetType().FullName);
        }

        public IActionResult Index()
        {
            logger.LogInformation("In Index of the HomeController", null);
            return View();
        }
        // Rest of the file here

I’ve added the logger factory via dependency injection, then logged a message whenever the Index file is served in the home controller. If you run this version of the code (available on the GitHub respository at tag p1), you will see the following in your Visual Studio output window:

20170216-01

It’s swamped by the Application Insights data, but you can clearly see the informational message there.

Deploy your App to Azure App Service

Publishing to Azure App Service is relatively simple – right-click on the project and select Publish… to kick off the process. The layout of the windows has changed from Visual Studio 2015, but it’s the same process. You can create a new App Service or use an existing one. Once you have answered all the questions, your site will be published. Eventually, your site will be displayed in your web browser.

Turn on Diagnostic Logging

  • Click View > Server Explorer to add the server explorer to your work space.
  • Expand the Azure node, the App Service node, and finally your resource group node.
  • Right-click the app service and select View Settings
  • Turn on logging and set the logging level to verbose:

20170216-02

  • Click Save to save the settings (the site will restart).
  • Right-click the app service in the server explorer again and this time select View Streaming Logs
  • Wait until you see that you are connected to the log streaming service (in the Output window)

Now refresh your browser so that it reloads the index page again. Note how you see the access logs (which files have been requested) but the log message we put into the code is not there.

The Problem and Solution

The problem is, hopefully, obvious. ASP.NET Core does not by default feed logs to Azure App Service. We need to enable that feature in the .NET Core host. We do this in the Program.cs file:

using System.IO;
using Microsoft.AspNetCore.Hosting;

namespace ExampleServer
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var host = new WebHostBuilder()
                .UseKestrel()
                .UseContentRoot(Directory.GetCurrentDirectory())
                .UseIISIntegration()
                .UseStartup<Startup>()
                .UseApplicationInsights()
                .UseAzureAppServices()
                .Build();

            host.Run();
        }
    }
}

You will also need to add the Microsoft.AspNetCore.AzureAppServicesIntegration package from NuGet for this to work. Once you have done this change, you can deploy this and watch the logs again:

20170216-03

If you have followed the instructions, you will need to switch the Output window back to the Azure logs. The output window will have been switched to Build during the publish process.

Adjusting the WebHostBuilder for the environment

It’s likely that you won’t want Application Insights and Azure App Services logging except when you are running on Azure App Service. There are a number of environment variables that Azure App Service uses and you can leverage these as well. My favorites are REGION_NAME (which indicates which Azure region your service is running in) and WEBSITE_OWNER_NAME (which is a combination of a bunch of things). You can test for these and adjust the pipeline accordingly:

using Microsoft.AspNetCore.Hosting;
using System;
using System.IO;

namespace ExampleServer
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var hostBuilder = new WebHostBuilder()
                .UseKestrel()
                .UseContentRoot(Directory.GetCurrentDirectory())
                .UseIISIntegration()
                .UseStartup<Startup>()
                .UseApplicationInsights();

            var regionName = Environment.GetEnvironmentVariable("REGION_NAME");
            if (regionName != null)
            {
                hostBuilder.UseAzureAppServices();
            }
                
            var host = hostBuilder.Build();

            host.Run();
        }
    }
}

You can download this code at my GitHub repository at tag p2.

Integrating OData and DocumentDb with Azure Functions

This is the finale in a series of posts that aimed to provide an alternative to Azure Mobile Apps based on Azure Functions and DocumentDb. I started with discussing how to run a CRUD HTTP API, then moved onto DocumentDb, handled inserts and replacements. Now it’s time to fetch data. Azure Mobile Apps uses a modified OData v3 query string to perform the offline sync and online querying of data. This is mostly because ASP.NET (which was the basis for the original service) has a nice OData library for it. OData is painful to use in our context, however. Firstly, there are some necessary renames – the updatedAt field is actually the DocumentDb timestamp, for example. The other thing is that there is no ready made library for turning an OData query string into a DocumentDb SQL statement. So I don’t have an “easy” way of fulfilling the requirement.

Fortunately, the Azure Mobile Apps Node SDK has split off a couple of libraries for more general use. The first is azure-query-js. This is a library for converting between a set of OData query parameters and an internal query structure. The second is azure-odata-sql, which is for turning a normalized OData query into SQL, based on Microsoft SQL or SQLite syntax. Neither of these libraries is particularly well documented, but they are relatively easy to use based on the examples used within the Azure Mobile Apps SDKs. We are going to need to modify the azure-odata-sql library to generate appropriate SQL statements for DocumentDB, so I’ve copied the source to the library into my project (in the directory odata-sql). My first stab at the getAllItems() method looks like this:

var OData = require('azure-query-js').Query.Providers.OData;
var formatSql = require('../odata-sql').format;

function getAllItems(req, res) {
    // DoumentDB doesn't support SKIP yet, so we can't do TOP either without some problems
    var query = OData.fromOData(
        settings.table,
        req.query.$filter,
        req.query.$orderby,
        undefined, //parseInt(req.query.$skip),
        undefined, //parseInt(req.query.$top),
        req.query.$select,
        req.query.$inlinecount === 'allpages',
        !!req.query.__includeDeleted);

    var sql = formatSql(OData.toOData(query), {
        containerName: settings.table,
        flavor: 'documentdb'
    });
    
    res.status(200).json({ query: req.query, sql: sql, message: 'getAll' });
}

As noted here, DocumentDB hasn’t added full support for SKIP/TOP statements, so we can’t use those elements. Once the support is available within DocumentDB, I just need to include that support in the odata-sql library and change the two parmeters to the fromOData() call.

So, what does this do? Well, first, it converts the request from the browser (or client SDK) from the jumble of valid OData query params into a Query object. That Query object is actually a set of functions to do the parsing. Then we use the toOData() method (from the azure-query-js library) to convert that Query object into a normalized OData query. Finally, we use a custom SQL formatter (based on the azure-odata-sql) library to convert it to a SQL statement. If you run this, you should get something like the following out of it:

getall-1

I can now see the SQL statements being generated. The only problem is that they are not actually valid SQL statements for DocumentDB. They are actually perfectly valid for Microsoft SQL Server or SQL Azure. We need to adjust the odata-sql library for our needs. There are a couple of things needed here. Our first requirement is around the updatedAt field. This is not updatedAt in DocumentDB – it’s _ts, and it’s a number. We can do this using regular expressions like this:

if (req.query.$filter) {
    while (/updatedAt [a-z]+ '[^']+'/.test(req.query.$filter)) {
        var re = new RegExp(/updatedAt ([a-z]+) '([^']+)'/);
        var results = re.exec(req.query.$filter);
        var newDate = moment(results[2]).unix();
        var newString = `_ts ${results[1]} ${newDate}`;
        req.query.$filter = req.query.$filter.replace(results[0], newString);
    }
}

I could have probably shrunk this code somewhat, but it’s clear as to what is going on. We loop around the filter while there is still an updatedAt clause, convert the date, then replace the old string with the new string. We need to do similar things with the $select and $orderby clauses as well – left out because I’m trying to make this simple.

In terms of the odata-sql library, most of what we want is in the helpers.js library. Specifically, in the case of DocumentDB, we don’t need the square brackets. That means the formatMember() and formatTableName() methods must be adjusted to compensate.

I found it easier to step through the code by writing a small test program to test this logic out. You can find it in todoitem\test.js. With Visual Studio Code, you can set breakpoints, watch variables and do all the normal debugging things to really understand where the code is going and what it is doing.

Now that the SQL looks good, I need to execute the SQL commands. I’ve got a version of queryDocuments() in the driver:

    queryDocuments: function (client, collectionRef, query, callback) {
        client.queryDocuments(collectionRef._self, query).toArray(callback);
    },

This is then used in the HTTP trigger getAllItems() method. I’ve included the whole method here for you:

function getAllItems(req, res) {
    // Adjust the query parameters for DocumentDB
    if (req.query.$filter) {
        while (/updatedAt [a-z]+ '[^']+'/.test(req.query.$filter)) {
            var re = new RegExp(/updatedAt ([a-z]+) '([^']+)'/);
            var results = re.exec(req.query.$filter);
            var newDate = moment(results[2]).unix();
            var newString = `_ts ${results[1]} ${newDate}`;
            req.query.$filter = req.query.$filter.replace(results[0], newString);
        }
    }
    // Remove the updatedAt from the request
    if (req.query.$select) {
        req.query.$select = req.query.$select.replace(/,{0,1}updatedAt/g, '');
    }

    // DoumentDB doesn't support SKIP yet, so we can't do TOP either
    var query = OData.fromOData(
        settings.table,
        req.query.$filter,
        req.query.$orderby,
        undefined, //parseInt(req.query.$skip),
        undefined, //parseInt(req.query.$top),
        req.query.$select,
        req.query.$inlinecount === 'allpages',
        !!req.query.__includeDeleted);

    var sql = formatSql(OData.toOData(query), {
        containerName: settings.table,
        flavor: 'documentdb'
    });

    // Fix up the object so that the SQL object matches what DocumentDB expects
    sql[0].query = sql[0].sql;
    sql[0].parameters.forEach((value, index) => {
        sql[0].parameters[index].name = `@${value.name}`;
    });

    // Execute the query
    console.log(JSON.stringify(sql[0], null, 2));
    driver.queryDocuments(refs.client, refs.table, sql[0])
    .then((documents) => {
        documents.forEach((value, index) => {
            documents[index] = convertItem(value);
        });

        if (sql.length == 2) {
            // We requested $inlinecount == allpages.  This means we have
            // to adjust the output to include a count/results field.  It's
            // used for paging, which DocumentDB doesn't support yet.  As
            // a result, this is a hacky way of doing this.
            res.status(200).json({
                results: documents,
                count: documents.length
            });
        } else {
            res.status(200).json(documents);
        }
    })
    .catch((error) => {
        res.status(400).json(error);
    });
}

Wrapping Up

So, there you have it. A version of the Azure Mobile Apps service written with DocumentDB and executing in dynamic compute on Azure Functions.

Of course, I wouldn’t actually use this code in production. Firstly, I have not written any integration tests on this, and there are a bunch of corner cases that I would definitely want to test. DocumentDB doesn’t have good paging support yet, so you are getting all records all the time. I also haven’t looked at all the OData methods that can be converted into SQL statement to ensure DocumentDB support. Finally, and this is a biggie, the service has a “cold start” time. It’s not very much, but it can be significant. In the case of a dedicated service, you spend that cold start time once. In the case of a dynamic compute Azure Function, you can spend that time continually. This isn’t actually a problem with DocumentDB, since I am mostly passing through the REST calls (adjusted). However, it can become a problem when using other sources. One final note is that I keep all the records in memory – this can drive up the memory requirements (and hence cost) of the Azure Function on a per-execution basis.

Until next time, you can find the source code for this project on my GitHub repository.

Updating Documents in DocumentDb

In my last few posts, I’ve been working on an Azure Mobile Apps replacement service. It will run in Azure Functions and use DocumentDb as a backing store. Neither of these requirements are possible in the Azure Mobile Apps server SDK today. Thus far, I’ve created a CRUD HTTP API, initialized the DocumentDb store and handled inserts. Today is all about fetching, but more importantly it is about replacing documents and handling conflict resolution.

The DocumentDb Driver

Before I get started with the code for the endpoint, I need to add some more functionality to my DocumentDb promisified driver. In the document.js file, I’ve added the following:

module.exports = {
    createDocument: function (client, collectionRef, docObject, callback) {
        client.createDocument(collectionRef._self, docObject, callback);
    },

    fetchDocument: function (client, collectionRef, docId, callback) {
        var querySpec = {
            query: 'SELECT * FROM root r WHERE r.id=@id',
            parameters: [{
                name: '@id',
                value: docId
            }]
        };

        client.queryDocuments(collectionRef._self, querySpec).current(callback);
    },

    readDocument: function (client, docLink, options, callback) {
        client.readDocument(docLink, options, callback);
    },

    replaceDocument: function(client, docLink, docObject, callback) {
        client.replaceDocument(docLink, docObject, callback);    
    }
};

My first attempt at reading a document used the readDocument() method. I would construct a docLink using the following:

var docLink = `${refs.table._self}${refs.table._docs}${docId}`;

However, this always resulted in a 400 Bad Request response from DocumentDb. The reason is likely that the _self link uses the shorted (and obfuscated) URI, whereas the Document Id I am using is a GUID and is not obfuscated. If you take a look at the response from DocumentDb, there is an id field and a _rid field. The _rid field is used in the document links. Thus, instead of using readDocument(), I’m using a queryDocuments() call on the driver to search for the Id. I’ve also promisified these calls in the normal manner using the Bluebird library.

Fetching a Record

The Azure Mobile Apps SDK allows me to GET /tables/todoitem/id – where id is the GUID. With the driver complete, I can do the following in the Azure Function table controller:

function getOneItem(req, res, id) {
    driver.fetchDocument(refs.client, refs.table, id)
    .then((document) => {
        if (typeof document === 'undefined')
            res.status(404).json({ 'error': 'Not Found' });
        else
            res.status(200).json(convertItem(document));
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

When doing this, I did notice that some semantics seem to have changed in the Azure Functions SDK. I can no longer use context.bindings.id and has to switch to using req.params.id. Aside from this small change in the router code, this code is relatively straight forward. I established the convertItem() and convertError() methods in my last article.

Replacing a Record

The more complex case is replacing a record. There is a little bit of logic around conflict resolution:

  • If there is an If-Match header, then ensure the version of the current record matches the If-Match header, otherwise return a 412 response.
  • If there is no If-match header, but the new record contains a version, return a 409 response.
  • Otherwise update the record

Because we want the version and updatedAt fields to be controlled as well, we need to ensure the new object does not contain those values when it is submitted to DocumentDb:

function replaceItem(req, res, id) {
    driver.fetchDocument(refs.client, refs.table, id)
    .then((document) => {
        if (typeof document === 'undefined') {
            res.status(404).json({ 'error': 'Not Found' });
            return;
        }

        var item = req.body, version = new Buffer(document._etag).toString('base64')
        if (item.id !== id) {
            res.status(400).json({ 'error': 'Id does not match' });
            return;
        }

        if (req.headers.hasOwnProperty('if-match') && req.header['if-match'] !== version) {
            res.status(412).json({ 'current': version, 'new': item.version, 'error': 'Version Mismatch' })
            return;
        }

        if (item.hasOwnProperty('version') && item.version !== version) {
            res.status(409).json({ 'current': version, 'new': item.version, 'error': 'Version Mismatch' });
            return;
        }

        // Delete the version and updatedAt fields from the doc before submitting
        delete item.updatedAt;
        delete item.version;
        driver.replaceDocument(refs.client, document._self, item)
        .then((updatedDocument) => {
            res.status(200).json(convertItem(updatedDocument));
            return;
        });
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

I’m using the same Base64 encoding for the etag in the current document to ensure I can do a proper match. I could get DocumentDb to do all this work for me – the options value in the driver replaceDocument() method allows me to specify an If-Match. However, to do that, I would need to still fetch the record (since I need the document link), so I may as well do the checks myself. This also keeps some load off the DocumentDb, which is helpful.

While this is almost there, there is one final item. If there is a conflict, the server version of the document should be returned. That means the 409 and 412 responses need to return convertItem(document) instead – a simple change.

Deleting a Record

Deleting a record does not delete a record. Azure Mobile Apps uses soft delete (whereby the deleted flag is set to true). This means that I need to use replaceDocument() again for deletions:

function deleteItem(req, res, id) {
    driver.fetchDocument(refs.client, refs.table, id)
    .then((document) => {
        if (typeof document === 'undefined') {
            res.status(404).json({ 'error': 'Not Found' });
            return;
        }

        var item = convertItem(document);
        delete item.updatedAt;
        delete item.version;
        item.deleted = true;
        driver.replaceDocument(refs.client, document._self, item)
        .then((updatedDocument) => {
            res.status(200).json(convertItem(updatedDocument));
            return;
        });
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

This brings up a point about the GetOneItem() method. It does not take into account the deleted flag. I need it to return 404 Not Found if the deleted flag is set:

function getOneItem(req, res, id) {
    driver.fetchDocument(refs.client, refs.table, id)
    .then((document) => {
        if (typeof document === 'undefined' || document.deleted === true)
            res.status(404).json({ 'error': 'Not Found' });
        else
            res.status(200).json(convertItem(document));
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

It’s a simple change, but important in getting the protocol right.

What’s left?

There is only one method I have not written yet, and it’s the biggest one of the set – the getAllItems() method. That’s because it deals with OData querying, which is no small task. I’ll be tackling that in my next article. Until then, get the current codebase at my GitHub repository.

Creating Documents in DocumentDB with Azure Functions HTTP API

Thus far in my story of implementing Azure Mobile Apps in a dynamic (consumption) plan of Azure Functions using DocumentDB, I’ve got the basic CRUD HTTP API stubbed out and the initialization of my DocumentDB collection done. It’s now time to work on the actual endpoints that my Azure Mobile Apps SDK will call. There are five methods to implement:

  • Insert
  • Update / Replace
  • Delete
  • Fetch a single record
  • Search

I’m going to do these in the order above. Before I do that, I need to take a look at what DocumentDB provides me. Azure Mobile Apps requires five fields to work properly:

  • id – a string (generally a GUID).
  • createdAt – the date the record was created, in ISO-8601 format.
  • updatedAt – the date the record was updated, in ISO-8601 format.
  • deleted – a boolean, if the record is deleted.
  • version – an opaque string for conflict resolution.

DocumentDB provides some of this for us:

  • id – a string (generally a GUID).
  • _ts – a POSIX / unix timestamp of the number of seconds since the epoch since the record was last updated.
  • _etag – a checksum / version identifier.

When we create a record, we need to convert the document that DocumentDB returns to us into the format that Azure Mobile Apps provides. I use the following routine:

/**
 * Given an item from DocumentDB, convert it into something that the service can used
 * @param {object} item the original item
 * @return {object} the new item
 */
function convertItem(item) {
    if (item.hasOwnProperty('_ts')) {
        item.updatedAt = moment.unix(item._ts).toISOString();
        delete item._ts;
    } else {
        throw new Error('Invalid item - no _ts field');
    }

    if (item.hasOwnProperty('_etag')) {
        item.version = new Buffer(item._etag).toString('base64');
        delete item._etag;
    } else {
        throw new Error('Invalid item - no _etag field');
    }

    // Delete all the known fields from documentdb
    if (item.hasOwnProperty('_rid')) delete item._rid;
    if (item.hasOwnProperty('_self')) delete item._self;
    if (item.hasOwnProperty('_attachments')) delete item._attachments;

    return item;
}

I’m using the moment library to do date/time manipulation. This is a very solid library and well worth learning about. In addition to the convertItem() method, I also need something to convert the error values that come back from DocumentDB. They are not nicely formed, so some massaging is in order:

/**
 * Convert a DocumentDB error into something intelligible
 * @param {Error} error the error object
 * @return {object} the intelligible error object
 */
function convertError(error) {
    var body = JSON.parse(error.body);
    if (body.hasOwnProperty("message")) {
        var msg = body.message.replace(/^Message:\s+/, '').split(/\r\n/);
        body.errors = JSON.parse(msg[0]).Errors;

        var addl = msg[1].split(/,\s*/);
        addl.forEach((t) => {
            var tt = t.split(/:\s*/);
            tt[0] = tt[0].replace(/\s/, '').toLowerCase();
            body[tt[0]] = tt[1];
        });

        delete body.message;
    }

    return body;
}

I had to work through the error object several times experimenting with the actual response to come up with this routine. This seems like the right code by experimentation. Whether it holds up during normal usage remains to be seen.

I’ve already written the createDocument() method in the DocumentDB driver:

module.exports = {
    createDocument: function (client, collectionRef, docObject, callback) {
        client.createDocument(collectionRef._self, docObject, callback);
    }
};

This is then promisifyed using the bluebird promise library. With this work done, my code for inserts becomes very simple:

function insertItem(req, res) {
    var item = req.body;

    item.createdAt = moment().toISOString();
    if (!item.hasOwnProperty('deleted')) item.deleted = false;

    driver.createDocument(refs.client, refs.table, item)
    .then((document) => {
        res.status(201).json(convertItem(document));
    })
    .catch((error) => {
        res.status(error.code).json(convertError(error));
    });
}

The item that we need to insert comes in on the body. We need to add the createdAt field and the deleted field (if it isn’t already set). Since this is an insert, we call createDocument() in the driver. If it succeeds, we return a 201 Created response with the new document (converted to the Azure Mobile Apps specification). If not, we return the error from DocumentDB together with the formatted object.

We can test inserts with Postman. For example, here is a successful insert:

insert-1

DocumentDB creates the id for me if it doesn’t exist. I convert the _ts and _etag fields to something more usable by the Azure Mobile Apps SDK on the way back to the client. If I copy the created object and push it again, I will get a conflict:

insert-2

Notice how DocumentDB does all the work for me? All I need to do is some adjustments on the output to get my insert operation working. I can use the Document Browser within the Azure Portal to look at the actual records.

In the next post, I’m going to move onto Update, Delete and Fetch all in one go.

Working with DocumentDb

In my last post, I introduced working with HTTP CRUD APIs with Azure Functions. My intent in all this is to create a proof of concept service that emulates the Azure Mobile Apps service, but using Azure Functions and the dynamic (or consumption-based) SKU. This means that you pay for the API only when it is being used, but it scales seamlessly as your needs grow. In addition, I’m going to make the backing store for this API a NoSQL store based on another Azure resource – DocumentDb.

Fortunately for me, DocumentDb has a nice Node.js driver. I’m going to promisify the callback-based SDK with bluebird. There are a number of samples available for the DocumentDb driver. For instance, here is my docdb-driver/database.js file:

module.exports = {
    createDatabase: function (client, databaseId, callback) {
        client.createDatabase({ id: databaseId }, callback);
    },

    deleteDatabase: function (client, databaseId, callback) {
        client.deleteDatabase(`dbs/${databaseId}`, callback);
    },

    findDatabaseById: function (client, databaseId, callback) {
        var qs = {
            query: 'SELECT * FROM root r WHERE r.id = @id',
            parameters: [
                { name: '@id', value: databaseId }
            ]
        };

        client.queryDatabases(qs).toArray(function (err, results) {
            if (err) {
                callback(err, null);
            } else {
                callback(null, (results.length === 0) ? null : results[0]);
            }
        });
    },

    listDatabases: function (client, callback) {
        client.readDatabases().toArray(callback);
    },

    readDatabase: function (client, database, callback) {
        client.readDatabase(database._self, callback);
    },

    readDatabases: function (client, databaseId, callback) {
        client.readDatabase(`dbs/${databaseId}`, callback);
    }
};

This is based on callbacks, rather than promises. So my docdb-driver/index.js file uses promisify to convert them to promises:

var Promise = require('bluebird');
var collection = require('./collection');
var database = require('./database');
var docops = require('./document');

var dbCache = {};

var createDatabase = Promise.promisify(database.createDatabase);
var findDatabaseById = Promise.promisify(database.findDatabaseById);

function ensureDatabaseExists(client, database) {
    if (database in dbCache) {
        return Promise.resolve(dbCache[database]);
    }

    return findDatabaseById(client, database).then((dbRef) => {
        if (dbRef == null) {
            return createDatabase(client, database).then((result) => {
                dbCache[database] = result;
                return result;
            });
        }
        dbCache[database] = dbRef;
        return dbRef;
    });
}

module.exports = {
    createCollection: Promise.promisify(collection.createCollection),
    listCollections: Promise.promisify(collection.listCollections),
    readCollection: Promise.promisify(collection.readCollection),
    readCollectionById: Promise.promisify(collection.readCollectionById),
    getOfferType: Promise.promisify(collection.getOfferType),
    changeOfferType: Promise.promisify(collection.changeOfferType),
    deleteCollection: Promise.promisify(collection.deleteCollection),

    createDatabase: createDatabase,
    deleteDatabase: Promise.promisify(database.deleteDatabase),
    ensureDatabaseExists: ensureDatabaseExists,
    findDatabaseById: findDatabaseById,
    listDatabases: Promise.promisify(database.listDatabases),
    readDatabase: Promise.promisify(database.readDatabase),
    readDatabases: Promise.promisify(database.readDatabases),

    createDocument: Promise.promisify(docops.createDocument)
};

I’m going to extend this driver package over time. Sometimes I use the straight API from the DocumentDb driver (see the readDatabase() method). Sometimes, however, I want to do something extra. The ensureDatabaseExists() method is an example of this. I want to find the database in the service and create it only if it doesn’t exist.

Back to the Azure Function I’m developing. DocumentDb mainly stores “documents” – JSON blobs of associated data. It organizes these documents into “collections” and collections into a “database”. In the Azure Mobile Apps equivalent, the collection would be a table and the individual rows or entities would be documents. My first requirement is to ensure that the database and collection are initialized properly (in todoitem/index.js):

var DocumentDb = require('documentdb');
var driver = require('../docdb-driver');

/**
 * Global Settings Object
 */
var settings = {
    host: process.env['DocumentDbHost'],
    accountKey: process.env['DocumentDbAccountKey'],
    database: 'AzureMobile',
    connectionPolicy: undefined,
    consistencyLevel: 'Session',
    pricingTier: 'S1',
    table: 'todoitem'
};

// Store any references we receive here as a cache
var refs = {
    initialized: false
};

/**
 * Routes the request to the table controller to the correct method.
 *
 * @param {Function.Context} context - the table controller context
 * @param {Express.Request} req - the actual request
 */
function tableRouter(context, req) {
    var res = context.res;
    var id = context.bindings.id;

    initialize(context).then(() => {
        switch (req.method) {
            case 'GET':
                if (id) {
                    getOneItem(req, res, id);
                } else {
                    getAllItems(req, res);
                }
                break;

            case 'POST':
                insertItem(req, res);
                break;

            case 'PUT':
                replaceItem(req, res, id);
                break;

            case 'DELETE':
                deleteItem(req, res, id);
                break;

            default:
                res.status(405).json({ error: "Operation not supported", message: `Method ${req.method} not supported`})
        }
    });
}

/**
 * Initialize the DocumentDb Driver
 * @param {Function.Context} context - the table controller context
 * @param {function} context.log - used for logging
 * @returns {Promise}
 */
function initialize(context) {
    if (refs.initialized) {
        context.log('[initialize] Already initialized');
    }

    context.log(`[initialize] Creating DocumentDb client ${settings.host} # ${settings.accountKey}`);
    refs.client = new DocumentDb.DocumentClient(
        settings.host,
        { masterKey: settings.accountKey },
        settings.connectionPolicy,
        settings.consistencyLevel
    );

    context.log(`[initialize] EnsureDatabaseExists ${settings.database}`);
    return driver.ensureDatabaseExists(refs.client, settings.database)
        .then((dbRef) => {
            context.log(`[initialize] Initialized Database ${settings.database}`);
            refs.database = dbRef;
            return driver.listCollections(refs.client, refs.database);
        })
        .then((collections) => {
            context.log(`[initialize] Found ${collections.length} collections`);
            const collection = collections.find(c => { return (c.id === settings.table); });
            context.log(`[initialize] Collection = ${JSON.stringify(collection)}`);
            if (typeof collection !== 'undefined') return collection;
            context.log(`[initialize] Creating collection ${settings.table}`);
            return driver.createCollection(refs.client, settings.pricingTier, refs.database, settings.table);
        })
        .then((collectionRef) => {
            context.log(`[initialize] Found collection`);
            refs.table = collectionRef;
            refs.initialized = true;
        });

    context.log('[initialize] Finished Initializing Driver');
}

Let's take this in steps. Firstly, I set up the settings. The important things here are the DocumentDbHost and the DocumentDbAccountKey. If you have created a DocumentDb within the Azure Portal, click on the Keys menu item. The DocumentDbHost is the URI field and the DocumentDbAccountKey is the PRIMARY KEY field. If you are running the Azure Function locally, then you will need to set these as environment variables before starting the func host. If you are running the Azure Function within Azure, you need to make these App Settings. An example of setting these locally in PowerShell:

$env:DocumentDbHost = "https://mydocdb.documents.azure.com:443/"
$env:DocumentDbAccountKey = "fuCZuSomeLongStringaLNKjIiMSEyaojsP05ywmevI7K2yCY9dYLRuCQPd3dMnvg=="
func run test-func --debug

When you use Postman (for example, a GET http://localhost:7071/tables/todoitem), you will see the initialize() method gets called. This method returns a Promise that, when resolved, will then allow the request to be continued. In the initialize() method, I short-circuit the initialization if it has already been initialized. If it has not been initialized, I fill in the refs object. This object will be used by the inidividual CRUD operations, so it needs to be filled in. The client, database, and collection that we need are found or created. At the end, we have resolve the promise by setting the initialized flag to true (thus future calls will be short circuited).

There is a race condition here. If two requests come in to a “cold” function, they will both go through the initialization together and potentially the “create database” and “create collection” will be duplicated, causing an exception in one of the requests. I’m sure I could fix this, but it’s a relatively rare case. Once the datbase and collection are created, the possibility of the condition goes away.

If you run this (either locally or within Azure Functions), you will see the following output in the log window:

function-docdb

If you’ve done something wrong, you will see the exception and you can debug it using the normal methods. Want a primer? I’ve written a blog post about it.

In the next post, I’ll cover inserting, deleting and updating records. Until then, check out the code on my GitHub repository.