NuGet Package To Manage CosmosDB Objects

Manage CosmosDB objects (Stored Procedure, Functions, Triggers …) with this NuGet package

In my Toss project, I decided to use CosmosDB as the main data store. Document-oriented Databases are fun to work with as you have to change the way you see the data and processing when compared to a relational database. NoSQL databases are often badly framed as “schemaless”, but there isn’t such thing as schemaless; the schema is just defined elsewhere, i.e., not on the database but on your application code.

There are 2 problems with this approach for the developers.

the data is persisted; so when you change your schema (property rename), you have to think of the existing data
there are some specific database objects, such as functions and triggers that need to be defined somewhere

That’s why I created this package. It’ll help you to manage your database objects and schema evolution along with your application code on your repository and it’ll be applied whenever and wherever you want (on app startup mostly).

Reading the embedded resources

I chose to use embedded JS file in the assembly for object definitions because -

If they are defined in JavaScript, it’s better to use .js file; the IDE will help the developer.
If it’s embedded, then the package user won’t have to think about securing the folder containing the definitions.

Reading the embedded ressource is fairly simple in C# and .NET Standard.

//read all the migration embbeded in CosmosDB/
Migrationsvarressources=migrationAssembly.GetManifestResourceNames().Where(r=>r.Contains(".CosmosDB.Migrations.")&&r.EndsWith(".js")).OrderBy(r=>r).ToList();
//for each migrationforeach(varmigrationinressources)
{
stringmigrationContent;
using(varstream = migrationAssembly.GetManifestResourceStream(migration)) {
using(varreader = newStreamReader(stream)) {
migrationContent = awaitreader.ReadToEndAsync();
}
}
// do something}

migrationAssembly is sent as a parameter of my method, so the user can add its migrations to the app ressources or to a library ressource.
I don’t really think about the performance here as this code is supposed to run only once per app lifecycle, so I prefer to keep it clear.

Applying the migrations

I decided to implement the strategy pattern - for each type of objects there is one strategy, so the users will be able to implement their own strategies.

This piece of code goes on the “//do something” from the previous code sample.

var parsedMigration = new ParsedMigrationName(migration);
var strategy = strategies.FirstOrDefault(s => s.Handle(parsedMigration));
if (strategy == null)
{
throw new InvalidOperationException(string.Format("No strategy found for migration '{0}", migration));
}
await client.CreateDatabaseIfNotExistsAsync(parsedMigration.DataBase);
if (parsedMigration.Collection != null)
{
await client.CreateDocumentCollectionIfNotExistsAsync(UriFactory.CreateDatabaseUri(parsedMigration.DataBase.Id), parsedMigration.Collection);
}
await strategy.ApplyMigrationAsync(client, parsedMigration, migrationContent);

Parsed migration is the class that checks and reads the resource name. Each resource must respect the documented convention: the “Test.js” file located in “CosmosDB/Migrations/TestDataBase/StoredProcedure/” is the content of the stored procedure called “Test” located in the database “TestDataBase” and in the collection.
Here, I created the required database and collection also.

Strategy implementation

For each kind of object I want to handle, I have to create an implementation of the strategy. Here is the one for the triggers.

internal class TriggerMigrationStrategy: IMigrationStrategy
{
public async Task ApplyMigrationAsync(IDocumentClient client, ParsedMigrationName migration, string content)
{
var nameSplit = migration.Name.Split('-');
Trigger trigger = new Trigger() {
Body = content, Id = nameSplit[2], TriggerOperation = (TriggerOperation) Enum.Parse(typeof(TriggerOperation), nameSplit[1]), TriggerType = (TriggerType) Enum.Parse(typeof(TriggerType), nameSplit[0])
};
await client.UpsertTriggerAsync(UriFactory.CreateDocumentCollectionUri(migration.DataBase.Id, migration.Collection.Id), trigger);
}
public bool Handle(ParsedMigrationName migration)
{
return migration.Type == "Trigger";
}
}

For the triggers, I need more information than the name, such as - type and operation. So, I created another convention for setting those on the file name.

{Type}-{Operation}-{Name}

Using the library

For using the library, you need to,

Install the package “Install-Package RemiBou.CosmosDB.Migration -Version 0.0.0-CI-20190126-145359” (it’s still a pre-release; I’ll try to have a better versioning strategy later).
Create the folders “CosmosDB” and “CosmosDB/Migrations”
Add “” to your project's csproj
Add your migration respecting the convention
Add this code when you want to run the migrations (I guess on App Startup.)
1. await new CosmosDBMigration(documentClient).MigrateAsync(this.GetType().Assembly);
2. // add .Wait() if your are in a not in an async context like the Startup
Run your app.

I’ll try later to automate the step 2 and 3 when installing the package.

Conclusion

This package was fun to build and design and I think I got something nice. Now, I have many features to do (read here the todo list) but before that, I have to set up the test the suit so the user will be reassured that the package is stable.

This article was published here.