Real-Time Azure Digital Twins: Building a Scalable NDJSON Solution with Event-Driven Architecture

Introduction

Modern industrial IoT demands real-time digital twin synchronization without traditional database overhead. This guide demonstrates building an event-driven Azure Digital Twins NDJSON solution using Azure Functions, Service Bus, and in-memory caching for lightning-fast performance and scalability.

Architecture Overview

Our solution leverages Azure Digital Twins for core twin management, Service Bus for enterprise messaging, Event Grid for real-time notifications, Azure Functions for serverless processing, and a Web API with MemoryCache for high-performance NDJSON data serving.

Step 1. Solution Setup.

dotnet new sln -n SmartFactoryTwins

dotnet new classlib -n TwinCore.Models

dotnet new functionsapp -n TwinProcessor.Functions --framework net8.0

dotnet new webapi -n TwinData.WebAPI --framework net8.0

dotnet new console -n TwinFactory.Generator --framework net8.0

Add references and packages.

dotnet sln add TwinCore.Models/TwinCore.Models.csproj

dotnet add TwinProcessor.Functions package Azure.DigitalTwins.Core

dotnet add TwinProcessor.Functions package Azure.Messaging.ServiceBus

dotnet add TwinData.WebAPI package Microsoft.Extensions.Caching.Memory

This creates a multi-project solution with enhanced naming for better organization. Each project handles specific responsibilities: models, event processing, API services, and twin generation.

Sample Output

Sample Output

Step 2. Core Models Implementation.

This creates and uploads DTDL models defining the structure and properties of digital twins. Models establish the schema for industrial assets like sensors and motors.

// TwinUpdateRequest.cs in TwinCore.Models

namespace TwinCore.Models
{
    public class TwinUpdateRequest
    {
        public string TwinId { get; set; } = string.Empty;

        public Dictionary<string, object> Properties { get; set; } = new();

        public DateTime LastUpdateTime { get; set; }

        public string EventType { get; set; } = string.Empty;
    }
}

// TwinDataModel.cs

namespace TwinCore.Models
{
    public class TwinDataModel
    {
        public string Id { get; set; } = string.Empty;

        public string ModelId { get; set; } = string.Empty;

        public Dictionary<string, object> Properties { get; set; } = new();

        public DateTime LastUpdateTime { get; set; }
    }
}

These models define essential data contracts for twin updates and data representation. They ensure type safety and consistent data flow across all solution components.

Step 3. DTDL Model Creation.

// ModelUploader.cs in TwinFactory.Generator

public class ModelUploader
{
    public async Task UploadIndustrialModels()
    {
        var models = new List<string>
        {
            GetTemperatureSensorModel(),
            GetMotorModel()
        };

        var response = await _client.CreateModelsAsync(models);
        Console.WriteLine($"Uploaded {response.Value.Count} models successfully");
    }

    private string GetTemperatureSensorModel()
    {
        return @"{
  ""@context"": ""dtmi:dtdl:context;3"",
  ""@id"": ""dtmi:factory:TemperatureSensor;1"",
  ""@type"": ""Interface"",
  ""displayName"": ""Temperature Sensor"",
  ""contents"": [
    { ""@type"": ""Property"", ""name"": ""name"", ""schema"": ""string"" },
    { ""@type"": ""Property"", ""name"": ""location"", ""schema"": ""string"" },
    { ""@type"": ""Property"", ""name"": ""temperature"", ""schema"": ""double"" },
    { ""@type"": ""Property"", ""name"": ""status"", ""schema"": ""string"" }
  ]
}";
    }

    // You may want to implement GetMotorModel here
}

Sample Output

Sample Output

Step 4. Event Processing with Azure Functions.

This implements dual event processing using Event Grid for real-time updates and Service Bus for reliable message processing. The function ensures no twin updates are lost.

public class EventProcessor
{
    private readonly DigitalTwinsClient _dtClient;
    private readonly ServiceBusProcessor _serviceBus;

    [Function("ProcessTwinEvents")]
    public async Task ProcessEventGrid([EventGridTrigger] EventGridEvent gridEvent)
    {
        var twinId = ExtractTwinId(gridEvent.Subject);
        var twin = await _dtClient.GetDigitalTwinAsync<BasicDigitalTwin>(twinId);

        // Send to Service Bus for reliability
        await _serviceBus.SendMessageAsync(new TwinUpdateRequest
        {
            TwinId = twinId,
            Properties = twin.Value.Contents,
            LastUpdateTime = DateTime.UtcNow,
            EventType = gridEvent.EventType
        });

        // Direct API notification for real-time updates
        await NotifyWebAPI(updateRequest);
    }

    [Function("ProcessServiceBusMessages")]
    public async Task ProcessServiceBus(
        [ServiceBusTrigger("twin-updates")] ServiceBusReceivedMessage message)
    {
        var updateRequest = JsonSerializer.Deserialize<TwinUpdateRequest>(message.Body);
        await ProcessTwinUpdate(updateRequest);
    }
}

Sample output

Sample output

Step 5. High-Performance Web API.

The Web API provides NDJSON endpoints with in-memory caching for ultra-fast data retrieval. Cache updates ensure real-time synchronization with Azure Digital Twins.

[ApiController]
[Route("api/[controller]")]
public class TwinsController : ControllerBase
{
    private readonly IMemoryCache _cache;
    private const string CACHE_KEY = "factory_twins";

    [HttpGet("ndjson")]
    public async Task<IActionResult> GetNDJSON()
    {
        var twins = await GetCachedTwins();
        var ndjsonBuilder = new StringBuilder();

        foreach (var twin in twins.Values)
        {
            ndjsonBuilder.AppendLine(JsonSerializer.Serialize(twin));
        }

        return Content(ndjsonBuilder.ToString(), "application/x-ndjson");
    }

    [HttpPost("cache/update")]
    public async Task<IActionResult> UpdateCache([FromBody] TwinUpdateRequest request)
    {
        var twins = await GetCachedTwins();

        twins[request.TwinId] = new TwinDataModel
        {
            Id = request.TwinId,
            Properties = request.Properties,
            LastUpdateTime = request.LastUpdateTime
        };

        _cache.Set(CACHE_KEY, twins, TimeSpan.FromHours(1));

        return Ok(new { Success = true, TwinId = request.TwinId });
    }
}

Sample output

Sample output

Step 6. Twin Factory Generator.

This automated generator creates 600 industrial twins for testing and demonstration. It supports multiple twin types with realistic property values.

public class TwinFactory
{
    public async Task CreateIndustrialTwins()
    {
        var twinTypes = new[] { "Motor", "TemperatureSensor", "PressureSensor" };

        for (int i = 1; i <= 600; i++)
        {
            var twinType = twinTypes[i % twinTypes.Length];
            var twinId = $"{twinType}-{i:D3}";

            var twin = new BasicDigitalTwin
            {
                Metadata = { ModelId = $"dtmi:factory:{twinType};1" },
                Contents = GenerateProperties(twinType)
            };

            await _client.CreateOrReplaceDigitalTwinAsync(twinId, twin);

            if (i % 100 == 0)
                Console.WriteLine($"Created {i} twins...");
        }
    }
}

Sample output

Step 7. Azure Infrastructure Setup.

This PowerShell script deploys the complete Azure infrastructure, including Digital Twins, Service Bus, Function Apps, and Web API, with proper naming conventions.

# deploy-smart-factory.ps1

param (
    [string] $ResourceGroup,
    [string] $Location,
    [string] $FactoryName
)

Create core resources

az group create --name $ResourceGroup --location $Location

az dt create --dt-name $FactoryName --resource-group $ResourceGroup

Set up Service Bus

$serviceBusNamespace = "$FactoryName-messaging"

az servicebus namespace create `
    --name $serviceBusNamespace `
    --resource-group $ResourceGroup `
    --sku Standard

az servicebus queue create `
    --name "twin-updates" `
    --namespace-name $serviceBusNamespace `
    --resource-group $ResourceGroup

Deploy Function App and Web API

az functionapp create \
  --name "$FactoryName-processor" \
  --resource-group $ResourceGroup \
  --runtime dotnet

az webapp create \
  --name "$FactoryName-api" \
  --resource-group $ResourceGroup \
  --runtime "DOTNETCORE|8.0"

Sample Output

Sample Output

Performance Benefits

This architecture delivers sub-second twin synchronization, horizontal scalability through Azure Functions, and cost-effective serverless operations. The dual processing approach ensures both real-time performance and enterprise-grade reliability with Service Bus messaging.

Conclusion

This event-driven Azure Digital Twins solution demonstrates modern cloud-native patterns for industrial IoT at scale. The combination of in-memory caching, Service Bus messaging, and serverless computing provides a robust foundation for real-time digital twin management in manufacturing environments.