Sending Event Data To Azure Event Hub

Introduction

 
In this article we are going to see how we can send data to an Azure event hub and how we can check if the event hub has received our data on the Azure portal. Consider a scenario where we are sending data from a weather station collected via sensors to an event hub for later processing.
 

Roadmap

  1. Azure Event Hubs Overview.
  2. Creating Azure event hub on azure portal.
  3. Writing code to send data to the even hub.

Overview

 
Microsoft has provided the Eventhub service on their cloud platform as a big streaming palform and event ingestion service. Eventhubs can receive and process millions of event data per second. Events of event data sent to the eventhubs can be transformed and stored by many real time analytics services or storage adapters.

Creating Azure event hub on Azure portal

 
Follow these steps to create an Event Hub on the Azure portal.
  1. Sign in to the Azure Portal.
  2. On the portal, click +New > Internet of Things > Event Hubs.

    Sending Event Data To Azure Event Hub

  3. In the "Create Namespace" blade, enter the name of your Event Hub in the name field, then choose the Standard Pricing Tier, and choose the desired subscription to create the Event Hub under it. For Resource group, select the "Create new" option and enter the resource group name as you wish. Select the location that is nearest to you and press "Create".

    Sending Event Data To Azure Event Hub

  4. After the "deployment succeeded" message on the Notification bar, it opens our namespace to show the details of the Event Hub. Press +Event Hub to add an event hub to the namespace.

    Sending Event Data To Azure Event Hub

  5. In the "Create Event Hub" blade, enter the name of the hub and click the "Create" button.

    Sending Event Data To Azure Event Hub

  6. Wait a few minutes. Our Event Hub is created and shown under the "Event Hubs" blade. For connection purposes, click the "Shared access policies" and choose "RootManageSharedAccesskey". It opens the SAS policy that consists of a connection string. Copy and paste the connection string on the clipboard.

Sending data to Event hub via Code

 
Considering the scenario we created for a weather station, we will be writing a console app, a service that records data and sends it to the event hub. For recording weather we will be using a random function for now that oscillates between a certain temperature and wind speed, and wind direction.
 
Make a .Net Core Console App.
 
Once you have made the app you will need to create a Class Library project which you will use as a means of sending the data. While the main project or the driver simply calls functions from that class library.
 
Add a class Library project named “Sender”.
 
Create a file by the name of WeatherData.Sender.cs. Create a Folder, name it Models. This folder will contains the class model which we will use to map our data before it is sent. Create a class in the Models folder named WeatherData.
 
Code for WeatherData.Sender.cs
  1. namespace Sender  
  2. {  
  3.     public interface IWeatherDataSender  
  4.     {  
  5.         Task SendDataAsync(WeatherData data);  
  6.     }  
  7.     public class WeatherDataSender : IWeatherDataSender  
  8.     {  
  9.         private EventHubClient _eventHubClient;  
  10.   
  11.         public WeatherDataSender(string eventHubConnectionString)  
  12.         {  
  13.             _eventHubClient = EventHubClient.CreateFromConnectionString(eventHubConnectionString);  
  14.         }  
  15.         public async Task SendDataAsync(WeatherData data)  
  16.         {  
  17.             EventData eventData = CreateEventData(data);  
  18.             await _eventHubClient.SendAsync(eventData);  
  19.         }  
  20.   
  21.        private static EventData CreateEventData(WeatherData data)  
  22.         {  
  23.             var dataAsJson = JsonConvert.SerializeObject(data);  
  24.             var eventData = new EventData(Encoding.UTF8.GetBytes(dataAsJson));  
  25.             return eventData;  
  26.         }  
  27.     }  
  28. }  
Code for WeatherData.cs
  1. namespace Sender.Models  
  2. {  
  3.     public class WeatherData  
  4.     {  
  5.         public int Temperature { getset; }  
  6.         public int WindSpeed { getset; }  
  7.         public Direction WindDirection { getset; }  
  8.   
  9.         public enum Direction   
  10.         {  
  11.             North, South, East, West   
  12.         }  
  13.   
  14.         public override string ToString()  
  15.         {  
  16.             return "{ Temp: " + Temperature.ToString() + " *C "   
  17.                  + "| Windspeed: " + WindSpeed.ToString() + " km/h "  
  18.                  + "| Wind Direction: " + WindDirection.ToString() + " }";  
  19.         }  
  20.     }  
  21. }  
Now add a class to the main driver project named SenderHelper.cs.
 
Code for SenderHelper.cs
  1. namespace WeatherApp  
  2. {  
  3.     public class SenderHelper  
  4.     {  
  5.         WeatherDataSender weatherDataSender;  
  6.           
  7.         public SenderHelper(string conn)  
  8.         {  
  9.             weatherDataSender = new WeatherDataSender(conn);  
  10.         }  
  11.           
  12.         public async Task SendDataAsync(WeatherData WeatherData)  
  13.         {  
  14.             try  
  15.             {  
  16.                 await weatherDataSender.SendDataAsync(WeatherData);  
  17.                 Console.WriteLine($"Sent data: {WeatherData}");  
  18.             }  
  19.             catch (Exception ex)  
  20.             {  
  21.                 Console.WriteLine($"Exception: {ex.Message}");  
  22.             }  
  23.         }  
  24.   
  25.         public async void DataSender()  
  26.         {  
  27.             await SendDataAsync(CreateWeatherData());  
  28.         }  
  29.   
  30.         public WeatherData CreateWeatherData()  
  31.         {  
  32.             return new WeatherData  
  33.             {  
  34.                 Temperature = 35, // in degree celcius  
  35.                 WindSpeed = 300, // Kmph  
  36.                 WindDirection = WeatherData.Direction.North  
  37.             };  
  38.         }  
  39.     }  
  40. }  
This class will be used as controller in the Main function to send data to the Event Hub.
 
Add the following code to your Program.cs
  1. public class Program  
  2.     {  
  3.         public static string ConnectionString = "<your event hub connection string>";  
  4.   
  5.         static void Main(string[] args)  
  6.         {  
  7.             SenderHelper helper = new SenderHelper(ConnectionString);  
  8.               
  9.             for(int i = 0; i < 5; i++)  
  10.             {  
  11.                 helper.DataSender();  
  12.             }  
  13.         }  
  14.     }  
This will send 5 events to the event hub. Now before the messages are sent to the eventhub your eventhub on the azure portal will look something like this,
 
Sending Event Data To Azure Event Hub 
 
Once you send events these graphs will change and they will look like this,
 
Sending Event Data To Azure Event Hub 
 
You have successfully sent data to your eventhub.
 

Use cases

 
The following scenarios are some of the scenarios where you can use Event Hubs,
  • Anomaly detection (fraud/outliers)
  • Application logging
  • Analytics pipelines, such as clickstreams
  • Live dashboarding
  • Archiving data
  • Transaction processing
  • User telemetry processing
  • Device telemetry streaming