Cognitive Services - Analyze An Image Using Computer Vision API With ASP.NET Core And C#

This article explains how to implement Computer Vision API in our application

Introduction

 
One of the important Cognitive Services API is Computer Vision API and it helps to access the advanced algorithms for processing images and returning valuable information. For example, by uploading an image or specifying an image URL, Microsoft Computer Vision algorithms can analyze visual content in different ways based on inputs and user choices. So we will get various information about the given image. We need a valid subscription key for accessing this feature.
 
Prerequisites
  1. Subscription key ( Azure Portal ).
  2. Visual Studio 2015 or 2017

Subscription Key Free Trail

 
If you don’t have Microsoft Azure Subscription and want to test the Computer Vision API because it requires a valid Subscription key for processing the image information. Don’t worry !! Microsoft gives a 7 day trial Subscription Key ( Click here ). We can use that Subscription key for testing purposes. If you sign up using the Computer Vision free trial, then your subscription keys are valid for the westcentral region (https://westcentralus.api.cognitive.microsoft.com ).
 

Requirements

 
These are the major requirements mentioned in the Microsoft docs.
  1. Supported input methods: Raw image binary in the form of an application/octet stream or image URL.
  2. Supported image formats: JPEG, PNG, GIF, BMP.
  3. Image file size: Less than 4 MB.
  4. Image dimension: Greater than 50 x 50 pixels.

Computer Vision API

 
First, we need to log into the Azure Portal with our Azure credentials. Then we need to create an Azure Computer Vision Subscription Key in the Azure portal.
 
Cognitive Services 
 
Click on “Create a resource” on the left side menu and it will open an “Azure Marketplace”. There, we can see the list of services. Click “AI + Machine Learning” then click on the “Computer Vision”. 
 

Provision a Computer Vision Subscription Key

 
After clicking the “Computer Vision”, it will open another section. There, we need to provide the basic information about Computer Vision API.
 
Cognitive Services 
  • Name
    Name of the Computer Vision API.

  • Subscription
    We can select our Azure subscription for Computer Vision API creation.

  • Location
    We can select our location of resource group. The best thing is we can choose a location closest to our customer.

  • Pricing tier
    Select an appropriate pricing tier for our requirement.

  • Resource group
    We can create a new resource group or choose from an existing one.
Now click on the MenothVision in dashboard page and it will redirect to the details page of MenothVision ( “Overview” ). Here, we can see the Manage Key ( Subscription key details ) & Endpoint details. Click on the Show access keys links and it will redirect to another page.

Cognitive Services 
 
We can use any of the subscription keys or regenerate the given key for getting image information using Computer Vision API.
 
Cognitive Services 
 

Endpoint

 
As we mentioned above the location is the same for all the free trial Subscription Keys. In Azure we can choose available locations while creating a Computer Vision API. We have used the following endpoint  in our code.
 
"https://westus.api.cognitive.microsoft.com/vision/v1.0/analyze" 
 

View Model

 
The following model will contain the API image response information.
  1. using System;  
  2. using System.Collections.Generic;  
  3. using System.Linq;  
  4. using System.Threading.Tasks;  
  5. namespace VisionApiDemo.Models {  
  6.     public class Detail {  
  7.         public List < object > celebrities {  
  8.             get;  
  9.             set;  
  10.         }  
  11.     }  
  12.     public class Category {  
  13.         public string name {  
  14.             get;  
  15.             set;  
  16.         }  
  17.         public double score {  
  18.             get;  
  19.             set;  
  20.         }  
  21.         public Detail detail {  
  22.             get;  
  23.             set;  
  24.         }  
  25.     }  
  26.     public class Caption {  
  27.         public string text {  
  28.             get;  
  29.             set;  
  30.         }  
  31.         public double confidence {  
  32.             get;  
  33.             set;  
  34.         }  
  35.     }  
  36.     public class Description {  
  37.         public List < string > tags {  
  38.             get;  
  39.             set;  
  40.         }  
  41.         public List < Caption > captions {  
  42.             get;  
  43.             set;  
  44.         }  
  45.     }  
  46.     public class Color {  
  47.         public string dominantColorForeground {  
  48.             get;  
  49.             set;  
  50.         }  
  51.         public string dominantColorBackground {  
  52.             get;  
  53.             set;  
  54.         }  
  55.         public List < string > dominantColors {  
  56.             get;  
  57.             set;  
  58.         }  
  59.         public string accentColor {  
  60.             get;  
  61.             set;  
  62.         }  
  63.         public bool isBwImg {  
  64.             get;  
  65.             set;  
  66.         }  
  67.     }  
  68.     public class Metadata {  
  69.         public int height {  
  70.             get;  
  71.             set;  
  72.         }  
  73.         public int width {  
  74.             get;  
  75.             set;  
  76.         }  
  77.         public string format {  
  78.             get;  
  79.             set;  
  80.         }  
  81.     }  
  82.     public class ImageInfoViewModel {  
  83.         public List < Category > categories {  
  84.             get;  
  85.             set;  
  86.         }  
  87.         public Description description {  
  88.             get;  
  89.             set;  
  90.         }  
  91.         public Color color {  
  92.             get;  
  93.             set;  
  94.         }  
  95.         public string requestId {  
  96.             get;  
  97.             set;  
  98.         }  
  99.         public Metadata metadata {  
  100.             get;  
  101.             set;  
  102.         }  
  103.     }  
  104. }  

Request URL


We can add additional parameters or request parameters ( optional ) in our API “endPoint” and it will provide more information for the given image.
  1. https://[location].api.cognitive.microsoft.com/vision/v1.0/analyze[?visualFeatures][&details][&language]  

Request parameters

 
Currently we can use 3 optional parameters.
  1. Visual Features
  2. Details
  3. Language
VisualFeatures

The name itself clearly mentions it returns Visual Features of the given image. If we add multiple values in a Visual Features parameter then put a comma for each value. The following are the Visual Features parameters in API.
  1. Categories
  2. Tags
  3. Description
  4. Faces
  5. ImageType
  6. Color
  7. Adult
Details

This parameter will return domain specific information whether it is Celebrities or Landmarks.
  • Celebrities
    If the detected image is of a celebrity it identifies it.

  • Landmarks
    If the detected image is of a landmark it identifies it. 
language

The service will return recognition results in specified language. Default language is english.

Supported languages.
  1. en – English, Default.
  2. zh – Simplified Chinese

Vision API Service


The following code will process and generate image information using Computer Vision API and its response is mapped into the “ImageInfoViewModel”. We can add the valid Computer Vision API Subscription Key into the following code.
  1. using Newtonsoft.Json;  
  2. using System;  
  3. using System.Collections.Generic;  
  4. using System.IO;  
  5. using System.Net.Http;  
  6. using System.Net.Http.Headers;  
  7. using System.Threading.Tasks;  
  8. using VisionApiDemo.Models;  
  9. namespace VisionApiDemo.Business_Layer {  
  10.     public class VisionApiService {  
  11.         const string subscriptionKey = "<Enter your subscriptionKey>";  
  12.         const string endPoint = "https://westus.api.cognitive.microsoft.com/vision/v1.0/analyze";  
  13.         public async Task < ImageInfoViewModel > MakeAnalysisRequest() {  
  14.             string imageFilePath = @ "C:\Users\Rajeesh.raveendran\Desktop\Rajeesh.jpg";  
  15.             var errors = new List < string > ();  
  16.             ImageInfoViewModel responeData = new ImageInfoViewModel();  
  17.             try {  
  18.                 HttpClient client = new HttpClient();  
  19.                 // Request headers.    
  20.                 client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);  
  21.                 // Request parameters. A third optional parameter is "details".    
  22.                 string requestParameters = "visualFeatures=Categories,Description,Color";  
  23.                 // Assemble the URI for the REST API Call.    
  24.                 string uri = endPoint + "?" + requestParameters;  
  25.                 HttpResponseMessage response;  
  26.                 // Request body. Posts a locally stored JPEG image.    
  27.                 byte[] byteData = GetImageAsByteArray(imageFilePath);  
  28.                 using(ByteArrayContent content = new ByteArrayContent(byteData)) {  
  29.                     // This example uses content type "application/octet-stream".    
  30.                     // The other content types you can use are "application/json"    
  31.                     // and "multipart/form-data".    
  32.                     content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");  
  33.                     // Make the REST API call.    
  34.                     response = await client.PostAsync(uri, content);  
  35.                 }  
  36.                 // Get the JSON response.    
  37.                 var result = await response.Content.ReadAsStringAsync();  
  38.                 if (response.IsSuccessStatusCode) {  
  39.                     responeData = JsonConvert.DeserializeObject < ImageInfoViewModel > (result, new JsonSerializerSettings {  
  40.                         NullValueHandling = NullValueHandling.Include,  
  41.                             Error = delegate(object sender, Newtonsoft.Json.Serialization.ErrorEventArgs earg) {  
  42.                                 errors.Add(earg.ErrorContext.Member.ToString());  
  43.                                 earg.ErrorContext.Handled = true;  
  44.                             }  
  45.                     });  
  46.                 }  
  47.             } catch (Exception e) {  
  48.                 Console.WriteLine("\n" + e.Message);  
  49.             }  
  50.             return responeData;  
  51.         }  
  52.         static byte[] GetImageAsByteArray(string imageFilePath) {  
  53.             using(FileStream fileStream = new FileStream(imageFilePath, FileMode.Open, FileAccess.Read)) {  
  54.                 BinaryReader binaryReader = new BinaryReader(fileStream);  
  55.                 return binaryReader.ReadBytes((int) fileStream.Length);  
  56.             }  
  57.         }  
  58.     }  
  59. }  
API Response – Based on the given Image
 
The successful json response.
  1. {  
  2.     "categories": [{  
  3.         "name""people_group",  
  4.         "score": 0.6171875,  
  5.         "detail": {  
  6.             "celebrities": []  
  7.         }  
  8.     }, {  
  9.         "name""people_many",  
  10.         "score": 0.359375,  
  11.         "detail": {  
  12.             "celebrities": []  
  13.         }  
  14.     }],  
  15.     "description": {  
  16.         "tags": ["person""sitting""indoor""posing""group""people""man""photo""woman""child""front""young""table""cake""large""holding""standing""bench""room""blue"],  
  17.         "captions": [{  
  18.             "text""a group of people sitting posing for the camera",  
  19.             "confidence": 0.9833507086594954  
  20.         }]  
  21.     },  
  22.     "color": {  
  23.         "dominantColorForeground""White",  
  24.         "dominantColorBackground""White",  
  25.         "dominantColors": ["White""Black""Red"],  
  26.         "accentColor""AD1E3E",  
  27.         "isBwImg"false  
  28.     },  
  29.     "requestId""89f21ccf-cb65-4107-8620-b920a03e5f03",  
  30.     "metadata": {  
  31.         "height": 346,  
  32.         "width": 530,  
  33.         "format""Jpeg"  
  34.     }  
  35. }  
Download
Output
 
Image information captured using Computer Vision API. For demo purposes, I have taken only a small amount of data even though you can get more information about the image.
 
Cognitive Services
Reference
  1. Computer Vision API
  2. Analysis an Image Docs
  3. API documentation
  4. TechBlog Article Content
See Also

You can download other ASP.NET Core source codes from MSDN Code, using the link, mentioned below.

Summary

 
From this article we have learned how to implement one of the important Cognitive Services APIs ( Computer Vision API ). I hope this article is useful for all Azure Cognitive Services API beginners.