Xamarin.Android - Recognize Celebrities By Camera Using Computer Vision

Introduction

In this article, you will learn how to consume cognitive services for recognizing a celebrity in Xamarin.Android by using a mobile camera. I hope you will learn some cool stuff in Xamarin using cognitive services.

Xamarin.Android - Recognize Celebrities By Camera

Prerequisites

Computer Vision API Key
Microsoft.Net.Http
Newtonsoft.Json

Computer Vision API keys

Computer vision services require special subscription keys. Every call to the Computer Vision API requires a subscription key. This key needs to be either passed through a query string parameter or specified in the request header.

To sign up for subscription keys, see Subscriptions. It's free to sign up. Pricing for these services is subject to change.

If you sign up using the Computer Vision free trial, your subscription keys are valid for the west-central region (https://westcentralus.api.cognitive.microsoft.com).

The steps given below are required to be followed in order to create a Celebrities Recognize app in Xamarin.Android, using Visual Studio.

Step 1 - Create an Android Project

Create your Android solution in Visual Studio or Xamarin Studio. Select Android and from the list, choose Android Blank App. Give it a name, like RecognizeCelebritiesbyCamera.

(ProjectName: RecognizeCelebritiesbyCamera)

Step 2 - Add References of Nuget Packages

First of all, in References, add the reference of Microsoft.Net.Http and Newtonsoft.Json using NuGet Package Manager, as shown below.

Step 3 - User Interface

Open Solution Explorer-> Project Name-> Resources-> Layout-> Main.axml and add the following code. The layout will have an ImageView in order to display the preview of Celebrities image. I also added a TextView to display the contents of the Celebrities and two buttons.

(FileName: Main.axml)

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent">
<ImageView
android:id="@+id/image"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:layout_above="@+id/group_button" />
<LinearLayout
android:layout_above="@+id/txtDescription"
android:id="@+id/group_button"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:orientation="horizontal"
android:weightSum="2">
<Button
android:id="@+id/btnCapture"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:layout_weight="1"
android:text="Capture" />
<Button
android:id="@+id/btnProcess"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:layout_weight="1"
android:text="Process" />
</LinearLayout>
<TextView
android:id="@+id/txtDescription"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:text="Description: "
android:textSize="20sp"
android:layout_alignParentBottom="true" />
</RelativeLayout>

Step 4 - Analysis Model Class

Add a new class in your project with name AnalysisModel.cs. Add the following properties to get the result set from JSON response with the appropriate namespace.

using System.Collections.Generic;
namespace RecoginizeCelebrities
{
public class AnalysisModel
{
public IList<Category> categories { get; set; }
public object adult { get; set; }
public IList<Tag> tags { get; set; }
public Description description { get; set; }
public string requestId { get; set; }
public Metadata metadata { get; set; }
public IList<Face> faces { get; set; }
public Color color { get; set; }
public ImageType imageType { get; set; }
}
public class FaceRectangle
{
public int left { get; set; }
public int top { get; set; }
public int width { get; set; }
public int height { get; set; }
}
public class Celebrity
{
public string name { get; set; }
public FaceRectangle faceRectangle { get; set; }
public double confidence { get; set; }
}
public class Detail
{
public IList<Celebrity> celebrities { get; set; }
public object landmarks { get; set; }
}
public class Category
{
public string name { get; set; }
public double score { get; set; }
public Detail detail { get; set; }
}
public class Tag
{
public string name { get; set; }
public double confidence { get; set; }
}
public class Caption
{
public string text { get; set; }
public double confidence { get; set; }
}
public class Description
{
public IList<string> tags { get; set; }
public IList<Caption> captions { get; set; }
}
public class Metadata
{
public int width { get; set; }
public int height { get; set; }
public string format { get; set; }
}
public class Face
{
public int age { get; set; }
public string gender { get; set; }
public FaceRectangle faceRectangle { get; set; }
}
public class Color
{
public string dominantColorForeground { get; set; }
public string dominantColorBackground { get; set; }
public IList<string> dominantColors { get; set; }
public string accentColor { get; set; }
public bool isBWImg { get; set; }
}
public class ImageType
{
public int clipArtType { get; set; }
public int lineDrawingType { get; set; }
}
}

Step 5 - Backend Code

Now, go to Solution Explorer-> Project Name-> MainActivity and add the following code with appropriate namespaces.

Note
Please replace your subscription key and your selected region address in the Main activity class.

(FileName: MainActivity)

using Android.App;
using Android.Widget;
using Android.OS;
using Android.Support.V7.App;
using Android.Graphics;
using System.Net.Http;
using System.IO;
using Android.Content;
using Android.Runtime;
using Android.Content.PM;
using Android;
using Android.Provider;
using System.Threading.Tasks;
using Newtonsoft.Json;
using System;
using System.Net.Http.Headers;
namespace RecoginizeCelebrities
{
[Activity(Label = "@string/app_name", Theme = "@style/AppTheme", MainLauncher = true)]
public class MainActivity : AppCompatActivity
{
const string subscriptionKey = "3407ad6140b240f58847194ebf0dc26d";
const string uriBase = "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0/analyze";
ImageView imageView;
Bitmap mBitMap;
int CAMERA_CODE = 1000, CAMERA_REQUEST = 1001;
ByteArrayContent content;
TextView txtDes;
Button btnProcess, btnCapture;
public override void OnRequestPermissionsResult(int requestCode, string[] permissions, [GeneratedEnum] Permission[] grantResults)
{
base.OnRequestPermissionsResult(requestCode, permissions, grantResults);
if (requestCode == CAMERA_CODE)
{
if (grantResults[0] == Permission.Granted)
Toast.MakeText(this, "Permission Granted", ToastLength.Short).Show();
else
Toast.MakeText(this, "Permission Not Granted", ToastLength.Short).Show();
}
}
protected override void OnActivityResult(int requestCode, [GeneratedEnum] Result resultCode, Intent data)
{
base.OnActivityResult(requestCode, resultCode, data);
if (requestCode == 0 && resultCode == Android.App.Result.Ok &&
data != null)
{
mBitMap = (Bitmap)data.Extras.Get("data");
imageView.SetImageBitmap(mBitMap);
byte[] bitmapData;
using (var stream = new MemoryStream())
{
mBitMap.Compress(Bitmap.CompressFormat.Jpeg, 100, stream);
bitmapData = stream.ToArray();
}
content = new ByteArrayContent(bitmapData);
}
}
protected override void OnCreate(Bundle savedInstanceState)
{
base.OnCreate(savedInstanceState);
// Set our view from the "main" layout resource
SetContentView(Resource.Layout.activity_main);
//Request runtime permission
if (CheckSelfPermission(Manifest.Permission.Camera) == Android.Content.PM.Permission.Denied)
{
RequestPermissions(new string[] { Manifest.Permission.Camera }, CAMERA_REQUEST);
}
txtDes = FindViewById<TextView>(Resource.Id.txtDescription);
imageView = FindViewById<ImageView>(Resource.Id.image);
btnProcess = FindViewById<Button>(Resource.Id.btnProcess);
btnCapture = FindViewById<Button>(Resource.Id.btnCapture);
btnCapture.Click += delegate
{
Intent intent = new Intent(MediaStore.ActionImageCapture);
StartActivityForResult(intent, 0);
};
btnProcess.Click += async delegate
{
await MakeAnalysisRequest(content);
};
}
public async Task MakeAnalysisRequest(ByteArrayContent content)
{
try
{
HttpClient client = new HttpClient();
// Request headers.
client.DefaultRequestHeaders.Add(
"Ocp-Apim-Subscription-Key", subscriptionKey);
string requestParameters =
"visualFeatures=Categories&details=Celebrities";
// Assemble the URI for the REST API method.
string uri = uriBase + "?" + requestParameters;
content.Headers.ContentType =
new MediaTypeHeaderValue("application/octet-stream");
// Asynchronously call the REST API method.
var response = await client.PostAsync(uri, content);
// Asynchronously get the JSON response.
string contentString = await response.Content.ReadAsStringAsync();
var analysesResult = JsonConvert.DeserializeObject<AnalysisModel>(contentString);
txtDes.Text = "Name: " + analysesResult.categories[0].detail.celebrities[0].name.ToString();
}
catch (Exception e)
{
Toast.MakeText(this, "" + e.ToString(), ToastLength.Short).Show();
}
}
}
}

Results of Analyzing the Celebrities

Just capture the image and hit "Analyze". You will get the same results as shown below.

14 Comments

Join the conversation! Your thoughts help the community grow.

נועה עין צבי6y ago
What should I do if I want to recognize persons from my own database?
- Ahsan Siddique6y ago
  Then you will need to develop your own service.
Sid Olu7y ago
Nice one, weldone.
- Ahsan Siddique7y ago
  Thank you Sid Olu.
Sarathlal Saseendran7y ago
Good thought Ahsan. Thanks for sharing
- Ahsan Siddique7y ago
  You are welcome sarathlal. I have published more stuff like this, but unfortunately all stuff is in review status. So stay tuned.
- Ahsan Siddique7y ago
  I also published this article for Xamarin.forms
- Ahsan Siddique7y ago
  Yeah I have done with face detection using face API just article is pending.