Introduction

In this article, I shall show you how to build a text recognition application by using the device camera. Through this tutorial, I would like to present to readers the amazing feature of Mobile Vision API: Text recognition by using a mobile camera.

Text Recognition

Mobile Vision is an API which helps us to find the text in pictures and video streams to observe the content checked in that.

Structure of Text

The text recognizer breaks the content into pieces, lines, and words.

  1. A word is a contiguous arrangement of alphanumeric characters on a similar vertical axis.
  2. A line is an adjoining set of words on a similar vertical axis.
  3. A chunk is a bordering set of text lines.


Step 1

Open Visual Studio. Go to New Project-> Templates-> Visual C#-> Android-> Blank app. Select Blank app. Give the project
a name like TextRecognizerByCamera.

ProjectName: TextRecognizerByCamera



Step 2

First of all, we need to add a component that is required for text recognition. Open Solution Explorer and go to Components -> Get More Components. In this way, you can move on Xamarin Components Store, then search for Vision and click "Add to App".


Step 3

We need a permission from the device because we shall be using the device’s camera to capture texts. Please add CAMERA permission to your AndroidManifest.xml.

Let's open Solution Explorer-> Properties-> AndroidManifest and let's add the code inside application tags.


Camera Permission
  1. <uses-permission android:name="android.permission.CAMERA" />  
  2. <application android:allowBackup="true" android:label="@string/app_name">  
  3.     <meta-data android:name="com.google.android.gms.vision.DEPENDENCIES" android:value="ocr" />   
  4. </application>
Step 4

Open Solution Explorer-> Project Name-> Resources-> Layout-> Main.axml and add the following code.

The layout will have a SurfaceView in order to display the preview frames captured by the camera. I also added a TextView to display the contents of the recognized text.

(FileName: Main.axml)

XAML Code
  1. <?xml version="1.0" encoding="utf-8"?>  
  2. <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" android:padding="16dp" android:layout_width="match_parent" android:layout_height="match_parent">  
  3.     <SurfaceView android:id="@+id/surface_view" android:layout_width="match_parent" android:layout_height="match_parent" />  
  4.     <TextView android:id="@+id/txtview" android:text="No Text" android:textSize="20sp" android:textColor="@android:color/white" android:layout_width="match_parent" android:layout_height="wrap_content" android:layout_alignParentBottom="true" />   
  5. </RelativeLayout>
Layout

Step 6

Open Solution Explorer-> Project Name-> MainActivity and add the following code to main activity and use appropriate namespaces.
Complete Code of Main Activity
  1. using Android.App;  
  2. using Android.Widget;  
  3. using Android.OS;  
  4. using Android.Views;  
  5. using Android.Gms.Vision;  
  6. using Android.Gms.Vision.Texts;  
  7. using Android.Util;  
  8. using Android.Graphics;  
  9. using Android.Runtime;  
  10. using Android.Support.V4.App;  
  11. using Android;  
  12. using Android.Content.PM;  
  13. using static Android.Gms.Vision.Detector;  
  14. using System.Text;  
  15. namespace CameraTextRecognizer {  
  16.     [Activity(Label = "CameraTextRecognizer", MainLauncher = true)]  
  17.     public class MainActivity: Activity, ISurfaceHolderCallback, IProcessor {  
  18.         private SurfaceView cameraView;  
  19.         private TextView txtView;  
  20.         private CameraSource cameraSource;  
  21.         private  
  22.         const int RequestCameraPermissionID = 1001;  
  23.         public override void OnRequestPermissionsResult(int requestCode, string[] permissions, [GeneratedEnum] Permission[] grantResults) {  
  24.             switch (requestCode) {  
  25.                 case RequestCameraPermissionID:  
  26.                     {  
  27.                         if (grantResults[0] == Permission.Granted) {  
  28.                             cameraSource.Start(cameraView.Holder);  
  29.                         }  
  30.                     }  
  31.                     break;  
  32.             }  
  33.         }  
  34.         protected override void OnCreate(Bundle savedInstanceState) {  
  35.             base.OnCreate(savedInstanceState);  
  36.             // Set our view from the "main" layout resource  
  37.             SetContentView(Resource.Layout.Main);  
  38.             cameraView = FindViewById < SurfaceView > (Resource.Id.surface_view);  
  39.             txtView = FindViewById < TextView > (Resource.Id.txtview);  
  40.             TextRecognizer txtRecognizer = new TextRecognizer.Builder(ApplicationContext).Build();  
  41.             if (!txtRecognizer.IsOperational) {  
  42.                 Log.Error("Main Activity", "Detector dependencies are not yet available");  
  43.             } else {  
  44.                 cameraSource = new CameraSource.Builder(ApplicationContext, txtRecognizer).SetFacing(CameraFacing.Back).SetRequestedPreviewSize(1280, 1024).SetRequestedFps(2.0 f).SetAutoFocusEnabled(true).Build();  
  45.                 cameraView.Holder.AddCallback(this);  
  46.                 txtRecognizer.SetProcessor(this);  
  47.             }  
  48.         }  
  49.         public void SurfaceChanged(ISurfaceHolder holder, [GeneratedEnum] Format format, int width, int height) {}  
  50.         public void SurfaceCreated(ISurfaceHolder holder) {  
  51.             if (ActivityCompat.CheckSelfPermission(ApplicationContext, Manifest.Permission.Camera) != Android.Content.PM.Permission.Granted) {  
  52.                 //Request permission  
  53.                 ActivityCompat.RequestPermissions(this, new string[] {  
  54.                     Android.Manifest.Permission.Camera  
  55.                 }, RequestCameraPermissionID);  
  56.                 return;  
  57.             }  
  58.             cameraSource.Start(cameraView.Holder);  
  59.         }  
  60.         public void SurfaceDestroyed(ISurfaceHolder holder) {  
  61.             cameraSource.Stop();  
  62.         }  
  63.         public void ReceiveDetections(Detections detections) {  
  64.             SparseArray items = detections.DetectedItems;  
  65.             if (items.Size() != 0) {  
  66.                 txtView.Post(() => {  
  67.                     StringBuilder strBuilder = new StringBuilder();  
  68.                     for (int i = 0; i < items.Size(); ++i) {  
  69.                         strBuilder.Append(((TextBlock) items.ValueAt(i)).Value);  
  70.                         strBuilder.Append("\n");  
  71.                     }  
  72.                     txtView.Text = strBuilder.ToString();  
  73.                 });  
  74.             }  
  75.         }  
  76.         public void Release() {}  
  77.     }  
  78. }

Output

Running this project, and scanning a text block on the paper, you may have result like this.