PowerApps AI - Object Detection


Artificial Intelligence, is the latest trend in the industry.  Machine learning and AI are interrelated.  Machine learning, is a technique where machines learn using various algorithms and tagged data.  Artificial Intelligence, is a technique, using which machines can execute tasks smartly, by applying machine learning.
PowerApps provides a rapid development environment to build custom business applications.  Furthermore, it automates business processes by leveraging flows and providing rich integration, with data sources using CDS, online, and on-premise.
AI builder, which is still in preview, provides an opportunity to integrate PowerApps and AI to automate processes, predict sales, and classify data, which will help in tagging data, by providing meaningful insights to organizations.  Object detection can be used to detect images and tag them, while form processing can help the insurance industry to process claims faster, by extracting data from documents.  This AI integration will turn out to be a power booster for O365.
You need not be a data scientist to use AI Builder, as all the algorithms are embedded in the tool.  The most important aspect is having data, it can be either text, images, or pdf.  In addition to this, understanding about associating model with data is required as well.  Choosing this correctly will enable machines to create correct patterns, learn, and help in achieving automation goals.


In today's instance, I will take object detection, as an example, and we will see how object detection model can tag images.


Common data source service is like a relational database, which stores data in the entity, similar to tables.  Rules and business logic can be applied to the data source itself, so there is no need to recreate these rules again in PowerApps.  There are various ways to do a mass upload of data onto CDS.  For example, CSV, JSON, XML, Azure, SQL Server and many more.

AI Builder(Preview)

To use AI, your data should be preloaded into CDS.  You may choose from one of the below models, depending on the requirement.
Please refer to GA dates to see the dates for general availability.
Model Category
Binary Classification Prediction
Text Classification Language
Form Processing Vision
Object Detection Vision
Object Detection
This AI model needs atleast 15 images.
We will use the below steps to achieve a working object detection model.

Add Image Name 

Create a Model Driven App to add data to AI Object Detection Image Entity.  Follow below steps
PowerApps AI - Object Detection
PowerApps AI - Object Detection
PowerApps AI - Object Detection
PowerApps AI - Object Detection 

AI Builder

Let's go to AI Builder Preview.  There may be some changes in steps or UI, when this becomes GA, but the overall concept should remain the same.
Click on Build. Select Object Detection.
PowerApps AI - Object Detection
PowerApps AI - Object Detection
Select the Entity and check Name 
PowerApps AI - Object Detection
Now we will be able to select orange.
PowerApps AI - Object Detection
Let's Add Images, upload atleast 15 images,
  PowerApps AI - Object Detection
Once the images are uploaded, use the selector to mark image for orange.  Once all the images are tagged.  Train your model.
PowerApps AI - Object Detection
Let's test the model.

Create Canvas App

PowerApps AI - Object Detection
Publish the App, upload image of an orange.  Here, I used a fresh orange from my fruit basket which I had not used to train.
As shown below the image of an orange is detected. 
PowerApps AI - Object Detection


In this article, we learned the basic concepts and steps to implement Object Detection model.