Receipt Processing Using AI Builder In PowerApps

Overview

 
This article is about receipt processing using AI Builder. During this session, we will cover how we can process receipts with the Power Apps Receipt Processing AI model.
 
During this session, we will cover the following things,
  • What is the Receipt Processing AI Model?
  • Real life use cases and examples
  • Key considerations before we proceed for receipt processing model
  • Best practices
  • Building Power Apps to scan the receipts and store the useful information to SharePoint lists
So, now let’s get started!
 
Overview of Receipt Processing AI Model
  • Receipt Processing is the Pre-Build model of AI Builder.
  • Receipt Processing uses the OCR (Optical Character Reading) technique to detect printed and handwritten text from receipts.

Real life use cases where we can use the Receipt Processing Model

 
Consider an example of an “Employee Expense” tracking system where we are keeping track of each employee's expenses throughout the year. The employee can simply upload the receipt through the Power Apps interface and all the information will be pre-populated in the SharePoint form. Then, Employees can submit their expenses with a single click for the next approval. All the information from the receipt will be detected automatically.
 
Consider the below example
 
Receipt Processing Using AI Builder In PowerApps
 
Here, with the easy interface of Power Apps, we can scan receipts and pre-populate information in the grid. From there, employees can submit the form for the next approval.
 

Key Considerations before proceeding

 
We need to consider the following things before we proceed for receipt processing AI Builder
  • Supported image format: JPEG, PNG, or PDF.
  • File size: < 20 MB.
  • Image dimensions must be between 50 x 50 pixels and 10000 x 10000 pixels.
  • PDF dimensions must be at most 17 x 17 inches, which is the equivalent of the Legal or A3 paper sizes or smaller. PDF
  • For PDF documents, only the first 200 pages are processed.
Best Practices
 
Receipt Processing Using AI Builder In PowerApps
 
Receipt Processing Using AI Builder In PowerApps
 
Receipt Processing Using AI Builder In PowerApps
 
Let’s get started!
 
At the end of this article series, we will be able to create the app like the following.
 
Users can upload the receipts using Power Apps.
 
Receipt Processing Using AI Builder In PowerApps
Once the image is processed, it will autodetect all fields.
 
Receipt Processing Using AI Builder In PowerApps
 
The data will be stored in SharePoint list at last
 
Receipt Processing Using AI Builder In PowerApps
 
Download Code Files: You can download the files from my GitHub Account.
 
So, now let’s get started!
 

Build List Schema

 
Consider the following list schema.
 
We have two SharePoint lists.
 
Employee receipts
 
Receipt Processing Using AI Builder In PowerApps
 
Purchased Items
 
Receipt Processing Using AI Builder In PowerApps
 
Basic information is stored in the Employee receipts list. Purchased items will be stored in the “Purchased Items” section.
 

Build Power Apps

 
There are two ways to create Power Apps.
 
From AI Builder Section,
 
Click on Receipt Processing from AI Builder
 
Receipt Processing Using AI Builder In PowerApps
 
Click on Use in App option.
 
Receipt Processing Using AI Builder In PowerApps
 
This will create new Power Apps with Phone Layout.
 
Receipt Processing Using AI Builder In PowerApps
 
Directly Using Canvas App
 
We can also create Power Apps using the Canvas App option.
 
Receipt Processing Using AI Builder In PowerApps
 
Select Tablet Layout
 
Receipt Processing Using AI Builder In PowerApps
 
This will create a blank Power Apps.
 
We will create Power Apps using the second option for Tablet layout.
 
Step 1
 
Make a data source connection with two SharePoint lists.
 
Receipt Processing Using AI Builder In PowerApps
 
Step 2
 
Add Receipt Processor AI Builder control in Power Apps.
 
Receipt Processing Using AI Builder In PowerApps
 
Receipt Processing Using AI Builder In PowerApps
 
Step 3
 
Add SharePoint OOTB form for the Employee Receipt list.
 
Receipt Processing Using AI Builder In PowerApps
 
Once we add form, this will look like below.
 
Step 4
 
Now, Add the following code on change event of event builder.
  1. ResetForm(Form1);  
  2. NewForm(Form1);  
  3. ClearCollect(  
  4.     PurchasedItemsCollection,  
  5.     ReceiptProcessor1.PurchasedItems  
  6. );  
  7. UpdateContext({Gridshow: true});  
Receipt Processing Using AI Builder In PowerApps
 
Step 5
 
Now, we need to pre-populate detected information to the form fields.
 
We can get the following fields from AI Builder control.
 
Property
Definition
MerchantName
Merchant name
MerchantAddress
Merchant address
MerchantPhone
Merchant phone number
TransactionDate
Transaction date
TransactionTime
Transaction time
PurchasedItems
The list of purchased items
  • Name: Name of the purchased item
  • Price: Price of the purchased item
  • Quantity: Quantity of the purchased item
  • TotalPrice: Total price of the purchased item
Subtotal
Subtotal
Tax
Tax
Tip
Tip
Total
Total
DetectedText
The list of all recognized lines of text on the receipt
 
So, we need to add Default value in every form field.
 
Receipt Processing Using AI Builder In PowerApps
 
Data Card Name
Default Event
Title_DataCard1
ReceiptProcessor1.MerchantName
MerchantAddress_DataCard1
ReceiptProcessor1.MerchantAddress
MerchantPhone_DataCard1
ReceiptProcessor1.MerchantPhone
TransactionDate_DataCard1
ReceiptProcessor1.TransactionDate
DateValue1
DataCardValue4
Transaction Time_DataCard1
ReceiptProcessor1.TransactionTime
Sub Total_DataCard1
ReceiptProcessor1.Subtotal
Tax_DataCard1
ReceiptProcessor1.Tax
Total_DataCard1
ReceiptProcessor1.Total
Tip_DataCard1
ReceiptProcessor1.Tip
 
Step 5
 
Add Datatable and set the following properties,
 
Data source = PurchasedItemsCollection (This is one which we created in Step 4)
 
Visible: Gridshow
 
Receipt Processing Using AI Builder In PowerApps
 
Step 6
 
Add the Save button.
 
Configure the following properties,
 
OnSelect: SubmitForm(Form1);
 
Step 7
 
Add Bulk Insert code for the multiple items submit at once to the Grid on form’s On Success.
 
Receipt Processing Using AI Builder In PowerApps
 
Bulk Insert Grid Data to Purchase item list. This will loop on the Grid and add the form's last submitted ID to Receipt ID Column.
  1. ForAll(  
  2.     PurchasedItemsCollection,  
  3.     Patch(  
  4.         'Purchased Items',  
  5.         Defaults('Purchased Items'),  
  6.         {  
  7.             Title: ThisRecord.Name,  
  8.             Price:ThisRecord.Price,  
  9.             'Total Price': ThisRecord.TotalPrice,  
  10.             Quantity: Value(ThisRecord.Quantity),  
  11.             'Receipt ID': Form1.LastSubmit.ID  
  12.         }  
  13.     )  
  14. );  
  15. UpdateContext({Gridshow:false});  
Step 8
 
Create Clear Button and add the following line of code over there.
 
Receipt Processing Using AI Builder In PowerApps 
  1. Clear(PurchasedItemsCollection);ResetForm(Form1);Reset(ReceiptProcessor1);UpdateContext({Gridshow:false});  
Let’s test!
 
The app will look like this.
 
Receipt Processing Using AI Builder In PowerApps
 
Scan Receipt. This will auto-populate fields.
 
Receipt Processing Using AI Builder In PowerApps
 
Receipt Processing Using AI Builder In PowerApps
 

Conclusion

 
This is how we can scan receipts using the Power Apps receipt Processing AI Builder Model and Store the Information to SharePoint list.
 
Happy Power Apping!!