Receipt Processing Using AI Builder In PowerApps

Overview

 

This article is about receipt processing using AI Builder. During this session, we will cover how we can process receipts with the Power Apps Receipt Processing AI model.

 

During this session, we will cover the following things,

• What is the Receipt Processing AI Model?
• Real life use cases and examples
• Key considerations before we proceed for receipt processing model
• Best practices
• Building Power Apps to scan the receipts and store the useful information to SharePoint lists

So, now let’s get started!

 

Overview of Receipt Processing AI Model

• Receipt Processing is the Pre-Build model of AI Builder.
• Receipt Processing uses the OCR (Optical Character Reading) technique to detect printed and handwritten text from receipts.

Real life use cases where we can use the Receipt Processing Model

 

Consider an example of an “Employee Expense” tracking system where we are keeping track of each employee's expenses throughout the year. The employee can simply upload the receipt through the Power Apps interface and all the information will be pre-populated in the SharePoint form. Then, Employees can submit their expenses with a single click for the next approval. All the information from the receipt will be detected automatically.

 

Consider the below example

 

 

Here, with the easy interface of Power Apps, we can scan receipts and pre-populate information in the grid. From there, employees can submit the form for the next approval.

 

Key Considerations before proceeding

 

We need to consider the following things before we proceed for receipt processing AI Builder

• Supported image format: JPEG, PNG, or PDF.
• File size: < 20 MB.
• Image dimensions must be between 50 x 50 pixels and 10000 x 10000 pixels.
• PDF dimensions must be at most 17 x 17 inches, which is the equivalent of the Legal or A3 paper sizes or smaller. PDF
• For PDF documents, only the first 200 pages are processed.

Best Practices

 

 

 

 

Let’s get started!

 

At the end of this article series, we will be able to create the app like the following.

 

Users can upload the receipts using Power Apps.

 

Once the image is processed, it will autodetect all fields.

 

 

The data will be stored in SharePoint list at last

 

 

Download Code Files: You can download the files from my GitHub Account.

 

So, now let’s get started!

 

Build List Schema

 

Consider the following list schema.

 

We have two SharePoint lists.

 

Employee receipts

 

 

Purchased Items

 

 

Basic information is stored in the Employee receipts list. Purchased items will be stored in the “Purchased Items” section.

 

Build Power Apps

 

There are two ways to create Power Apps.

 

From AI Builder Section,

 

Click on Receipt Processing from AI Builder

 

 

Click on Use in App option.

 

 

This will create new Power Apps with Phone Layout.

 

 

Directly Using Canvas App

 

We can also create Power Apps using the Canvas App option.

 

 

Select Tablet Layout

 

 

This will create a blank Power Apps.

 

We will create Power Apps using the second option for Tablet layout.

 

Step 1

 

Make a data source connection with two SharePoint lists.

 

 

Step 2

 

Add Receipt Processor AI Builder control in Power Apps.

 

 

 

Step 3

 

Add SharePoint OOTB form for the Employee Receipt list.

 

 

Once we add form, this will look like below.

 

Step 4

 

Now, Add the following code on change event of event builder.

ResetForm(Form1);
NewForm(Form1);
ClearCollect(
    PurchasedItemsCollection,
    ReceiptProcessor1.PurchasedItems
);
UpdateContext({Gridshow: true});

 

Step 5

 

Now, we need to pre-populate detected information to the form fields.

 

We can get the following fields from AI Builder control.

 

Property	Definition
MerchantName	Merchant name
MerchantAddress	Merchant address
MerchantPhone	Merchant phone number
TransactionDate	Transaction date
TransactionTime	Transaction time
PurchasedItems	The list of purchased items Name: Name of the purchased item Price: Price of the purchased item Quantity: Quantity of the purchased item TotalPrice: Total price of the purchased item
Subtotal	Subtotal
Tax	Tax
Tip	Tip
Total	Total
DetectedText	The list of all recognized lines of text on the receipt

 

So, we need to add Default value in every form field.

 

 

Data Card Name	Default Event
Title_DataCard1	ReceiptProcessor1.MerchantName
MerchantAddress_DataCard1	ReceiptProcessor1.MerchantAddress
MerchantPhone_DataCard1	ReceiptProcessor1.MerchantPhone
TransactionDate_DataCard1	ReceiptProcessor1.TransactionDate
DateValue1	DataCardValue4
Transaction Time_DataCard1	ReceiptProcessor1.TransactionTime
Sub Total_DataCard1	ReceiptProcessor1.Subtotal
Tax_DataCard1	ReceiptProcessor1.Tax
Total_DataCard1	ReceiptProcessor1.Total
Tip_DataCard1	ReceiptProcessor1.Tip

 

Step 5

 

Add Datatable and set the following properties,

 

Data source = PurchasedItemsCollection (This is one which we created in Step 4)

 

Visible: Gridshow

 

 

Step 6

 

Add the Save button.

 

Configure the following properties,

 

OnSelect: SubmitForm(Form1);

 

Step 7

 

Add Bulk Insert code for the multiple items submit at once to the Grid on form’s On Success.

 

 

Bulk Insert Grid Data to Purchase item list. This will loop on the Grid and add the form's last submitted ID to Receipt ID Column.

ForAll(
    PurchasedItemsCollection,
    Patch(
        'Purchased Items',
        Defaults('Purchased Items'),
        {
            Title: ThisRecord.Name,
            Price:ThisRecord.Price,
            'Total Price': ThisRecord.TotalPrice,
            Quantity: Value(ThisRecord.Quantity),
            'Receipt ID': Form1.LastSubmit.ID
        }
    )
);
UpdateContext({Gridshow:false});

Step 8

 

Create Clear Button and add the following line of code over there.

 

 

Clear(PurchasedItemsCollection);ResetForm(Form1);Reset(ReceiptProcessor1);UpdateContext({Gridshow:false});

Let’s test!

 

The app will look like this.

 

 

Scan Receipt. This will auto-populate fields.

 

 

 

Conclusion

 

This is how we can scan receipts using the Power Apps receipt Processing AI Builder Model and Store the Information to SharePoint list.

 

Happy Power Apping!!