Multi-Layout Form Processing Using AI Builder

Introduction

 
Sometimes, we have a scenario where we have a different layout for our invoices and forms. In this article, we will talk about how we can process multi-layout forms with the single AI builder form processing model. Let's get started!
 
The scenario and real-life use case
 
Let's consider an example of one of the finance organizations. They are generating a different invoice for each organization every month. The invoice layout is different for each company. During this entire article series, we will build our form processing a model with two different layouts of invoices, and based on that we will train our model. So, whenever we upload any new invoices from the power apps, it will pre-populate every field within the form and store all information within the SharePoint list.
 
Different Form Layouts.
 
Consider the following two invoices for the “Learn2All” and “PP Trainings” Organization.
 
Both companies have different Layouts
 
 
 
Multi Layout Form Processing using AI Builder
 
You can see here; some fields are available only within “PP Trainings” invoice.
 
Now, we need to build one generalize AI Model considering all the fields.
 
There are best practices we need to know before we get started!
 
Group documents with the same layout inside a collection. Don't use documents with different layouts in the same collection.
 
Multi Layout Form Processing using AI Builder
 
Use documents with primary text. Don't use documents with lots of images or checkboxes.
 
Use simple documents. Don't use files containing multiple documents.
 
Multi Layout Form Processing using AI Builder
 
Group documents with the same layout inside a collection. Don't use documents with different layouts in the same collection.
 
Multi Layout Form Processing using AI Builder
 
5+ documents with the same layout needed.
 

Overall Procedure for Form Processing

 
Step 1
 
Teach the model about your documents
 
Multi Layout Form Processing using AI Builder
 
Step 2
 
Tag documents and then train your model
 
Multi Layout Form Processing using AI Builder
 
Step 3
 
Publish and use your new AI model
 
Multi Layout Form Processing using AI Builder
 
Now let’s get started!
  • Open Power Apps Portal.
  • From Left Navigation Click on AI Builder and Expand Build Section.
Multi Layout Form Processing using AI Builder 
  • Click on Form Processing. Give the name for your Form Processing Model.
Multi Layout Form Processing using AI Builder 
  • Define Fields and Tables.
Multi Layout Form Processing using AI Builder 
  • Here, we will take the maximum no of fields from both invoices and define them by clicking on the + icon.
  • In a similar manner, it goes to the Tables section and defines the table and its field. Click on the new Table and give the name of your table. It also defines all the columns.
Multi Layout Form Processing using AI Builder 
  • Click on Next.
  • Click on New Collection.
Multi Layout Form Processing using AI Builder 
  • Change the name of the collection and click on the + icon to upload documents.
Multi Layout Form Processing using AI Builder 
  • Upload 5 documents for each type of invoice.
Multi Layout Form Processing using AI Builder 
  • We created two collections with both types of invoices and uploaded 5 documents in each collection.
Multi Layout Form Processing using AI Builder 
  • Click on Analyze
Multi Layout Form Processing using AI Builder 
  • This will show the following pop up.
  • Now, we need to tag our documents.
Multi Layout Form Processing using AI Builder 
  • Select any collection to tag documents.
  • When we open invoices, this will show a dotted line around the detected words. Make sure you clicked on “Show detected words” from the top navigation.
  • To tag the fields select fields and map appropriate fields like this.
Multi Layout Form Processing using AI Builder 
Multi Layout Form Processing using AI Builder 
Multi Layout Form Processing using AI Builder 
  • Follow the same procedure for all the fields.
  • Now, to detect a table, click on the table icon and click on “Tag table”
Multi Layout Form Processing using AI Builder
  • Select the table name.
Multi Layout Form Processing using AI Builder 
  • Then define each column as well for the table.
Multi Layout Form Processing using AI Builder 
  • Once all the fields from the form are detected this will show a blue tick like the below image.
Multi Layout Form Processing using AI Builder 
  • Follow the same procedure for all collection and all documents.
  • If any of the fields is not a part of the form then, Click on the ellipses after the Form name and select “Not available in documents”.
Multi Layout Form Processing using AI Builder 
  • Once you train all documents, click on Next.
Multi Layout Form Processing using AI Builder 
  • This will show the model summary. Now, click on Train.
Multi Layout Form Processing using AI Builder 
  • This will show “Your Model is Training”.
Multi Layout Form Processing using AI Builder 
  • Click on “Go to Details Page”.
Multi Layout Form Processing using AI Builder 
  • Here, you can Quick Test the document. I’m uploading one sample.
Multi Layout Form Processing using AI Builder 
  • All the fields from “Learn2All” have been detected perfectly.
Multi Layout Form Processing using AI Builder 
  • Now, make sure the model is Published.
In my next article, we will learn how we can consume Form Processing AI Builder in Power Apps. Stay tuned to the next session.
 

Conclusion

 
This is how we can Build, Train, and Publish Multi Layout forms using a Single Form Processing AI Model. Happy Power Apping!