Next Gen Computer Vision Capabilities with Project Florence Foundation Model

Introduction

This article will help you learn how to unleash your creativity and innovation using Computer Vision in the Azure AI Studio. I also will explore the different capabilities of Computer Vision Studio.

Computer Vision 

Computer Vision is an area of artificial intelligence that deals with visual perception and its unified service that offers innovative vision studio capabilities with the Project Florence Foundation model. It provides your apps the ability to read text, analyze images and detect faces with technology like OCR and Spatial Analysis.

Project Florence

Project Florence is a Microsoft AI Cognitive Services initiative and also advances the state of the art of computer vision technologies like OCR, Spatial Analysis, and Image Analysis.

Project Florence, which helps to develop the next-generation framework for visual recognition.

Computer Vision Studio Capabilities

Computer Vision Studio offers different capabilities

• Video summary and frame locator: Video search and summarization uses a combination of natural language processing and computer vision techniques to analyze the content of a video.
  It can quickly and concisely summarize the main points of a video and allows you to search for specific moments within the video, making it easy to find relevant content.
• Remove backgrounds from images: Easily remove the background and preserve foreground elements in your image.
• Search photos with image retrieval: Retrieve specific moments within your photo album. For example, you can query a wedding you attended last summer, your pet, or your favorite city. Search for images based on the content of the image itself rather than relying solely on manually assigned keywords or tags.
• Add captions to images: Generate a human-readable sentence that describes the content of an image.
• Add dense captions to images: Generate human-readable captions for all important objects detected in your image.
• Detect common objects in images: Detect and extract bounding boxes based on thousands of recognizable objects and living beings.
• Extract common tags from images: Extract tags based on thousands of recognizable objects, living beings, scenery, and actions.
• Detect sensitive content in images: Detect sensitive content in images so you can moderate their usage in your applications. You can use this feature to detect when images contain Adult, Racy, or Gory content.
• Recognize and analyze shelf products: Find specific objects within a single image for use cases, such as locating products on shelves, merchandise on a store display, or items in an assembly line.
• Detect faces in an image: Detect the location of one or more human faces in images, along with attributes such as pose, face mask, and facial landmarks.Azure.

Computer Vision Steps

We have the following steps to create a Computer Vision Service in the Azure Portal.

Go to the Azure portal and sign in with your Azure account.

Search "Computer Vision" in the search bar and select "Computer Vision" from the search results.

Click on the Create button for Computer Vision.

In the Basics tab, provide the following information Choose the Subscription.

Then Create a Resource Group named testRG.

Choose the Region as East US and type the name as retailvision26.

Select the Pricing tier as Free F0 tier.

Check the  Acknowledge box for accepting the terms and conditions.

Click the Next button on the Computer Vision Page.

Click Review + Create button.

You will get a validation message as Validation Passed.

Click Create button.

Deployment started initializing in a minute or two this became successful.

Click Go to Resource button, and then you will be able to find the Vision Studio.

Click Vision Studio from the Computer Vision page.

Glimpses of Vision Studio

Video Summary and frame locator

This feature is in the PREVIEW mode and chooses a video clip to see the summarization and frame locator capabilities. Also, the users can upload the file in the Blob container from the Azure Portal.

Here the users clicked the Datacenter video that showcases a data center, depicting the hardware that powers it and the people that maintain it.

Videos that have been uploaded to Vision Studio will be stored in your account for 48 hours. After that, it will be deleted automatically.

You can see that Computer Vision is used in various use cases like Retail, Factory, and Warehouse.

Click Run a test and then choose the option to locate specific frames using a search query.

In the Search Query, the users type the query as a Person with a laptop and then click the Run button.

In the Results section, Vision Studio completely searched the query, and it will match the query within the specific time frame, which satisfies the query as a Person with a laptop.

Remove backgrounds from images

This feature is in the PREVIEW mode, and the users can upload the images in the Blob container from the Azure Portal.

This feature easily removes backgrounds from the images.

Now the users can see the background tree images are removed from the original image.

Search photos with image retrieval

This feature is in the PREVIEW mode, and the users can upload the images in the Blob container from the Azure Portal.

Here the users can choose the Nature gallery from the menu.

The user wants to a retrieve query as Dew drops on a plant and then clicks the Search button.

The Query results vary from most relevant in the dataset to least relevant. Use the slider to view more or less images based on their relevance to the retrieved query.

Summary

In this article, we successfully learned and deployed the Computer Vision Studio. We explored the different Vision Studio Capabilities, which become a Visual treat for human beings. Using Vision Studio, the possibilities, and opportunities for innovation are unlimited.

Thanks for reading this article!!!