Face and Emotion Detection using Azure Cognitive Services SDK on Python

Introduction

In today's digital age, understanding and analyzing human emotions have become crucial in various fields such as marketing, healthcare, and entertainment. With advancements in artificial intelligence (AI), face and emotion detection technologies have gained significant momentum. Microsoft Azure Cognitive Services provides a powerful solution for developers to integrate face and emotion detection capabilities into their applications effortlessly. In this article, we will explore how to utilize Azure Cognitive Services SDK on Python to detect faces and emotions, opening up a world of possibilities for emotion-driven applications.

Understanding Azure Cognitive Services

Azure Cognitive Services is a comprehensive set of AI services provided by Microsoft Azure, enabling developers to incorporate AI capabilities into their applications without the need for extensive machine learning expertise. Among its rich collection of services, Azure Cognitive Services offers Face API, which allows for detecting and analyzing faces within images or video streams, and Emotion API, which accurately recognizes emotions expressed by individuals.

Setting up Azure Face resource

Go to Azure Portal and search Face API, then click on Create.

Choose the subscription, resource group, region, pricing tier, and type the resource name. Then, click on Review + create.

Once the resource is created, go to Keys and Endpoint to copy your credentials.

Detecting Faces and Recognizing Emotions using Azure Cognitive Services SDK on Python

You need to install the Azure Cognitive Services SDK. You can do this by running the following command in your Python environment:

pip install azure-cognitiveservices-vision-face

After the Azure Cognitive Services SDK was installed, we can import the needed libraries.

from azure.cognitiveservices.vision.face import FaceClient
from msrest.authentication import CognitiveServicesCredentials
from PIL import Image, ImageDraw, ImageFont

Create the FaceClient using the API Key and Endpoint you copied previously.

APIKEY = ""
ENDPOINT = ""

face_client = FaceClient(ENDPOINT, CognitiveServicesCredentials(APIKEY))

To test choose any image with at least one person that is showing his face.

single_image_name = "funny-curly-haired-woman-glasses-taking-selfie.jpg"
image = open(single_image_name, 'r+b')

To see the available attributes we can return, check this page.

To mitigate potential misuse that can subject people to stereotyping, discrimination, or unfair denial of services, Microsoft is retiring Face API attributes that predict emotion, gender, age, smile, facial hair, hair, and makeup. Read more about this decision here. Microsoft will also retire the Snapshot API, which allowed biometric data transfer from one Face subscription to another. Existing customers have until 30 June 2023 to use the emotion, gender, age, smile, facial hair, hair, and makeup attributes and the Snapshot API through Face API before they are retired.

face_attributes = ['emotion']
detected_faces = face_client.face.detect_with_stream(image, return_face_attributes = face_attributes)
if not detected_faces:
	raise Exception('No face detected from image {}'.format(single_image_name))

Next, we will create the methods to get the rectangle coordinates and the emotion of the detected faces.

def get_rectangle(face_dictionary):
    rect = face_dictionary.face_rectangle
    left = rect.left
    top = rect.top
    right = left + rect.width
    bottom = top + rect.height

    return ((left, top), (right, bottom))

 We can have more than one emotion from the response, but in this method, we are getting the emotion with the highest level of confidence.

def get_emotion(emotion):
    max_emotion_value = 0.0
    emotion_type = None

    for emotion_name, emotion_value in vars(emotion).items():
        if emotion_name == "additional_properties":
           continue

        if emotion_value > max_emotion_value:
           max_emotion_value = emotion_value
           emotion_type = emotion_name

    return emotion_type

For this example, we are going to use Arial font. You can use whatever you want. First, you must download the TTF file for that font.

def detect_faces_and_emotions(image_name: str, detected_faces, font_size_percentage = 5):
    img = Image.open(image_name)

    font_size = round(img.height * font_size_percentage / 100)
    font = ImageFont.truetype('arial.ttf', font_size)

    draw = ImageDraw.Draw(img)
    for face in detected_faces:
        rect = get_rectangle(face)
        draw.rectangle(rect, outline='red')
        face_emotion = get_emotion(face.face_attributes.emotion)
        draw.text(rect[0], face_emotion, 'white', font=font)

    return img

Finally, call the detect_faces_and_emotions method passing the image name, detected faces response, and the font size percentage (5 by default).

detect_faces_and_emotions(single_image_name, detected_faces, 3)

You can find the full source code here.

Conclusion

The combination of face and emotion detection using Azure Cognitive Services SDK on Python opens up a world of possibilities in various domains. By integrating these capabilities into applications, developers can unlock the potential of emotional intelligence, enabling personalized user experiences, sentiment analysis, and more. With Azure Cognitive Services, harnessing the power of AI for face and emotion detection has never been easier, empowering developers to create innovative and empathetic solutions that better understand and respond to human emotions.

Thanks for reading

Thank you very much for reading. I hope you found this article interesting and may be useful in the future. If you have any questions or ideas you need to discuss, it will be a pleasure to collaborate and exchange knowledge.