How Is The Communication Between Machine Learning Models And Applications Performed In Production Environment

In this article, we’ll learn about how communication is done between the machine learning models and the full-fledged application that uses components of artificial intelligence. In today’s world where artificial intelligence is adding value to multitudes of domains, let us learn how software applications are enabled with AI.

Different methods of deployment were discussed in the previous article, Machine Learning Workflow and Methods of deployment. With deployment, the model can be made available for usage through a web application or software. Data can be inputted by the user into the model and predictions can be received back through the application from the model.

Let us take an instance of a recommender system such as an eCommerce website or YouTube, which takes in data from users that will be used to predict the product or video the user would like. The Model will take in data from the application and provide predictions using which different services, products, music, and videos can then be recommended to the users. This has been possible with Artificial Intelligence, Machine Learning, and Deep Learning. The major contribution to the rise of an app like Spotify goes to the recommendation system which helps users explore similar music of their choice which might have been impossible for any regular person to find.

So how does this work behind the scene? This article discusses it in brief.

All these applications that millions of users use, run on the production environment. What is production environment do you ask? Known as Production or the Production Environment, it refers to the application's location for the public and the intended users. Bugs should be fixed before launching the system to the production, with all testing and evaluation maintaining all the necessary requirements of coding standards and practices on the production code.

Now, how does all the communication within the application and model work in production? To answer this question, we need to know about endpoint. The endpoint is basically an interface to model. In our case of Machine Learning Models, the endpoint is the gateway that facilitates the communication between the application and the model. To summarize, the functioning of this interface we call endpoints, we can point it down as following.

Endpoint enables the application to send data from the user to the model
The prediction output from the model is then received back which is based upon the data inputted from the user.

The image below explains this communication channel visually.

To understand this with a program, let us consider ml_app.py which is a python program. In this program, the endpoint is similar to the function call that helps connect model and application whereas the function itself is a model which performs training and evaluation. The entire program ml_app.py represents the application. The description association of the elements of the sample program to model, application, and endpoint can be seen below.

def main():
    #Obtain data from user
    input_user_data = get_user_data()

    #Receive predictions based upon the data fetched from user
    prediction = ml_model(input_user_data)

    #Showcase prediction to user
    showcase_prediction_to_user(predictions)

def ml_model(user_data):
    loaded_data = load_user_data(user_data)

In the program above, the program ml_app.py represents the application. The Function ml_model can be specified as the model and the function call ‘prediction’ represents the endpoint.

In Line 6,

prediction = ml_model(input_user_data)

This is the function call to ml_model which is the endpoint.

In Line 12,

def ml_model(user_data):

It defines the model.

Similarly, the Python program ml_app.py represents the application.

From the above example, we can realize that the application similar to the python program itself, displays the prediction of the model to the application user. This will be enabled by the communication channel or interface of the endpoint – here, the function call which accepts the user data as the input and then returns the prediction of the model which is completely based upon this input accessed through the endpoint. Here, the input argument is the user data and the returned value from the function call is the prediction from the model.

This example reaffirms how the endpoint is basically just an interface between the application and the model that enables users to obtain predictions from the deployed model which is based upon the data retrieved from the user.

How does endpoint facilitate the communication between the model and application?

The communication between the model and application happens through the interface we know as endpoint such that the endpoint is an API. API is basically an abbreviation of Application Programming Interface that are a set of rules which enables programs to communicate with each other and in our case, the model and the application. The API uses REST Architecture, which gives a framework for the rules and constraints that need to be adhered to for the communication to occur. REST API uses HTTP requests and responses in order to provide communication through the endpoint for application and model.

HTTP Request

The HTTP (Hypertext Transfer Protocol) request sent from the application to the model comprises four components.

Endpoint

The endpoint would be in form of URL ie. Web address.

HTTP Method

Out of four HTTP Methods, GET, POST, PUT and DELETE, for deployment only POST method is used in the application.

HTTP Headers

Information such as the format of the data possessed in the message for the receiving program is contained in the HTTP headers.

Message

In our case, data of the user that is the input for the model is present in the message for deployment.

Similarly, HTTP Response that is sent to the application from the model consists of three constituents.

HTTP Status Code

When the model successfully receives and processes the data from the user, the status code starts with 2 such as 200.

HTTP Headers

Information such as the format of the data in the message which is passed to the receiving program is present in the HTTP Headers.

Message

The prediction provided by the model is present in the message.

This prediction is finally presented to the user via an application that is enabled by the endpoint (interface) to connecting the model and application through REST API. Moreover, the data from the user would need to be in the format of JSON or CSV with certain ordering based upon the model and the final predictions will also be returned in the JSON or CSV format with the proper ordering which is based upon the model used.

Conclusion

Hence, in this article, we learned about the model, application, and endpoints on the production environment and how data from users are shared to models and the predicted values obtained to showcase the users. The image with the block diagram visually represented such application and the illustration of the program was done to understand the ideology of endpoint, model, and application. Then, we learned how endpoint facilitates the communication between application and model. Finally, HTTP Request and HTTP Responses’ constituents' parts were discussed.