Getting Started With Machine Learning .NET (ML.NET)

Introduction

In Build 2018, Microsoft introduced the preview of ML.NET (Machine Learning .NET) which is a cross-platform, open-source Machine Learning Framework. Yes, now it is easy to develop our own Machine Learning application or develop custom modules using Machine Learning Framework. ML.NET is a Machine Learning framework that was mainly developed for .NET developers. We can use C# or F# to develop ML.NET applications. ML.NET is open source and cross-platform and can run on Windows, Linux, and macOS. ML.NET is still in development and now we can use the preview version to work and play with ML.NET.

ref link

Introducing ML.NET: Cross-platform, Proven and Open Source Machine Learning Framework

In this article, we will see how to develop our first ML.NET application to predict the item stock quantity.

Machine Learning Regression

In this sample program, we will be using the Machine Learning Regression of ML.NET to predict the Item Stock. Regression is a statistical method to find the relation between variables, for example, in our demo program we will be predicting the stock item based on the existing stock dataset.

Ref link

Regression, Taxi fare predictor (regression)

Things to know before starting ML.NET

Initialize the Model

For working with Machine Learning, first, we need to pick our best-fit machine learning algorithm. Machine learning has clustering, regression, classification and anomaly detection modules. Here in this article, we will be using the regression model for predicting the item in stock.

Train

We need to train the Machine Learning Model. Training is the process of analyzing input data by model. The training is mainly used for the model to learn the pattern and save it as a trained model. For example, we will be creating a CSV file in our application and in the CSV file, we will be giving the stock details such as ItemID, Location, InQTY, OutQTY, ItemType, and TotalStock quantity. We give around 100 records in the CSV file as a sample with all necessary details. We need to give this CSV file as an input to our model. Our model needs to be trained and using this data our model needs to be analyzed to predict the result. Once our model is trained and understands the pattern of the input the resultant model needs to be saved to predict the result.

Score

The score is also called the prediction, where the score needs to have the same column as the trained model. The score generates the result based on the trained model.

Evaluate

Evaluate will be performed after the model training is completed. To evaluate the trained model it will be compared with the test data to compare and predict the final result to be produced.

Ref link - https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/machine-learning-modules

Prerequisites:

Make sure you have installed all the prerequisites on your computer. If not, then download and install Visual Studio 2017 15.6 or later with the ".NET Core cross-platform development" workload installed.

Code Part

Step 1 - Create a C# Console Application

After installing the prerequisites, click Start >> Programs >> Visual Studio 2017 >> Visual Studio 2017 on your desktop. Click New >> Project. Select Visual C# >> Windows Desktop >> Console APP (.Net Framework). Enter your project name and click OK.

Getting Started with Machine Learning DotNet (ML.NET)

Step 2 – Add Microsoft ML package

Right-click on your project and click on Manage NuGet Packages.

Select the Browse tab and search for Microsoft.ML

Click on Install, I accept it and wait until the installation is complete.

We can see Microsoft.ML package has been installed and all the references for Microsoft.ML has been added to our project references.

Step 3 – Creating Train and Evaluate Data

Now, we need to create a Model training and evaluate the dataset. For creating this we will add two CSV files, one for training and one for the evaluation. We will create a new folder called data in our project to add our CSV files.

Add Data Folder

Right-click the project and Add New Folder and name the folder as “Data”

Creating a Train CSV file

Right-click the Data folder click on Add >> New Item >> select the text file and name it as “StockTrain.csv”

Select the properties of the “StockTrain.csv” change the Copy to Output Directory to “Copy always”

Add your CSV file data like below.

Here, we have added the data with the following fields.

Note that we need to fix the label and the features.

Label

A Label is a field that we need to predict in our sample. Here TotalStockQty is the Label where we will be predicting the total remaining Stock Quantity of an item. We make the other field features.

ItemID - Stock Item ID (Feature)
Loccode – Warehouse Location of the Item (Feature)
InQty – Total Item received in location (Feature)
OutQty – Total Item delivered from the location (Feature)
ItemType - ItemType “In” means as local manufactured and “out” means as outsourced. (Feature)
TotalStockQty - Total no of item stock in location. (Label)

Note

We need a minimum of 100 records of data to be added to train our Model.

Like this, we will add one more CSV file and add similar information for evaluating. We have created one more CSV file named “StockTest.csv”

Step 4 – Creating Class for Input Data and Prediction

Now, we need to create a class for Input Data and prediction. For doing this right-click our project and add a new class and name it as “ItemStock.cs”

In our class, first, we need to import the Microsoft.ML.Runtime.API

using Microsoft.ML.Runtime.Api;

Next, we need to add all our columns like our CSV file in the same order in our class and set the column as 0 to 5.

public class ItemStock
{
[Column("0")]
public string ItemID;
[Column("1")]
public float Loccode;
[Column("2")]
public float InQty;
[Column("3")]
public float OutQty;
[Column("4")]
public string ItemType;
[Column("5")]
public float TotalStockQty;
}

Now, we need to create a prediction class and, in this class, we need to add our prediction column. The prediction column is the same as our label. Our label is “TotalStockQty”. Our aim of this program is to predict and display the final result of the Total Stock quantity from the trained model. We add “TotalStockQty” as a prediction column in our prediction class.

Note

Important to be noted is in the prediction column we need to set the column name as the “Score” and also set the data type as the float.

In the regression model, the scoring column contains the predicted results.

public class itemStockQtyPrediction
{
[ColumnName("Score")]
public float TotalStockQty;
}

Step 5 – Program.cs

To work with ML.NET, we open our “program.cs” file and first, we import all the needed ML.NET references.

using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Models;
using Microsoft.ML.Trainers;
using Microsoft.ML.Transforms;

Also import the below to your program.cs file.

using System.IO;

Dataset Path

We set the Traincsv data, Evaluate data and Model data path. For the traindata we give “StockTrain.csv” path and do the same for the evaluation CSV file.

The final trained model needs to be saved for evaluating and producing results. For this, we set the modelpath with the “Model.zip” file. The trained model will be saved in the zip file automatically during the runtime of the program in our bin folder with all needed files.

static readonly string _Traindatapath = Path.Combine(Environment.CurrentDirectory, "Data", "StockTrain.csv");
static readonly string _Evaluatedatapath = Path.Combine(Environment.CurrentDirectory, "Data", "StockTest.csv");
static readonly string _modelpath = Path.Combine(Environment.CurrentDirectory, "Data", "Model.zip");

Change the Main method to async Task Main method like the below code,

static async Task Main(string[] args)
{
}

Before doing this, we need to perform 2 important tasks to successfully run our program

The first is to set Platform Target as x64. The ML.NET only runs in x64, for doing this right-click the project and select properties >> Select Build and change the Platform target to x64.

In order to run with our async Task Main method, we need the change the Language version to C#7.1

In the Project Properties >> Build tab >> click on the Advanced button at the bottom and change the Language Version to C#7.1

Working with Training Model

First, we need to train the model and save the model to the zip file. For this in our main method we call the predictionModel method and pass the ItemStock and itemStockQtyPrediction class and return the model to the main method:

static async Task Main(string[] args)
{
PredictionModel<ItemStock, itemStockQtyPrediction> model = await TrainourModel();
}
public static async Task<PredictionModel<ItemStock, itemStockQtyPrediction>> TrainourModel()
{
}

Train and Save Model

In the above method, we add the function to train the model and save the model to the zip file.

LearningPipeline

In training, the first step will be working the LearningPipeline().

The LearningPipeline loads all the training data to train the model.

TextLoader

The TextLoader is used to get all the data from a train CSV file for training and here we set the useHeader: true to avoid reading the first row from the CSV file.

ColumnCopier

In the training model, the important part is the predicted values to be defined. In our example, we have set the TotalStockQty as the label. In the training mode, we need to set the prediction label by using the ColumnCopier. We can see in our below code part in ColumnCopier we have given the TotalStockQty and Label.

CategoricalOneHotVectorizer

The training model needs numeric features to transform the categorical data like our ItemID, ItemType into numbers.

ColumnConcatenator

Next, we add all our featured columns to be trained and evaluated.

Adding Learning Algorithm

FastTreeRegressor

The learner will train the model. We have selected regression for our sample and we will be using FastTreeRegressor learner. FastTreeRegressor is one of the regression learners provided by the ML.NET. Here we add the FastTreeRegressor to our pipeline.

Train and Save Model

Finally, we will train and save the model from this method.

public static async Task<PredictionModel<ItemStock, itemStockQtyPrediction>> TrainourModel()
{
var pipeline = new LearningPipeline
{
new TextLoader(_Traindatapath).CreateFrom<ItemStock>(useHeader: true, separator: ','),
new ColumnCopier(("TotalStockQty", "Label")),
new CategoricalOneHotVectorizer(
"ItemID",
"ItemType"),
new ColumnConcatenator(
"Features",
"ItemID",
"Loccode",
"InQty",
"OutQty",
"ItemType"),
new FastTreeRegressor()
};
PredictionModel<ItemStock, itemStockQtyPrediction> model = pipeline.Train<ItemStock, itemStockQtyPrediction>();
await model.WriteAsync(_modelpath);
return model;
}

Evaluate Model

Next, we need to evaluate the model by checking the model with the test data and predicting the final results. For this, we call the method to Evaluate our main method and pass the trained model to this method.

static async Task Main(string[] args)
{
PredictionModel<ItemStock, itemStockQtyPrediction> model = await TrainourModel();
Evaluate(model);
}

In the Evaluate method, we pass the trained model. We need to evaluate the model with our sample CSV test data. In this method using textloader, we load all the data from the test data CSV file for evaluating the model.

Using RegressionEvaluator method we evaluate the model with the testdata and produce evaluation metrics

We display both the RMS (Root mean Squared) and RSquared metrics value.

RMS is one of evaluation metrics where the lower RMS value is treated as the better model.

RSquared is another evaluation metric where the value will be between 0 to 1. If the value is closer to 1 is the better model.

private static void Evaluate(PredictionModel<ItemStock, itemStockQtyPrediction> model)
{
var testData = new TextLoader(_Evaluatedatapath).CreateFrom<ItemStock>(useHeader: true, separator: ',');
var evaluator = new RegressionEvaluator();
RegressionMetrics metrics = evaluator.Evaluate(model, testData);
Console.WriteLine($"Rms = {metrics.Rms}");
Console.WriteLine($"RSquared = {metrics.RSquared}");
}

Prediction Results

Now its time for us to produce the result of predicted results by model. For this we will add one more class and, in this Class, we will give the inputs.

Create a new Class named “ItemStocks.cs“

We add the values to the ItemStock Class which we already created and defined the columns for Model training. Here, we have created stock input and we have given the values as below,

Note

In each Stock, we haven’t given the input for the TotalStockQty and we have set the TotalStockQty = 0 as the model needs to predict the result from the trained model and produce the result.

static class ItemStocks
{
internal static readonly ItemStock stock1 = new ItemStock
{
ItemID = "Item001",
Loccode = 1,
InQty = 100,
OutQty = 10,
ItemType = "IN",
TotalStockQty = 0 // predict it. Actual Total Stock Quantity is = 90
};
internal static readonly ItemStock stock2 = new ItemStock
{
ItemID = "Item003",
Loccode = 4,
InQty = 6000,
OutQty = 1200,
ItemType = "IN",
TotalStockQty = 0 // predict it. Actual Total Stock Quantity is = 4800
};
}

We can see in our StockTrain.csv file we have added the values of above stock1 and we can see as we have given TotalStockQty as 90 and the Model needs to predict this result and produce the result.

Same for Stock2 -- we have added the values in the StockTrain.csv file, we can see as we have given TotalStockQty as 4800 and the Model needs to predict this result and produce the result.

Produce the Model Predicted results

In our program's main method, we will add the below code at the bottom after Train and Evaluate method call to predict the Stock Quantity details and display the final predicted results from the model to users in the command window.

Here we display both Stock1 and Stock2 prediction results. The prediction.TotalStockQty will have the model predicted results and we display the results to the end-users of the predicted Stock Quantity with actual input given by us.

Console.WriteLine("-------First Prediction Vale : ----------------" );
itemStockQtyPrediction prediction = model.Predict(ItemStocks.stock1);
Console.WriteLine("Item ID {0}", ItemStocks.stock1.ItemID);
Console.WriteLine("Predicted Stock: {0}, actual Stock Qty: 90 ", prediction.TotalStockQty);
Console.WriteLine(" ");
Console.WriteLine("----------Next Prediction : -------------");
Console.WriteLine(" ");
prediction = model.Predict(ItemStocks.stock2);
Console.WriteLine("Item ID {0}", ItemStocks.stock2.ItemID);
Console.WriteLine("Predicted Stock: {0}, actual Stock Qty: 4800 ", prediction.TotalStockQty);
Console.ReadLine();

Build and Run

When we run the program, we can see the result in the command window like below.

We can see the RMS value as 0.052 as we already discussed in this article if the Rms vale is near to 0 then the model is good in prediction and also, we can see the RSquared =0.999. As we already discussed in this article, if the RSquared value is near to 1 then the model is good and we can see the predicted result of both the Stock1 and Stock2 as both have predicted the same Stock quantity as the actual one. Our Model is very good in prediction and the predicted value is exactly the same as the actual value.

Trained Model Information

After we build and run the program we can see the Model.zip file in the Data folder inside the Application root bin folder. We can also see our StockTrain and StockTest CSV files. The Model will be trained with this CSV file and the trained model result will be stored in Model.zip file for predicting the result.

Conclusion

ML.NET (Machine Learning DotNet) is a great framework for all the dotnet lovers who are all looking to work with machine learning. Now the only preview version of ML.NET is available and I can’t wait until the release of the public version of ML.NET. Here in this article, I have used the regression model to predict and display the Total Stock Quantity. If you are .Net lovers, not aware of Machine Learning and looking forward to working with machine learning then ML.Net is for you. Hope you all enjoy reading this article and see you soon with another post.

18 Comments

Join the conversation! Your thoughts help the community grow.

Nildo Araujo1y ago
Great job. Congratulations!
sudhakar reddy5y ago
Hi Syed, how do we predict the values from Database table data instead of .CSV file, do you have any examples regarding that.
- somnath chaturvedi5y ago
  Just read to database instead of csv
Shawn Eary6y ago
Not sure if I'm missing something, but is there somewhere I can go to specify how many seconds TrainourModel will run? Since your post, Microsoft has released an ML.NET Builder and it allows you to specify how long to train the model, but when it is done writing the code, I don't see where it is saving the option that specifies how long the training should take place.
Sourav Kumar Das7y ago
Nice Article.
Shakeel Khan7y ago
Nice and clearly described
sreenivasa k7y ago
Hi , it was nice article thanks for sharing.
Eduardo Gandorphi7y ago
Hi Syed, i wais trying your sample out but after using my data i keep getting this message in the console 'Not training a calibrator because it is not needed.' and then a TargetInvocationExceptoion is thrown. could you help me?
Manoj Kumawat7y ago
Hi, It's a great article. There is several algorithms for using in pipeline so how can i find suitable algorithm for a problem. It will be great if you suggest any article about it or you write an article. Thanks.
jagath prasad8y ago
new FastTreeRegressor() is not working properly and it throws error for me.
Mayur Mathkar8y ago
Very nice article..Thanks a lot..
Prabu Elavarasan8y ago
ItemID had conflicts with ItemId also just for my convenience i modified the location code as string instead of float. So finally i modified the itemid prop and Loc code data type as float then everything went well. Thanks much Shanu.
Prabu Elavarasan8y ago
ItemID had conflicts with ItemId also just for my convenience i modified the location code as string instead of float. So finally i modified the itemid prop and Loc code data type as float then everything went well. Thanks much Shanu.
Prabu Elavarasan8y ago
Still trying to clear it...
- Syed Shanu8y ago
  It seems like you ahve error at this line new ColumnConcatenator( "Features", "ItemID", "Loccode", "InQty", "OutQty", "ItemType"), .Check well as you have same column name in Class .Iam also planing for a webinar in this topic hope that will be helful to get to know more in detail.
Prabu Elavarasan8y ago
Hi Shanu, I just followed your instruction as is. However, it ends up with the Exception "ArgumentOutOfRangeException: Column 'Features' has invalid source types: All source columns must have the same type. Source types: 'Vec<R4, 5>, Text, R4, R4, Vec<R4, 2>'.Parameter name: Column"
Naresh Singhal8y ago
Awesome article , It's really very good to read ..Thanks Sir Syed Shanu
Sundaram Subramanian8y ago
Fabulous writing. It was a great introduction to ML