Machine learning

Elavarasan R

7y2kRobotics & Hardware

What is the different between traing set and test set ? can give a real time example?

3 Replies

Know the answer? Post it — somebody with the same question will find it here.

Mehreen Tahir7y ago
Accepted answer
Usually you need two types of datasets when performing supervised learning:
The dataset that has the input data with correct/expeced output which would be used to train your model. This dataset is known as training set.

The data you are going to apply your model to. This is the data in which you are interested for the output of your model and want the "expected" output.

So there're going to be two phases while performing machine learning. Training Phase, in which you will present your data from training set and train your model, by pairing the input with expected output. For example we are interested in predicting the type of diabetes a patient has. Our training data will contain all the information related to patient i.e. height, weight, BMI, age, blood sugar level etc. along with the expected out either type A or B. Now comes the Test Phase, in which you'll apply your freshly-developed model to the real-world data and get the results. In this phase, we will feed our model with the test set that will contain the similar patiet information and our model will predict either patient is suffering from type A diabetes or type B. This phase will help you estimate how well your model has been trained. Most commonly the partition ratio of training and test set is 70/30.
Rohit Gupta7y ago
A dataset can be divided into 3 parts:
1. Test Set
It is the part of the dataset which is used for Black box testing, i.e. here the motive is to test the model on data that the model has never seen before. Here we use metrics like the Confusion Matrix, accuracy score and F1 score.
2. Training Set
It is the part of the dataset which is used to train the model and make the hypothesises more accurate
3. Validation set
In machine learning, when we generate a model, we need to validate the model before sending it for the final testing. So to validate the working of the model we use the validation set

For Example:
Let us take the MNIST cloth dataset

Here we would like to generate a model that is able to predict the type and genre of the cloth.
So to do so, we would divide the whole fo the dataset into a ratio of 7:3 where 70% of the data is the training+Validation Data and 30% is the Test data

In Python to segregate the training and test data, we use the sklear.model_selection.train_test_split()
X_train, X_test, y_train, y_test = train_test_split(
... X, y, test_size=0.3, random_state=0)
The above code demostrates how to divide the dataset in the 7:3 ratio
Md Sarfaraj7y ago
Go to below link https://stats.stackexchange.com/questions/19048/what-is-the-difference-between-test-set-and-validation-set