In this article, you will learn how to evaluate and understand the performance of your regression models to make informed decisions on your next steps.
A regression model predicts the continuous value from a set of input features. For example: predicting the weight of a fetus, given the input features such as the week of pregnancy, mother’s height, sex of the fetus, etc.
For more information about regression, check out this article.
What are regression evaluation metrics in general?
Once a model is trained, you will need an unused set of input records, with true feature values to evaluate its performance. This is typically achieved by splitting available data into train and test sets. You will then need to pass these inputs through your model, get a set of predictions, and then compare these predictions to the set of true values. Regression evaluation metrics achieve this through various ways, while providing an overall estimate of how well the model is performing. Both train/test split and evaluation of metrics are automatically handled by the AI & Analytics Engine.
For more information about train/test split, check out this article.
For more information about regression evaluation metrics, check out this article.
How to evaluate regression models in the Engine?
You must first have a trained regression model.
Basic evaluation:
-
Navigate to the leaderboard page
-
On that page, you will see a short summary of your top models
-
Each model will show a prediction quality, ranging (usually) from 0% to 100%
Summary of a model from the leaderboard page
Advanced evaluation:
The Engine has several advanced evaluation metric displays.
-
Navigate to the selected model details → Insights
-
Within “insights”, advanced metrics will be displaying the following:
-
Prediction Quality, Prediction Error and Percentage error
-
The advanced evaluation metrics' display
b. Prediction error breakdown table: This table allows “splitting” the range of predicted values. The approximated error metrics are displayed for each split
Prediction errors breakdown table
c. A detailed view which includes:
i. a full list of error metrics such as “Mean Absolute Error”, “Mean absolute Percentage Error”, etc
A full list of regression error metrics of a trained model, evaluated on a test set
ii. plots of the predicted vs. actual values, the residuals distribution and the residuals vs. predicted
values
Sample distribution of residuals |
Sample predicted vs. actual feature value |
Sample residuals vs. predicted feature. |