Custom model Evaluation Metric

In building machine learning models, There comes a time you might want to evaluate your model to know it's a performance using a metric you created. while scikit-learn with all it's beauty provides the evaluation metrics we need, there comes a time we might need to create our own metrics. scikit-learn makes this easy by providing a function make_scorer.
to create a custom metric, all you need to do is create a function containing the metric definition and convert it to a scorer function using scikit-learn make_scorer. we will use the make_regression dataset to show an example of creating a custom metric below.

let's begin by importing the neccessary libraries

# Load libraries from sklearn.metrics import make_scorer, r2_score from sklearn.model_selection import train_test_split from sklearn.linear_model import Ridge from sklearn.datasets import make_regression 

Next we generate features and the target matrix.

features, target = make_regression(n_samples = 100,n_features = 4, random_state = 1) 

random_state parameter ensures we get the same distribution when we use the same random_state value.

let's continue by creating the training and testing set using train_test_split

X_train, X_test, y_train, y_test = train_test_split(features, target, test_size=0.10, random_state=1) 

then create our custom metric function.

def custom_metric(X_test, target_predicted): # Calculate r-squared score  r_squared = r2_score(X_test, target_predicted) # Return r-squared score  return r_squared 

next convert the custom metric into a scorer metric

# Make scorer and define that higher scores are better score = make_scorer(custom_metric, greater_is_better=True) 

greater_is_better signifies that higher score is better

Create the regression and train.

classifier = Ridge() # Train ridge regression model model = classifier.fit(X_train, y_train) 

Finally apply the custom metric to check model performance.

score(model, X_test, y_test)