How to use the templated ML solution for customer churn prediction?

This article details the steps to use the templated ML solution for customer churn prediction.

How to create a customer churn prediction application?

1. Choose an existing project, or create a new one. Then on the project home page, under “Start New > App Builder”, select the “Use Template” option:

2. In the next screen showing available templates, choose “Customer Churn Prediction”:

3. Next, a dialogue will pop up with a pre-filled app name. Modify if necessary and click “Create”.

Upon clicking on “Create”, you will enter the app builder. On the left side of the app builder, you can see the list of steps you need to follow. On the right side, you can provide the necessary information. You can also click on any step on the left side, to go back to the previous steps and make changes.

The bottom right of the app builder has two action buttons:

1. Save and continue takes you to the next step in the app building process.

2. Save and Exit allows you to continue the app building process later. You can find the saved app in Draft status in the app listing page of your project:

Upon clicking on the draft app, you can see the steps completed and yet to be completed to finish the app building process, and click on “Continue building app” to continue where you left off:

---

How to provide input datasets to the customer churn prediction template?

The first step in the process of using the template is to provide the input datasets.

In the data upload sections, you would first need to provide the transactions dataset.

1. Click “Add dataset”:

2. This opens the “Add dataset” dialogue. If your dataset was not previously imported into the Engine, you can import it using one of the several data connectors, ranging from file upload to databases:

The rest of the process is the same as the generic process to import a dataset into the Engine.

If you have already imported the required dataset into the project before starting the customer churn template, you can choose it from the list of datasets in the project:

3. Once the transactions dataset is added, the Engine asks you to identify the columns that contain the:

• ID of the customer who made the transaction (each customer is assumed to have a unique ID)

• the date of the transaction, and

• the transaction amount

4. Click “Save and continue” to proceed to adding the customer information dataset (such as date of birth, gender, profession, etc.). The process to add this dataset is the same as the previous case. Here, you will be required to identify the column containing the customer IDs matching with the customer IDs in the transactions dataset:

This completes the process of providing input datasets to the customer churn template app.

---

How to define churn?

When using the customer churn prediction template, you can define "churn" as either a period of very few or no transactions, or transactions that add up to a very small amount. This means that your business needs to determine whether a customer has churned based on their level of transaction activity.

Businesses also need to decide when they want to be informed if a customer's transaction activity is decreasing. This is important so that they have enough time to implement customer retention strategies.

You can define two parameters in the next step. Enter the numbers specific to your business problem and click “Save and continue” to proceed further:

---

How to define contributing factors?

On the churn template, contributing factors (features) that can potentially foretell churn are computed from both the transactions dataset and the customer information dataset (if provided).

From the transactions dataset, the Engine automatically generates stats such as total amount credited/ debited, transactions count, and maximum interval between transactions in the specified time intervals.

The only required user input is specifying the time windows over which these stats need to be computed.

The time window of “Most recent 30 days” is chosen by default. You can edit / remove this default selection, and also add more time windows:

There are two options to add more time windows:

1. “Most recent N days” (N configurable to any number)

2. “Pick from a calendar” to select an M-day period ending K days ago (M, K are configurable to any number)

For the latter, you need to pick a date range from the calendar relative to the current date (the exact dates chosen here will not be used in the computation, it will depend on the date at which the prediction is made). If the current date is Feb 27, 2023 and you want a 30-day period ending 15 days ago, you will need to select the date range of Jan 14, 2023 to Feb 12, 2023:

As for how to decide what time windows to add in this section, that is entirely based on your domain knowledge and business intelligence. If you suspect that your customer’s activity in the previous two fortnights gives good indication that they are highly likely to churn after the specified period, then you should provide the time windows as “most recent 14 days” and “14-day period ending 14 days ago”.

From the customer information dataset, you can choose a sub-set of columns that you may want to consider as useful in predicting churn:

---

How to select & train your churn models?

Selecting models to train for the churn template is easy. You can either:

1. Direct the Engine to automatically find the top K algorithms (K configurable, default = 1) based on an analysis of your datasets, or

2. Choose specific algorithms (such as XGBoost, Random Forest, Logistic Regression etc.) to train, if you know the most suitable algorithms that work for you.

You can then click on “Start building” to commence the entire process. You will be taken to the App details page where you can see the progress of various steps:

When one or more models have been trained and evaluated and are ready for prediction while others are still being trained/evaluated, you can see a status indicator showing the number of models being trained/evaluated:

---

How to access model insights for a model under the churn template?

Once a model has finished training and evaluation, you will be able to access its insights. To access insights for models trained under the churn template, click on the “Models” tab of the particular churn template app’s detail page:

This will take you to the model leaderboard. In the section ranking trained models by their performance, click on the model that you are interested in:

This will take you to the model’s detail page, where you can access all insights under the “Insights” section:

---

How to understand your churn model performance?

To understand your churn model’s performance, follow the steps in the previous section to access the insights section of the model. Then click on the “Performance” card under the insights section:

This will take you to the detailed performance page of the model. Here, you can access the binary classification evaluation metrics, the precision-recall curve and the ROC curve:

---

How to make predictions from your churn model?

To make predictions from your churn model, you will need to wait for one or more models to finish training. You will then see the “Make predictions” button on your churn template app’s summary page. This will enable you to run predictions with the model that performs the best, among the ones fully trained and evaluated:

As shown in the above screenshot, a message suggesting you to wait will appear above the “Make prediction” button if there are models still being trained and evaluated. This is to ensure that you run predictions with the best model, in case the best model turns out to be the one that is yet to be trained/evaluated. When all models have been trained and evaluated, the best performing model on the test set will be highlighted:

You can also go to the model leaderboard and click on the “Use model to predict” button either from the top performing models' cards or by hovering over any of the rows in the model performance ranking table below:

 

In the next step, you need to choose what kind of predictions you want to make: "One-off prediction" or "Schedule periodic predictions":

---

How to schedule churn predictions?

When you want to schedule churn predictions, it means you want to give the ML pipeline built by the churn template access to your live data, to output the following periodically:

• List of currently active customers

• Their predicted probability of churn, as per the definition you provided while setting up the app.

To do this, you first need a live database containing up-to-date transaction data and customer-info data. Once you have this, click on “Make predictions” (as explained in the previous section). Then in the dialog asking how you would like to make your predictions, the “Schedule periodic predictions” option and continue:

Select the model that you want to predict with. The model with the best prediction quality is recommended and chosen by default.

In the next step, you will provide a connection to a database table/collection containing the latest transactions data. Provide to your data connection by choosing the database server type, and enter the credentials and details of the transactions table:

Choose the type of data connection

 

Provide access credentials, database name, and table/collection name

 

When your connection is successful, you can continue to the next step:

In the next step, you can similarly connect to your customer information data. You must provide a customer information data connection if you provided it while building the app. If you did not provide it while building the app, you will not be asked for it.

Then, specify your output schedule:

In the final step, you can optionally specify where to export the prediction outputs to.

You can add any number of output destinations here. There are several kinds of output destinations available:

• As a new dataset on a project.

• As a new table on a database. (Note, that for this option, any existing table with the same name will be flushed and a new table will be created with the latest predictions, every time the prediction runs)

• Updating an existing table on a database.

When you finish this process, you can see the details of the prediction output schedule you created:

Prediction pipeline schedule details

 

When you go back to your app summary, you can see your new prediction schedule. Every time the prediction runs successfully, it will generate a new entry in this section:

---

How to make a one-off prediction?

In addition to making scheduled predictions, you can also make one-off predictions from your churn models:

In one-off predictions, you provide a batch of transactions for a long enough period (going back to at least to the start of the earliest of the time windows you defined while setting up the contributing factors section) and want to know who will churn, looking at the historical trends.

First, confirm the model you want to use to make predictions:

There are two options here:

1. You can re-use the transactions data that you originally uploaded.

2. You can upload a new dataset.

In both cases, the app will generate predictions for the period after the last transaction in the data, based on the timelines defined by you while setting up the churn definition, for all customers who were active at the end date of the transactions dataset.

If you choose to upload a new dataset, you will be taken to the usual data import flow.

You will be required to provide a dataset for the new batch of transactions for which you want to predict. If you used a customer information dataset while setting up the app, you will be required to provide one corresponding to the customers in the new batch. In both cases, you will need to click “add dataset” and then choose your data sources.

Click “Add dataset” to add the transactions and the customer information data

 

You can upload either new data or choose from existing datasets in the project: