Methodological Issues: Should we use all of the available data modeling CLV?

[chenx2018]

As a general question for modeling CLV, should I use all of the available data (around 5 years) or just use some recent data (recent 2 years) to build the Pareto/NBD model?

The data is from a jewelry retailer, and the mean inter-purchase time is around 5 months.

I have read "Counting your customers: who are they and what will they do next" paper in 1987 and it suggests using the recent 2 years data. I am not sure this is appropriate in my situation.

Any suggestions? Thanks!

[pschil]

You should fit the latent attrition models in this package on cohorts. Cohorts are groups of customers defined by the time they joined your company (=did their first purchase) and are sometimes further split by acquisition channel or type of customer (b2c vs b2b). It is assumed that customers in different cohorts substantially differ between each other (ie early adopter vs late adopters, life-cycle stage, etc). For more information see here: wikipedia

Hence, you should first divide your data in cohorts (time of acquisition) and fit a separate Pareto/NBD model on each cohort.
Example:

Cohort time span: 3 month
Cohort-1: All customers (and their transactions) that made their first purchase between 01/01/2015 and 31/03/2015
Cohort-2: All customers (and their transactions) that made their first purchase between 01/04/2015 and 30/06/2015
....
Cohort-N: All customers (and their transactions) that made their first purchase between 01/07/2020 and 30/09/2020

What is the best time-span to define a cohort depends on your specific data set and industry. Common are 1-month, 3-month, or 1-year.

Once you have broken your data into different cohorts, a Pareto/NBD model is fit on all the data that is available for this cohort.
To assess the goodness of fit and prediction quality, you may want to split the cohort-wise data in an "estimation" and "holdout" period using the parameter estimation.end in clvdata(). The model is then only fit on the data in the estimation period and can be compared to the actual data in the holdout period.

From the above example, you could split the data for Cohort-1 at 31.12.2018 which will fit the model on the data from 01.01.2015 to 31.12.2018 and then assess the performance in the holdout data which ranges from 01.01.2019 until last date. Pseudo code:

clv.cohort.1 <- clvdata(data.cohort1, estimation.end="31.12.2018")
# fit on data until 31.12.2018
p.cohort.1 <- pnbd(clv.cohort.1)
# assess prediction performance on whole holdout data (1.1.2019-now)
predict(p.cohort.1)[, mean(abs(actual.x - CET))] # MAE

Once you are happy with your model fit and want to measure/predict its current CLV, standing at the last possible timepoint (="now"), you fit the model on all the data that is available for this cohort and then predict any period ahead. CLV for the pnbd standard model is calculated from DERT*mean.spending.per.transaction and is therefore independent of the prediction horizon. This is because, by definition, DERT is an analytical expression from the end of the fitting period until infinity (because DERT is from the end of the fitting period, is also why you fit the model on all of the cohort's data for the final CLV assessment, without any holdout period)

Because there is less data available for some cohorts (earlier cohort have more data, later cohorts have less) there may be different confidence in the estimated parameter values of different cohorts.

[mmeierer]

To put your question into context. Here is the respective paragraph of the cited paper:

In your particular case, I would probably make sense to use as much data as possible for each cohort. However, without seeing more descriptive statistics, this is quite hard to judge.

As @pschil highlighted, these kinds of probablistic models are usually used with cohort data. This also means, that you are likely to adapt the training period depending on how long you do observe the respective cohort.

If you were able to estimate the Pareto/NBD or an alternative model on your data, it would be nice if you share the tracking plots for a few cohorts. See https://cran.r-project.org/web/packages/CLVTools/vignettes/CLVTools.pdf (page 9).

[chenx2018]

Thanks for your replying! From @pschil 's comment, I understand that CLV is independent of the prediction horizon. The following is my experiment result,

# modeling -----------------------------------
clv_dat <- clvdata(
  cohort_dat,
  date.format = "ymd", 
  time.unit = "w", 
  name.id = "cust", 
  name.date = "date",
  name.price = "sales", 
  estimation.split = "2019-07-31"
)

summary(clv_dat)

# Transaction Data Summary 
                               # Estimation        Holdout          Total       
# Number of customers                -                 -                2678        
# First Transaction in period        2018-01-11        2019-08-01       2018-01-11  
# Last Transaction in period         2019-07-31        2020-01-25       2020-01-25  
# Total # Transactions               4865              1193             6058        
# Mean # Transactions per cust       1.817             2.150            2.262       
# (SD)                               2.900             2.920            4.056       
# Total # zero repeaters             1934              2123             1706        
# Percentage # zero repeaters        0.722             0.793            0.637       
# Mean Interpurchase time            12.888            5.955            19.399      
# (SD)                               14.372            5.117            17.939

# parameter estimation
pnbd_est <- pnbd(clv.data = clv_dat)

# Coefficients:
#   Estimate Std. Error  z-val Pr(>|z|)    
#   r      0.41675    0.04149 10.044   <2e-16 ***
#   alpha 11.69813    0.98180 11.915   <2e-16 ***
#   s      0.13469    0.01964  6.858    7e-12 ***
#   beta   0.53906    0.29153  1.849   0.0644 .  

summary(pnbd_est)
plot(pnbd_est)
plot(pnbd_est, cumulative = TRUE)

# mae error = 0.5264825
predict(pnbd_est)[, mean(abs(actual.x - CET))]

Here are some plots,

Based on the tracking plot, I think adding time-varying coefficients, such as seasonal dummy, could improve the model fit. May I know if there is other way to tune parameters to get improvement? Any suggestions? Thanks again!

[pschil]

CLV is independent of the prediction horizon

CLV in the models without time-varying covariates is independent from prediction horizon and is a long-term prediction. See the following expression where DERT is the expression in the integral. As you see, it is from T (fitting end) until Inf (=prediction horizon) and accounts for the probability of being alive (S, the survival function) and involves continuous discounting (time value of money).

Regarding your tracking plot:
Are you sure that there are only transactions of one cohort in cohort_dat? Judging from your tracking plot, it looks as if new customers are joining (making their first purchase) somewhere around 01/2019. The tracking plot should look more like the one in the pdf linked by Markus, page 9: Customers of the cohort first come alive and then the cohort gradually dies out over time.

Regarding time-varying covariates:
It could make sense to add these if your business is driven by seasonal patterns or events such as christmas, birthdays, holidays etc. In order to predict future transactions and calculate CLV you have to supply the covariates also for the future: To calculate customers current CLV, standing at "now", you also have to supply the covariates for the future. You can either only use covariates which are certain (seasonality, holidays, birthday, etc) or also use covariates which need to be predicted (say temperature) what adds more noise to your CLV predictions.

Beware that CLV for the PNBD with time-varying covariates does depend on the the prediction horizon very much. The DERT expression until infinity cannot be derived for time-varying models because the time-varying covariate data is only given for a limited period. Instead of DERT the expression is called DECT and is calculated based on the covariate data, from fitting end until prediction end. CLV then is again calculated as DECT * mean.spending.per.transaction.

See also part of this discussion for how to handle and supply time varying covariate data: #127 (comment)

[chenx2018]

Thanks for your suggestion @pschil! Regarding this question,

Are you sure that there are only transactions of one cohort in cohort_dat? Judging from your tracking plot, it looks as if new customers are joining (making their first purchase) somewhere around 01/2019.

Actually, I did not use cohort data to fit the model. What I did is to split the transaction data (the whole data) into two parts:

• training set: customer making their first purchase from 2018-01-11 to 2019-07-31.
• testing set: transaction data from 2019-08-01 to "now".

Your feeling is absolutely correct: there are new customers joining in early 2019.

Then I try to build the cohort-wise data with 1 year time-span and get the following tracking plot:

Now I get really confused about cohort-approach and regular approach (i.e. use all transaction data without doing cohort). Which one is appropriate? As you mentioned before,

It is assumed that customers in different cohorts substantially differ between each other.

If I only use time span to define cohort and assume customers in different cohorts behaving super different, then the only reason to explain the cause of the difference is the first purchase time. However, in my situation, I am not sure different first purchase time will lead to different customer's behavior. The pnbd-model parameters for different cohort data as follows,

cohort	r	alpha	s	beta	lambda	mu
joining in 2017	0.7100672	56.49091	0.1604819	2.1884981	0.0125696	0.0733297
joining in 2018	0.6951503	51.14220	77.5453719	9425.1005082	0.0135925	0.0082275
joining in 2019	1.3543031	70.42756	0.2480876	0.6737219	0.0192297	0.3682344

Maybe I should use likelihood ratio test to formally compare them. The summary of transaction shows: around 60% customers only did single purchase and gone, and the inter-purchase time is around 8 months. I also did some CLV comparison for cohort-approach and regular approach.

For earlier cohort (first joining in 2017), two approaches provided the similar predicted CLV.

For later cohort (first joining in 2019), the predicted CLV is quite different.

Moreover, since I would like to assign CLV-rank for every customer, if using cohort-approach, is it meaningful to compare individual's predicted CLV in different cohort?

Many Thanks!

[pschil]

I would say cohort-wise fitting is the regular approach.
Customers are maybe not "super different" in their behavior but different acquisition channels, promotions, or brand image and awareness etc may still attract different customer types 1 year later.

In some sense the customers really are "all the same" because they all start buying and then gradually die. But the customers in cohort 1 started earlier with this than the customers in cohort 2 and are therefore already some time ahead. This puts customers in different cohorts in different life-cycle stages and gives each cohort a different average retention rate: The earlier cohorts have already died out more and likely only a self-selection of customers that like your product are left in older cohorts. That is one important way in how cohorts differ in their "behavior".

Related to this you should also not forget that the PNBD model has only a mere 4 parameters. That gives it some flexibility to account for heterogeneity through the gamma distributions but it is still limited. While individual customer-level information is used (recency, frequency, time alive), the same 4 parameters are shared across all customers and, if only one model is fit on all data, the same parameters are also shared across cohorts. You would end up with an average purchase rate (r/alpha) and an average dropout rate (s/beta) that are the average over all your cohorts - but from your fitted parameters at least the dropout rate for the 2019 cohort seems to be distinctly different from all the others, so fitting on the full data does not seem to be a good idea as it will bias the purchase and dropout rate for all the other cohorts.

Fitting the model over all cohorts at once has shown to bias CLV estimates at least in the contractual setting where death is actually observable (Fader and Hardie, 2010).

Maybe I should use likelihood ratio test to formally compare them

Rather not: LR tests are meant to test different models on the same dataset while you are actually testing the same model on different datasets (cohort vs full data). Likelihood ca: "How well the model is explained by the data", LR test ca: "How well different models are explained by the (same) data".
You could use it for example to test the pnbd no covariates model versus the pnbd model with static covariates - on the same data.

around 60% customers only did single purchase and gone

Zero-repeaters are often found in b2c settings and there exist zero-inflated models that specifically account for that. On my own retail data, I have found that zero-repeaters are in almost all cases correctly predicted as "death" with the standard pnbd model and it is very rare that a customer that only transacted once and then disappeared for a long time to suddenly come back. So zero-repeaters at least for me did not turn out to be an issue with the standard models.
But beware that your data has to span long enough to actually observe re-purchases which differs by industry/good. An example are cars which are purchased every say 5 years and having only 3 years of data will result in many zero-repeaters which in fact are not but they simply did not have time to come back and buy another car.
In your case the mean inter-purchase time seems to indicate that the data spans long enough to view re-purchases.

is it meaningful to compare predicted CLV in different cohort?

Yes, they are all monetary values discounted to the same point, and in my opinion, it is just a fact that that different cohorts (=different time with your company or different business segment) hold different values to your company. Depending on your business, older cohorts may have already "spent their money with you" and will move on what makes them less valuable (say a clothing brand targeting 20-30 year olds) or had more time to develop some form of relationship with you and will now buy more valuable products while all the less profitable customers have left this old cohort (say a bank where a customer of 15years takes out its first mortage at the age of 35years).

But individual-level CLV from standard latent attrition models admittedly has its limitations as it is very much only the "average CLV" of a customer defined by its RFM pattern. The aggregated CLV (customer equity) then again is however often very accurate as it is the sum of averages.

[chenx2018]

@pschil Thanks for your very detailed explanation! It helps a lot!