Aggregating across permnos within a permco

[chenandrewy]

@ralphkoijen made me realize that we may not be aggregating market equity across permnos with a permco 🤦‍♂️

It seems like this is becoming more of a problem. Among stocks with B/M, multi-permno issues affect only about 1% of permnos, but they now affect about 4% of total market cap:

I say about, because this was a pretty quick calculation that relies on intermediate outputs of our code here.

Then, valuation ratios for this 4% of market cap may be way off. For example, the market value of permno 90319 is $827b, but the total market cap of the company (Google) is roughly double that:

We should check and fix this. From looking at the code this morning, I did not see any aggregation across permnos.

All of the market cap values used in signals seem to come from https://github.com/OpenSourceAP/CrossSection/blob/1aa9f0e27e7f89882d8664547031edef0e4ef9ec/Signals/Code/DataDownloads/I_CRSPmonthly.do

All it does to calculate mve_c is

gen mve_c = (shrout * abs(prc)) // Common shares outstanding * Price

and it seems like all the signals use mve_c as market equity.

[chenandrewy]

As part of addressing this issue, @tomz23 looked into what is done in the literature with aggregating market equity across permnos within a permco. It seems like people generally ignored this issue.

Novy-Marx and Velikov calculate B/M using permno-level market equity. See market equity calcuation and B/M calc.

The Bali et al textbook discusses in detail market cap and B/M calculations in Ch 9 and 10. But they do not mention permno vs permco or dual-class stock.

This raises the question of: should the repo replicate the literature's methods or should we do the correct calculation? When Fama-French 1992 say

should we do what it says or do what others have done?

We decided to follow what Fama-French say instead of what is (mostly) done in practice. This ensures that the dataset is useful in the present day when mega caps with dual-class shares can significantly distort calculations. At the same time the code still produces results very close to the original papers in the original sample periods, at least.

[chenandrewy]

This bug/issue is addressed for the Predictors/ here 08f29ab. We basically:

1. Calculate mve_permco in addition to mve_c (permno-level) in DataDownloads
2. For all accounting-based predictors, we use mve_permco instead of mve_permno.
   We left all other predictors alone. The other predictors are either purely market-price based (so the permno-level market cap makes sense) or use market prices as a filter (which the ideal method is unclear so we try to match the original). The detailed list is in the Claude Code plan for deploying the fix (Claude's motivation for the fix is not quite right but the code checks out).

Still want to address this issue for some important Placebos/ (e.g. Campbell et al's Failure Probability) (@idreesm18 ).

[chenandrewy]

After 08f29ab, we realized we were moving too fast and not carefully testing the issue patches. We went back and slowly redid the updates including this issue fix.

A more careful fix is implemented in af38b81. The philsophy is the same as before: use permco market equity (mve_permco) only if market equity is used to scale an accounting variable. But this time we were more careful in applying this rule. You can see how exactly the rule was applied in https://github.com/OpenSourceAP/CrossSection/blob/af38b815b0cb8f5fc287260ebc1f9d0affcb0557/Docs/mve_permco-plan2.md.

Overall, t-stats are either little changed or in two instances closer to the original papers.