Why The Best Predictor of Future Stock Market Returns is Useless

On the Importance of Supply and Demand

Photo: Pixabay

In one of my favorite animal experiments, researchers wanted to see how pond cichlids would change their mating behavior as the sex ratio (number of males relative to females) was altered in a controlled environment.  Since cichlids are known for their monogamy and don’t desert their partners often, the zoologists thought that small changes to the sex ratio wouldn’t have a large impact on cichlid mating behavior.  Jon Birger summarizes what the experimenters found in Date-onomics: How Dating Became a Lopsided Numbers Game:

Increasing the ratio of male cichlids to females from 6:6 to 7:5 cut the male desertion rate in half—from 22 percent to 11 percent.  It also rendered females choosier about males and made the successful male suitors more protective of their females.  The end result was a kind of underwater patriarchy—one in which male cichlids fought each other for access to mates, jealously guarded their females after mating, and…made greater investments in parenting (be it through direct parenting effort or via the providing of resources such as food or protection).

When there were more males than females, females had more relative “market power” and were able to get more commitment and parenting resources from their mates.  However, when the experimenters ran the reverse experiment, altering the ratios of males to females to 5:7, this all changed.  With more females relative to males, the male desertion rate skyrocketed to 51% and many of the male cichlids took on multiple female mates.  The “market power” reversed and the males got the upper hand.

You might think that this behavior is isolated to cichlids and other animals, but Birger found the same pattern in human mating after examining the sex ratios of different dating markets across the U.S.  As the theory predicts, those areas with more females (relative to males) had a higher percentage of short-term dating encounters and those with more males (relative to females) had more relationships and committed men.  This was true whether you examined a big city, a small town, or a college/university.

Birger’s work illustrates how markets, even those as complex as dating markets, can be partially determined by one of the simplest idea in economics—supply and demand.  While you might think this is far-fetched, the basic forces of supply and demand are also one of the best predictors of future stock market returns.  In particular, when average investor allocation to stocks is high, returns for the next 10 years are low, and when average investor allocation to stocks is low, returns for the next 10 years are high.  Using survey data from the American Association of Individual Investors, we can see the negative relationship between average investor allocation to equities (“AvgEquityShare”) and future returns clearly:

As you can see, when AvgEquityShare is higher future returns are lower and vice versa.  Anecdotally this idea makes sense.  For example, BusinessWeek’s “The Death of Equities” issue came out in 1979, the year before the best back-to-back decades in U.S. stock market history, while the DotCom bubble was the point of maximum stock demand right before a decade of low returns.

I originally came across this idea in an incredible post from the blogger Jesse Livermore (pseudonym) in December 2013.  Jesse’s post is far more technical and in-depth than what I have time to cover here, but it illustrates how AvgEquityShare is far more predictive of future stock market returns than many traditional valuation metrics.  [Quick aside:  Though the AAII data I use does not include bank holdings, which is an important and critical difference from Jesse’s method, I still think we find similar conclusions.]

With that being said, though the AvgEquityShare model outperforms “Buy and Hold”, I will also show how it is useless for a typical investor.  To start, here is how the AvgEquityShare model works:

  1. Start by investing in stocks (S&P 500).
  2. When demand gets too high (>70% average equity allocation) => sell your stocks and move into bonds (5yr Treasuries).
  3. Stay in bonds until demand gets too low (<50% average equity allocation) => sell your bonds and buy back into stocks.
  4. Repeat steps 2-3.

That’s it.  I chose the 70% upper limit and 50% lower limit to have round numbers that also corresponded to different return regimes (aka I data-mined this using backtests).  If you run this model you will find that from 1987 to 2018, $1 would have grown into $43 compared to only $24 for “Buy and Hold” (almost 2x better dollar growth), with a far better drawdown profile:

As you can see, the worst loss you could have experienced for the AvgEquityShare model is half (-23.2%) of what “Buy and Hold” delivered (-50.9%).  Though I could show you many other performance metrics that illustrate how much better the AvgEquityShare is than “Buy and Hold” it wouldn’t matter.  Why?  Because when we dig into the details we realize that the AvgEquityShare model would’ve been near impossible to hold for any typical investor.

As an example, consider how the AvgEquityShare (orange line) performed compared to “Buy and Hold” (teal line) during the DotCom bubble:

As you can see, the model moved to bonds in September 1996 and didn’t get back into stocks until October 2002!  Though it outperformed in the end, at what psychological cost?  Can you imagine how hard it would be to sit in bonds for 4 years while your friends (some of them dumber than you) got rich along the way?  It would be near impossible.  The same thing happened in the GFC when this model signaled to get out of stocks in May 2006 and get back in during November 2008.  Yes, this also outperformed “Buy and Hold” over that period, but who could’ve waited it out?

The Hardest Discipline

Even when we have reasonable evidence that a particular investment strategy will work, the hardest discipline is sticking to that strategy through thick and thin.  It is so difficult because that little voice in your head says, “What if this signal doesn’t work anymore?  What if it was just a data-mined result?  What if I’m wrong?”  That little doubt is all it takes to turn the hardest resolve into the emotional putty that has destroyed generations of investors.

With that being said, I am going to issue a warning on what I am about to say.  I do not recommend you changing your investment strategy based on the following information:

  • The AAII AvgEquityShare model I discussed above sold out of stocks in January 2018 after being fully invested every single month since November 2008.

This happened because the AvgEquityShare got above 70%.  It currently (as of September 2018) stands at 69.88%.  Does this mean that a crash is coming?  Maybe.  Or maybe in a few years.  Or maybe the model doesn’t hold anymore.  If you think there is some chance this model is right and you want to see the data source, I will provide the phrase found in Dante’s famed inferno:

Abandon hope all ye who enter here.

I can only show you the door.  You are the one that has to walk through it.  Thank you for reading!

➤ You can follow Of Dollars And Data via Email (1 weekly newsletter) or Twitter.

This is post 93.  Any code I have related to this post can be found here with the same numbering: https://github.com/nmaggiulli/of-dollars-and-data


Full Disclosure: Nothing on this site should ever be considered to be advice, research or an invitation to buy or sell any securities, please see my Terms & Conditions page for a full disclaimer.

OfDollarsAndData.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com and affiliated sites.

43 Responses

  1. Anonymous commented on Oct 09

    Hi Nick. This reminds me the Dual Momentum Investing book by Gary Antonacci. Any views ton that strategy? He uses a 12 month lookback on a monthly basis to choose between All world (ex SP500), SP500 or Bonds.

  2. Anonymous commented on Oct 09

    Totally agree it would take some huge stones to hang with that. Super interesting though. Thanks for publishing.

  3. Anonymous commented on Oct 09

    Hi Nick. This reminds me of Gary Antonacci book Dual Momentum Investing. Gary uses a 12 month look back to choose between SP500, All World ExSP500 OR Bonds. Any thoughts on this strategy? I’d love to see them compared.

    • Nick Maggiulli commented on Oct 16

      Never looked into his book, but I will consider it in the future. Thank you for bringing to my attention.

  4. Anonymous commented on Oct 10

    Where do I track the AVGEquityshare model and how many % investment in equities and bonds there are?

  5. Anonymous commented on Oct 15

    Another amazing post. Just showing support to another awesome human named nick ;). Keep writing!

  6. Anonymous commented on Oct 17

    Is there an international (ex-US developed and emerging markets) equivalent of this? Would be curious to see where those are today..

    • Nick Maggiulli commented on Oct 18

      No clue. The data is probably far harder to come by if it exists. Given U.S. equities represent about half of total equities, it is a decent proxy.

  7. Anonymous commented on Oct 17

    Is there an international (ex-US developed and emerging markets) equivalent of this? Would be interesting to see where we are today..

  8. Anonymous commented on Oct 18

    I love this and all your posts. Thanks for putting in the time to educate us all.

  9. Anonymous commented on Oct 27

    To reiterate, this is clearly data mined. (As Nick has admitted.) It doesn’t matter that round numbers of 50% and 70% were used. To show the strategy is robust, a test of a range of parameters would be needed, to show that the pattern is robust to other cut-offs. The future is likely to be different than the past, to enough of a degree that there can be no known optimal parameters ahead of time.

    That said, the strategy has some passing similarities to moving average trading strategies, which are quite robust across parameters (it really doesn’t matter what moving average lengths you use–they all perform broadly similarly, other than higher transaction frequencies for shorter moving averages).

    Note that the data set in this post only encompasses two very clean bear markets (dot com crash and financial crash). How would it perform in the 50’s, 60’s, 70’s, etc.? Jesse Livermore’s post seems to suggest it might work, but it hasn’t been tested.