Over the last couple of weeks I had an important project running. Together with the help of some trading friends we created a S&P500 survivorship bias free database starting 1990. I did a similar exercise for Nasdaq 100 stocks already see (here and here). With this post I want to share some of my S&P500 specific findings that might be relevant for you.
My trading systems focus on stocks of a particular index. I like to do that, because trading S&P500 or NDX100 stocks ensures you that there is enough liquidity (at least for my account size). Furthermore trade execution should be smooth with narrow bid/ask spreads. At least that’s my believe and as Van Tharp says: we don’t trade the market we trade our believes about the market.
For backtesting my strategies I purely rely on Norgate’s premiumdata service. They provide me with current data as well as delisted data. I’m very satisfied with their service. Hence I build my survivorship bias free database based on Norgate’s premiumdata. I’ve heard Norgate is going to release a service for providing historical index constitution during next year. I’m not associated with them, though I can strongly recommend their service. Data just hassle-free in good quality!
Numbers for the numbers lover
Let me give you some raw facts about the index (from 1990 until November 2010)
- About 1006 stocks were part of the index
- 402 stocks that used to be part of the index are delisted now .
- Only 189 stocks survived staying in the index from 1990 until today.
- About 5.7% of stocks enter/leave the index every year. That’s less than half the value of the NDX100 turn-over rate.
The impact of survivorship bias
In order to understand the impact of survivorship bias I created a simple swing trading system (posted at the end). Here are the entry rules in short:
- Long only
- Entry when RSI2 < 30
- Exit when RSI2 > 70
- 10 Positions, each 10% equity
- Portfolio ranking: low RSI2
- No commission / slippage
- Explanation of the key performance indicators being used (link)
I run the test with two different data sets. First with today’s S&P500 stocks (NO) and second with survivorship bias free adjusted data (YES). The results are about twice as good using today’s index members only!
Using today’s S&P500 stocks to backtest strategies will guide you towards wrong decisions in your system design & development process.
#include <SP500_member.afl>; SetOption("CommissionAmount",0.00); SetBacktestMode( backtestRegular); SetPositionSize( 10, spsPercentOfEquity ); SetOption("MaxOpenPositions",10); SetOption("MaxOpenLong", 10); SetTradeDelays( 0, 0, 0, 0 ); Member = SP500member(Name(),DateNum()); RoundLotSize = 0; Short = Cover = False; Buy = member AND RSI(2)<30 AND Year()>=1990; Sell = RSI(2)>70; PositionScore= 100 - RSI(2);