This post is about the learning how survivorship is impacting system design with a focus on rotational trading. As posted here, the last couple of weeks I’ve worked on creating a NASDAQ 100 database that doesn’t have a survivorship bias. It’s been quite a tricky and tedious task. Tricky, because I first had to figure out how to do the technical implementation. Tricky, because it takes a lot of time to transform the raw data into Amibroker AFL code.
Let me first describe the actual problem! Often times you test an idea based on stocks being part of a particular index, e.g. Nasdaq 100. Why does one focus his or her testing on an index? Because the members of this particular index are BIG. And size often correlates with the number of stocks being traded, so liquidity shouldn’t be an issue. Also because, often institutional investors focus on these stocks as well. Furthermore being part of a particular index required companies to comply with higher accounting standards etc.
It’s well known that a system should be tested over an extended period of time (bars) in order to ensure the system has been through different kind of markets (bull, bear, side, low/high volatility, etc.). So you are inclined to include as many as possible historical bars. Typically I use about 3000 daily bars for my test (~ 12 years). So let’s check how many of today’s NDX-100 members have been part of the NDX-100 in the past.
So effectively I tested my strategies against a VERY different environment than the one that existed in reality, e.g. only 60 of today’s members were part of the index in 2005. To say it differently, 40 stocks got dismissed because of poor stock performance, bankruptcy or acquisition.
A few weeks ago I presented a very simple rotational system in order to introduce the idea and to give the interested reader a starting point. The rules of this system: only in the market when the QQQQ’s are trading above MA30 or MA200, Ranking: TSI
Let’s review the impact of survivorship bias:
- A performance drop of about 50% in absolute as well as risk adjusted performance
- Significantly higher max. draw down
The results consider only trades starting 01.01.2001. The same test going back to 1995 resulted in about the same performance drop.
Data quality: I got historical index constitution all the way back 1995. The completeness until 1997 is very good, before that i miss a very few stocks.
Technical implementation: all historical data is maintained in a Microsoft Excel sheet. I then turned this data into one gigantic Amibroker AFL function. So this function can be called with ticker name + date and will then return TRUE/FALSE depending if this ticker has been part of the index at the given date. Once I did this, it has become extremely simple and easy to test existing systems. Just needed to add a call function statement and run the system against the newly created watchlist (containing all current + historical tickers). Within minutes i got the results (after i spend many many days building the database).
I’m strongly convinced that testing these kinds of systems without a survivorship free database has severe impact on system design and performance. The decisions you take during the system designed can be flawed as your data suggest a much more positive environment than the one existed in reality. So I encourage you to either build your own survivorship free database or get one from me! As said, i build that database for my personal use. Though I’m not in the business of selling software or data. So donate 500$ to UNICEF (link), send me the UNICEF donation confirmation and i will send you an email containing: Excel file, Amibroker AFL file as well as Watchlist file (historical stock price data isn’t included) . This way you can do something good to your trading business and support UNICEF in their effort to build schools for children in Africa.
I’m not going to build any kind of system based on an index I don’t have survivorship free data for.