Friday, April 20, 2012

The life and death of a strategy

Sometimes it is instructive to look back at some strategies that used to thrive, and then quite suddenly contracted a chronic illness that ultimately led to its demise. It gives us a sense of the unreliability of backtests and curb our over-confidence, which is always useful when dealing with the financial markets.

One good example is a well-known strategy that we called "buy-on-gap". In its simplest version, just buy at the open 100 stocks within the S&P500 which have the lowest returns from their previous day's lows to the current day's open, provided that these returns are lower than one standard deviation. (The standard deviation is computed as the 90-day moving standard deviation of close-to-close returns of a stock.) Exit such  long positions at the day's close.

Many traders know of variants of this strategy, and I started trading it around the beginning of 2007, and in fact, it formed part of my first fund's portfolio of strategies. You can see the cumulative return chart below (click to enlarge) from 2007/01/03-2008/10/29. The APR is 19%, unlevered. The Sharpe ratio is 1.4 and the maximum drawdown is just 4%. Note that Lehman Brothers went bankrupt on 2008/09/15, and this is a long-only strategy, yet the performance was spectacular in September-October 2008. We were patting ourselves furiously on the back.




Now look what happened after this happy period.
 The APR was -6%. 2008/10/29 turned out to be the high watermark.

I have seen some strategies that have the opposite behavior: poor performance prior to 2009, and stellar performance since then. Was there a structural break in the market due to the financial crisis? Was this due to the advent of high frequency trading? The declining volume in the equities market? I will leave these deep questions to financial economists. The only lesson I have learned from this and other examples is that, once a strategy is in decline for some time, it seldom comes back to health, and the best course of action is to bury it swiftly.

63 comments:

Pi said...

Hi Ernie. Nice strategy. Just wanted a clarification. The return from yest low to today open could be flat, negative or positive. So by lowest 100 you are implying we rank the 500 stocks by % chng, remove those whose absolute value is greater than the 1SD value, and then choose the bottom 100. A lot of these will have negative returns frm yest low.. so thats acceptable ?

Also, are the stocks then sold by end of day ?

Ernie Chan said...

Hi Pi,
No, by lowest 100 I mean first apply the criteria that their returns must be < -1*std, and then pick the lowest 100 among the candidates. Of course, there may not be 100 candidates, in which case we will buy fewer stocks.

Yes, the portfolio is liquidated at the close.
Ernie

Pi said...

So in essence you are picking the weakest stocks.. and expecting mean reversion to happen for the basket as a whole.. or for the basket to outperform the overall index per se..

Ernie Chan said...

That's correct. It is a mean-reverting strategy.
Ernie

Anonymous said...

Hi Ernie,

To be honest I do not see how one can use such a strategy. In my opinion this is simply going to boil down to luck. I think an improvement on this may to group n stocks in a basket and buy the stock that performed the least in the prior period. I believe ensembling things like correlation with some more advanced machine learning models would be the way to go. Note I have had success with that type of strategy in medium frequency trading (holding time a few minutes).

Thanks

Ernie Chan said...

Hi Anon,
This strategy exploits a particular inefficiency in the opening auction price of equities. Over a period of about 2 years of live trading, it has a consistently high winning ratio, and numerous traders have traded this strategy. However, this inefficiency has disappeared, or at least greatly diminished, since the financial crisis. In fact, maybe it disappeared because too many traders learn about it.

The strategy you suggested with a few minutes holding period are in a completely different category. There is no reason why we cannot trade both types of strategies successfully.

In contrast to you, I have never had any success with machine learning based trading models.

Ernie

Anonymous said...

Please define "swiftly". :-)

Ernie Chan said...

Hi Anon,
"Swiftly" is somewhat subjective, depending on whether you think the drawdown is due to a specific event that won't repeat. But if there is no discernible reason for the decline, and the drawdown is longer than the the max drawdown duration in the backtest, then it is time for drastic cuts in leverage.
Ernie

Alpha said...

hi Ernie; what is the rationale for being long only? why not trade it on both sides?

William said...

I think it's important to always be creating new strategies, even if your returns are doing well with current ones.

Like you indicated, inefficiencies last only as long as you stay out of the radar, that way nobody replicates it. For example:

Same problem happened at my university store. I found the the price of a salad (same amount of food, plus two additional snacks) was was cheaper than the pitas themselves. I told a few friends about it and two weeks later, all my friends had started buying the salads. Two weeks later, they raised the price of the salads...

Lesson learned: If you spot an inefficiency, stay off the radar and keep quiet...

vix said...

Hi Ernie,
May I suggest a conspiracy theory that since the financial crisis there is greater government involvement/manipulation in the markets. Somone supports yen, someone supports swiss frank, someone supports interest rates. It can be that someone supports stocks as well (or does it indirectly). And while supporting stocks they can really go for an easy target: buying those stocks that on openings actually trade higher, not those that fall deeper than the others. The capital concentrates in selected instruments that are being propelled beyond reasonable (a fruit, anyone?), ignoring broad market.

Ken said...

Hi Ernie -- first off, loved the book, which then clued me into your blog! Also, I greatly enjoy the content in addition to the community comments -- great intelligent and often challenging stuff which makes me research more! The email notice of a new post is always a welcome arrival!

On this topic, irrespective of a particular strategy, I agree many strategies can become unusable once they are "known". There is, though, an interesting judgement between: is my strategy just not working in the short term (temporary drawdown), or is it now overused and unusable. That may be difficult to detect in the medium term, but hopefully backtests illustrating prior drawdown periods may be a metric of comparison. We can't abandon each strategy quickly when we just "think" it's dead.

William -- funny example and spot on, that's the lesson here! And is also why I wouldn't expect to read of great (completed ready-to-trade) strategies in the literature. As long as we are all doing something a bit different, the markets work.

Regards.

Ernie Chan said...

Hi Alpha,
It turns out that the short side of this strategy does not perform very well. This seems to be a general characteristic of equities trading, which is not surprising considering the positive expected return of the market.
Ernie

Ernie Chan said...

Hi William,
Agreed!
Ernie

Ernie Chan said...

Hi Vix,
Interesting thought. Zerohedge recently mentioned (http://www.zerohedge.com/news/why-market-slowly-dying) that retail trading has decreased ever since 2008. So this inefficiency may have died with retail trading.
Ernie

Ernie Chan said...

Hi Ken,
Thanks for your kind words on my book. I agree that it is difficult to tell if a strategy is just suffering from short-term drawdown. But I believe it is always better to be safe than sorry once the drawdown duration approaches the max duration in backtest.
Ernie

Anonymous said...

Hello,

This probably means that this strategy was followed by some big funds then which may have abandon it for some reason or actually started to fade it when they realized retail follows it. The strategy is too simplistic too work for a long time. Simple = no edge as anyone can figure it out.

Ernie Chan said...

Anon,
Your explanation for the demise of this strategy may be true, though I personally stick to my explanation above: that there are too few retail traders nowadays who used to drive down the prices of stocks in a panic at the open to allow for this opportunity.

I disagree that it loses profitability because it is too simple. You can make it more complicated with all sorts of hedges, risk management measures, and mathematical analysis. It won't change the basic fact that as long as there are no retail participation, professional traders cannot hope to be profitable, because they will just be competing among themselves.

Ernie

Anonymous said...

Hi Ernie,

I understand your argument but this makes no sense to me: why would retail traders push prices down on the open when in a bull market most indicators they use would signal a buy instead of a sell?

IMO this had to do with the proliferation of pairs trading funds during that period. Most of those funds went bust during the bear market although they continued the strategy for as long as they could and as a result the effect you saw of it being profitable during the market drop. When those funds ran out of money the strategy stopped working.

Anonymous said...

Hi Ernie,

I understand your argument but this makes no sense to me: why would retail traders push prices down on the open when in a bull market most indicators they use would signal a buy instead of a sell?

IMO this had to do with the proliferation of pairs trading funds during that period. Most of those funds went bust during the bear market although they continued the strategy for as long as they could and as a result the effect you saw of it being profitable during the market drop. When those funds ran out of money the strategy stopped working.

Anonymous said...

Insightful post as always!

Wondering what your and everyone else's thoughts are on whether such a strategy is feasible for retail traders (100K). I mean is it really cost effective to trade 100 names intraday?

Ernie Chan said...

Hi Anon,
Your comment is very thought-provoking, even though I disagree with all your points!

First, by retail traders, I am thinking of those who do not trade using technical indicators or otherwise algorithmically. My strategy bought stocks at the open when the market is down, sometimes very down. So obviously most traders are selling, and we can hypothesize that a sizeable portion of these sellers were retail traders. If, as you said, in a bull market every body would be buying, then why would the market be down at all at the open? I believe these retail selliing exaggerated the down move at the open. But nowadays, they no longer participate in the market, so we no longer have exaggerated down moves.

Second, far from the "bankruptcy" of pair traders that you suggested, mean-reverting stat arb strategies actually worked very well in 2008, as you can find out by backtesting the Andrew Lo toy strategy referenced in my article http://epchan.blogspot.ca/2012/01/what-worked-in-2011.html. I believe it is because of this great success that many more institutional traders have adopted this strategy, so they all buy at the open when the market is indicated down, again alleviating the exaggerated down move and wiping out everybody's profits.
Ernie

Ernie Chan said...

Anon,
As I suggested in my post, this is just the simplest version of the strategy. You can modify it so that it is suitable for a $100K account. In fact, I personally traded a version of it in an $100K account (with 4x leverage) initially, and with good success.

Ernie

Anonymous said...

"My strategy bought stocks at the open when the market is down, sometimes very down. So obviously most traders are selling, and we can hypothesize that a sizeable portion of these sellers were retail traders. If, as you said, in a bull market every body would be buying, then why would the market be down at all at the open? "

I think your argument doesn't make sense? Why would retail traders sell at the open of a bull market, especially those that do not use technical analysis? Most of these people follow the media and media always say buy, buy...

I think the other anon tried to argue that the market is lower at the open because of arbitrage by funds where the sell outperforming stocks and buy underperforming ones. I think this is a better explanation than yours.

Ernie Chan said...

Hi Anon,
Actually, I think you are right that this gapping down may have to do with stat arb funds selling stocks that outperform in recent days, because if I impose that condition, the Sharpe ratios improve across the board, even though it is still weaker in the second part of the data. So I think my point about increasing participation of stat arb funds due to their superior performance during the financial crisis is also right.

Thanks for the debate -- this is a great example of how thinking about the root cause of a strategy's decline can inspire ways to rejuvenate it!

Ernie

Anonymous said...

Generally-speaking about life & death of strategies, why not to apply some form of long term momentum to theorical equities-line of your strategies, choosing the best or simply turnig off when performances are negative?
There seems to be some evidences about classic market anomalies, recursively picking the best past performer: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1364813

Regards,
Lorenzo

Ken said...

Thanks Lorenzo for this good article !
Do you have any others articles on data snooping/anomalies/model selection ?

Thanks

SP is only joking said...

Hi Ernie & Community -

Good insight here regarding regression & trading. I've been thinking about logit/probit type regressions in trading but don't have experience with these types of models. I have googled around but there doesn't seem to be a lot of research in this area.

If anyone has any papers or comments on the application of these types of regressions in trading I would be interested. Please feel free to share here or you can mail me at algotr8der@gmail.com. Thanks.

Anonymous said...

Hi Ernie,

It's a great article, thanks. I have one question: in the strategy above, u used previous low to open gap to decide what stocks to buy at open. But how do you buy at open after you already know the open price?

John

Ernie Chan said...

Hi John,
As with any strategy, there will be slippage costs involved. In this case, you can use pre-market prices to estimate the open, or you can trade just after the open.
Ernie

Zack said...

Hi Ernie,

Great article, thanks for all the hard work. I'm curious, if a strategy's performance begins to languish and you take stop trading it, do you continue to monitor its performance through simulation and consider trading it again if conditions become favorable or do you completely do away with it in favor of developing a newer strategy? I guess the middle ground here to developing an entirely new strategy would be to adjust parameters of the older strategy instead.

Thanks,

Zack

Ernie Chan said...

Hi Zack,
In my experience, adjusting the parameters of an old strategy that stopped working won't make it work again in the short term. I would backtest the strategy in 6-12 months and see if performance has returned, and if not, tweak the trading rules.
Ernie

Kevin said...

I don't think 2009 is the only regime change we have seen. I believe the Flash Crash in 2010 is another.
I developed a strategy in college as part of my economics thesis, which used an event study to identify a set of recommendations (upgrades, downgrades) that had statistical and economic significance on the underlying stock in an intraday basis. This led to a strategy of creating a portfolio which buys/sells stocks which received new analyst recommendations at the open, and closing all positions at the close. The backtest from 2001 to 2009 worked very well. Live trading started in 2009, results matched backtested results. Results began to weaken in late 2009, and all of a sudden, the strategy stopped working after the flash crash. I believe the market structure has been altered since May 2010 and the flash crash, although I do not know why.

Ernie Chan said...

Kevin,
Interesting!
We traded a similar strategy based on companies' guidance. The results also weakened in 2009, and we kill that strategy long before flash crash.
Ernie

Anonymous said...

Hi Ernie,

I was testing similar strategies and I found without stop loss they cannot be profitable. Did you apply any stop or simply closed positions at close? Did you find any correlation with the market capitalization of the stocks?

Thanks

Jack

Ernie Chan said...

Hi Jack,
If you are referring to the earnings recommendations or guidance strategies, then yes, we have tested stop losses. But stop loss invited "stop-hunting" strategies, and it was not successful.

There was some difference in performance between large vs small cap stocks, though I don't remember which one is better, and I don't know if this difference persists.

Ernie

Anonymous said...

Hi Ernie,

I am referring to stops on the long positions opened at open according to this gap strategy.
Without stops it is not profitable, unless by trading 100 stocks it is ( i tested with less).
How can they hunt your stops?
Are your stops visible in the book? They should not be?

Thanks
Jack

Ernie Chan said...

Hi Jack,
If you are referring to the gap strategy, then we have never found stops to be useful. It was profitable when we traded it without stops nor profit caps.

Stop-hunting can hurt you even without others knowing exactly where your stops are. For details, you can read the Trading & Exchanges book on my Recommended Book list on the right sidebar, section 11.4.

Ernie

jack said...

Hi Ernie,

thank you for the reference.
Have you tried to trade the same strategy but using the price in the middle of the day as the open?
This would basically be an intraday reversal. I suppose it would not work as well because traders had more time to react to such a price configuration, since it developed during the past half of the day, while at open there is less time for reaction, so fast (automatic) traders can have an edge.

Jack

jack said...

Hi Ernie,

thank you for the reference.
Have you tried to trade the same strategy but using the price in the middle of the day as the open?
This would basically be an intraday reversal. I suppose it would not work as well because traders had more time to react to such a price configuration, since it developed during the past half of the day, while at open there is less time for reaction, so fast (automatic) traders can have an edge.

Jack

Ernie Chan said...

Hi Jack,
To your question: yes, I have. It is equally profitable.
Ernie

Anonymous said...

Thanks for one of the best blogs about quantitative trading - thought provoking articles and the best comments in the blog sphere around!

I agree too many traders knowing a strategy/edge will kill it naturally because you can only trade a limited volume on it.

But why 'it seldom comes back to health'. If the edge isn't working anymore people should stop trading it.

Maybe the mob (90% of traders loose) is still keeping the worn out strategy?

For me it is one of the biggest task to decide when to leave a trading system. I use a combination of DD-Duration and DD-depth with a grain of guts feeling.

Walter W.

Ernie Chan said...

Hi Walter,
Sometimes strategies die because of competition. Other times they die because of structural change in the market (e.g. decimalization, change in short-sale rules, rise of dark pools, etc.) Structural changes may do irreversible damage to strategies.

I also rely on drawdowns and gut-feel (some sense of the fundamental reasons) to decide when to drop a strategy.

Ernie

GekkoQuant said...

Hi Ernie,
I love your blog, especially the community that you've built up as reflected by the many thought provoking comments.

I have been modeling the variance of asset returns in FTSE 100 sectors trying to separate it into two components, an environmental component(ie macroeconomic news) and a truly random random component.

It is interesting that in the period 2000 to June 2008 the model says that the noise is virtually always 100% environmental, however come July only 65% is of the noise environmental. There is a definite transition in market microstructure from June 2008 on wards.

Btw October 2011 only 30% of the noise is environmental, the other 70% is random, possibly caused by HFT trading price action rather than fundamentals.

Once i've written a paper / blog post on it in more detail I'll post it in the comments.

Ernie Chan said...

Hi GekkoQuant,
Those are very interesting results -- thanks for sharing!

Here is another way to look at the "regime change" after the 2008 financial crisis: http://www.columbia.edu/~kd2371/papers/unpublished/mom4.pdf

It turns out that after every stock market crash, the market switched from momentum to severe mean regression for an extended period of time. Increased "random noise" like you suggested would lead to increased mean reversion, since "environmental noise" do not reverse easily.

Perhaps the changes we observed is a consequence of this fundamental change, or maybe there are other factors such as heightened risk aversion, decreased volume/liquidity, and higher frequency trading.
Ernie

Alpha said...

hi Ernie,

what do you think about this idea to address the decaying of strategies:

- Calculate pessimistic stats; such as drawdown, annual return, sharpe; using VaR monte carlo simulation.

- Do periodic assessment of the strategy (i.e. monthly or quarterly)

- Stop trading a strategy as soon as the pessimistics levels are reached.

regards.

Ernie Chan said...

Hi Alpha,
Your suggestion seems sensible. I suppose Monte Carlo simulations can tell you when the distribution of prices or returns has really changed if the actual returns ended up on the tail end of the distributions, indicating a regime shift.
Ernie

mich said...

i know a lot of strategies which you may be interested. just let me know.

athena said...

wow very nice article! very helpful for a researcher like me

GekkoQuant said...

Hello again,

I've implemented this strategy (or at least my interpretation of what you outlined) at http://gekkoquant.com/2012/06/01/trading-strategy-buy-on-gap-epchan/

I hope you don't mind this comment.

From what i've implemented it looks like this strategy still works well. One slight difference that I think might exist is the standard deviation test, I check that the stock is less than or equal to the mean return + 1 standard deviation.

Another slight difference is that i'm using the 2011 S&P constituents (so have survival bias).

Thanks for this great post.

What sort of values in the sharpe ratio/win ratio/max drawdown performance metrics do you look for before taking a strategy live?

Ernie Chan said...

Hi Gekkoquant,
Yes, the strategy apparently worked well in 2012, but I don't know about 2009-2010.

I look for a Sharpe ratio > 1 before trading live. Max drawdown is less important since it depends on leverage. If we have only 1 strategy, then Sharpe ratio needs to be even higher, at least 1.5 in backtest.

Ernie

Anonymous said...

Hi, Ernie,

In one of your comment, you said "It is equally profitable" for buying in the mid of the day. Were you using low from previous day? And ignoring the price data in the morning session?

Thanks,

S.ink

Ernie Chan said...

Hi S.ink,
Yes and yes.
Ernie

Anonymous said...

Hi, Ernie,

Thanks. I should have rephrased my question to "are you using the previous day low and open of the noon bar?" I don't have access to intraday data and can't really run the test with any other time except daily open.

Also, what value did you used for slippage in your backtest (daily open entry)? When I looked at the 5min charts, the open is so chaotic. In your opinion, what level of slippage should I expected?


Thanks,

S.ink

Ernie Chan said...

Hi S.ink,
I also don't use historical intraday data except high and low. However, that doesn't mean you cannot enter intraday.

Due to the uncertainty of the data at open, 5bps for large cap, 10 bps for small cap tcost will be prudent.

Ernie

yu nam said...

Hi, Ernie,

Thanks for all the replies. But I think I don't quite understand what you mean. Because if you are using just daily open-hi-low-close, how can you back test using noon open price?

Does that mean when you trade intraday, you rank your stock using daily open and daily low(-1)?. But trade at noon on a different price, even though the ranking might be different at noon?

Or do you redo the ranking at noon, using the noon "open" price with previous day low? Then trade at noon. I do feel more comfortable with that because the instance I observed, its quiet after 11:00AM.

Sorry, I am a total newbie at this kind of topic (therefore such explicit questions).


Thanks,

S.ink

yu nam said...

Hi, Ernie,

What is tcost mean in your post? Commission only, commission & slippage? Is the bps for 1 way or round trip?


Thanks,

S.ink

Ernie Chan said...

Hi S.ink,
If your strategy says "Buy if the price is lower than the previous day's low", and your backtest shows that today's low is indeed lower than previous day's low, then you know the execution price is the previous day's low. You don't need intraday prices to do this kind of backtesting.

However, you won't be able to test a strategy that can only execute at 1pm, of course, without intraday historical data.
Ernie

Ernie Chan said...

My tcost is one-way, include slippage and commission.

yu nam said...

Hi, Ernie,

In your comment, you said

"Buy if the price is lower than the previous day's low, and your backtest shows that today's low is indeed lower than previous day's low, then you know the execution price is the previous day's low. You don't need intraday prices to do this kind of backtesting.".

But Andrew Lo's strategy is "buy the open"?


Thanks,

S.ink

Ernie Chan said...

S,
I believe you have lost track of the thread of conversation.

The thread started with the question "Have you tried to trade the same strategy but using the price in the middle of the day as the open?
This would basically be an intraday reversal. "

implying this is an intraday variation of Lo's strategy.

E.

yu nam said...

Hi, Ernie,

Yes, that's the comment I was referring to. I am not sure at what point the ranking happen, and using what data point. I can think of three methods:

1) use the rank at open using open(0) and low(-1), and buy at noon_open(0) ... this seems random to me
2) rank at middle of the day using noon_open(0) and low(-1) and buy noon_open(0) ... but this confused me when you comment "Buy if the price is lower than the previous day's low,",
3) rank using before_noon_low(0) and noon_open(0), but noon_open(0) >= before_noon_low(0), since there is no reason to believe the market will gap at noon. However, we only care about its relative rank?

But after reading your replies, I don't think I understand how this variation work. Can you elaborate it a little bit?


Regards,

S.ink

Ernie Chan said...

Hi S,
Actually, intraday variations may not involve ranking. But I will leave it to your imagination to find out the exact strategy that is possible and profitable, lest I trample on someone else' intellectual property!
Ernie