tag:blogger.com,1999:blog-35364652Thu, 29 Jan 2015 14:00:46 +0000StrategiesAutomated trading platformsfactor modelBook reviewsQuantitative TradingQuantitative investment and trading ideas, research, and analysis.http://epchan.blogspot.com/noreply@blogger.com (Ernie Chan)Blogger198125tag:blogger.com,1999:blog-35364652.post-7540053347751873883Thu, 08 Jan 2015 14:06:00 +00002015-01-08T09:07:17.311-05:00StrategiesTrading with Estimize and I/B/E/S earnings estimates data<i>By Yang Gao</i><br /><i><br /></i><a href="http://www.estimize.com/">Estimize</a> is an online-community utilizing 'wisdom of crowds' to offer intelligence about market. It contains a wide range of crowd-sourced estimates from over 4,500 buy-side, sell-side and individual analysts. <a href="http://www.estimize.com/api">Studies</a> (from Deustche Bank and Rice University among others) show estimates from Estimize are more accurate than estimates from traditional sell-side analysts.<br /><br />The first strategy we tested is a <a href="http://www.deltixlab.com/research">mean reversion strategy</a> developed by the quantitative research team from Deltix using Estimize’s data. This strategy is based on the idea that post-earning-announcement prices typically revert from the short-term trend driven by the more recent Estimize estimates just before the announcement. We backtested this strategy with S&P100 over the period between 2012/01/01 and 2013/12/31. (Even though Estimize has 2014 data, we do not have the corresponding survivorship-bias-free price data from the Center for Research in Securities Prices that includes the closing bid and ask prices.) With 5bp one-way transaction cost, we found that the backtest shows a Sharpe ratio of 0.8 and an average annual return of 6%. The following figure is the cumulative P&L of the strategy based on $1 per stock position.<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-qVu21wMrgSo/VK2WXibzHNI/AAAAAAAADIA/Pv3IqQxxpFY/s1600/Estimize1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://1.bp.blogspot.com/-qVu21wMrgSo/VK2WXibzHNI/AAAAAAAADIA/Pv3IqQxxpFY/s1600/Estimize1.png" height="266" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Cumulative P&L of Deltix Mean Reversion Strategy with Estimize </td></tr></tbody></table>It surprised us that a mean-reverting instead of a momentum strategy was used in conjunction with Estimize data, since earnings estimates and announcements typically generate price momentum. In order to show that this return is really driven by the information in Estimize and not simply due to price reversal, we provide a benchmark mean-reverting strategy that uses prices alone to generate signal:<br /><br />1. Find long period T and short period T_s, where T is average period of the reporting of all the quarterly estimates and T_s is average period of the reporting of the latest 20% of all estimates.<br />2. Calculate stock return R over T and Rs over T_s, and let delta = R - Rs<br />3. Buy stocks with delta > 0 at close before an earnings announcement and exit the positions next morning at the open after the announcement.<br />4. Sell stocks with delta < 0 at close before an earnings announcement and exit the positions next morning at the open after the announcement.<br />5. Hedge net exposure with SPY during the entire holding period.<br /><div><br /></div>This benchmark shows no significant positive return and so it does seem that there is useful information in the Estimize data captured by Deltix’s mean-reversion strategy.<br /><br />Next, we compare the traditional earnings estimates from I/B/E/S gathered from sell-side Wall Street analysts to the crowd-sourced Estimize estimates. Backtest showed the same Deltix mean reversion strategy described above but using I/B/E/S estimates gave negative return over the same S&P100 universe and over the same 2012-2013 period, again supporting the thesis that Estimize estimates may be superior.<br /><br />Since Deltix's mean reversion strategy gives negative returns on I/B/E/S data, it is natural to see if a momentum strategy would work instead: if the short-term average estimate is higher than the long-term average estimate (i.e. analogous to delta < 0 above), we expect the price to move up and vice verse.<br /><br />The backtest result of this momentum strategy over the same universe and time period is quite promising: with 5bp transaction cost, the Sharpe ratio = 1.5 and average annual return = 11%. The following figure is the daily P&L of the strategy based on $1 per stock position<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-ZuJ0fCQ9cvQ/VK2XKj6cG8I/AAAAAAAADII/I6dSd0nThrM/s1600/Estimize2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://2.bp.blogspot.com/-ZuJ0fCQ9cvQ/VK2XKj6cG8I/AAAAAAAADII/I6dSd0nThrM/s1600/Estimize2.png" height="266" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"> Cumulative P&L of momentum Strategy with I/B/E/S</td></tr></tbody></table><br /><br />We tried the same momentum strategy using Estimize data over 2012-2013, and it generated negative returns this time. This is not surprising since we found earlier that the mean reversion strategy using Estimize data generated positive returns.<br /><br />We proceeded to backtest this momentum strategy over the S&P100 using out-of-sample I/B/E/S data between 2010 and 2012, and unfortunately the strategy failed there too. The following figure is the daily P&L of the strategy from 2010-2014.<br /><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-56RwcWrknCs/VK2Xq5Jo2FI/AAAAAAAADIU/nDp4O_9Jp3I/s1600/Estimize3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://3.bp.blogspot.com/-56RwcWrknCs/VK2Xq5Jo2FI/AAAAAAAADIU/nDp4O_9Jp3I/s1600/Estimize3.png" height="266" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Cumulative P&L of momentum Strategy with I/B/E/S </td></tr></tbody></table><div><br /></div><div>So how would Deltix’s mean-reversion strategy with Estimize data work over this out-of-sample period? Unfortunately, we won’t know because Estimize didn't start collecting data until the end of 2011. The following table is a summary on the annual returns comparing different strategies using different data sets and periods.</div><div><br /></div><div><table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; margin-left: 5.4pt; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;"> <tbody><tr style="height: 21.4pt; mso-yfti-firstrow: yes; mso-yfti-irow: 0;"> <td style="border: solid windowtext 1.0pt; height: 21.4pt; mso-border-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">Strategies<o:p></o:p></span></div></td> <td style="border-left: none; border: solid windowtext 1.0pt; height: 21.4pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">Mean-Reversion<o:p></o:p></span></div></td> <td style="border-left: none; border: solid windowtext 1.0pt; height: 21.4pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">Momentum<o:p></o:p></span></div></td> </tr><tr style="height: 23.35pt; mso-yfti-irow: 1;"> <td style="border-top: none; border: solid windowtext 1.0pt; height: 23.35pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">Estimize <o:p></o:p></span></div><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">(2012.01-2013.12)<o:p></o:p></span></div></td> <td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; height: 23.35pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;"> 6%<o:p></o:p></span></div></td> <td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; height: 23.35pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;"> -9%<o:p></o:p></span></div></td> </tr><tr style="height: 21.4pt; mso-yfti-irow: 2;"> <td style="border-top: none; border: solid windowtext 1.0pt; height: 21.4pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">I/B/E/S <o:p></o:p></span></div><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">(2012.01-2013.12)<o:p></o:p></span></div></td> <td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; height: 21.4pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;"> -17%<o:p></o:p></span></div></td> <td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; height: 21.4pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;"> 11%<o:p></o:p></span></div></td> </tr><tr style="height: 22.25pt; mso-yfti-irow: 3; mso-yfti-lastrow: yes;"> <td style="border-top: none; border: solid windowtext 1.0pt; height: 22.25pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">I/B/E/S<o:p></o:p></span></div><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;">(2010.01-2011.12)<o:p></o:p></span></div></td> <td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; height: 22.25pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;"> 1.8%<o:p></o:p></span></div></td> <td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; height: 22.25pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal"><span style="font-family: "Helvetica","sans-serif"; font-size: 8.0pt;"> -6.4%<o:p></o:p></span></div><div class="MsoNormal"><br /></div></td> </tr></tbody></table></div><div><br /></div><div>As a result, we cannot conclude that Estimize data is consistently better than I/B/E/S data in terms of generating alpha: it depends on the strategy deployed. We also cannot decide which strategy – mean-reversion or momentum – is consistently better: it depends on the time period and the data used. The only conclusion we can reach is that the short duration of the Estimize data coupled with our lack of proper price data in 2014 means that we cannot have a statistically significant backtest. This state of inconclusiveness will of course be cured in time.</div><div><br /></div><div>_________</div><div>Yang Gao, Ph.D., is a research intern at <a href="http://epchan.com/">QTS Capital Management, LLC.</a></div><div><br /></div><div>===</div><div><b>Industry Update</b></div><div>(No endorsement of companies or products is implied by our mention.)</div><div><ul><li>There is a good discussion comparing <a href="http://quantconnect.com/">Quantconnect</a> to Quantopian <a href="http://www.quora.com/Quantitative-Finance/What-are-the-advantages-and-disadvantages-of-Quantopian-v-Quantconnect">here</a>.</li><li>For FX traders, <a href="http://rizm.io/">Rizm</a> offers a comparable service as Quantconnect and Quantopian as it is directly connected to FXCM.</li><li>Quantopian now offers free fundamental data from MorningStar. Also, check out their <a href="https://www.quantopian.com/posts/announcing-the-quantopian-managers-program">Quantopian Managers Program</a> where you can compete to manage real money.</li></ul><div>===</div></div><div><b>Workshop Update</b></div><div><br /></div><div>Our next online workshop will be <a href="http://www.epchan.com/workshops/">Millisecond Frequency Trading</a> on March 25-26. It is for traders who are interested in intraday trading (even if not at millisecond frequency) and who want to defend <i>against </i>certain HFT tactics.</div><div><br /></div><div>===</div><div><b>Managed Account Program Update</b></div><div><b><br /></b></div><div><div>Our <a href="http://www.epchan.com/accounts/">FX Managed Account program</a> had a strong finish in 2014, with annual net return of 69.86%.</div><div><br /></div><div>===</div><div>Follow me on Twitter: @chanep</div></div><div><br /></div>http://epchan.blogspot.com/2015/01/trading-with-estimize-and-ibes-earnings.htmlnoreply@blogger.com (Ernie Chan)50tag:blogger.com,1999:blog-35364652.post-7929659524799719952Fri, 14 Nov 2014 19:46:00 +00002014-11-17T15:25:15.855-05:00Automated trading platformsRent, don’t buy, data: our experience with QuantGo (Guest Post)<i>By Roger Hunter</i><br /><i><br /></i>I am a quant researcher and developer for QTS Partners, a commodity pool Ernie (author of this blog) founded in 2011. I help Ernie develop and implement several strategies in the pool and various separate accounts. I wrote this article to give insights into a very important part of our strategy development process: the selection of data sources.<br /><br />Our main research focus is on strategies that monitor execution in milliseconds and that hold for seconds through several days. For example, a strategy that trades more than one currency pair simultaneously must ensure that several executions take place at the right price and within a very short time. Backtesting requires high quality historical intraday quote and trade, preferably tick data for testing. Our initial focus was futures and after looking at various vendors for the tick data quality and quantity we needed, we chose Nanex data which is aggregated at 25ms. This means, for example, that aggressor flags are not available. We purchased several years of futures data and set to work.<br /><br />Earlier this year we needed to update our data and discovered that Nanex prices had increased significantly. We also needed quotes and trades, and data for more asset classes including US equities and options.<br /><br />We looked at TickData.com which has good data but is very expensive and you pay up-front per symbol. There are other services like Barchartondemand.com and XIgnite.com where you pay based on your monthly usage (number of data requests made) which is a model we do not like. We ended up choosing <a href="http://tinyurl.com/oxv4d6k">QuantGo.com</a>, where you have unlimited access to years of global tick or bar data for a fixed monthly subscription fee per data service.<br /><br />On QuantGo, you get computer instances in your own secure and private cloud built on Amazon AWS with on-demand access to a wide range of global intraday tick or bar data from multiple data vendors. Since you own and manage the computer instances you can choose any operating system, install any software, access the internet or import your own data. With QuantGo the original vendor data must remain in the cloud but you can download your results, this allows QuantGo to rent access to years of data at affordable monthly prices.<br /><br />All of the data we have used so far is from AlgoSeek (one of QuantGo’s data vendors). This data is survivorship bias-free and is exactly as provided by the exchanges at the time. Futures quotes and trades download very quickly on the system. I am testing options strategies, which is challenging due to the size of the data. The data is downloaded in highly compressed form which is then expanded (by QuantGo) to a somewhat verbose text form. Before the price split, a day of option quotes and trades for AAPL was typically 100GB in this form. Here is a data sample from the full Options (OPRA) data:<br /><br />Timestamp, EventType, Ticker, OptionDetail, Price, Quantity, Exchange, Conditions<br />08:30:02.493, NO_QUOTE BID NB, LLEN, PUT at 7.0000 on 2013-12-21, 0.0000, 0, BATS, F<br />08:30:02.493, NO_QUOTE ASK, LLEN, CALL at 7.0000 on 2013-12-21, 0.0000, 0, BATS, F<br />09:30:00.500, ROTATION ASK, LLEN, PUT at 2.0000 on 2013-07-20, 0.2500, 15, ARCA, R<br />09:30:00.500, ROTATION BID, LLEN, PUT at 2.0000 on 2013-07-20, 0.0000, 0, ARCA, R<br />09:30:00.507, FIRM_QUOTE ASK NB, LLEN, PUT at 5.0000 on 2013-08-17, 5.0000, 7, BATS, A<br />09:30:00.508, FIRM_QUOTE BID NB, LLEN, PUT at 6.0000 on 2013-08-17, 0.2000, 7, BATS, A<br /><br />These I convert to a more compact format, and filter out lines we don't need (e.g. NO_QUOTE, non-firm, etc.)<br /><br />The quality of the AlgoSeek data seems to be high. One test I have performed is to record live data and compare it with AlgoSeek. This is possible because the AlgoSeek historical data is now updated daily, and is one day behind for all except options, which varies from two days to five (they are striving for two, but the process involves uploading all options data to special servers --- a significant task). Another test is done using OptionNET Explorer (ONE). ONE data is at 5-minute intervals and the software displays midpoints only. However, by executing historical trades, you can see the bid and ask values for options at these 5-minute boundaries. I have checked 20 of these against the AlgoSeek data and found exact agreement in every case. In any event, you are free to contact the data vendors directly to learn more about their products. The final test of data quality (and of our market model) is the comparison of live trading results (at one contract/spread level) with backtests over the same period.<br /><br />The data offerings have recently expanded dramatically with more data partners and now include historical data from (QuantGo claims) "every exchange in the world". I haven't verified this, but the addition of elementized, tagged and scored news from Acquire Media, for example, will allow us to backtest strategies of the type discussed in Ernie's latest book.<br /><br />So far, we like the system. For us, the positives are:<br /><br />1. Affordable Prices. The reason that the price has been kept relatively low is that original vendor data must be kept and used in the QuantGo cloud. For example, to access years of US data we have been paying<br />Five years of US Equities Trades and Quotes (“TAQ”) is $250 per month<br />Five years of US Equities 5 minute Bars $75 per month<br />Three Years of US Options 1 minute bars $100 per month. <br />Three Year of CME, CBOT, NYMEX Futures Trades and Quotes $250 per month<br /><br />2. Free Sample Data. Each data service has free demo data which is actual real historical data where I can select data from the demo date range. This allowed me to view and work with the data before subscribing.<br /><br />3. One API. I have one API to access different data vendors. QuantGo gives me a java GUI, python CLI and various libraries (R, Matlab, Java).<br /><br />4. On-Demand. The ability to select the data we want "on demand" via a subscription from a website console at any time. You can select data for any symbol and for just a day or for several years.<br /><br />5. Platform not proprietary. We can use any operating system or software with the data as it is being downloaded to virtual computers we fully control and manage.<br /><br />Because all this is done in the cloud, we have to pay for our cloud computer usage as well. While cloud usage is continuing to drop rapidly in price it is still a variable cost and it needs to monitored. QuantGo does provide close to real-time billing estimates and alarms you can preset at dollar values. <br /><br />I was at first skeptical of the restriction of not being able to download the data vendor’s tick or bar data, but so far this hasn't been an issue as in practice we only need the results and our derived data sets. I'm told that if you want to buy the data for your own computers, you can negotiate directly with the individual data vendor and will get a discount if you have been using it for a while on QuantGo.<br /><br /><br />As we use the windows operating system we access our cloud computers with Remote Desktop and there have been some latency issues, but these are tolerable. On the other hand, it is a big advantage to be able to start with a relatively small virtual machine for initial coding and debugging, then "dial up" a much larger machine (or group of machines) when you want to run many compute and data intensive backtests. While <a href="http://tinyurl.com/oxv4d6k">QuantGo</a> is recently launched and is not perfect, it does open up the world of the highest institutional quality data to those of us who do not have the data budget of a Renaissance Technologies or D.E. Shaw.<br /><br /><span style="color: #333333; font-family: Georgia, serif; font-size: x-small;"><span style="background-color: white; line-height: 20.799999237060547px;">===</span></span><br /><b>Industry Update</b><br />(No endorsement of companies or products is implied by our mention.)<br /><ul><li>A new site for jobs in finance was recently launched: <a href="http://www.financejobs.co/">www.financejobs.co</a>.</li><li>A new software package Geode by Georgica Software can backtest tick data, and comes with a fairly rudimentary fill simulator.</li><li><a href="http://quantopian.com/">Quantopian.com</a> now incorporates a new IPython based research environment that allows interactive data analysis using minute level pricing data in Python.</li></ul>===<br /><div class="separator" style="background-color: white; clear: both; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;"><b>Workshops Update</b></div><br style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;" /><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">My next online <a href="http://www.epchan.com/workshops/">Quantitative Momentum Strategies</a> workshop will be held on December 2-4. Any reader interested in futures </span><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">trading</span><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;"> </span><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">in general would benefit from this course.</span><br /><br style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;" />===<br /><b>Managed Account Program Update</b><br /><br />Our FX Managed Account program had an unusually profitable month in <a href="http://www.epchan.com/accounts/">October</a>.<br /><br />===<br /><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">Follow me on Twitter: @chanep</span><br /><div><br /></div>http://epchan.blogspot.com/2014/11/rent-dont-buy-data-our-experience-with.htmlnoreply@blogger.com (Ernie Chan)93tag:blogger.com,1999:blog-35364652.post-4015011819012923001Fri, 05 Sep 2014 12:16:00 +00002014-09-05T08:16:21.597-04:00Moving Average Crossover = Triangle Filter on 1-Period ReturnsMany traders who use technical analysis favor the Moving Average Crossover as a momentum indicator. They compute the short-term minus the long-term moving averages of prices, and go long if this indicator just turns positive, or go short if it turns negative. This seems intuitive enough. What isn't obvious, however, is that MA Crossover is nothing more than an estimate of the recent average compound return.<br /><br />But just when you might be tempted to ditch this indicator in favor of the average compound return, it can be shown that the MA Crossover is also a triangle filter on the 1-period returns. (A triangle filter in signal processing is a set of weights imposed on a time series that increases linearly with time up to some point, and then decreases linearly with time up to the present time. See the diagram at the end of this article.) Why is this interpretation interesting? That's because it leads us to consider other, more sophisticated filters (such as the least square, Kalman, or wavelet filters) as possible momentum indicators. In collaboration with my former <a href="http://epchan.com/workshops" target="_blank">workshop</a> participant Alex W. who was inspired by this <a href="http://www.lyxor.com/fileadmin/_fileup/lyxor_wp/document/51/files/assets/downloads/publication.pdf" target="_blank">paper</a> by Bruder <i>et. al.</i>, we present the derivations below.<br /><br />===<br /><br />First, note that we will compute the moving average of log prices y, not raw prices. There is of course no loss or gain in information going from prices to log prices, but it will make our analysis possible. (The exact time of the crossover, though, will depend on whether we use prices or log prices.) If we write MA(t, n1) to denote the moving average of n1 log prices ending at time t, then the moving average crossover is MA(t, n1)-MA(t, n2), assuming n1< n2. By definition,<br /><br />MA(t, n1)=(y(t)+y(t-1)+...+y(t-n1+1))/n1<br />MA(t, n2)=(y(t)+y(t-1)+...+y(t-n1+1)+y(t-n1)+...+y(t-n2+1)/n2<br /><br />MA(t, n1)-MA(t, n2)<br />=[(n2-n1)/(n1*n2)] *[y(t)+y(t-1)+...+y(t-n1+1)] - (1/n2)*[y(t-n1)+...+y(t-n2+1)] <br />=[(n2-n1)/n2] *MA(t, n1)-[(n2-n1)/n2]*MA(t-n1, n2-n1)<br />=[(n2-n1)/n2]*[MA(t, n1)-MA(t-n1, n2-n1)]<br /><br />If we interpret MA(t, n1) as an approximation of the log price at the midpoint (n1-1)/2 of the time interval [t-n1+1, t], and MA(t-n1, n2-n1) as an approximation of the log price at the midpoint (n2-n1-1)/2 of the time interval [t-n1, t-(n2-n1)], then [MA(t, n1)-MA(t-n1, n2-n1)] is an approximation of the total return over a time period of n2/2. If we write this total return as an average compound growth rate r multiplied by the period n2/2, we get<br /><br />MA(t, n1)-MA(t, n2) ≈ [(n2-n1)/n2]*(n2/2)*r<br /><br />r ≈ [2/(n2-n1)]*[MA(t, n1)-MA(t, n2)]<br /><br />as shown in Equation 4 of the paper cited above. (Note the roles of n1 and n2 are reversed in that paper.)<br /><br />===<br /><br />Next, we will show why the MA crossover is also a triangle filter on 1-period returns. Simplifying notation by fixing t to be 0,<br /><br />MA(t=0, n1)<br />=(y(0)+y(-1)+...+y(-n1+1))/n1<br />=(1/n1)*[(y(0)-y(-1))+2(y(-1)-y(-2))+...+n1*(y(-n1+1)-y(-n1))]+y(-n1)<br /><br />Writing the returns from t-1 to t as R(t), this becomes<br /><br />MA(t=0, n1)=(1/n1)*[R(0)+2*R(-1)+...+n1*R(-n1+1)]+y(-n1)<br /><br />Similarly,<br /><br />MA(t=0, n2)=(1/n2)*[R(0)+2*R(-1)+...+n2*R(-n2+1)]+y(-n2)<br /><br />So MA(0, n1)-MA(0, n2)<br />=(1/n1-1/n2)*[R(0)+2*R(-1)+...+n1*R(-n1+1)]<br /> -(1/n2)*[(n1+1)*R(-n1)+(n1+2)*R(-n1-1)+...+n2*R(-n2+1)]<br />+y(-n1)-y(-n2)<br /><br />Note that the last line above is just the total cumulative return from -n2 to -n1, which can be written as<br /><br />y(-n1)-y(-n2)=R(-n1)+R(-n1-1)+...+R(-n2+1)<br /><br />Hence we can absorb that into the expression prior to that<br /><br />MA(0, n1)-MA(0, n2)<br />=(1/n1-1/n2)*[R(0)+2*R(-1)+...+n1*R(-n1+1)]<br /> -(1/n2)*[(n1+1-n2)*R(-n1)+(n1+2-n2)*R(-n1-1)+...+(-1)*R(-n2+2)]<br />=(1/n1-1/n2)*[R(0)+2*R(-1)+...+n1*R(-n1+1)]<br /> +(1/n2)*[(n2-n1-1)*R(-n1)+(n2-n1-2)*R(-n1-1)+...+R(-n2+2)]<br /><br />We can see the coefficients of R's from t=-n2+2 to -n1 form the left side of an triangle with positive slope, and those from t=-n1+1 to 0 form the right side of the triangle with negative slope. The plot (click to enlarge) below shows the coefficients as a function of time, with n2=10, n1=7, and current time as t=0. The right-most point is the weight for R(0): the return from t=-1 to 0.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-om1XEg4jujg/VADhixi23BI/AAAAAAAAC3s/QDQrZ3iue1Y/s1600/triangleFilter.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-om1XEg4jujg/VADhixi23BI/AAAAAAAAC3s/QDQrZ3iue1Y/s1600/triangleFilter.png" height="141" width="320" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;">Q.E.D. Now I hope you are ready to move on to a wavelet filter!</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">P.S. It is wonderful to be able to check the correctness of messy algebra like those above with a simple Matlab program!</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">===</div><div class="separator" style="clear: both; text-align: left;"><b>New Service Announcement</b></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">Our firm QTS Capital Management has recently launched a <a href="http://epchan.com/accounts" target="_blank">FX Managed Accounts</a> program. It uses one of the mean-reverting strategies we have been trading successfully in our fund for the last three years, and is still going strong despite the low volatility in the markets. The benefits of a managed account are that clients retain full ownership and control of their funds at all times, and they can decide what level of leverage they are comfortable with. Unlike certain offshore FX operators, QTS is a CPO/CTA regulated by the National Futures Association and the Commodity Futures Trading Commission.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">===</div><div class="separator" style="clear: both; text-align: left;"><b>Workshops Update</b></div><br />Readers may be interested in my next workshop series to be held in London, November 3-7. Please follow the link at the bottom of <a href="http://epchan.com/workshops" target="_blank">this page</a> for information.<br /><br />===<br />Follow me on Twitter: @chanephttp://epchan.blogspot.com/2014/09/moving-average-crossover-triangle.htmlnoreply@blogger.com (Ernie Chan)141tag:blogger.com,1999:blog-35364652.post-6480832994536541528Mon, 18 Aug 2014 14:33:00 +00002014-08-18T10:50:42.661-04:00Kelly vs. Markowitz Portfolio OptimizationIn my <a href="http://www.amazon.com/dp/0470284889/ref=as_sl_pd_tf_lc?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=0470284889&adid=0DEF116W4JR65B8KSV6H&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" target="_blank">book</a>, I described a very simple and elegant formula for determining the optimal asset allocation among N assets:<br /><br />F=C<sup>-1</sup>*M (1)<br /><div class="MsoNormal"><o:p></o:p></div><br />where F is a Nx1 vector indicating the fraction of the equity to be allocated to each asset, C is the covariance matrix, and M is the mean vector for the excess returns of these assets. Note that these "assets" can in fact be "trading strategies" or "portfolios" themselves. If these are in fact real assets that incur a carry (financing) cost, then excess returns are returns minus the risk-free rate.<br /><br />Notice that these fractions, or weights as they are usually called, are not normalized - they don't necessarily add up to 1. This means that F not only determines the allocation of the total equity among N assets, but it also determines the overall optimal leverage to be used. The sum of the absolute value of components of F divided by the total equity is in fact the overall leverage. Thus is the beauty of Kelly formula: optimal allocation and optimal leverage in one simple formula, which is supposed to maximize the compounded growth rate of one's equity (or equivalently the equity at the end of many periods).<br /><br />However, most students of finance are not taught Kelly portfolio optimization. They are taught Markowitz mean-variance portfolio optimization. In particular, they are taught that there is a portfolio called the <i>tangency portfolio </i>which lies on the efficient frontier (the set of portfolios with minimum variance consistent with a certain expected return) and which maximizes the Sharpe ratio. Left unsaid are<br /><br /><ul><li>What's so good about this tangency portfolio?</li><li>What's the real benefit of maximizing the Sharpe ratio?</li><li>Is this tangency portfolio the same as the one recommended by Kelly optimal allocation?</li></ul><div>I want to answer these questions here, and provide a connection between Kelly and Markowitz portfolio optimization.</div><div><br /></div><div>According to Kelly and Ed Thorp (and explained in my book), F above not only maximizes the compounded growth rate, but it also maximizes the Sharpe ratio. Put another way: the maximum growth rate is achieved when the Sharpe ratio is maximized. Hence we see why the tangency portfolio is so important. And in fact, <b>the tangency portfolio is the same as the Kelly optimal portfolio F</b>, except for that fact that the tangency portfolio is assumed to be normalized and has a leverage of 1 whereas F goes one step further and determines the optimal leverage for us. Otherwise, the percent allocation of an asset in both are the same (assuming that we haven't imposed additional constraints in the optimization problem). How do we prove this?</div><div><br /></div><div>The usual way Markowitz portfolio optimization is taught is by setting up a constrained <i>quadratic </i>optimization problem - quadratic because we want to optimize the portfolio variance which is a quadratic function of the weights of the underlying assets - and proceed to use a numerical quadratic programming (QP) program to solve this and then further maximize the Sharpe ratio to find the tangency portfolio. But this is unnecessarily tedious and actually obscures the elegant formula for F shown above. Instead, we can proceed by applying Lagrange multipliers to the following optimization problem (see <a href="http://faculty.washington.edu/ezivot/econ424/portfolioTheoryMatrix.pdf">http://faculty.washington.edu/ezivot/econ424/portfolioTheoryMatrix.pdf</a> for a similar treatment):</div><div><br /></div><div>Maximize Sharpe ratio = F<sup>T</sup>*M/(F<sup>T</sup>*C*F)<sup>1/2 </sup>(2)</div><div class="MsoNormal"><o:p></o:p></div><div><br /></div><div>subject to constraint F<sup>T</sup>*<b>1</b>=1 (3)</div><div class="MsoNormal"><o:p></o:p></div><div><br /></div><div>(to emphasize that the <b>1</b> on the left hand side is a column vector of one's, I used bold face.)</div><div><br /></div><div>So we should maximize the following unconstrained quantity with respect to the weights F<sub>i </sub>of each asset i and the Lagrange multiplier λ:</div><div><br /></div><div>F<sup>T</sup>*M/(F<sup>T</sup>*C*F)<sup>1/2 - </sup>λ(F<sup>T</sup>*<b>1-</b>1) (4)</div><div class="MsoNormal"><o:p></o:p></div><div><br /></div><div>But taking the partial derivatives of this fraction with a square root in the denominator is unwieldy. So equivalently, we can maximize the logarithm of the Sharpe ratio subject to the same constraint. Thus we can take the partial derivatives of </div><div><br /></div><div>log(F<sup>T</sup>*M)-(1/2)*log(F<sup>T</sup>*C*F)<sup> - </sup>λ(F<sup>T</sup>*<b>1-</b>1) (5)</div><div class="MsoNormal"><o:p></o:p></div><div><br /></div><div>with respect to F<sub>i</sub>. Setting each component i to zero gives the matrix equation</div><div><br /></div><div>(1/F<sup>T</sup>*M)M-(1/F<sup>T</sup>*C*F)C*F=λ<b>1 </b>(6)</div><div class="MsoNormal"><o:p></o:p></div><div><b><br /></b></div><div>Multiplying the whole equation by F<sup>T </sup>on the right gives</div><div><br /></div><div>(1/F<sup>T</sup>*M)F<sup>T</sup>*M-(1/F<sup>T</sup>*C*F)F<sup>T</sup>*C*F=λF<sup>T</sup>*<b>1 </b>(7)</div><div class="MsoNormal"><o:p></o:p></div><div><b><br /></b></div><div>Remembering the constraint, we recognize the right hand side as just λ. The left hand side comes out to be exactly zero, which means that λ is zero. A Lagrange multiplier that turns out to be zero means that the constraint won't affect the solution of the optimization problem up to a proportionality constant. This is satisfying since we know that if we apply an equal leverage on all the assets, the maximum Sharpe ratio should be unaffected. So we are left with the matrix equation for the solution of the optimal F:</div><div><br /></div><div>C*F=(F<sup>T</sup>*C*F/F<sup>T</sup>*M)M (8)</div><div class="MsoNormal"><o:p></o:p></div><div><br /></div><div>If you know how to solve this for F using matrix algebra, I would like to hear from you. But let's try an <i>ansatz </i>F=C<sup>-1</sup>*M as in (1). The left hand side of (8) becomes M, the right hand side becomes (F<sup>T</sup>*M/F<sup>T</sup>*M)M = M as well. So the ansatz works, and the solution is in fact (1), up to a proportionality constant. To satisfy the normalization constraint (3), we can write</div><div><br /></div><div>F=C<sup>-1</sup>*M / <span style="line-height: 115%;">(</span><b>1</b><sup>T</sup><span style="line-height: 115%;">*C</span><sup>-1</sup><span style="line-height: 115%;">*M)</span><span style="font-size: 13.5pt; line-height: 115%;"> </span>(9)</div><div><br /></div><div>So there, the tangency portfolio is the same as the Kelly optimal portfolio, up to a normalization constant, and without telling us what the optimal leverage is.</div><div><br /></div><div>===</div><div><b style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">Workshop Update:</b></div><div><b style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;"><br /></b><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">Based on popular demand, I have revised the dates for my online </span><a href="http://www.epchan.com/my-workshops/" style="background-color: white; color: #999999; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px; text-decoration: none;" target="_blank">Mean Reversion Strategies</a> <span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">workshop to be </span><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">August 27-29. </span></div><div class="MsoNormal"><sub><o:p></o:p></sub></div><div><br /></div><div>===</div><div><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;">Follow me </span><a href="https://twitter.com/intent/follow?original_referer=http%3A%2F%2Fepchan.blogspot.ca%2F&region=follow_link&screen_name=chanep&tw_p=followbutton&variant=2.0" style="background-color: white; color: #999999; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px; text-decoration: none;" target="_blank">@chanep</a><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.799999237060547px;"> on Twitter.</span></div><div><br /></div><div class="MsoNormal"><o:p></o:p></div><div><br /></div><div><br /></div><br /><div class="MsoNormal"><o:p></o:p></div><div class="MsoNormal"><o:p></o:p></div><div class="MsoNormal"><o:p></o:p></div><div class="MsoNormal"><o:p></o:p></div><div class="MsoNormal"><o:p></o:p></div><div class="MsoNormal"><o:p></o:p></div><div class="MsoNormal"><o:p></o:p></div><div class="MsoNormal"><o:p></o:p></div>http://epchan.blogspot.com/2014/08/kelly-vs-markowitz-portfolio.htmlnoreply@blogger.com (Ernie Chan)56tag:blogger.com,1999:blog-35364652.post-8388741534565461886Wed, 02 Jul 2014 18:51:00 +00002014-07-02T14:51:54.542-04:00Another "universal" capital allocation algorithmFinancial engineers are accustomed to borrowing techniques from scientists in other fields (e.g. genetic algorithms), but rarely does the borrowing go the other way. It is therefore surprising to hear about this <a href="http://www.pnas.org/content/early/2014/06/11/1406556111.abstract" target="_blank">paper</a> on a possible mechanism for evolution due to natural selection which is inspired by universal capital allocation algorithms.<br /><br />A capital allocation algorithm attempts to optimize the allocation of capital to stocks in a portfolio. An allocation algorithm is called <i>universal </i>if it results in a net worth that is "similar" to that generated by the best constant-rebalanced portfolio with fixed weightings over time (denoted CBAL* below), chosen in hindsight. "Similar" here means that the net worth does not diverge exponentially. (For a precise definition, see this very readable <a href="http://arxiv.org/pdf/1107.0036.pdf" target="_blank">paper</a> by Borodin, <i>et al</i>. H/t: Vladimir P.)<br /><br />Previously, I know only of one such universal trading algorithm - the Universal Portfolio invented by Thomas Cover, which I have described <a href="http://epchan.blogspot.ca/2007/01/universal-portfolios.html" target="_blank">before</a>. But here is another one that has proven to be universal: the exceedingly simple <a href="http://www.cis.upenn.edu/~mkearns/finread/helmbold98line.pdf" target="_blank">EG algorithm</a>.<br /><br />The EG ("Exponentiated Gradient") algorithm is an example of a capital allocation rule using "multiplicative updates": the new capital allocated to a stock is proportional to its current capital multiplied by a factor. This factor is an exponential function of the return of the stock in the last period. This algorithm is both greedy and conservative: greedy because it always allocates more capital to the stock that did well most recently; conservative because there is a penalty for changing the allocation too drastically from one period to the next. This multiplicative update rule is the one proposed as a model for evolution by natural selection.<br /><br />The computational advantage of EG over the Universal Portfolio is obvious: the latter requires a weighted average over all possible allocations at every step, while the former needs only know the allocation and returns for the most recent period. But does this EG algorithm actually generate good returns in practice? I tested it two ways:<br /><br />1) Allocate between cash (with 2% per annum interest) and SPY.<br />2) Allocate among SP500 stocks.<br /><br />In both cases, the only free parameter of the model is a number called the "learning rate" η, which determines how fast the allocation can change from one period to the next. It is generally found that η=0.01 is optimal, which we adopted. Also, we disallow short positions in this study.<br /><br />The benchmarks for comparison for 1) are, using the notations of the Borodin paper,<br /><br />a) the buy-and-hold SPY portfolio <b>BAH</b>, and<br />b) the best constant-rebalanced portfolio with fixed allocations in hindsight <b>CBAL*</b>.<br /><br />The benchmarks for comparison for 2) are<br /><br />a) a constant rebalanced portfolio of SP500 stocks with equal allocations <b>U-CBAL</b>,<br />b) a portfolio with 100% allocation to the best stock chosen in hindsight <b>BEST1</b>, and<br />c) <b>CBAL*</b>.<br /><br />To find CBAL* for a SP500 portfolio, I used Matlab Optimization Toolbox's constrained optimization function <i>fmincon</i>.<br /><br />There is also the issue of SP500 index reconstitution. It is complicated to handle the addition and deletion of stocks in the index within a constrained optimization function. So I opted for the shortcut of using a subset of stocks that were in SP500 from 2007 to 2013, tolerating the presence of surivorship bias. There are only 346 such stocks.<br /><br />The result for 1) (cash vs SPY) is that the CAGR (compound annualized growth rate) of EG is slightly lower than BAH (4% vs 5%). It turns out that BAH and CBAL* are the same: it was best to allocate 100% to SPY during 2007-2013, an unsurprising recommendation in hindsight.<br /><br />The result for 2) is that the CAGR of EG is higher than the equal-weight portfolio (0.5% vs 0.2%). But both these numbers are much lower than that of BEST1 (39.58%), which is almost the same as that of CBAL* (39.92%). (Can you guess which stock in the current SP500 generated the highest CAGR? The answer, to be revealed below*, will surprise you!)<br /><br />We were promised that the EG algorithm will perform "similarly" to CBAL*, so why does it underperform so miserably? Remember that similarity here just means that the divergence is sub-exponential: but even a polynomial divergence can in practice be substantial! This seems to be a universal problem with universal algorithms of asset allocation: I have never found any that actually achieves significant returns in the short span of a few years. Maybe we will find more interesting results with higher frequency data.<br /><br />So given the underwhelming performance of EG, why am I writing about this algorithm, aside from its interesting connection with biological evolution? That's because it serves as a setup for another, <i>non</i>-universal, portfolio allocation scheme, as well as a way to optimize parameters for trading strategies in general: both topics for another time<br /><br />===<br /><b>Workshops Update:</b><br /><b><br /></b>My next online workshop will be on <a href="http://www.epchan.com/my-workshops/" target="_blank">Mean Reversion Strategies</a>, August 26-28. <a href="http://www.nbs.ntu.edu.sg/Executive_Education/NTU_SGX_Centre_for_Financial_Education/Documents/ATC-%20MEAN%20REVERSION%20STRATEGIES.pdf" target="_blank">This</a> and the <a href="http://www.nbs.ntu.edu.sg/Executive_Education/NTU_SGX_Centre_for_Financial_Education/Documents/ATC-%20QUANTITATIVE%20MOMENTUM%20STRATEGIES.pdf" target="_blank">Quantitative Momentum</a> workshops will also be conducted live at Nanyang Technological University in Singapore, September 18-21.<br /> <b><br /></b><b>===</b><br />Do follow me <a href="https://twitter.com/intent/follow?original_referer=http%3A%2F%2Fepchan.blogspot.ca%2F&region=follow_link&screen_name=chanep&tw_p=followbutton&variant=2.0" target="_blank">@chanep</a> on Twitter, as I often post links to interesting articles there.<br /><br /><b>===</b><br />*The SP500 stock that generated the highest return from 2007-2013 is AMZN.<br /><div class="MsoNormal"><o:p></o:p></div><div class="MsoNormal"><o:p></o:p></div>http://epchan.blogspot.com/2014/07/another-universal-capital-allocation.htmlnoreply@blogger.com (Ernie Chan)27tag:blogger.com,1999:blog-35364652.post-5826648922585539936Fri, 09 May 2014 14:16:00 +00002014-05-09T10:16:50.125-04:00factor modelStrategiesShort Interest as a FactorReaders of zerohedge.com will no doubt be impressed by this <a href="http://www.zerohedge.com/sites/default/files/images/user5/imageroot/2014/02/20140221_SI.png" target="_blank">chart</a> and the accompanying <a href="http://www.zerohedge.com/news/presenting-most-shorted-stocks" target="_blank">article</a>:<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><img alt="" src="http://www.zerohedge.com/sites/default/files/images/user5/imageroot/2014/02/20140221_SI.png" height="207" style="margin-left: auto; margin-right: auto;" title="Cumulative returns of most shorted stocks relative to SPX" width="400" /></td></tr><tr><td class="tr-caption" style="text-align: center;">Cumulative Returns of Most Shorted Stocks in 2013</td></tr></tbody></table><br />Indeed, short interest (expressed as the number of shares shorted divided by the total number of shares outstanding) has long been thought to be a useful factor. To me, the counter-intuitive wisdom is that the more a stock is shorted, the better is its performance. You might explain that by saying this is a result of the "short squeeze", when there is jump in price perhaps due to news and stock lenders are eager to sell the stock they own. If you have borrowed this stock to short, your borrowed stock may be recalled and you will be forced to buy cover at this most inopportune time. But this is an unsatisfactory explanation, as this will result only in a short term (upward) momentum in price, not the sustained out-performance of the most shorted stocks. This long-term out-performance seems to suggest that short sellers are less informed than the average trader, which is odd.<br /><br />Whatever the explanation, I am intrigued to find out if short interest really is a good factor to incorporate into a comprehensive factor model over the long term.<br /><br />The result? Not particularly impressive. It turns out that 2013 was one of the best years for this factor (hence the impressive chart above). For that year, a daily-rebalanced long-short portfolio (long 50 most shorted stocks and short 50 least shorted stocks in the SPX) returned 6.9%, with a Sharpe ratio of 2 and a Calmar ratio of 2.9. However, if we extend our backtest to 2007, the APR is only 2.8%, with a Sharpe ratio of 0.5 and a Calmar ratio of 0.3. This backtest was done using survivorship-bias-free data from CRSP, with short interest data provided by Compustat.<br /><br />Here is the cumulative returns chart from 2007-2013:<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-5toLi7XwczY/U2UDA39_9II/AAAAAAAACPw/WuWnENUnHzQ/s1600/Cumulative+Returns+of+Short+Interest+Factors+2007+to+2013.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://3.bp.blogspot.com/-5toLi7XwczY/U2UDA39_9II/AAAAAAAACPw/WuWnENUnHzQ/s1600/Cumulative+Returns+of+Short+Interest+Factors+2007+to+2013.png" height="240" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Cumulative Returns of LS Portfolio based on Short Interest: 2007-2013</td></tr></tbody></table><br /><br />Interesting, trying this on the SP600 small-cap universe yielded negative returns, possibly meaning that short-sellers of small caps do have superior information.<br /><br />I promise, this will be the last time I talk about factors in a while!<br /><br />===<br /><b>Tech Update:</b><br /><b><br /></b>I was shocked to learn that Matlab now offers licenses for just $149 - the so-called <a href="http://www.mathworks.com/products/matlab-home/" target="_blank">Matlab Home</a> (h/t: Ken H.) In addition, its Trading Toolbox now offers API connection to Interactive Brokers, in addition to a few other brokerages. I am familiar with both Matlab and R, and while I am impressed by the large number of free, sophisticated statistical packages in R, I still stand by Matlab as the most productive platform for developing our own strategies. The Matlab development (debugging) environment is just that much more polished and easy-to-use. The difference is bigger than Microsoft Word vs. Google Docs.<br /><ul></ul>A reader Ravi B. told me that there is a website called www.seasonalgo.com if you want to try out different seasonal futures strategies.<br /><ul></ul>Finally, a startup at <a href="http://inovancetech.com/">inovancetech.com</a> offers machine learning algorithms to help you find the best combination of technical indicators for trading FX.<br /><ul></ul>===<br /><b>Workshops Update:</b><br /><br />I am now offering the <i>Millisecond Frequency Trading</i> (MFT) Workshop as an online course on June 26- 27. Previously, I have only offered it live in London and to a few institutional investors. It has two main parts:<br /><br />Part 1: introducing techniques for traders who want to <i>avoid</i> HFT predators.<br /><br />Part 2: how to backtest a strategy that requires tick data with millisecond resolution using Matlab.<br /><br />The example strategy used is based on order flow. For more details, please visit <a href="http://epchan.com/my-workshops">epchan.com/my-workshops</a>.<br /><br />Additionally, I will be teaching the Mean Reversion and Momentum (but not MFT) workshops in Hong Kong on June 17-20.<br /><br />http://epchan.blogspot.com/2014/05/short-interest-as-factor.htmlnoreply@blogger.com (Ernie Chan)144tag:blogger.com,1999:blog-35364652.post-3952097687344986066Thu, 27 Mar 2014 22:17:00 +00002014-03-27T18:17:10.890-04:00factor modelStrategiesUpdate on the fundamentals factors: their effect on small cap stocksIn my last <a href="http://epchan.blogspot.com/2014/02/fundamental-factors-revisited-with.html">post</a>, I reported that the fundamental factors used by Lyle and Wang seem to generate no returns on SP500 large cap stocks. These fundamental factors are the growth factor return-on-equity (ROE), and the value factor book-to-market ratio (BM).<br /><br />I have since studied the effect of these factors on SP600 small cap stocks since 2004, using a survivorship-bias-free database combining information from both Compustat and CRSP. This time, the factors do produce an annualized average return of 4.7% and a Sharpe ratio of 0.8. Though these numbers are nowhere near the 26% return that Lyle and Wang found, they are still statistically significant. I have plotted the equity curve below.<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-mLfHcjclBek/UyR0kAisYpI/AAAAAAAACMo/bKoz9C8dFcs/s1600/compustatFundamentFactors4_SML.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="2004-2013" border="0" src="http://4.bp.blogspot.com/-mLfHcjclBek/UyR0kAisYpI/AAAAAAAACMo/bKoz9C8dFcs/s1600/compustatFundamentFactors4_SML.jpg" height="240" title="" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Equity curve of long-short small-cap portfolio based on regression on ROE and BM factors (2004-2013)</td></tr></tbody></table>One may wonder whether ROE or BM is the more important factor. So I run a simpler model which uses one factor at a time to rank stocks every day. We buy stocks in top decile of ROE, and short the ones in the bottom decile. Ditto for BM. I found an annualized average return of 5% with a Sharpe ratio of 0.8 using ROE only, and only 0.8% with a Sharpe ratio of 0.09 using BM only. The value factor BM is almost completely useless! Indeed, if we were to first sort on ROE, pick the top and bottom deciles, and then sort on BM, and pick the top and bottom halves, the resulting average return is almost the same as sorting on ROE alone. I plotted the equity curve for sorting on ROE below.<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-Ut4v9wWKCkU/UyR5UdD-gXI/AAAAAAAACM0/kgtLyAKud9I/s1600/compustatFundamentFactors5_SML_ROEonly.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://2.bp.blogspot.com/-Ut4v9wWKCkU/UyR5UdD-gXI/AAAAAAAACM0/kgtLyAKud9I/s1600/compustatFundamentFactors5_SML_ROEonly.jpg" height="240" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Equity curve of long-short small-cap portfolio based on top and bottom deciles of ROE (2004-2013)</td></tr></tbody></table><br />Notice the sharp drawdown from 2008-05-30 to 2008-11-04, and the almost perfect recovery since then. This mirrors the behavior of the equity market itself, which raises the question of why we bother to construct a long-short portfolio at all as it provides no hedge against the downturn. It is also interesting to note that this factor does not exhibit "momentum crash" as explained in a previous <a href="http://epchan.blogspot.com/2013/07/momentum-crash-and-recovery.html">article</a>: it does not suffer at all during the market recovery. This means we should not automatically think of a fundamental growth factor as similar to price momentum.<br /><br />My conclusion was partly corroborated by I. Kaplan who has written a <a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2407303" target="_blank">preprint</a> on a similar topic. He found that a long-short portfolio created using the ratio EBITA/Enterprise Value on large caps generates a Sharpe ratio of about 0.6 but with very little drawdown unlike the ROE factor that I studied above as applied to small caps.<br /><br />As Mr. Kaplan noted, these results are in some contradiction not only with Lyle and Wang's paper, but also with the widely circulated <a href="http://faculty.chicagobooth.edu/tobias.moskowitz/research/JF_12021_TMcomments.pdf" target="_blank">paper</a> by Cliff Asness <i>et al</i>. These authors found the the BM factor works in practically every asset class. Of course, the timeframe of their research is much longer than my focus above. Furthermore, they have excluded financial and penny stocks, though I did not find such restrictions to have great impact in my study of large cap portfolios. In place of a fundamental growth factor, these authors simply used price momentum over an 11-month period (skipping the most recent month), and found that this is also predictive of future quarterly returns.<br /><br />Finally, we should note that the ROE and BM factors here are quite similar to the Return-on-Capital and Earnings Yield factors used by Joel Greenblatt in his famous "<a href="http://www.amazon.com/gp/product/0470624159/ref=as_li_qf_sp_asin_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0470624159&linkCode=as2&tag=quantitativet-20" target="_blank">Little Book That Still Beats The Market</a>". One wonders if those factors suffer a similar drawdown during the financial crisis.<br /><br />===<br /><br />My online Momentum Workshop will be offered on May 5-7. Please visit epchan.com/my-workshops for registration details. Furthermore, I will be teaching my Mean Reversion, Momentum, and Millisecond Frequency Trading workshops in Hong Kong on June 17-20.<br /><div><br /></div><br />http://epchan.blogspot.com/2014/03/update-on-fundamentals-factors-their.htmlnoreply@blogger.com (Ernie Chan)67tag:blogger.com,1999:blog-35364652.post-2028420659889950202Sat, 08 Feb 2014 16:47:00 +00002014-02-15T11:24:55.090-05:00Automated trading platformsfactor modelStrategiesFundamental factors revisited, with a technology updateContrary to my tradition of alerting readers to new and fancypants <a href="http://epchan.blogspot.com/search/label/factor%20model" target="_blank">factors</a> for predicting stock returns (while not necessarily endorsing any of them), I report that <a href="http://www.hbs.edu/faculty/Publication%20Files/13-050_14a48ef4-5d37-453d-9102-66bd4548f581.pdf" target="_blank">Lyle and Wang</a> have recently published new research demonstrating the power of two very familiar factors: book-to-market ratio (BM) and return-on-equity (ROE).<br /><br />The model is simple: at the end of each calendar quarter, compute the log of BM and ROE for every stock based on the most recent earnings announcement, and regress the next-quarter return against these two factors. One subtlety of this regression is that the factor loadings (log BM and ROE) and the future returns for stocks within an industry group are pooled together. This makes for a cross-sectional factor model, since the factor loadings (log BM and ROE) vary by stock but the factor returns (the regression coefficients) are the same for all stocks within an industry group. (A clear elucidation of cross-sectional vs time-series factor models can be found in Section 17.5 of <a href="http://www.amazon.com/dp/1441977864?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=1441977864&adid=160RSP2N7R0PWA0KC4XN&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" target="_blank">Ruppert</a>.) If we long stocks within the top decile of expected returns and short the bottom decile and hold for a quarter, the expected annualized average returns of this model is an eye-popping 26% or so.<br /><br />I have tried to replicate these results, but unfortunately I couldn't. (My program generated a measly, though positive, APR.) The data requirement and the program are both quite demanding. I am unable to obtain the 60 quarters of fundamental data that the authors recommended - I merely have 40. I used the 65 industry groups defined by the GIC industry classifications, while the authors used the 48 Fama-French industry groups. Finally, I am unsure how to deal with stocks which have negative book values or earnings, so I omit those quarterly data. If any of our readers are able to replicate these results, please do let us know.<br /><br />The authors and I used Compustat database for the fundamental data. If you do not have subscription to this database, you can consider a new, free, website called <a href="http://thinknum.com/">Thinknum.com</a>. This website makes available all data extracted from companies' SEC filings starting in 2009 (2011 for small caps). There is also a neat integration with R described <a href="http://www.r-bloggers.com/thinknum-a-new-interactive-public-database-and-graphing-tool/" target="_blank">here</a>.<br /><br /><b>*** Update ***</b><br /><b><br /></b>I forgot to point out one essential difference between the method in the cited paper and my own effort: the paper used the entire stock universe except for stocks cheaper than $1, while I did my research only on SP500 stocks (Hat tip to Prof. Lyle who clarified this). This turns out to be of major importance: a to-be-published paper by our reader I. Kaplan reached the conclusion that "Linear models based on value factors do not predict future returns for the S&P 500 universe for the past fifteen years (from 1998 to 2013)."<br /><b><br /></b><br />===<br /><br /><span style="font-family: inherit;">Speaking of new trading technology platforms that provide historical data for backtesting (other than Thinknum.com and the previously mentioned Quantopian.com), here is another interesting one: <a href="https://quantgo.com/?affid=17d">QuantGo.com</a>. It provides institutional intraday historical data through its data partners from 1 minute bars to full depth of book in your own private cloud running on Amazon EC2 account for a low monthly rate. They give unlimited access to years of historical data for a monthly data access fee, for examples US equities Trades and Quotes (TAQ) for an unlimited number of years are $250 per month of account rental, OPRA TAQ $250 permonth and tagged news is $200. Subscribers control and manage their own computer instances, so can install and use whatever software they want on them to backtest or trade using the data. The only hitch is that you are not allowed to download the vendor data to your own computer, it has to stay in the private cloud.</span><br /><br />===<br /><br /><span style="font-family: inherit;">Follow <a href="https://twitter.com/intent/follow?original_referer=http%3A%2F%2Fepchan.blogspot.ca%2F&region=follow_link&screen_name=chanep&tw_p=followbutton&variant=2.0" target="_blank">@chanep</a> to receive my occasional tweets on interesting quant trading industry news and articles.</span><br /><br />===<br /><br /><span style="color: #333333; font-family: inherit;"><span style="line-height: 20.796875px;">My online Mean Reversion Strategies Workshop will be offered on April 1-3. Please visit <a href="http://epchan.com/my-workshops">epchan.com/my-workshops</a> for registration details. Furthermore, I will be teaching my Mean Reversion, Momentum, and Millisecond Frequency Trading workshops in London on March 17-21, and in Hong Kong on June 17-20.</span></span><br /><div><br /></div>http://epchan.blogspot.com/2014/02/fundamental-factors-revisited-with.htmlnoreply@blogger.com (Ernie Chan)90tag:blogger.com,1999:blog-35364652.post-6972923671523898326Wed, 08 Jan 2014 15:03:00 +00002014-02-07T09:40:00.511-05:00factor modelStrategiesVariance Risk Premium for Return ForecastingFolklore has it that VIX is a reasonable leading indicator of risk. Presumably that means if VIX is high, then there is a good chance that the future return of the SP500 will be negative. While I have found some evidence that this is true when VIX is particularly elevated, say above 30, I don't know if anyone has established a negative correlation between VIX and future returns. (<i>Contemporaneous </i>VIX and SP500 levels do have a very nice linear relationship with negative slope.)<br /><br />Interestingly, the situation is much clearer if we examine the Variance Risk Premium (VRP), which is defined as the difference between a model-free implied volatility (of which VIX is the most famous example) and the historical volatility over a recent period. The relationship between VRP and future returns is examined in a paper by <a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2292496" target="_blank">Chevallier and Sevi</a> in the context of OVX, which is the CBOE Crude Oil Volatility Index. They have found that there is a statistically significant negative linear relationship between VRP and future 1-month crude oil futures (CL) returns. The historical volatility is computed over 5-minute returns of the most recent trading day. (Why 5 minutes? Apparently this is long enough to avoid the artifactual volatility induced by bid-ask bounce, and short enough to truly sample intraday volatility.) If you believe in the prescience of options traders, it should not surprise you that the regression coefficient is negative (i.e. a high VRP predicts a lower future return).<br /><br />I have tested a simple trading strategy based on this linear relationship. Instead of using monthly returns, I use VRP to predict daily returns of CL. It is very similar to a mean-reverting Bollinger band strategy, except that here the "Bollinger bands" are constructed out of moving first and third quartiles of VRP with a 90-day lookback. Given that VRP is far from normally distributed, I thought it is more sensible to use quartiles rather than standard deviations to define the Bollinger bands. So we buy a front contract of CL and hold for just 1 day if VRP is below its moving first quartile, and short if VRP is above its moving third quartile. It gives a decent average annual return of 17%, but performance was poor in 2013.<br /><br />Naturally, one can try this simple trading strategy on the E-mini SP500 future ES also. This time, VRP is VIX minus the historical volatility of ES. Contrary to folklore, I find that if we regress the future 1 day ES return against VRP, the regression coefficient is positive. This means that an increase of VIX relative to historical volatility actually predicts an increase in ES! (Does this mean investors are overpaying for put options on SPX for portfolio protection?) Indeed, the opposite trading rules from the above give positive returns: we should buy ES if VRP is above its moving third quartile, and short ES if VRP is below its moving first quartile. The annualized return is 6%, but performance in 2013 was also poor.<br /><br />As the authors of the paper noted, whether or not VRP is a strong enough stand-alone predictor of returns, it is probably useful as an additional factor in a multi-factor model for CL and ES. If any reader know of other volatility index like VIX and OVX, please do share with us in the comments section!<br /><br />===<br /><br /><span style="font-family: inherit;"><span style="background-color: white; color: #333333; line-height: 20.796875px;">My online Backtesting Workshop will be offered on February 18-19. Please visit </span><a href="http://epchan.com/my-workshops" style="background-color: white; color: #999999; line-height: 20.796875px; text-decoration: none;" target="_blank">epchan.com/</a><a href="http://epchan.com/my-workshops" style="background-color: white; color: #999999; line-height: 20.796875px; text-decoration: none;" target="_blank">my-workshops </a><span style="background-color: white; color: #333333; line-height: 20.796875px;">for registration details. Furthermore, I will be teaching my Mean Reversion, Momentum, and Millisecond Frequency Trading workshops in London on March 17-21, and in Hong Kong on June 17-20.</span></span>http://epchan.blogspot.com/2014/01/variance-risk-premium-for-return.htmlnoreply@blogger.com (Ernie Chan)55tag:blogger.com,1999:blog-35364652.post-940091923041235791Fri, 15 Nov 2013 14:45:00 +00002013-11-15T11:50:19.534-05:00StrategiesCointegration Trading with Log Prices vs. PricesIn my recent <a href="http://www.amazon.com/dp/1118460146/ref=as_li_qf_sp_asin_til?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=1118460146&adid=00W7DZWKQ8DNBM541YD6&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" target="_blank">book</a>, I highlighted a difference between cointegration (pair) trading of price spreads and log price spreads. Suppose the price spread hA*yA-hB*yB of two stocks A and B is stationary. We should just keep the <b>number of shares</b> of stocks A and B fixed, in the ratio hA:hB, and short this spread when it is much higher than average, and long this spread when it is much lower. On the other hand, for a stationary log price spread hA*log(yA)-hB*log(yB), we need to keep the <b>market values</b> of stocks A and B fixed, in the ratio hA:hB, which means that at the end of every bar, we need to rebalance the shares of A and B due to price changes.<br /><br />For most cointegrating pairs that I have studied, both the price spreads and the log price spreads are stationary, so it doesn't matter which one we use for our trading strategy. However, for an unusual pair where its log price spread cointegrates but price spread does not (Hat tip: Adam G. for drawing my attention to one such example), the implication is quite significant. A stationary price spread means that prices differences are mean-reverting, a stationary log price spread means that returns differences are mean-reverting. For example, if stock A typically grows 2 times as fast as B, but has been growing 2.5 times as fast recently, we can expect the growth rate differential to decrease going forward. We would still short A and long B, but we would exit this position when the growth rates of A vs B return to a 2:1 ratio, and not when the price spread of A vs B returns to a historical mean. In fact, the price spread of A vs B should continue to increase over the long term.<br /><br />This much is easy to understand. But thanks to a reader Ferenc F. who referred me to a paper by <a href="http://www.jstor.org/stable/4480875" target="_blank">Fernholz and Maguire</a>, I realize there is a simple mathematical relationship between stock A and B in order for their log prices to cointegrate.<br /><br />Let us start with a formula derived by these authors for the change in log market value P of a portfolio of 2 stocks: d(logP) = hA*d(log(yA))+hB*d(log(yB))+gamma*dt.<br /><br />The gamma in this equation is<br /><br />gamma=1/2*(hA*varA + hB*varB), where varA is the variance of stock A <u>minus</u> the variance of the portfolio market value, and ditto for varB.<br /><br />Note that this formula holds for a portfolio of any two stocks, not just when they are cointegrating. But if they are in fact cointegrating, and if hA and hB are the weights which create the stationary portfolio P, we know that d(logP) cannot have a non-zero long term drift term represented by gamma*dt. So gamma must be zero. Now in order for gamma to be zero, the <b>covariance </b>of the two stocks must be positive (no surprise here) and equal to the <b>average of the variances</b> of the two stocks. I invite the reader to verify this conclusion by expressing the variance of the portfolio market value in terms of the variances of the individual stocks and their covariance, and also to extend it to a portfolio with N stocks. This cointegration test for log prices is certainly simpler than the usual CADF or Johansen tests! (The price to pay for this simplicity? We must assume normal distributions of returns.)<br /><br /><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.796875px;">===</span><br /><br style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.796875px;" /><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.796875px;">My online Quantitative Momentum Strategies workshop will be offered on December 2-4. Please visit </span><a href="http://epchan.com/my-workshops" style="background-color: white; color: #999999; font-family: Georgia, serif; font-size: 13px; line-height: 20.796875px; text-decoration: none;" target="_blank">epchan.com/</a><a href="http://epchan.com/my-workshops" style="background-color: white; color: #999999; font-family: Georgia, serif; font-size: 13px; line-height: 20.796875px; text-decoration: none;" target="_blank">my-workshops </a><span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20.796875px;">for registration details.</span>http://epchan.blogspot.com/2013/11/cointegration-trading-with-log-prices.htmlnoreply@blogger.com (Ernie Chan)45tag:blogger.com,1999:blog-35364652.post-5933365541814525630Thu, 24 Oct 2013 20:07:00 +00002013-11-14T10:56:50.835-05:00StrategiesHow Useful is Order Flow and VPIN?Can short-term price movement be predicted? (I am speaking of seconds or minutes here.) This is a question not only relevant to high frequency traders, but to every long-term investor as well. Even if one plans to buy and hold a stock for years, nobody likes to suffer short-term negative P&L immediately after entry into position.<br /><br />One short-term prediction method that has long found favor with academic researchers and traders alike is order flow. Order flow is just signed transaction volume: if a transaction of 100 shares is classified as a "buy", the order flow is +100; if it is classified as a "sell", the order flow is -100. This might strike some as rather strange: every transaction has a buyer and seller, so what does it mean by a "buy" or a "sell"? Well, the "buyer" is defined as the one who is the "aggressor", i.e. one that is using a market order to buy at the ask price. (And vice versa for the seller, whom I will henceforth omit in this discussion.) The intuitive reason why a series of large "buy" market orders are predictive of short-term price increase is that if someone is so eager to go long, s/he is likely to know something about the market that others don't (either due to superior fundamental knowledge or technical model), so we better join her/him! Such superior traders are often called "informed traders", and their order flow is often called "toxic flow". Toxic, that is, to the uninformed market maker.<br /><br />In theory, if one has a tick data feed, one can tell whether an execution is a "buy" or "sell" by comparing the trade price with the bid and ask price: if the trade price is equal to the ask, it is a "buy". This is called the "Quote Rule". But in practice, there is a hitch. If the bid and ask prices change quickly, a buy market order may end up buying at the bid price if the market has fortuitously moved lower since the order was sent. Besides, perhaps 1/3 of trading in the US equities markets take place in dark pools or via hidden orders, so the quotes are simply invisible and order flow non-computable. So this classification scheme is not foolproof. Therefore, a number of researchers (see "<a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1695596" target="_blank">Flow Toxicity and Volatility in a High Frequency World</a>" by Easley, et. al.) proposed an alternative, "easier", method to compute order flow. Instead of checking the trade price of each tick, they just need the "open" and "close" trade prices of a bar, preferably a volume bar, and assign a fraction of the volume in that bar to "buy" or "sell" depending on whether the close price is higher or lower than the open price. (The assignment formula is based on the cumulative probability density of a Gaussian distribution, which incidentally models price changes of volume bars, but not time bars, pretty well.) The absolute difference between buy and sell volume expressed as a fraction of the total volume is called "VPIN" by the authors, or <i>Volume-Synchronized Probability of Informed Trading</i>. The higher VPIN is, the more likely we will experience short-term momentum due to informed trading.<br /><br />Theory and intuition aside, how well does order flow work in practice as a short-term predictor in various markets? And how predictive is VPIN as compared to the old Quote Rule? In my experience, while this indicator is predictive of price change, the change is often too small to overcome transaction costs including the bid-ask spread. And more disturbingly, in those markets where both Quote Rule and VPIN should work (e.g. futures markets), VPIN has so far underperformed Quote Rule, despite (?) it being patented and highly touted. I have informally polled other investment professionals on their experience, and the answer usually come back indifferent as well.<br /><br />Do you have live experience with VPIN? Or more generally, do you find strategies built using volume bars superior to those using time bars? If so, please leave us your comments!<br /><br />===<br /><br />My online Quantitative Momentum Strategies workshop will be offered in December. Please visit <a href="http://epchan.com/my-workshops" target="_blank">epchan.com/</a><a href="http://epchan.com/my-workshops" target="_blank">my-workshops </a>for registration details.<br /><br /><br />http://epchan.blogspot.com/2013/10/how-useful-is-order-flow-and-vpin.htmlnoreply@blogger.com (Ernie Chan)26tag:blogger.com,1999:blog-35364652.post-2820464674684285744Tue, 20 Aug 2013 13:08:00 +00002013-08-20T09:08:49.279-04:00Guest Post: A qualitative review of VIX F&O pricing and hedging modelsBy Azouz Gmach<br /><br /><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">VIX Futures & Options are one of the most actively traded index derivatives series on the Chicago Board Options Exchange (CBOE). These derivatives are written on S&P 500 volatility index and their popularity has made volatility a widely accepted asset class for </span><span lang="FR"><a href="http://www.quantshare.com/item-908-capital-allocation-based-on-the-expected-market-volatility-vix"><span lang="EN-US" style="color: #1155cc; font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">trading, diversifying and hedging instrument</span></a></span><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";"> since their launch. VIX Futures started trading on March 26<sup>th</sup>, 2004 on CFE (CBOE Future Exchange) and VIX Options were introduced on Feb 24<sup>th</sup>, 2006.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">VIX Futures & Options</span></b><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">VIX (Volatility Index) or the ‘Fear Index’ is based on the S&P 500 options volatility. Spot VIX can be defined as square root of 30 day variance swap of S&P 500 index (SPX) or in simple terms it is the 30-day average implied volatility of S&P 500 index options. The VIX F&O are based on this spot VIX and is similar to the equity indexes in general modus operandi. But structurally they have far more differences than similarities. While, in case of equity indices (for example SPX), the index is a weighted average of the components, in case of the VIX it is sum of squares of the components. This non-linear relationship makes the spot VIX non-tradable but at the same time the derivatives of spot VIX are tradable. This can be better understood with the analogy of Interest Rate Derivatives. The derivatives based on the interest rates are traded worldwide but the underlying asset: interest rate itself cannot be traded.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">The different relation between the VIX derivatives and the underlying VIX makes it unique in the sense that the overall behavior of the instruments and their pricing is quite different from the equity index derivatives. This also makes the pricing of VIX F&O a complicated process. A proper statistical approach incorporating the various aspects like the strength of trend, mean reversion and volatility etc. is needed for modeling the pricing and behavior of VIX derivatives.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Research on Pricing Models</span></b><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">There has been a lot of research in deriving models for the VIX F&O pricing based on different approaches. These models have their own merits and demerits and it becomes a tough decision to decide on the most optimum model. In this regards, I find the work of <i>Mr. Qunfang Bao</i> titled</span><span lang="FR"><a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2257099"><span style="color: black; font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman"; text-decoration: none; text-underline: none;"> </span></a><a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2257099"><i><span lang="EN-US" style="color: #1155cc; font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">‘Mean-Reverting Logarithmic Modeling of VIX’</span></i></a></span><i><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";"> </span></i><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">quite interesting. In his research, Bao not only revisits the existing models and work by other prominent researchers but also comes out with suggestive models after a careful observation of the limitations of the already proposed models. The basic thesis of Bao’s work involves mean-reverting logarithmic dynamics as an essential aspect of Spot VIX.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">VIX F&O contracts don’t necessarily track the underlying in the same way in which equity futures track their indices. VIX Futures have a dynamic relationship with the VIX index and do not exactly follow its index. This correlation is weaker and evolves over time. Close to expiration, the correlation improves and the futures might move in sync with the index. On the other hand VIX Options are more related to the futures and can be priced off the VIX futures in a much better way than the VIX index itself.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Pricing Models</span></b><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">As a volatility index, VIX shares the properties of mean reversion, large upward jumps & stochastic volatility (<i>aka </i>stochastic vol-of-vol). A good model is expected to take into consideration, most of these factors.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">There are roughly two categories of approaches for VIX modeling. One is the Consistent approach and the other being Standalone approach.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";"> I. Consistent Approach: - </span></b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">This is the pure diffusion model wherein the inherent relationship between S&P 500 & VIX is used in deriving the expression for spot VIX which by definition is square root of forward realized variance of SPX.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";"> II. Standalone Approach:</span></b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";"> - In this approach, the VIX dynamics are directly specified and thus the VIX derivatives can be priced in a much simpler way. This approach only focuses on pricing derivatives written on VIX index without considering SPX option.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Bao in his paper mentions that the standalone approach is comparatively better and simpler than the consistent approach.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">MRLR model</span></b><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">The most widely proposed model under the standalone approach is MRLR (Mean Reverting Logarithmic Model) model which assumes that the spot VIX follows a Geometric Brownian motion process. The MRLR model fits well for VIX Future pricing but appears to be unsuited for the VIX Options pricing because of the fact that this model generates no skew for VIX option. In contrast, this model is a good model for VIX futures.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">MRLRJ model</span></b><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Since the MRLR model is unable to produce implied volatility skew for VIX options, Bao further tries to modify the MRLR model by adding jump into the mean reverting logarithmic dynamics obtaining the Mean Reverting Logarithmic Jump Model (MRLRJ). By adding upward jump into spot VIX, this model is able to capture the positive skew observed in VIX options market.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">MRLRSV model</span></b><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Another way in which the implied volatility skew can be produced for VIX Options is by including stochastic volatility into the spot VIX dynamics. This model of Mean Reverting Logarithmic model with stochastic volatility (MRLRSV) is based on the aforesaid process of skew appropriation.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Both, MRLRJ and MRLRSV models perform equally well in appropriating positive skew observed in case of VIX options.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><b><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">MRLRSVJ model</span></b><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Bao further combines the MRLRJ and MRLRSV models together to form MRLRSVJ model. He mentions that this combined model becomes somewhat complicated and in return adds little value to the MRLRJ or MRLRSV models. Also extra parameters are needed to be estimated in case of MRLRSVJ model.</span><o:p></o:p></div><div class="Normal1" style="text-align: justify; text-justify: inter-ideograph;"><br /></div><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-bidi-language: AR-SA; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: FR;">MRLRJ & MRLRSV models serve better than the other models that have been proposed for pricing the VIX F&O. Bao in his paper, additionally derives and calibrates the mathematical expressions for the models he proposes and derives the hedging strategies based on these models as well. Quantifying the Volatility skew has been an active area of interest for researchers and this research paper addresses the same in a very scientific way, keeping in view the convexity adjustments, future correlation and numerical analysis of the models etc. While further validation and back testing of the models may be required, but Bao’s work definitely answers a lot of anomalous features of the VIX and its derivatives.</span><br /><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-bidi-language: AR-SA; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: FR;"><br /></span><span style="font-family: Times New Roman, serif;"><span style="line-height: 18px;">---</span></span><br /><span style="font-family: inherit;"><span style="color: #222222;">Azouz Gmach works for </span><span lang="FR"><a href="http://www.quantshare.com/"><span lang="EN-US">QuantShare</span></a></span>, a technical/fundamental analysis software.</span><br /><span style="font-family: inherit;"><br /></span><span style="font-family: inherit;">===</span><br /><span style="font-family: inherit;">My online Mean Reversion Strategies workshop will be offered in September. Please visit <a href="http://epchan.com/my-workshops" target="_blank">epchan.com/</a></span><a href="http://epchan.com/my-workshops" target="_blank">my-workshops </a>for registration details.<br /><br />Also, I will be teaching a new course <a href="http://www.globalmarkets-training.co.uk/mft.html" target="_blank">Millisecond Frequency Trading (MFT)</a> in London this October.<br /><br />-Ernie<br /><div class="Normal1"><o:p></o:p></div><div class="Normal1"><o:p></o:p></div><span style="font-family: "Times New Roman","serif"; font-size: 12.0pt; line-height: 115%; mso-ansi-language: EN-US; mso-bidi-font-size: 11.0pt; mso-bidi-language: AR-SA; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: FR;"><br /></span>http://epchan.blogspot.com/2013/08/guest-post-qualitative-review-of-vix-f.htmlnoreply@blogger.com (Ernie Chan)20tag:blogger.com,1999:blog-35364652.post-1191810480197060939Tue, 16 Jul 2013 20:52:00 +00002013-11-14T10:57:23.838-05:00StrategiesMomentum Crash and Recovery<div class="separator" style="clear: both; text-align: left;">In my <a href="http://www.amazon.com/dp/1118460146/ref=as_li_qf_sp_asin_til?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=1118460146&adid=08VMM7B00KMRE1S0CTQ2&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" target="_blank">book</a> I devoted considerable attention to the phenomenon of "<a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1914673" target="_blank">Momentum Crashes</a>" that professor Kent Daniel discovered. This refers to the fact that momentum strategies generally work very poorly in the immediate aftermath of a financial crisis. This phenomenon apparently spans many asset classes, and has been around since the Great Depression. Sometimes it lasted multiple decades, and at other times these strategies recovered during the lifetime of a momentum trader. So how have momentum strategies fared after the 2008 financial crisis, and have they recovered?</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">First, let's look at the <a href="http://www.aftllc.com/dti.html" target="_blank">Diversified Trends Indicator</a> (formerly the S&P DTI index), which is a fairly generic trend-following strategy applied to futures. Here are the index values since inception (click to enlarge):</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-JfRmMjCwv4E/UeWsCBHi59I/AAAAAAAABL0/S950Ze4xehs/s1600/DTITR2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="197" src="http://1.bp.blogspot.com/-JfRmMjCwv4E/UeWsCBHi59I/AAAAAAAABL0/S950Ze4xehs/s320/DTITR2.jpg" width="320" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;">and here are the values for 2013:</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-w4Xu3tyDm08/UeWsGeRsE9I/AAAAAAAABL8/vWXCGSM02wI/s1600/DTITR3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="193" src="http://3.bp.blogspot.com/-w4Xu3tyDm08/UeWsGeRsE9I/AAAAAAAABL8/vWXCGSM02wI/s320/DTITR3.jpg" width="320" /></a></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"></div><div class="separator" style="clear: both; text-align: left;">After suffering relentless decline since 2009, it has finally shown positive returns YTD!</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">Now look at a momentum strategy on the soybean futures (ZS) that I have been working on. Here are the cumulative returns from 2009 to 2011 June:</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-xu58C-5g_Do/UeWsZSrSwYI/AAAAAAAABME/7ZcBRgNclDg/s1600/ZS_mom_cumret_pre201207.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="143" src="http://1.bp.blogspot.com/-xu58C-5g_Do/UeWsZSrSwYI/AAAAAAAABME/7ZcBRgNclDg/s320/ZS_mom_cumret_pre201207.jpg" width="320" /></a></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">and here the cumulative returns since then:</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-Z_GH_4z1WMA/UeWs_z7BRgI/AAAAAAAABMM/u323OozyiQo/s1600/ZS_mom_cumret_post201207.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="143" src="http://1.bp.blogspot.com/-Z_GH_4z1WMA/UeWs_z7BRgI/AAAAAAAABMM/u323OozyiQo/s320/ZS_mom_cumret_post201207.jpg" width="320" /></a></div><br />The difference is stark!<br /><br />Despite evidences that indeed momentum strategies have enjoyed a general recovery, we must play the part of skeptical financial scientists and look for alternative theories. If any reader can tell us an alternative, plausible explanation why ZS should start to display trending behavior since July 2011, but not before, please post that in the comment area. The prize for the best explanation: I will disclose in private more details about this strategy to that reader. (To claim the prize, please include the last 4 digit of your phone number in the post for identification purpose.)<br /><br />===<br />Upcoming events:<br /><ol><li><div style="text-align: left;">I will be teaching an online workshop on Momentum Strategies from July 30 - August 1. Registration info can be found <a href="http://www.epchan.com/my-workshops/" target="_blank">here</a>.</div></li><li><div style="text-align: left;">My friend Dr. Haksun Li is offering a Certificate in Quantitative Investment series of <a href="http://cqi.sg/en/blog/" target="_blank">courses</a>. </div></li></ol><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"></div><div align="left"></div>http://epchan.blogspot.com/2013/07/momentum-crash-and-recovery.htmlnoreply@blogger.com (Ernie Chan)79tag:blogger.com,1999:blog-35364652.post-9220248282547424281Sat, 25 May 2013 13:58:00 +00002013-05-25T09:58:48.494-04:00My new book on Algorithmic Trading is outA reader (Hat tip: Ken) told me that my new book <a href="http://www.amazon.com/gp/product/1118460146/ref=as_li_qf_sp_asin_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1118460146&linkCode=as2&tag=quantitativet-20" target="_blank">Algorithmic Trading: Winning Strategies and Their Rationale</a> is now available for purchase at Amazon.com. The difference with my previous book? A lot more sample strategies with an emphasis on their "rationale", and more advanced techniques. It covers stocks, futures, and FX. A big thank-you to my editors, reviewers, and you, the reader, for your on-going support.<br /><br />And when you are done with it, please post a review on Amazon whether you like it or hate it!<br /><br />Also, I am now offering a live online course on <a href="http://www.epchan.com/my-workshops/" target="_blank">Backtesting</a> in June. It covers in excruciating details the various nuances of conducting a correct backtest and the numerous pitfalls one can encounter when backtesting different types of strategies and asset classes. For syllabus and registration details, please visit my <a href="http://www.epchan.com/my-workshops/" target="_blank">website</a>.<br /><br /><br /><br />http://epchan.blogspot.com/2013/05/my-new-book-on-algorithmic-trading-is.htmlnoreply@blogger.com (Ernie Chan)245tag:blogger.com,1999:blog-35364652.post-8590048457355940240Fri, 03 May 2013 12:31:00 +00002013-05-03T14:45:42.398-04:00Book reviewsStrategiesNonlinear Trading StrategiesI have long been partial to <a href="http://epchan.blogspot.ca/2011/04/many-facets-of-linear-regression.html" target="_blank">linear strategies</a> due to their simplicity and relative immunity to overfitting. They can be used quite easily to profit from mean-reversion. However, there is a serious problem: they are quite <a href="http://epchan.blogspot.ca/2013/03/what-can-quant-traders-learn-from.html" target="_blank">fragile</a>, <em>i.e.</em> vulnerable to tail risks. As we move from mean-reverting strategies to momentum strategies, we immediately introduce a nonlinearity (stop losses), but simultaneously remove certain tail risks (except during times when markets are closed). But if we want to enjoy anti-fragility and are going to introduce nonlinearities anyway, we might as well go full-monty, and consider options strategies. (It is no surprise that Taleb was an options trader.)<br /><br />It is easy to see that options strategies are nonlinear, since options payoff curves (value of an option as function of underlying stock price) are plainly nonlinear. I personally have resisted trading them because they all seem so complicated, and I abhor complexities. But recently a reader recommended a little book to me: Jeff Augen's "<a href="http://www.amazon.com/dp/0137029039/ref=as_li_qf_sp_asin_til?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=0137029039&adid=13WZN3EAQWT423HNK9FD&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" target="_blank">Day Trading Options</a>" where the Black-Scholes equation (and indeed any equation) is mercifully absent from the entire treatise. At the same time, it is suffused with qualitative ideas. Among the juicy bits:<br /><br />1) We can find distortions in the 2D implied volatility surface (implied volatility as z-axis, expiration months as x, and strike prices as y) which may mean revert to "smoothness", hence presenting arbitrage opportunities. These distortions are present for both stock and stock index options.<br /><br />2) Options are underpriced intraday and overpriced overnight: hence it is often a good idea to buy them at the market open and sell them at market close (except on some special days! See 4 below.). In fact, there are certain days of the week where this distortion is the most drastic and thus favorable to this strategy.<br /><br />3) Certain cash instruments have unusually high kurtosis, but their corresponding option prices consistently underprice such tail risks. Thus structures such as strangles or backspreads can often be profitable without incurring any left tail risks.<br /><br />4) If there is a long weekend before expiration day (e.g. Easter weekend), the time decay of the options value over 3 days is compressed into an intraday decline on the last trading day before the weekend.<br /><br />Now, as quantitative traders, we have no need to take his word on any of these assertions. So, onward to backtesting!<br /><br />(For those who may be stymied by the lack of affordable historical intraday options data, I recommend Nanex.net.)<br /><br />===<br /><br />There are still 2 slots available in my online <a href="http://www.epchan.com/my-workshops/" target="_blank">Mean Reversion Strategies</a> workshop in May. The workshop will be conducted <em>live</em> via Adobe Connect, and is limited to a total of 4 participants. Part of the workshop will focus on how to avoid getting hurt when a pair or a portfolio of instruments stop cointegrating.http://epchan.blogspot.com/2013/05/nonlinear-trading-strategies.htmlnoreply@blogger.com (Ernie Chan)29tag:blogger.com,1999:blog-35364652.post-5669688286675515947Thu, 04 Apr 2013 15:16:00 +00002013-04-04T18:40:56.340-04:00Automated trading platformsAn Integrated Development Environment for High Frequency StrategiesI have come across many software platforms that allow traders to first specify and backtest a strategy and then, with the push of a button, turn the backtest strategy into a live trading program that can automatically submit orders to their favorite broker. (See all my articles on this topic <a href="http://epchan.blogspot.com/search/label/Automated%20trading%20platforms" target="_blank">here</a>.) I called these platforms "Integrated Development Environment" (IDE) in my <a href="http://www.amazon.com/gp/product/1118460146/ref=as_li_qf_sp_asin_il_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1118460146&linkCode=as2&tag=quantitativet-20" target="_blank">new book</a>, and they range from the familiar and retail-oriented (e.g. MetaTrader, NinjaTrader, TradeStation), to the professional but skills-demanding (e.g. ActiveQuant, Marketcetera, TradeLink), and finally to the comprehensive and industrial-strength (e.g. Deltix, Progress Apama, QuantHouse, RTD Tango). Some of these require no programming skills at all, allowing you to construct strategies by dragging-and-dropping, others use some simple scripting languages like Python, and yet others demand full-blown programming abilities in Java, C#, or C++. But which of these allow us to backtest and execute high frequency strategies?<br /><br />To state the obvious: backtesting HFstrategies is quite hard. The volume of data is one issue. But in addition, the execution details are very important to such strategies: details such as the exact exchange/venue to which we are routing our orders, the precise state of the order book that triggers our orders, the order types we are using, and finally the probability of getting filled if we use non-marketable orders. Messing up one of these details and the backtest will be far from realistic. I often tell people that it is easier to paper trade a HF strategy than to backtest one. While many of the platforms I reported above do allow backtesting using tick data, I don't know that they enable backtesting using the full order book and choice of execution venue. With this background, I am happy to report I have recently come across just such a platform called <a href="http://www.limebrokerage.com/services/marketdata/simulation" target="_blank">Lime Strategy Studio</a>. <br /><br /><strike>First, the bad news. LimeTrader is useful only to traders who trade with Lime Brokerage, as it is configured to send live orders to Lime only.</strike> [<strong>UPDATE:</strong> I have since learned that there are adapters available for 3rd party brokers.] However, if you are going to trade HF stocks and futures strategies, why not go with Lime, since they provide you with a comprehensive API, direct ultra-low latency feeds from the exchanges, and allow (nay, insist on) colocation either at the exchanges or at their data center at a reasonable fee? (Full Disclosure: I have no current business relationship with Lime, though I was a customer.) Another piece of bad news: the specification of the strategy must be in C++. <br /><br />But once you get over these two hurdles, the benefits are manifold. Every detail that you can specify for a live trading strategy can be specified for the backtest and paper trading. As I said, these details may include order type, trading venue, state of order book, and even statistics of the order book, not to mention fundamental data such as earnings, corporate actions, and other user-provided data such as news. A fill simulator is included for your non-marketable orders. As with other IDEs, once you backtested a strategy in its every detail and are satisfied with the performance metrics, you can go live (either for paper or production trading) with the push of a button. <br /><br />If any reader know of other IDEs that have similar features and useful for backtesting HF strategies, please let us know!<br /><br />===<br /><br />Speaking of HF strategies, traders often lament the ultra-high secrecy around them and the difficulty of gathering knowledge in this field. A friend (hat tip: Dave) referred me to this <a href="http://www.math.stevens.edu/~ifloresc/Research/Publications/ProjectpricevolFinalwithDragos.pdf" target="_blank">paper</a> by Prof. Dragos Bozdog <em>et. al.</em> that gives a flavor of what sort of modeling may be involved. I find it very readable and thought-provoking.<br /><br />===<br /><br />There are still 2 slots available in my online <a href="http://www.epchan.com/my-workshops/" target="_blank">Mean Reversion Strategies workshop</a> scheduled for May. <br /><br /><br /><br />http://epchan.blogspot.com/2013/04/an-integrated-development-environment.htmlnoreply@blogger.com (Ernie Chan)77tag:blogger.com,1999:blog-35364652.post-7257486152642926538Thu, 14 Mar 2013 09:36:00 +00002013-04-29T09:06:30.231-04:00Book reviewsWhat Can Quant Traders Learn from Taleb's "Antifragile"?It can seem a bit ironic that we should be discussing Nassim Taleb's best-seller "<a href="http://www.amazon.com/gp/product/1400067820/ref=as_li_tf_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1400067820&linkCode=as2&tag=quantitativet-20" target="_blank">Antifragile</a>" here, since most algorithmic trading strategies involve predictions and won't be met with approval from Taleb. Predictions, as Taleb would say, are "fragile" -- they are prone to various biases (e.g. data snooping bias) and the occasional Black Swan event will wipe out the small cumulative profits from many correct bets. Nevertheless, underneath the heap of diatribes against various luminaries ranging from Robert Merton to Paul Krugman, we can find a few gems. Let me start from the obvious to the subtle:<br /><br />1) Momentum strategies are more antifragile than mean-reversion strategies.<br /><br />Taleb didn't say that, but that's the first thought that came to my mind. As I argued in many places, mean reverting strategies have natural profit caps (exit when price has reverted to mean) but no natural stop losses (we should buy more of something if it gets cheaper), so it is very much subject to <i>left </i>tail risk, but cannot take advantage of the unexpected good fortune of the <i>right </i>tail. Very fragile indeed! On the contrary, momentum strategies have natural stop losses (exit when momentum reverses) and no natural profit caps (keep same position as long as momentum persists). Generally, very antifragile! Except: what if during a trading halt (due to the daily overnight gap, or circuit breakers), we can't exit a momentum position in time? Well, you can always buy an option to simulate a stop loss. Taleb would certainly approve of that.<br /><br />2) High frequency strategies are more antifragile than low frequency strategies.<br /><br />Taleb also didn't say that, and it has nothing to do with whether it is easier to predict short-term vs. long-term returns. Since HF strategies allow us to accumulate profits much faster than low frequency ones, we need not apply any leverage. So even when we are unlucky enough to be holding a position of the wrong sign when a Black Swan hits, the damage will be small compared to the cumulative profits. So while HF strategies do not exactly benefit from right tail risk, they are at least robust with respect to left tail risk.<br /><br />3) Parameter estimation errors and vulnerability to them should be explicitly incorporated in a backtest performance measurement.<br /><br />Suppose your trading model has a few parameters which you estimated/optimized using some historical data set. Based on these optimized parameters, you compute the Sharpe ratio of your model on this same data. No doubt this Sharpe ratio will be very good, due to the in-sample optimization. If you apply this model with those optimized the parameters on out-of-sample data, you would probably get a worse Sharpe ratio which is more predictive. But why stop at just two data sets? We can find N different data sets of the same size, calculate the optimized parameters on each of them, but compute the Sharpe ratios over the N-1 out-of-sample data sets. Finally, you can average over all these Sharpe ratios. If your trading model is fragile, you will find that this Sharpe ratio is quite low. But more important than Sharpe ratios, you should compute the maximum drawdown based on each set of parameters, and also the maximum of all these max drawdowns. If your trading model is fragile, this maximum of maximum drawdowns is likely to be quite scary.<br /><br />The scheme I described above is called cross-validation and is well-known before Taleb, though his book reminds me of its importance.<br /><br />4) Notwithstanding 3) above, a true estimate of the max drawdown is impossible because it depends on the estimate of the probability of rare events. As Taleb mentioned, even in case of a normal distribution, if the "true" standard deviation is higher than your estimate by a mere 5%, the probability of a 6-sigma event will be increased by 5 times over your estimate! So really the only way to ensure that our maximum drawdown will not exceed a certain limit is through <a href="http://epchan.blogspot.ca/2010/04/how-do-you-limit-drawdown-using-kelly.html" target="_blank">Constant Proportion Portfolio Insurance</a>: trading risky assets with Kelly-leverage in a limited liability company, putting money that you never want to lose in a FDIC-insured bank, with regular withdrawals from the LLC to the bank (but not the other way around).<br /><br />5) Correlations are impossible to estimate/predict. The only thing we can do is to short at +1 and buy at -1.<br /><br />Taleb hates Markowitz portfolio optimization, and one of the reasons is that it relies on estimates of covariances of asset returns. As he said, a pair of assets that may have -0.2 correlation over a long period can have +0.8 correlation over another long period. This is especially true in times of financial stress. I quite agree on this point: I believe that manually assigning correlations with values of +/-0.75, +/-0.5, +/-0.25, 0 to entries of the correlation matrix based on "intuition" (fundamental knowledge) can generate as good <i>out-of-sample</i> performance as any meticulously estimated numbers.The more fascinating question is whether there is indeed mean-reversion of correlations. And if so, what instruments can we use to profit from it? Perhaps this <a href="http://web-docs.stern.nyu.edu/salomon/docs/derivatives/GSAM%20-%20NYU%20conference%20042106%20-%20Correlation%20trading.pdf" target="_blank">article</a> will help.<br /><br />6) Backtest can only be used to reject a strategy, not to predict its success.<br /><br />This echoes the point made by commenter Michael Harris in a previous <a href="http://epchan.blogspot.ca/2013/01/the-pseudo-science-of-hypothesis-testing.html" target="_blank">article</a>. Since historical data will never be long enough to capture all the possible Black Swan events that can occur in the future, we can never know if a strategy will fail miserably. However, if a strategy already failed in a backtest, we can be pretty sure that it will fail again in the future.<br /><br />===<br /><br />The online "Quantitative Momentum Strategies” workshop that I mentioned in the previous <a href="http://epchan.blogspot.ca/2013/02/a-workshop-webinar-and-question.html" target="_blank">article</a> is now fully booked. Based on popular demand, I will offer a "Mean Reversion Strategies" workshop in May. Once again, it will be conducted in real-time through Skype, and the number of attendees will be similarly limited to 4. See <a href="http://www.epchan.com/my-workshops/" target="_blank">here</a> for more information.<br /><br /><br /><br /><br /><br /><br />http://epchan.blogspot.com/2013/03/what-can-quant-traders-learn-from.htmlnoreply@blogger.com (Ernie Chan)34tag:blogger.com,1999:blog-35364652.post-6471196189794970005Mon, 18 Feb 2013 20:41:00 +00002013-02-18T15:42:29.397-05:00A workshop, a webinar, and a questionThere is a workshop on the 25th of February titled "<a href="http://ieor.columbia.edu/financial-engineering-practitioners-seminar-market-turbulence-monetization-and-universality" target="_blank">Market turbulence; monetization; and universality</a>" by Mike Lipkin at Columbia University that promises to be interesting to those traders who have a physics background. Mike is a former colleague of mine at Cornell's Laboratory of Atomic and Solid State Physics, and I fondly remember the good old days when we all hunched over the theory group's computers while day-dreaming of our future. Mike has since gone on to become an options market-maker at the American Stock Exchange and an Adjunct Associate Professor at Columbia. He <a href="http://www.math.nyu.edu/faculty/avellane/PowerLaw.pdf" target="_blank">published</a> some very interesting research on the "stock pinning" phenomenon near options expirations, i.e. stock prices often converge to the nearest strike prices of their options just before expirations.<br /><br />---<br /><br />If we want to trade directly on various FX ECNs such as HotspotFX or EBS, perhaps because we want to run some <a href="http://epchan.blogspot.ca/2012/03/high-frequency-trading-in-foreign.html" target="_blank">HFT strategies</a>, we will need to be sponsored by a prime broker. However, since the Dodd-Frank act has been in full force, no prime brokers that I know of are willing to take on customers with less than $10M assets. (I often feel that the CFTC's primary goal is to prevent small players like myself from ever competing with bigger institutions. Of course, their stated goal is to "protect" us from financial harm ....) The only exception may be CitiFX TradeStream ECN. Has any reader ever traded on this market? Any reviews or comments will be most welcome.<br /><br />---<br /><br />I am now offering an online workshop "Quantitative Momentum Strategies” to a select number of traders and portfolio managers. It will be conducted in real-time through Skype, and the number of attendees will be limited to 4. See <a href="http://www.epchan.com/my-workshops/" target="_blank">here</a> for more information.http://epchan.blogspot.com/2013/02/a-workshop-webinar-and-question.htmlnoreply@blogger.com (Ernie Chan)46tag:blogger.com,1999:blog-35364652.post-3180335172467105051Sun, 03 Feb 2013 16:38:00 +00002014-02-07T09:32:28.228-05:00factor modelStrategiesA stock factor based on option volatility smirkA <a href="http://epchan.blogspot.ca/2013/01/the-pseudo-science-of-hypothesis-testing.html?showComment=1359481217691#c3157683415976459113" target="_blank">reader</a> pointed out an interesting <a href="http://www.ruf.rice.edu/~yxing/option-skew-FINAL.pdf" target="_blank">paper </a>that suggests using option volatility smirk as a factor to rank stocks. Volatility smirk is the difference between the implied volatilities of the OTM put option and the ATM call option. (Of course, there are numerous OTM and ATM put and call options. You can refer to the original paper for a precise definition.) The idea is that informed traders (<i>i.e.</i> those traders who have a superior ability in predicting the next earnings numbers for the stock) will predominately buy OTM puts when they think the future earnings reports will be bad, thus driving up the price of those puts and their corresponding implied volatilities relative to the more liquid ATM calls. If we use this volatility smirk as a factor to rank stocks, we can form a long portfolio consisting of stocks in the bottom quintile, and a short portfolio with stocks in the top quintile. If we update this long-short portfolio weekly with the latest volatility smirk numbers, it is reported that we will enjoy an annualized excess return of 9.2%.<br /><br />As a standalone factor, this 9.2% return may not seem terribly exciting, especially since transaction costs have not been accounted for. However, the beauty of factor models is that you can combine an arbitrary number of factors, and though each factor may be weak, the combined model could be highly predictive. A search of the keyword "factor" on my blog will reveal that I have talked about many different factors applicable to different asset classes in the past. For stocks in particular, there is a <a href="http://epchan.blogspot.ca/2012/01/what-worked-in-2011.html" target="_blank">short term factor</a> as simple as the previous 1-day return that worked wonders. Joel Greenblatt's famous "<a href="http://www.amazon.com/gp/product/0470624159/ref=as_li_qf_sp_asin_il_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0470624159&linkCode=as2&tag=quantitativet-20" target="_blank">Little Book that Beats the Market</a>" used 2 factors to rank stocks (return-on-capital and earnings yield) and generated an APR of 30.8%.<br /><br />The question, however, is how we should combine all these different factors. Some factor model aficionados will no doubt propose a linear regression fit, with future return as the dependent variable and all these factors as independent variables. However, my experience with this method has been unrelentingly poor: I have witnessed millions of dollars lost by various banks and funds using this method. In fact, I think the only sensible way to combine them is to simply add them together with equal weights. That is, if you have 10 factors, simply form 10 long-short portfolios each based on one factor, and combine these portfolios with equal capital. As <a href="http://www.amazon.com/dp/0374275637/ref=as_li_qf_sp_asin_til?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=0374275637&adid=1QE6Q9J63V9WFKMZWCFH&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" target="_blank">Daniel Kahneman</a> said, "Formulas that assign equal weights to all the predictors are often superior, because they are not affected by accidents of sampling".<br /><br /><br />http://epchan.blogspot.com/2013/02/a-stock-factor-based-on-option.htmlnoreply@blogger.com (Ernie Chan)36tag:blogger.com,1999:blog-35364652.post-9161010336089772883Wed, 02 Jan 2013 14:15:00 +00002013-01-02T11:05:02.639-05:00The Pseudo-science of Hypothesis TestingBacktesting trading strategies necessarily involves a very limited amount of historical data. For example, I seldom test strategies with data older than 2007. Gathering longer history may not improve predictive accuracy since the market structure may have changed substantially. Given such scant data, it is reasonable to question whether the good backtest results (e.g. a high annualized return R) we may have obtained is just due to luck. Many academic researchers try to address this issue by running their published strategies through standard statistical hypothesis testing.<br /><div><br /></div><div>You know the drill: the researchers first come up with a supposedly excellent strategy. In a display of false modesty, they then suggest that perhaps a null hypothesis can produce the same good return R. The null hypothesis may be constructed by running the original strategy through some random simulated historical data, or by randomizing the trade entry dates. The researchers then proceed to show that such random constructions are highly unlikely to generate a return equal to or better than R. Thus the null hypothesis is rejected, and thereby impressing you that the strategy is somehow sound.</div><div><br /></div><div>As statistical practitioners in fields outside of finance will tell you, this whole procedure is quite meaningless and often misleading.</div><div><br /></div><div>The probabilistic syllogism of hypothesis testing has the same structure as the following simple example (devised by Jeff Gill in his paper "The Insignificance of Null Hypothesis Significance Testing"):</div><div><br /></div><div>1) If a person is an American then it is highly unlikely she is a member of Congress.</div><div>2) The person is a member of Congress.</div><div>3) Therefore it is highly unlikely she is an American.</div><div><br /></div><div>The absurdity of hypothesis testing should be clear. In mathematical terms, the probability we are really interested in is the conditional probability that the null hypothesis is true given an observed high return R: P(H<span style="font-size: xx-small;">0</span>|R). But instead, the hypothesis test merely gives us the conditional probability of a return R given that the null hypothesis is true: P(R|H<span style="font-size: xx-small;">0</span>). These two conditional probabilities are seldom equal.</div><div><br /></div><div>But even if we can somehow compute P(H<span style="font-size: xx-small;">0</span>|R), it is still of very little use, since there are an infinite number of potential H<span style="font-size: xx-small;">0</span>. Just because you have knocked down one particular straw man doesn't say much about your original strategy.</div><div><br /></div><div>If hypothesis testing is both meaningless and misleading, why do financial researchers continue to peddle it? Mainly because this is <i>de rigueur</i> to get published. But it does serve one useful purpose for our own private trading research. Even though a rejection of the null hypothesis in no way shows that the strategy is sound, a failure to reject the null hypothesis will be far more interesting.</div><div><br /></div><div>(For other references on criticism of hypothesis testing, read Nate Silver's bestseller "<a href="http://www.amazon.com/dp/159420411X/ref=as_li_qf_sp_asin_til?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=159420411X&adid=1T2D70JWEBA1DVTW0MQD&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" target="_blank">The Signal and The Noise</a>". Silver is of course the statistician who correctly predicted the winner of all 50 states + D.C. in the 2012 US presidential election. The book is highly relevant to anyone who makes a living predicting the future. In particular, it tells the story of one Bob Voulgaris who makes $1-4M per annum betting on NBA outcomes. It makes me wonder whether I should quit making bets on financial markets and move on to sports.)</div>http://epchan.blogspot.com/2013/01/the-pseudo-science-of-hypothesis-testing.htmlnoreply@blogger.com (Ernie Chan)88tag:blogger.com,1999:blog-35364652.post-563761588722470222Thu, 29 Nov 2012 16:48:00 +00002012-11-29T13:23:45.665-05:00The Importance of 2 (as Sharpe Ratio)A reader <a href="http://epchan.blogspot.com/2010/04/how-do-you-limit-drawdown-using-kelly.html?showComment=1352433899601#c2233375091631222369" target="_blank">ezbentley</a> recently pointed out a little-noticed fact in the <a href="http://www.edwardothorp.com/sitebuildercontent/sitebuilderfiles/KellyCriterion2007.pdf" target="_blank">derivation</a> of Kelly's formula: if we apply the optimal Kelly leverage, then the standard deviation of the annualized <i>compounded </i>growth rate of your equity is none other than the Sharpe ratio (Sdev=S). This fact is of mild interest in itself, but its implication has relevance to another interesting fact of behavioral finance, so I will reproduce our discussions here.<br /><br />Suppose our strategy has an annualized Sharpe ratio of 2. According to the above result, Sdev=2 as well. This may startle some of us: a standard deviation of 200% of our compounded growth rate g - wouldn't ruin be very likely? But check out g itself: g=S^2/2, so g=2 when S=2, which means that g itself is exactly 200%. A Sdev of 200% here means that if the growth rate drops one standard deviation below its mean, we will still manage not to lose money for the year. Another way to put this is that there is a 84.1% chance that our annual return will be greater than 0, based on the Gaussian distribution.<br /><br />It gets better if S goes above 2. For example, at S=3, g=4.5, but Sdev is just 3. So you can see that as S goes above 2, a 1 standard deviation fluctuation of g below the mean will still get you a positive number: profitable for the year.<br /><br />This is a very interesting result: this means that S=2 is really an important threshold in more ways that I realized. From behavioral finance experiments, we already know that humans demands $2 profits for $1 risk. Given the universal desire of portfolio managers not to lose money on the year, it turns out that the demand of a Sharpe ratio of at least 2 is quite rational!<br /><br />===<br /><br />Now, time for a couple of public service announcements:<br /><br />1) Those who are looking for a way to connect Matlab to Interactive Brokers should check out <a href="http://undocumentedmatlab.com/ib-matlab/">undocumentedmatlab.com</a>. The creator of this product has an accompanying book, and the documentation for the product is excellent.<br /><br />2) <a href="http://www.nag.com/numeric/MB/manual64_23_1/pdf/GENINT/product.html" target="_blank">NAG</a> sells high performance Matlab toolboxes for those who prefer alternatives to the native ones.<br /><br />3) <a href="https://twitter.com/FIXGlobalOnline" target="_blank">Here</a> is the Twitter feed for FIXGlobal Online, the magazine from the creator of the FIX Protocol, an order submission standard. Interesting breaking news from the global finance scene.http://epchan.blogspot.com/2012/11/the-importance-of-2-as-sharpe-ratio.htmlnoreply@blogger.com (Ernie Chan)80tag:blogger.com,1999:blog-35364652.post-7972226156398551363Thu, 25 Oct 2012 20:31:00 +00002013-04-29T09:07:27.440-04:00StrategiesA leveraged ETFs strategyIn a <a href="http://epchan.blogspot.ca/2009/07/are-triple-leveraged-etfs-suitable-for.html" target="_blank">post</a> some years ago, I argued that leveraged ETF (especially the triple leveraged ones) are unsuitable for long-term holdings. Today, I want to present research that suggests leveraged ETF can be very <i>suitable </i>for <i>short</i>-term trading.<br /><br />The research in question was just <a href="http://ssrn.com/abstract=2161057" target="_blank">published</a> by Prof. Pauline Shum and her collaborators at York University. Here is the simplest version of the strategy: if a stock market index has experienced a return >= 2% since the previous day's close up to the current time at 2:15pm ET, then buy this index (via its futures, ETFs, or stock components) right away, and exit at the close with a market-on-close order. Vice versa if the return is <= -2%. The annualized average return from June 2006 to July 2011 was found to be higher than 100%.<br /><br />Now this strategy is actually quite well-known among institutional traders, although this is the first time I see the backtest results published. The reason why it works is also quite well-known: it has to do with the fact that every leveraged ETF need to rebalance at the market close in order to keep its leverage constant (at x2 or x3, depending on the fund). If the market index goes up, the fund needs to buy the component stocks; otherwise, it needs to sell stocks. If there is major market movement (with absolute return >= 2%) since the previous close, then the amount of stocks that need to be bought or sold will be correspondingly larger, resulting in momentum in all those stocks near the close. This strategy aims to front-run this rebalancing to take advantage of the anticipated momentum.<br /><br />It has been estimated that if the market moves by 1%, the rebalancing could account for up to 16.8% of the market-on-close volume, so the induced momentum can be substantial. Now who is paying for this profits for those momentum traders? Why, the buy-and-hold investors, of course. This loss for the ETFs shows up as their tracking errors, resulting in a cost of as much as 5% per annum for the buy-and-hold investors. Yet another reason we should not be one of those investors!<br /><br />As Prof. Shum pointed out, if you trade this strategy live today, you will likely get a lower return, because of all those momentum traders who drove up the price way before the close. However, there may be an ameliorating factor at work here: this momentum is proportional to the NAVof the ETFs. As their NAV goes up with time (either due to additional subscriptions or positive market returns), the returns of this strategy should also increase.<br /><br />===<br />Now for some public service announcements:<br /><br />1) A company called Level 3 Data Corp sells proprietary data indicating buying and selling pressure on stocks. Their internal backtests show that adding these data to some common stock trading strategies essentially double their returns. An explanatory <a href="http://www.youtube.com/watch?v=me1g3PU7nzI" target="_blank">video</a> is available, and I heard they are offering 3-month free trials.<br /><br />2) The London Systematic Traders (LST) Club has asked me to say a few words about their new initiative to build a London centric collaborative community of traders, developers and researchers.<br /><br />LST aims to be at the intersection of traders, developers and quants with a strong emphasis community building and on knowledge exchange, providing a trading networks with a very specific focus on systematic, algorithmic (i.e. automated) or quantitative trading.<br /><br />Membership is free and open to everybody with an interest in the above topics.<br /><br /><a href="http://www.meetup.com/London-Systematic-Traders/">http://www.meetup.com/London-Systematic-Traders/</a><br /><br />On Friday, Nov 23, I expect to be hosting a Q&A session with members of the LST (see 2 above) at the Apex Hotel in London. All are welcome. Please visit their website for details.<br /><br />3) I will be conducting my <a href="http://www.technicalanalyst.co.uk/training/backtestingEC.htm" target="_blank">Backtesting</a> and <a href="http://www.technicalanalyst.co.uk/training/statarb.htm" target="_blank">Statistical Arbitrage</a> workshops in London, Nov 19-22, and look forward to seeing some of our readers there!http://epchan.blogspot.com/2012/10/a-leveraged-etfs-strategy.htmlnoreply@blogger.com (Ernie Chan)34tag:blogger.com,1999:blog-35364652.post-4949175638420541423Mon, 08 Oct 2012 15:04:00 +00002013-04-29T09:07:56.510-04:00Automated trading platformsStrategiesOrder flow as a predictor of returnOrder flow is signed transaction volume: if an order is executed at the ask price, the incremental order flow is +(order size); if executed at the bid price, it is -(order size). In certain markets where traders can only buy and sell from market makers but not from each other, a positive order flow means that traders are net buyers of a security. But even in markets where everyone can place and fill orders on a common order book, a positive order flow indicates that informed traders (those willing to aggressively get into a position) are eagerly acquiring a security.<br /><br />The neat thing about order flow is that it has proven to be a good momentum indicator. That is to say, a positive flow predicts a positive future return. This might seem trivially obvious, but you have remember that generally speaking, a positive past return by no means predicts a positive future return. That FX order flow possesses this predictive power was shown by Evans and Lyons in a series of <a href="http://www.bis.org/publ/bppdf/bispap02j.pdf" target="_blank">papers</a>, but this indicator is useful in many other markets, and at many different time scales. For example, in a <a href="http://www.people.hbs.edu/estafford/Papers/AFS.pdf" target="_blank">paper</a> by Coval and Stafford, it was shown that if you can tease out the order flow of a stock due to mutual funds' trading alone, you can also predict its future return up to, say, a quarter. This paper not only shows that order flow is predictive, but that sometimes a specific kind of order flow (in this case, that of mutual funds only) is sometimes more predictive than general order flow. In many cases, traders find that by counting only order flow due to institutional traders, or order flow due to large orders, they can better predict future returns. (No wonder institutional traders are trying their darnedest to break up their orders into small chunks, or to trade in dark pools!) I recently also heard that order flow into sector ETFs can be predictive of that sector's return. If any reader has read papers or has experience with this type of sector rotation model, please leave a comment!<br /><br />Despite the proven usefulness of order flow, not too many retail traders utilize it. The reason is simple: it can be hard to measure. In FX in particular, many markets do not report trade information, or they report with a sufficient delay such that the information has no predictive utility. Even for markets that report instantaneous trade information, you would need a good piece of software to capture every bid, ask, trade, and trade size, and store them in an array, in order to compute order flow, an operation that most retail trading software cannot accomplish. However, this barrier to entry may just mean that there are still decent alpha to be extracted from this indicator.<br /><br />Now, a bunch of public service announcements ...<br /><br />===<br /><br />A new algorithmic trading platform called Rizm designed for retail traders is now available. You can sign up for their beta trial <a href="http://equametrics.com/" target="_blank">here</a>.<br /><br />===<br /><br /><a href="https://app.quantopian.com/posts/ernie-chans-gold-vs-gold-miners-stat-arb" target="_blank">Quantopian</a> has created an event-driven version of my <a href="http://epchan.blogspot.ca/2011/06/when-cointegration-of-pair-breaks-down.html" target="_blank">gold/gold-miners arb strategy</a> with source codes and analysis available. I find that the performance metrics clear and useful: better than the output from my own backtest programs! (Quantopian is a platform where you can share backtest results and codes with other traders.)<br /><br />===<br /><br /><a href="http://arb-maker.com/" target="_blank">Arbmaker</a> is a platform for pair traders, and it incorporates software for cointegration tests, has integrated data feed from many vendors, and allows automated order submission to Interactive Brokers. Neural networks and Kalman filter are also included.<br /><br />===<br /><br />Finally, I will be giving a talk titled "Backtesting and Its Pitfalls" at the World MoneyShow at the Metro Toronto Convention Centre on Saturday, October 20. Interested readers can register <a href="https://secure.moneyshow.com/msc/toms/registration.asp?sid=TOMS12&scode=029492" target="_blank">here</a>.<br /><br />http://epchan.blogspot.com/2012/10/order-flow-as-predictor-of-return.htmlnoreply@blogger.com (Ernie Chan)20tag:blogger.com,1999:blog-35364652.post-6370834161133230242Sat, 04 Aug 2012 20:56:00 +00002012-08-04T17:05:31.235-04:00An options workshop and other miscellanyI confess I have always found it hard to trade options. This is despite having read some of the "bibles" of options trading, including Lawrence McMillan's <a href="http://www.amazon.com/dp/0735201978?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=0735201978&adid=1J02QNSXDTHFC1DVSZ6C&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" rel="nofollow" target="_blank">Options as a Strategic Investment</a> and Euan Sinclair's <a href="http://www.amazon.com/dp/0735201978?tag=quantitativet-20&camp=14573&creative=327641&linkCode=as1&creativeASIN=0735201978&adid=1J02QNSXDTHFC1DVSZ6C&&ref-refURL=http%3A%2F%2Fepchan.blogspot.ca%2F" rel="nofollow" target="_blank">Option Trading: Pricing and Volatility Strategies and Techniques</a>. Partly that is because I prefer simple strategies, and options strategies are rarely simple. Partly that is because I was brought up on stocks, but stock options are depressingly illiquid. Most successful options traders that I know of prefer to trade index options instead, an area that I unfortunately have no intuition at all. Papers and books written by options professionals on this topic tend to be dense with equations, and worse, they seldom focus on the practical side of trading.<br /><br />That's why I am pleased to learn that Larry Connors, whose books I enjoy due to their simplicity of exposition, is presenting his first ever quantitative index options trading seminars. Interested traders can register for his free preview webinars on August 9 and 15 <a href="http://presentations.tradingmarkets.com/1580745/connors-research-quantified-options-trading-strategy-course?utm_source=Chan" rel="nofollow" target="_blank">here</a>, or a pre-recorded preview <a href="http://presentations.tradingmarkets.com/1580804/1st-quantified-options-trading-strategies-summit-preview-video?utm_source=Chan" target="_blank">here</a>.<br /><br />===<br /><br />Speaking of seminars, readers in Asia may be interested to know that my own workshops on<a href="http://www.technicalanalyst.co.uk/training/backtestingEC.htm" target="_blank"> Backtesting</a> and <a href="http://www.technicalanalyst.co.uk/training/statarb.htm" target="_blank">Statistical Arbitrage</a> will be held in Hong Kong on October 2-5. The same workshops will be held in London on November 19-22.<br /><br />(I enjoy giving those workshops very much, because many of the participants are institutional traders whose knowledge and points of view are very much at the cutting edge. Past participants include quants and traders from, in no particular order, Goldman Sachs, Morgan Stanley, Royal Bank of Scotland, Bank of America, UBS, Societe Generale, Deutsche Bank, BNP Paribas, JP Morgan, Barclays, Citigroup, Blackrock, and various other Asian and European hedge funds, energy companies, banks, and asset managers. I humbly submit that the in-class discussions are sometimes more interesting than my prepared materials.)<br /><br />===<br /><br />I <a href="http://epchan.blogspot.ca/2012/03/high-frequency-trading-in-foreign.html" target="_blank">wrote</a> some time ago about those FX brokers or ECNs where algo-traders can colocate their trading programs to lower latency for a reasonable price. There are also similar options for futures algo-traders. For e.g. <a href="http://ticktotrade.com/" target="_blank">Optimus Trading Group</a> provides a market data service called Rithmic which is colocated at the major futures exchanges, and traders can colocate with Rithmic to reduce latency. Of course, traders can also directly colocate at the new <a href="http://www.cmegroup.com/globex/files/CME-Co-Location-Services-Overview.pdf" target="_blank">CME data center in Aurora, IL</a>. I suspect, though, that the cost of the latter option will be considerable.<br /><br />====<br /><br />Finally, as a quant trader, I nevertheless read macroeconomic analyses occasionally, if only to figure out why some of my strategies suddenly start to fail. One website that provides interesting analysis of the energy markets is oilprice.com. In particularly, this <a href="http://oilprice.com/Interviews/Global-Trade-Likely-to-Collapse-if-Romney-Wins-Interview-with-Mike-Shedlock.html" rel="nofollow" target="_blank">interview</a> with economic commentator Mike Shedlock is unusually detailed and thoughtful.http://epchan.blogspot.com/2012/08/an-options-workshop-and-other-miscellany.htmlnoreply@blogger.com (Ernie Chan)39tag:blogger.com,1999:blog-35364652.post-7328811305218889285Tue, 10 Jul 2012 18:53:00 +00002013-04-29T09:08:09.695-04:00StrategiesExtracting roll returns from futuresFutures returns consist of two components: the returns of the spot price and the "roll returns". This is kind of obvious if you think about it: suppose the spot price remains constant in time (and therefore has zero return). Futures with different maturities will still have different prices at any point in time, and yet they must all converge to the same spot price at expirations, which means they must have non-zero returns during their lifetimes. <span style="background-color: white;">This roll return is in action every day, not just during the rollover to the next nearest contract. For some futures, the magnitude of this roll return can be very large: it averages about -50% annualized for VX, the volatility futures. Wouldn't it be nice if we can somehow extract this return?</span><br /><br />In theory, extracting this return should be easy: if a future is in backwardation (positive roll return), just buy the future and short the underlying asset, and vice versa if it is in contango. Unfortunately, shorting, or even buying, an underlying asset is not easy. Except for precious metals, most commodity ETFs that hold "commodities" actually hold only their futures (e.g. USO, UNG, ...), so they are of no help at all in this arbitrage strategy. Meanwhile, it is also a bit inconvenient for us to go out and buy a few oil tankers ourselves.<br /><br />But in arbitrage trading, we often do not need an exact arbitrage relationship: a statistical likely relationship is good enough. So instead of using a commodity ETF as a hedge against the future, we can use a commodity-producer ETF. For example, instead of using USO as a hedge, we can use XLE, the energy sector ETF that holds energy producing companies. These ETFs should have a higher degree of correlation with the spot price than do the futures, and therefore very suitable as hedges. In cases where the futures do not track commodities (as in the case of VX), however, we have to look harder to find the proper hedge.<br /><br />Which brings me to this fresh-off-the-press <a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2094510" target="_blank">paper by David Simon and Jim Campasano</a>. (Hat tip: Simon T.) This paper suggests a trading strategy that tries to extract the very juicy roll returns of VX. The hedge they suggest is -- you guessed it! -- the ES future. In a nutshell:<span style="background-color: white;"> if VX is in contango (which is most of the time), just short both VX and ES, and vice versa if VX is in backwardation. </span><br /><span style="background-color: white;"><br /></span><br /><span style="background-color: white;">Why does ES work as a good hedge? Of course, its very negative correlation with VX is the major factor. But one should not overlook the fact that ES also has a very small roll return (about +1.5% annualized). In other words, if you want to find a future to act as a hedge, look for ones that have an insignificant roll return. (Of course, if we can find a future that has high correlation with your original future but which has a high roll return of the opposite sign, that would be ideal. But we are seldom that lucky.)</span><br /><span style="background-color: white;"><br /></span><br /><span style="background-color: white;">P.S. The reader Simon who referred me to this paper also drew my attention to an apparent contradiction between its conclusion and my earlier blog post: <a href="http://epchan.blogspot.ca/2011/01/shorting-vix-calendar-spread.html" target="_blank">Shorting the VIX Calendar Spread</a>. This paper says that it is profitable to short VX </span><span style="background-color: white;">when it is in contango </span><span style="background-color: white;">and hedge with short ES, while I said it may not be profitable to short the front contract of VX when it is in contango and hedge with long back contract of VX. Both statements are true: hedging with the back contract of VX brings very little benefit because both the front and back contracts are suffering from very similar roll returns, so there is little return left when you take opposite positions in them!</span><br /><br />http://epchan.blogspot.com/2012/07/extracting-roll-returns-from-futures.htmlnoreply@blogger.com (Ernie Chan)28