Friday, November 27, 2015

Predicting volatility

Predicting volatility is a very old topic. Every finance student has been taught to use the GARCH model for that. But like most things we learned in school, we don't necessarily expect them to be useful in practice, or to work well out-of-sample. (When was the last time you need to use calculus in your job?) But out of curiosity, I did a quick investigation of its power on predicting the volatility of SPY daily close-to-close returns. I estimated the parameters of a GARCH model on training data from December 21, 2005 to December 5, 2011 using Matlab's Econometric toolbox, and tested how often the sign of the predicted 1-day change in volatility agree with reality on the test set from December 6, 2011 to November 25, 2015. (One-day change in realized volatility is defined as the change in the absolute value of the 1-day return.) A pleasant surprise: the agreement is 58% of the days.

If this were the accuracy for predicting the sign of the SPY return itself, we should prepare to retire in luxury. Volatility is easier to predict than signed returns, as every finance student has also been taught. But what good is a good volatility prediction? Would that be useful to options traders, who can trade implied volatilities instead of directional returns? The answer is yes, realized volatility prediction is useful for implied volatility prediction, but not in the way you would expect.

If GARCH tells us that the realized volatility will increase tomorrow, most of us would instinctively go out and buy ourselves some options (i.e. implied volatility). In the case of SPY, we would probably go buy some VXX. But that would be a terrible mistake. Remember that the volatility we predicted is an unsigned return: a prediction of increased volatility may mean a very bullish day tomorrow. A high positive return in SPY is usually accompanied by a steep drop in VXX. In other words, an increase in realized volatility is usually accompanied by a decrease in implied volatility in this case. But what is really strange is that this anti-correlation between change in realized volatility and change in implied volatility also holds when the return is negative (57% of the days with negative returns). A very negative return in SPY is indeed usually accompanied by an increase in implied volatility or VXX, inducing positive correlation. But on average, an increase in realized volatility due to negative returns is still accompanied by a decrease in implied volatility.

The upshot of all these is that if you predict the volatility of SPY will increase tomorrow, you should short VXX instead.

====

Industry Update
  • Quantiacs.com just launched a trading system competition with guaranteed investments of $2.25M for the best three trading systems. (Quantiacs helps Quants get investments for their trading algorithms and helps investors find the right trading system.)
  • A new book called "Momo Traders - Tips, Tricks, and Strategies from Ten Top Traders" features extensive interviews with ten top day and swing traders who find stocks that move and capitalize on that momentum. 
  • Another new book called "Algorithmic and High-Frequency Trading" by 3 mathematical finance professors describes the sophisticated mathematical tools that are being applied to high frequency trading and optimal execution. Yes, calculus is required here.
My Upcoming Workshop

January 27-28: Algorithmic Options Strategies

This is a new online course that is different from most other options workshops offered elsewhere. It will cover how one can backtest intraday option strategies and portfolio option strategies.

March 7-11: Statistical Arbitrage, Quantitative Momentum, and Artificial Intelligence for Traders.

These courses are highly intensive training sessions held in London for a full week. I typically need to walk for an hour along the Thames to rejuvenate after each day's class.

The AI course is new, and to my amazement, some of the improved techniques actually work.

My Upcoming Talk

I will be speaking at QuantCon 2016 on April 9 in New York. The topic will be "The Peculiarities of Volatility". I pointed out one peculiarity above, but there are others.

====

QTS Partners, L.P. has a net return of +1.56% in October (YTD: +11.50%). Details available to Qualified Eligible Persons as defined in CFTC Rule 4.7.

====

Follow me on Twitter: @chanep




100 comments:

  1. Great analysis. Something I've long been interested in.

    You found the GARCH volatility prediction to be 58% accurate and, based on the paradoxical and counterintuitive relationship between changes in implied and realized volatility, you conclude one should short VXX on predictions of increased volatility in SPY. Your reasoning rings true.

    Have you backtested your suggestion, that is, a strategy that shorts VXX on a daily positive GARCH volatility prediction (and vice versa)?

    ReplyDelete
  2. Thank you, Stephen.

    Yes, I backtested this short trade. The CAGR (after 5bps one way tcost) is 67%, with Sharpe ratio of 1.2, on the test set.

    Ernie

    ReplyDelete
  3. For anyone interested in watching NYU's daily prediction, here's the URL: http://vlab.stern.nyu.edu/analysis/VOL.SPX:IND-R.GARCH

    ReplyDelete
  4. Hi Ernie,

    "The CAGR (after 5bps one way tcost) is 67%,"

    These strategies have usually large drawdown. Do you recall the level?

    ReplyDelete
  5. VXX has only been around for six years. And, correct me if I'm wrong, but the CAGR for SPY during that same period is about 19%. So Ernie, that's a pretty nice strategy.

    ReplyDelete
  6. Hi Michael,
    The max drawdown is 34%.

    Also, correction: the CAGR is 78%, with a Sharpe of 1.3.

    Ernie

    ReplyDelete
  7. Hi M,
    If by intraday strategy, you meant generating the signal at the close, but wait until next open to enter, the CAGR is lower (30%), and the Sharpe is lower too (0.8). The drawdown, however, increases.
    Ernie

    ReplyDelete
  8. Ernie, thanks for the reply.

    Isn't the period "from December 6, 2011 to November 25, 2015" sort of special in the sense that volatility has been below its historical mean?

    I have noticed that volatility-based strategies do not perform well during volatility spikes. Have you noticed the same?

    Best.

    ReplyDelete
  9. Hi Michael,
    Sure, a short volatility strategy usually loses when there is a spike in volatility. But selectively shorting volatility on certain days based on this strategy still greatly outperforms short-and-hold VXX.
    Ernie

    ReplyDelete
  10. Hey Ernie - I have been putting quite a bit of research into volatility. Predicting it on a short term basis (i.e. 60-min) is relatively easy. I take it you guys are talking about daily volatility but at any rate I am not able to correlate an expansion in RV with a drop in IV. Here are two screenshots - the upper panel shows the E-Mini futures, the center is my own volatility indicator which uses a simple BB to show expansion - on the bottom are the VX futures:

    http://screencast.com/t/D3IiDOFYrZ

    http://screencast.com/t/oh0xtrjEK8A

    I see very little correlation here - what am I missing?

    ReplyDelete
  11. Hi Evil Speculator,

    We must be very careful in comparing apples-to-apples.

    I define realized volatility as the square of the day's SPY return from previous close to today's close (this is consistent with the GARCH model's definition).

    I then compute the 1-day change of the realized volatility with the 1-day change in VXX from previous close to today's close.

    From 20051221-20151125, these two changes have the same sign only 35% of the days.

    Ernie

    ReplyDelete
  12. Agreed - crap in, crap out as the saying goes - thanks for clearing that up, Ernie. Is there a particular reason why you're using a SquaredReturn instead for example a LogReturn?

    ReplyDelete
  13. Hi Evil Speculator,
    The squared return is actually the square of the log return.

    Log return, of course, is signed, and we need an unsigned quantity as representative of realized volatility.

    You can try abs(ret) as well, but it will give the same result.

    Ernie

    ReplyDelete
  14. Hey again - ha, I actually was wondering if that's what you were doing :-)

    Cool beans. I don't use GARCH but will get up to speed on it. Personally I use volatility cycles as produced by the indicator I posted. I take it GARCH uses those two data series as input in order to arrive at a prediction of volatility? Have you guys ever tried that on a more short term chart, like perhaps an hourly? In terms of volatility cycles I'm getting high predictability > 80 percentile. It drops quite a bit when shifting upward toward the daily.

    ReplyDelete
  15. GARCH takes the lagged squared returns, and the previous predictions of volatility (conditional variance) as input.

    I haven't tried it on hourly. Good to know that it may work better on shorter time frames!
    Thanks for the tip.

    Ernie

    ReplyDelete
  16. Yes, predicting volatility expansion on the hourly is actually relatively easy. Unfortunately I don't see the IV correlation that you describe on that time frame - the dynamics are different. What I'm seeing however is that realized volatility of VXX (or the VIX futures) resolves more directional than in equities. Here's a recent example:

    http://screencast.com/t/VIaNm54AtT9

    Pay attention in particular to the highlighted areas. Note that realized volatility on the VIX futures features less overlap and a clear directional vector. Meanwhile the tape on the equities side in those areas is a lot more noisy. So this supports your idea of trading the VXX or the VIX futures as opposed to equities.

    ReplyDelete
  17. "VXX less noisy than SPY on hourly bars" - good tip to have, thanks!

    Ernie

    ReplyDelete
  18. Hi Ernie, I found your blog when I searched for Kelly formula. Thanks for your excellent blogs on Quantitative Trading and especially on Kelly formula. I have a few questions regarding Kelly formula. Please refer to my blog: 2015-12-14 Kelly formula for 盈富基金
    http://edwardyuinvest.blogspot.hk/2015/12/2015-12-14-kelly-formula-for.html
    1. Which m (mean annual return) should I used? Yahoo's or Morningstar's ?
    2. The standard deviation quoted in yahoo and morningstar, is it a percentage? Can it be used in the formula
    f=m/s^2
    3. Can the formula f=q/s be used instead, where q is the Sharpe Ratio ?
    4. I cannot find risk data (i.e. m, s, q) for non-ETF stocks such as 0823.HK. Are there any free site that gives risk data for HK stocks? (Yahoo, Google, Morningstar?)
    Thanks, Edward Yu

    ReplyDelete
  19. Hi Edward,
    1) We should use the longest lookback for mean return. I.e. 5 year is better than 3 year.
    2) Yes, percentage can be used (once converted to decimals). But make sure both mean returns and standard deviation of returns are annualized.
    3) Yes, annualized Sharpe ratio can be used as you wrote.
    4) To compute s, all you need to do is to download the daily returns of the stock into a spread sheet, and use the Excel function for standard deviation.

    Ernie

    ReplyDelete

  20. Hi Erine

    I have some questions of using the CQG time Sales data for testing. The resolution of the timestamp (col 3) is only up to min, and there are many Bid, Ask and Transaction records within each minute. As I found the testing time of my trading program is quite slow to loop through all
    the rows in this time sales data file, especially if I need to do few years backtest, so I m thinking of sampling only one B/A/T record in each minute for simulation, so what is the right way to do it? If I used the aggregated lot (col 5) to represent Bid vol and the Ask vol within this one min interval, what should the Bid and Ask price I should pick for simulation, same for the transaction (T)

    Thanks

    Carol

    CQG Time Sales Data

    J14 20140101 1700 12039 B N N 21
    J14 20140101 1700 12039 B N N 19
    :
    :
    J14 20140101 1700 12040 T N N 1
    J14 20140101 1700 12040 A N N 9
    J14 20140101 1700 12039 B N N 15
    J14 20140101 1700 12039 B N N 14
    J14 20140101 1700 12036 B N N 1
    :
    :
    J14 20140101 1701 12038 T N N 1
    J14 20140101 1701 12040 A N N 23
    J14 20140101 1701 12040 A N N 22
    :
    :
    J14 20140101 1701 12038 B N N 1
    J14 20140101 1701 12039 T N N 5
    J14 20140101 1701 12040 A N N 10
    J14 20140101 1701 12040 A N N 9

    ReplyDelete
  21. Hi Carol,
    You have put your finger on the biggest problem with CQG's "tick" data!
    I put "tick" in quotes because if the time stamps are 1-min, the data cannot be considered tick data, as we have no way to ascertain that the ticks are in chronological order. So CQG data is really 1-min bar data, available free on Quantopian.com (stocks, maybe not futures).

    Nanex.net has data with 25ms time stamps. Tickdata.com or quantGo.com both have data with 1ms time stamps. If you buy data directly from the exchanges, the time stamps are mostly in microseconds.

    Ernie

    ReplyDelete
  22. hi Ernie, the contest is interesting; but no one successful trader would submit his/her profitable system to that company. To me that website offering is somewhat unethical. They should open a live trading contest instead; and let the Accounting do its job.

    ReplyDelete
  23. Hi Alpha,
    Thanks for the feedback.
    As usual, a mention on my Industry Update section does not imply endorsement.
    Ernie

    ReplyDelete
  24. Hi, Ernie, just came off your chat with traders session and i have two questions i would love to ask since you have seen so many models and strategies:

    Of the many strategies/models you have tested and developed, how have multiple vs single entry/exit strategies tested out?
    i.e. does a single entry/exit strategy usually prove to be more profitable or does a multiple entry/exit stratgy provide much better risk adjusted returns?

    Second question:
    OF the many futures models you have seen, how is contango best managed if taking a long term view/outlook on an instrument?

    Many thanks in advance!!
    Wei Kai

    ReplyDelete
  25. Hi Wei Kai,
    1) Single entry/exit is suitable for momentum strategies. (Why wait for a worse price to enter if you think prices will move in same direction?) Multiple entries/exits are suitable for mean reverting strategies. (See p. 72 of my book Algorithmic Trading.)

    2) Contango is "managed" by shorting the future on average over the long term.

    Ernie

    ReplyDelete
  26. Dear Ernie

    Huge fan of yours and avid student of your books and blogs.
    I code in Matlab and trade with IB. I was wondering if you have an update recommendation on how to best connect the two for a true ATS. These days there seems to be a few options between Matlab's own trading toolbox as well as exchangeapi and undocumented matlab. Do you use Matlab for placing the trades yourself in a truly automated fashion?

    Finally what do you suggest as far as data feed? Do you rely on IB or connect Matlab to other sources?

    Thanks a bunch, Luis

    ReplyDelete
  27. Hi Luis,
    Thank you for your kind words.

    I still use Matlab to trade with IB for some very low frequency strategies. But for other strategies, we mostly use C# with IB's .NET API.

    The 3 Matlab-IB API's you mentioned are the only ones I know of.

    Ernie

    ReplyDelete
  28. Thanks Ernie

    Do you still develop and backtest those other strategies in Matlab and then compiled them into C#?

    Luis

    ReplyDelete
  29. Hi Luis,

    Yes, I continue to backtest strategies on Matlab. My partner Roger independently backtest the same strategies on C# as verification, and the same codes are used for execution.

    Ernie

    ReplyDelete
  30. Hi Ernie,

    What are the pitfalls of long-short equity strategies?

    Thanks.

    ReplyDelete
  31. Long-short equity strategy is a very general term. It can encompass strategies like pair-trading, fundamental long-short, or factor-based models. So we can't discuss the specific pitfalls unless we know what exactly the underlying strategy is.

    Generally speaking, long-short strategies are market-neutral. Because they are market-neutral, they cannot benefit from the long term positive drift of the stock market. Also, because it requires a substantial short position, and shorting stocks is subject to many type of restrictions and friction (e.g. hard-to-borrow, uptick rule), it is often costly and inefficient to implement. Finally, while we can buy-and-hold a long-only portfolio, one must rebalance a long-short portfolio periodically in order to maintain market or dollar neutrality. This rebalancing incurs transaction costs, and is subject to vega risks.

    Ernie

    ReplyDelete
  32. Hi Ernie,

    Thank you for quick response.

    I mean the pitfalls of factor-based models (price momentum, financial ratios).

    ReplyDelete
  33. The pitfall of a factor model is that investors may no longer consider that factor to carry risk. For example, market cap should to be a factor, but nowadays people do not believe small cap stocks, when averaged over all stocks in this class, present any greater risk than large cap. So market cap is no longer driving returns.

    Ernie

    ReplyDelete
  34. Hi Ernie,

    Does stocks pairs trading outperform factor models?

    Thanks

    ReplyDelete
  35. Not necessarily - again, it depends on the precise implementation method, and the specific factors used.

    Ernie

    ReplyDelete
  36. Ernie

    When is your next book coming out and what will it focus on? Please give us a preview.

    Should we expect some new strategies?
    A bit more Matlab code?
    Perhaps some machine learning approaches you found useful?

    Looking forward Luis

    ReplyDelete
  37. Luis,
    The topics will include techniques and asset classes that I have not discussed before.
    Certainly there will be new strategies and Matlab codes.
    It should be out by 2017 Q1.
    Ernie

    ReplyDelete
  38. Hi Ernie,

    What is fundamental long-short?
    Is it different from factor models?
    Thanks.

    ReplyDelete
  39. Fundamental long-short uses fundamental factors. So it is one type of factor models.
    The other type would be statistical factors.

    Ernie

    ReplyDelete
  40. Hi Ernie,

    What are statistical factors?
    Thanks.

    ReplyDelete
  41. Statistical factors are the result of principal component analysis. Please see the textbook by Ruppert et al on my Recommended Books list on the right sidebar of this blog.

    Ernie

    ReplyDelete
  42. Hi Ernie,

    For fundamental factors model, usually how many factors do we use?

    Thanks.

    ReplyDelete
  43. Hi Ernie,

    What are some advantage and disadvantage of trading arbitrage of two different months of oil future contract? If this is bad choice, what other future should I consider?

    Thanks,
    -HK

    ReplyDelete
  44. Also, you should search my blog for my many articles on factor models for examples.

    Ernie

    ReplyDelete
  45. Hi HK,
    You are essentially trading a calendar spread, and there are many strategies for that. The main question is: which do you believe will be the roll returns of the two legs?

    Ernie

    ReplyDelete
  46. Lets make a little tweak to Ernie's strategy.
    Many tend to forget that the VIX is implied volatility for only ATM options, so going for it VIX, is actually betting on the middle of the distribution which means missing the tails, and missing the convexity effect that leverages the volatility jump for you! It means almost going linear... aside from paying a hefty premium ATM. In "No Small Probabilities Are Not An Attractive Sell" Taleb (2013) showed that when ATM volatility doubles, Deep OTM can earn from 5 to 7,686 times their daily theta decay. When volatility triples it can go up almost nine fold over when they double.

    So, I would say, to enhance the returns one should buy Deep OTM options or short Deep ATM (if the preference allows).

    ReplyDelete
  47. Interesting point!

    It is true that OTM options have higher leverage, but they are also more expensive if measured in terms of implied volatility. (See Zhang, Xiaoyan, Zhao, Rui and Xing, Yuhang. 2008. What Does Individual Option Volatility Smirk Tell Us About Future Equity Returns?)

    Also, the strategy I described can be implemented by trading an ETF (VXX), whereas I am not aware that there is an ETF that implements OTM options. Trading ETF has the virtue that you can lever it quite a few times, beyond Reg T's 2x overnight limit if you have "portfolio margin" at various brokers (e.g. Interactive Brokers).

    Ernie

    ReplyDelete
  48. Spot on!
    Since OTM have higher IV, the art of forecasting comes to the rescue (or so we think?). Superior forecasting skills are required to those of the market makers, otherwise the volatility will be priced in. Hence, we should decompose the volatility signal to check. When I do it, historically around 50-55% of volatility is contained in the frequency range 1-4 days, and some 20-25% is contained in the interval 4-8 days, 10-15% in 8-16 days. It means that we would probably need a different time scale (lower frequency) to outsmart the market! ****Which makes sense since probably everybody uses GARCH to forecast the volatility. Though, I haven't tested this proposition... I did the one with Deep OTM. It's not very efficient unless you can buy in bulk and narrow the liquidity component of the price. The thing is, the monster kurtosis makes it very attractive if you can survive in the long term and have deep pockets. For example, under the mathematical expectation (corrected for look ahead bias) I backtested "the lazy option strategy" that buys Deep OTM S&P options with expiration date later than 6M, it returned over 700% in December 2008, 170%+ in June 2010, 115+% in July 2011, 250+% in September 2011, 350+% in August 2002, etc. Of course, it loses as well 10-20% any given month, so it has a RM problem... but there is much space to improve it...

    S.

    Stefan

    ReplyDelete
  49. The main problem of the rudimentary model I initially did (for fun) was, it had no signal component or any type of weighting rule, so it would naturally buy a big position at the top of the frenzy which contained extrapolated expectations and even higher IV, and of course, would end up being a losing trade that would dampen the winnings and kill the RM measurements on the downside vol. The reason I believe such trading could be very successful is that does not exactly play the short range-high frequency game (which is very crowded!), where the noise tends to be high and the expectation built in from the market consensus is already incorporated into the price.

    ReplyDelete
  50. Hi Stefan,

    Did you use real bid ask data to achieve such good return? It might be interesting to add skewness as a factor to your strategy.

    PL

    ReplyDelete
  51. I used real bid/ask daily data I acquired from a provider for around 1,000 bucks (I know I probably got ripped of but that didn't matter). It tested a version of numeraire portfolio with allocation 90% bonds/10% options.

    Three assets: Cash, Bonds and Options
    90% of the portfolio is allocated to 10Y T-Bonds. 10% is divided on monthly basis over the year to be used to buy options. Cash is only used as temporary account when a transaction has taken place but no rebalancing was done. On the first day of the trading month, our portfolio is rebalanced. The strategy searches for available option pairs. If there are, it buys the maximum amount to be invested, if there are none, it stays inactive. It buys only options that are actively traded and liquid. The price of purchase is the mid point on bid/ask. It sells the options only if: a.) Potential sell price is higher than the exercise price b.) If an option is not sold or exercised before 30 days to expiration Selling price is calculated as the mathematical expectation adjusted for the price. %x quantile price * by it's empirical probability.
    The strategy includes a (terrible) momentum based adjustment to account for an increase or drop in volatility.

    Results:
    http://www.slideshare.net/stefanbolta/long-straddle-strangle-deep-otm

    Three things I really like about it:
    1) It leaves much space for improvement, especially if it can be boosted by a Regime Switching model, so that a change in allocation is allowed.
    2) Trading costs are irrelevant. I priced them at usd 10 per trade!
    3) It has a pretty strong empirical foundation to stay profitable! The conclusions of Earnie's first book lay it out pretty nicely some of the advantages if offers over institutional investors.

    Overall, it can further be upgraded on larger scale with very little risk on capital!

    The main problem: It needs a substantial amount of capital, and especially so at these paltry interest rates.

    Stefan.

    ReplyDelete
  52. Hi Ernie,

    Have you ever heard about ADVFN?

    They provide free financial data.

    ReplyDelete
  53. No, I haven't. Thanks for the heads-up.
    Ernie

    ReplyDelete
  54. Hi Ernie,

    If you had to speculate, what type of strategies and methods do you think RenTech is using in it's Medallion fund?

    Many thanks,
    Peter

    ReplyDelete
  55. Hi Peter,
    I speculate that it uses quite high frequency strategies.
    Ernie

    ReplyDelete
  56. Hi Ernie

    What roundtrip cost would you assume for a very liquid fx future like eurusd on cme?

    ReplyDelete
  57. Roundtrip cost is broker commissions (plus exchange and regulatory fees) plus the bid-ask spread. The bid-ask spread for a very liquid future is typically the minimum price increment, i.e. 1 tick.

    Ernie

    ReplyDelete
  58. Hi Ernie,

    How much does compustat point-in-time fundamental database cost?

    Thanks.

    ReplyDelete
  59. I had free access to Compustat through my university affiliation, so I am afraid I don't know know its commercial price.
    However, you can also get fundamental data through Quandl.com's Sharadar Core US Fundamentals database, which is under a few hundred dollars.
    Ernie

    ReplyDelete
  60. Hi Ernie,

    Thank you for quick response.

    Is Quandl also point-in-time?
    How often do they update?

    ReplyDelete
  61. Quandl is a consolidator of many data vendors, so you have to ask that specific vendor whether it is point-in-time. I believe Sharadar is, but you should ask them directly.

    Ernie

    ReplyDelete
  62. Hi Ernie,

    Thank you for the information about Quandl.com's Sharadar Core US Fundamentals database.

    It looks very good.

    Have you heard any comments about backtesting using this database?

    Thanks.

    ReplyDelete
  63. I have used Sharadar for backtesting, and haven't encountered any issue.

    Ernie

    ReplyDelete
  64. Yes, we were hit by the highest volatility since 2008 in one of the FX pairs we are trading.

    Ernie

    ReplyDelete
  65. Hi Erine,

    Is shorting a pair of 3x or 2x etf (example short both YANG and YINN) a good idea with cut lose if one raises over 100%? Thanks.

    -HK

    ReplyDelete
  66. Hi HK,
    To find out if it is a good idea, why not backtest it?
    Ernie

    ReplyDelete
  67. Dear Ernie,

    Yeah I should just backtest some pairs and try different time frames to see the result. Just backtest historical data is confident enough? Or I should use at least one other math model to statically prove it is not just random luck?

    -HK

    ReplyDelete
  68. Hi HK,
    If your pair is supposed to be mean reverting, you can certainly run an ADF on it to verify that before backtesting. You can also compute half life of mean reversion using methods described in my 2 books, and see if it is too long for practical trading.
    Ernie

    ReplyDelete
  69. Hi Ernie,

    Is fundamental data from Bloomberg ok to use, compared with compustat database?

    Thanks.

    ReplyDelete
  70. While I haven't used BBG's fundamental data, I don't see why it should be inferior to Compustat.
    Ernie

    ReplyDelete
  71. Ernie have you found anything useful in the HFT algo book you mentioned? A lot of Calculus.

    ReplyDelete
  72. The last 2 chapters of the HFT book is particularly useful.
    Ernie

    ReplyDelete
  73. Hi Ernie,

    In Algorithmic Trading, at page 80, you wrote that if e(t) < -sqrt(Q), we long, and if e(t) > sqrt(Q), we short.

    Could we do normalization first, to have e(t)/sqrt(Q), then setting threshold as numbers?

    I find the threshold could be much bigger than 3. I think it depends on how we set Ve. You set Ve = 0.001 here.

    Thanks.

    ReplyDelete
  74. You can certainly enter based on a multiple of sqrt(Q), ie. entryZscore*sqrt(Q).
    Yes, the strategy as written is quite sensitive to the initial guess Ve. MATLAB's Econometrics Toolbox has state space model functions that allow you to optimize Ve quite easily.

    Ernie

    ReplyDelete
  75. Hi Ernie,

    Thank you for quick response.

    However, according to Montana's paper you mentioned, it seems we can fix Ve, and then optimize delta between 0 and 1.

    Thanks.

    ReplyDelete
  76. What other method is there to "fix" Ve except to feed past data into a model?

    Ernie

    ReplyDelete
  77. Hi Ernie,

    In Montana's paper, at page 2825, Ve is fixed at 1.
    At page 2828, in the beginning of section 6,Experimental results, he wrote "We
    have tested the system using a grid of values for the
    smoothing parameter delta described in Section 3,"

    It seems he only optimized delta ( which controls the speed of change of beta), and set Ve = 1.

    Thanks.

    ReplyDelete
  78. I don't think Ve=1 will work in our case, and frankly, I don't know how he come up with that initial value. I would use MLE to estimate Ve.

    Ernie

    ReplyDelete
  79. Hi Ernie,

    Would you please give more details about estimation of Ve?

    Is this estimation procedure independent of choosing delta?

    So we can pick Ve first, then optimizing delta.

    Thanks.

    ReplyDelete
  80. Delta and Ve can be found in the same MLE optimization (i.e. multivariate optimization).

    For details, please google MATLAB's Econometrics Toolbox's documentation on State Space Models.

    Ernie

    ReplyDelete
  81. Hi Ernie,

    Thank you for the information.

    To create the model via ssm class in MATLAB's Econometrics Toolbox, do we need to refer to "Implicitly Create Time-Varying State-Space Model"?

    ReplyDelete
  82. No, we are assuming that parameters analogous to Ve or delta are constant, not time varying.

    Ernie

    ReplyDelete
  83. Hi Ernie,

    Can stocks pairs trading still make profit?

    Thanks.

    ReplyDelete
  84. Since I haven't trade stock pairs for a while, I would not want to speculate on whether other traders are still successful with this strategy.

    Ernie

    ReplyDelete
  85. Apologies in advance if someone already mentioned this above (long thread). But Anonymous above said this:

    "Many tend to forget that the VIX is implied volatility for ***only ATM options***, so going for it VIX, is actually betting on the middle of the distribution which means missing the tails, and missing the convexity effect that leverages the volatility jump for you! It means almost going linear.."

    Actually that is not true - what Anonymous is talking about is the old VIX. Since Sept 22 2003 the formulate changed and the old VIX became the VXO. Lifted off the CBOE:

    "New Formula for Calculation of VIX. ****The new formula that will take into account a broader range of strike prices (rather than using only near-the-money strikes as the original-formula index did)****. Each strike price will be weighted, with at-the-money strikes having the most weight. The new formula is intended to make VIX a better index for investors who manage risks associated with the growing markets for volatility and variance swaps"

    Here's the VXO:

    "The CBOE is continuing to calculate and disseminate the volatility index introduced in 1993 based on trading of S&P 100 (OEX) options. This index has a price history dating back to 1986, which remains the same. As of September 22, 2003, the name was modified -- the original-formula index is now known as the CBOE S&P 100 Volatility IndexSM and is now disseminated under the new ticker symbol VXO (prior to September 2003 it was the "original" VIX Index)."

    On top of that the VIX is comprised of the VIN and VIF - they are not using the front month contract, but I'm certain most of you are aware of that ;-)

    ReplyDelete
  86. Hi Evil Speculator:

    Agreed! That's for pointing that out - indeed, VIX uses OTM options with 23-37 days tenor for averaging these days. (See http://www.cboe.com/micro/vix/vixwhite.pdf)

    Ernie

    ReplyDelete
  87. Hi Ernie,

    Thank you for quick response.

    May I ask what kind of trading strategies for stocks your fund are still using?

    Thanks.

    ReplyDelete
  88. We currently trade event-driven stock models in our fund.

    Ernie

    ReplyDelete
  89. Hi Ernie,

    To start off, I wanted to mention that "Algorithmic Trading" was a great read, and very clearly written. It was the first book I read on algo trading, and it inspired me to start trading a personal account (I use IB with backtests in Matlab, and I'm currently working on setting up a C# platform to systematically suggest trades). For that, I thank you, and I'm eagerly awaiting your next book!

    With regard to this post, perhaps there's something that I'm missing here, but my understanding of the strategy is the following:
    This strategy hinges the assumption that, more often than not, an increase in realized S&P volatility is accompanied by a decrease in implied S&P volatility. Thus, if you can somewhat accurately predict increases in realized S&P volatility (e.g. by fitting a GARCH model), then you can use this as a trading signal to short implied volatility.

    If my above summary is correct, then my question is thus: why are you using VXX as a proxy for implied S&P volatility? Wouldn't you want to use the actual VIX index? (i.e. modify the strategy so that when GARCH predicts a vol increase, short 1 VX future or something).
    I suggest this because VXX holds a varying allocation to M1 and M2 VIX futures contracts. Contango of VIX futures contracts causes the VXX to decay over time. This is why VXX returns have a significantly lower (more negative) skew than VIX returns.

    The reason why this additional contango-driven drag on VXX returns is relevant here, is that the main assumption of the strategy breaks down when we use the VIX as a proxy for implied vol (which I would argue is a more appropriate measure). In fact, over most of the time periods that I looked at (happy to provide more detail if you like), increases in realized S&P volatility are accompanied by virtually equal numbers of positive and negative movements in the VIX. I've found the following:

    - Of the days in which we experience an increase in realized S&P Volatility (i.e. " "abs(ret(t)) > abs(ret(t-1))"), around 57% of those days show a negative VXX return, which one could argue is an acceptable percentage to base a strategy on.
    - Of that same subset of days, around only 51% of those days show a negative VIX return. I would probably not want to base a strategy on 51%, especially if my realized S&P Vol predictions are only 58% accurate.

    For this strategy to work, my GARCH model needs to be an accurate prediction of realized S&P Vol on the same days that realized S&P Vol is an accurate prediction of implied S&P Vol.

    My guess is that this strategy is not profitable due to of the ability of RV to predict IV (which isn't significant if we use the VIX index as our measure of IV). In stead, I believe it is profitable for the same reason that so-called "convexity capture" strategies are profitable: because outright shorting ETFs with negative drag on returns tends to yield decent risk-adjusted returns, especially if we add a hedge.

    Let me know if you notice any holes in my logic. I'm not 100% sure of my analysis, as I'm still relatively new to the world of systematic investing :).

    ReplyDelete
  90. Hi Rob,
    Thank you for your kind words!

    Actually, upon further research, I have determined that on average, there is zero correlation between changes in implied vs. realized volatility on the SPX. However, there is a negative correlation on days when there is positive return on SPX. So I agree with your calculations.

    However, I don't think I wrote in my article that I computed the correlation between the change realized volatility and the change in VXX. I only stated that we can buy VXX if we predict that realized volatility is going down. Which turns out to be wrong without an additional prediction on the direction of the SPX price change.

    Check out my talk at QuantCon 2016. It has the correct reasoning, and a few suggested strategies to take advantage the contango of VX futures. But the upshot is that it is no longer straightforward to trade VX after 2013.

    Ernie

    ReplyDelete
  91. Thanks for the quick reply! I was too late to sign up for this year's QuantCon, and unfortunately registration it is now closed, and I don't think it's possible to see your talk through their website. If it is saved somewhere, could you provide me with a link that I can use to view your talk?

    ReplyDelete
  92. Rob,
    QuantCon will make available the videos (at a price) soon.
    Ernie

    ReplyDelete
  93. Its well known and very easy to see that VIX is highly correlated with SP500 by just looking at the chart. The most of the crucial changes happen intraday. So, I dont think daily models are useful. I studied all kinds of Autoregressive Models (ARIMA,GARCH, VECM, etc) and none of them are used by successful traders. Its mostly technical analysis which is also not easy to use profitably.
    And there are so many factors. For example, tech stocks crashed on June/July and it caused SPX to decline which caused VIX to spike. There is no way to predict that using a GARCH model.
    If GARCH were useful, all the traders would study that and it would be on the front page of WSJ.
    In any event, its best to start trading with a little of real money.

    ReplyDelete
  94. Hi Naverno1,
    I disagree with your suggestion that GARCH isn't useful. If you read page 127 of my book Machine Trading, you will see that GARCH is able to predict the sign of the change of realized volatility of SPY with out-of-sample accuracy of 66%. For USO, it is 67%. I listed the accuracy for several other instruments too - all highly statistically significant.

    We have incorporated GARCH models in our trading strategies that manage tens of millions of dollars, and continue to find that they add value.


    Ernie

    ReplyDelete
  95. I went through every comment on this article and it is amazing to see that almost 3 years later it is still very relevant. Markets are moving in a massive daily range and still trading close to all-time high prices. To consider how futures quant systems may fair during a volatility spike we need to do some calculations regarding drawdown values. Taking the Emini S&P Futures market (ES) as an example: Algorithmic Trading During Volatility Spike

    ReplyDelete
  96. Hi Ernie,

    I find it interesting that in the recent market rebound short term VIX futures have declined substantially (as expected), but the medium and longer term contracts haven't budged. Do you find the VIX term structure useful in any way for your trading? If so, what intuitive interpretation do you think can we draw from the VIX term structure in terms of forecasting volatility or returns of strategies that depend on it?

    ReplyDelete
  97. Hi,
    VX is typically in contango, so this term structure is the norm, except in times of crisis. Since people thinks all is well in the world, this is expected.
    Best,
    Ernie

    ReplyDelete