Thursday, April 07, 2016

Mean reversion, momentum, and volatility term structure

Everybody know that volatility depends on the measurement frequency: the standard deviation of 5-minute returns is different from that of daily returns. To be precise, if z is the log price, then volatility, sampled at intervals of τ, is 

volatility(τ)=√(Var(z(t)-z(t-τ)))

where Var means taking the variance over many sample times. If the prices really follow a geometric random walk, then Var(τ)≡Var((z(t)-z(t-τ)) ∝ τ, and the volatility simply scales with the square root of the sampling interval. This is why if we measure daily returns, we need to multiply the daily volatility by √252 to obtain the annualized volatility.

Traders also know that prices do not really follow a geometric random walk. If prices are mean reverting, we will find that they do not wander away from their initial value as fast as a random walk. If prices are trending, they wander away faster. In general, we can write

Var(τ)  ∝ τ^(2H)

where H is called the "Hurst exponent", and it is equal to 0.5 for a true geometric random walk, but will be less than 0.5 for mean reverting prices, and greater than 0.5 for trending prices.

If we annualize the volatility of a mean-reverting price series, it will end up having a lower annualized volatility than that of a geometric random walk, even if both have exactly the same volatility measured at, say, 5-min bars. The opposite is true for a trending price series.  For example, if we try this on AUDCAD, an obviously mean-reverting time series, we will get H=0.43.

All of the above are well-known to many traders, and are in fact discussed in my book. But what is more interesting is that the Hurst exponent itself can change at some time scale, and this change sometimes signals a shift from a mean reversion to a momentum regime, or vice versa. To see this, let's plot volatility (or more conveniently, variance) as a function of τ. This is often called the term structure of (realized) volatility. 

Start with the familiar SPY. we can compute the intraday returns using midprices from 1 minutes to 2^10 minutes (~17 hrs), and plot the log(Var(τ)) against log(τ). The fit, shown below,  is excellent. (Click figure to enlarge). The slope, divided by 2, is the Hurst exponent, which turns out to be 0.494±0.003, which is very slightly mean-reverting.




But if we do the same for daily returns of SPY, for intervals of 1 day up to 2^8 (=256) days, we find that H is now 0.469±0.007, which is significantly mean reverting. 




Conclusion: mean reversion strategies on SPY should work better interday than intraday.

We can do the same analysis for USO (the WTI crude oil futures ETF). The intraday H is 0.515±0.001, indicating significant trending behavior. The daily H is 0.56±0.02, even more significantly trending. So momentum strategies should work for crude oil futures at any reasonable time scales.


Let's turn now to GLD, the gold ETF. Intraday H=0.505±0.002, which is slightly trending. But daily H=0.469±0.007: significantly mean reverting! Momentum strategies on gold may work intraday, but mean reversion strategies certainly work better over multiple days. Where does the transition occur? We can examine the term structure closely:




We can see that at around 16-32 days, the volatilities depart from straight line extrapolated from intraday frequencies. That's where we should switch from momentum to mean reversion strategies.

One side note of interest: when we compute the variance of returns over periods that straddle two trading days and plot them as function of log(τ), should τ include the hours when the market was closed? It turns out that the answer is yes, but not completely.  In order to produce the chart above where the daily variances initially fall on the same straight line as the intraday variances, we have to count 1 trading day as equivalent to 10 trading hours. Not 6.5 (for the US equities/ETF markets), and not 24. The precise number of equivalent trading hours, of course, varies across different instruments.

===


Industry Update
Upcoming Workshops

There are a lot more to mean reversion strategies than just pairs trading. Find out how to thrive in the current low volatility environment favorable to this type of strategies.

111 comments:

  1. How did you calculate the 10 hours = 1 day, simply by fitting the 1 day point onto your linear fit, solving for it to put it on the line? Isn't there a risk one will wash out significant overnight trending/mean-reversion that way? And, I suppose more generally, are there any references for how to deal with overnight gaps when trying to do trading on the 3-5 day timeframe using intraday data?

    ReplyDelete
  2. Hi one_fell_swoop,

    You are right that 10 hours= 1 day is determined by trying different numbers, and finding the one where intraday and daily variances coincide at the same T where they overlap.

    Indeed, this does not capture seasonal mean reversion such as what may occur only overnight. This study is only concerned with "time-invariant" mean reversion, or "homogeneous" mean reversion.

    The general way to deal with overnight gap is that you can to treat the overnight bar differently from the intraday bars, by multiplying its variances by some adjustment factor. This factor is determined by a procedure outlined above. (Instead of adjusting the equivalent time, you can adjust the equivalent variance.)

    Ernie

    ReplyDelete
  3. Hi Ernie,

    how long is the window length over which you calculate the variance?
    You only state "variance over many sample times"... Or do you want to keep it a secret? :)

    I figured, if the slope is near 1.0, varying the window length may push the slope above or below 1 randomly. How do you handle this? (But SPY seems always to be mean reverting on an interday basis?!)

    BR!

    ReplyDelete
  4. Hi BR,

    I computed the variances over 20130116-20160115.

    Obviously, a different period might give a different answer. But within this period, you can see that I have included the standard errors (SE) in the slope. For many of the results (though not SPY intraday), a slope of 1 is not within 2x SE.

    Ernie

    ReplyDelete
  5. Hello Ernie,

    Interesting article. Is there a problem with the x-axis of the GLD chart? 16 and 32 days cannot have a base 2 log between 13 and 15, if I get this right of course.

    ReplyDelete
  6. Thanks, Michael.

    The units of x-axis (t) is actually in minutes. Apologies for not making that explicit.

    So at log2(t)=14, we have t=2*14/10/16 days=27 days. (The factor of 10 due to the equivalence of 10 trading hours and 1 trading day.)

    Ernie

    ReplyDelete
  7. Actually, I meant t=2*14/10/60 days=27 days above!

    ERnie

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete
  9. Hi Ernie!
    Nice artical again, but I have one remark. Hurst exponent is different for brownian motions with different distributions. You can simulate it with Monte Carlo and test out.

    ReplyDelete
  10. Hi Danil,
    Are you referring to fractional Brownian motion?
    Ernie

    ReplyDelete
  11. I refer to the article of this guy http://mechanicalforex.com/2016/03/the-hurst-exponent-and-forex-trading-instruments.html

    ReplyDelete
  12. Hi there, Dr Chan,

    Have you considered using this measurement of "momentum or MR indicator" as possible half life for cointegrated residues?
    Or, have you/do you use the Hurst exponent in cointegrated pairs or residues in order to further filter them as "more" mean reversal if they show an H<.5?

    Thanks

    ReplyDelete
  13. Hi Eduardo,
    No, the transition time scale does not indicate how long it would take for a price series to mean-revert. For that, you have to use Ornstein-Uhlenbeck equation.

    As we approach the transition time from the mean reverting region, we should expect the half life to approach infinity.

    But yes, the lower the Hurst exponent, the shorter the half life of mean reversion.

    Ernie

    ReplyDelete
    Replies
    1. Thanks for the answer, Dr.!
      I too use the OU equation for HL of Mr pairs trading.

      What I tried to imply was that, maybe the transition time scale is an additional filter for the filtering of possible pairs in the half life part. Like, if a pair has a HL (OU Calculated) of 20 days, but the transition time scale shows that in approximately 10 days there is a shift from MR to Momentum, maybe one should not hold the MR trade in the pair/etf/etc for more than the latter, in this case, 10 days.

      I may be wrong, just thinking out loud here

      Thanks

      Delete
  14. Hi Ernie,

    No matter intraday or swing trades, is that pair or more than pair kind of mean reverse strategies would need much smaller chance to cut lose than momentum? Seem like even the best momentum traders would cut lose 4 out of 10 times which is mentally hard to cut lose that often.

    -HK

    ReplyDelete
  15. Hi HK,
    I do not recommend stop loss for mean reversion strategy in general, unless it is never expected to be triggered. See last chapter of my second book Algorithmic Trading.
    Ernie

    ReplyDelete
  16. Hi Eduardo,
    That's a good point. Yes, you can use the transition time as an upper limit to your holding period.
    Ernie

    ReplyDelete

  17. Hi there, Great blog you have there, really. I learned so much from your posts already so please juse keep up the good work! :)

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. Great article! Very helpful information. I was trying to reproduce the SPY interday plot. I used log daily price of SPY. But seems I get a different result. For example, the first data point, 1 day lag, the log variance I got is -9.61, but your plot shows should be around -8.9. Am I missing anything? Thanks

    ReplyDelete
  20. Hi Ernie,

    Have you tried Digital Signal Processing (DSP) trading strategies?

    Are they profitable?

    Thanks.

    ReplyDelete
    Replies
    1. Also interested in hearing Ernie's opinion on this..

      (the financial hacker blog / Zorro platform lead programmer's blog talks extensively about this subject, very interesting)

      Best

      Delete
    2. This comment has been removed by the author.

      Delete
  21. Hi Dawei,
    Thanks for your kind words.

    Did you use consolidated close price for your daily plot? If so, they always exaggerate mean reversion, due to noise (see my article on Beware of Low Frequency Data, April 2015). I use midprice for my intraday plots.

    Also, I don't know if you use the same data period as my 20130116-20160115 for SPY?

    In any case, one data point isn't that important. What's important is the Hurst exponent, which averages out the noise.

    Ernie

    ReplyDelete
  22. Hi Eduardo,
    Many techniques in trading can be considered Digital Signal Processing. E.g. Kalman filter, wavelets, etc. Some of them are useful, others are not. So one needs to be specific about the technique.
    Ernie

    ReplyDelete
    Replies
    1. Sorry, I was referring specifically to high/low bypass filters and the very well documented FFT.

      Here's the link from another blog http://www.financial-hacker.com/build-better-strategies-part-2-model-based-systems/

      Delete
  23. Hi Ernie,

    Have you read books (Digital Signal Processing) written by John Ehlers?

    http://www.mesasoftware.com/ehlers_books.htm

    He mentioned Sinewave indicator in his books.

    Thanks.

    ReplyDelete
  24. No, I haven't read John Ehlers.
    Do these indicators work for you?
    Ernie

    ReplyDelete
  25. Hi Ernie,

    His books provide codes for Tradestation and Multicharts.

    He also sells his codes on his website.

    I just begin to read his books and test his codes.

    Basically, he said his indicators can tell Trend mode and Cycle mode.

    ReplyDelete
  26. Hi Eduardo,
    Thanks for the link.

    No, I haven't tried hi/lo pass filters or FFT in trading. I have tried wavelets, but with no major gains.

    Ernie

    ReplyDelete
  27. Dr. Chan
    Thank you very much for your reply. I was using the same date period SPY data. Since I was trying to do the interday plot, so I used SPY daily price data. As you suggested, I changed to midprice. (I used 0.5*(High+Low)). But the hurst exponent I got is only about 0.387.

    ReplyDelete
  28. Hi Dawei,
    Midprice does not mean the mid of high and low. It means the mid of bid and ask at the market close.
    Ernie

    ReplyDelete
  29. Hi Ernie,

    May I ask what is your intraday trends strategy of E-mini S&P 500 futures?

    Thanks.

    ReplyDelete
  30. Please feel free to email me to discuss that.

    Ernie

    ReplyDelete
  31. Hi Ernie,
    I plotted the autorcorrelation of daily returns of USO and found a statistically significant peak at a lag of 1 day. However the value is negative indicating mean reversion. So I tested two simple strategies to confirm this daily MR behavior and the difference in equity curves confirms this small daily MR behavior. Not sure how to connect autocorrelation of returns with the Hurst parameter and your results.
    Thanks,

    Steve

    ReplyDelete
  32. Hi Steven,
    Your result is indeed contrary to that from Hurst exponent.
    However, as an additional test, I would suggest you test using the midprice of the bid-ask at market close, not the consolidated close.

    As I wrote before on this blog, consolidated closes have a tendency to reveal false mean reversion that nobody can trade on.

    Ernie

    ReplyDelete
  33. Hi Ernie,

    Do you trade long/short equity strategies?


    Thanks.


    ReplyDelete
  34. Hi Ernie,

    What kinds of factors do you use in Long/Short equity strategies?

    Thanks.


    ReplyDelete
  35. Hi Ernie,

    Why don't you use fundamental factors, such as ROE?

    For lagged returns, do you use PCA?

    Thanks.

    ReplyDelete
  36. We prefer short term trading, hence fundamental factors are not of much help.

    We don't use PCA currently, but it is under active research.

    Ernie

    ReplyDelete
  37. Hi Ernie,

    May I ask how long is "short-term" trading in your definition and strategies?

    Thanks.

    ReplyDelete
  38. Hi Ernie,

    Do we usually include intercept when we run linear regression?

    Thanks.

    ReplyDelete
  39. Yes, including intercept is usually recommended, unless you have a fundamental reason not to.
    Ernie

    ReplyDelete
  40. Hi Ernie,

    For stocks pairs trading, do we need to include intercept when we run rolling linear regression?

    Thanks.

    ReplyDelete
  41. It depends on the exact pair. But generally I won't.
    Ernie

    ReplyDelete
  42. Hi Ernie,

    Many thanks for a great blog!

    I'm looking for an FX broker to trade a G10 strategy. Do you think IB is the best broker for an account of around $100k? What other alternatives should I look at? Do you have a good set up to recommend for low latency?

    Many thanks

    ReplyDelete
  43. Yes, IB is as good as any in terms of FX for accounts <= $100K.

    For low latency connection, ask Sam at speedytradingservers.com.

    Ernie

    ReplyDelete
  44. Just curious, what options would you look at for accounts >$100K?

    ReplyDelete
  45. If you are an Eligible Contract Participant, as defined by CFTC, then you can open any prime broker account, and access any FX ECN directly such as HotspotFX, LMAX, EBS, etc.

    Ernie

    ReplyDelete
  46. Hi Ernie,

    What is the reasonable transaction cost assumption (including commissions, spread, and slippage) for S&P 500 stocks?

    IB is the broker. Thanks.

    ReplyDelete
  47. We typically assume 5bps one way transaction cost for SP500 stocks.

    Ernie

    ReplyDelete
  48. Hi Ernie,

    It seems intraday long-short mean reverting strategy does notwork for SP500 stocks recently.

    Is that right? Thanks.

    ReplyDelete
  49. I agree it is hard to make long-short stock strategies work this year (see hedge fund reports also on this category).

    Ernie

    ReplyDelete
  50. Hi Dr,

    Barging in this subject about SP500 and plain vanilla LS..

    Have you ever tried to pair/triplet trade them in a very short term? Not HFT sub ms, us, but minutes wise, instead of EOD.

    I had some success in doing so in 2014 and 2015, although not in the US Market (didn't try it). Looking for really small distortions from the average mean, using real time bid and ask can get you in and out very fast, may me worthwhile if your costs are not high. I stopped doing so because as a retail trader my discounts were not big enough to make it worthwhile, gross profit was real, nevertheless.

    ReplyDelete
  51. Hi Eduardo,
    No, we haven't tried day trading stocks pairs like you suggested. I agree there may still be opportunities there.
    Ernie

    ReplyDelete
  52. Hi Ernie,

    Do you know why it is hard to make long-short stock strategies work this year?

    How do we deal with it? Thanks.

    ReplyDelete
  53. Typically, long-short strategies depend on volatility to earn returns. Volatility in the stock market has been very low in the last few months.

    You can always run a short volatility strategy in this market condition.

    Ernie

    ReplyDelete
  54. Hi Ernie,

    Thank you for quick response!

    May I ask what are short volatility strategies you would recommend?

    Thanks.

    ReplyDelete
  55. See the VX strategy discussed around Figure 5.12 in my book Algorithmic Trading.

    ERnie

    ReplyDelete
  56. Hi Ernie,

    I find that intraday long-short mean reverting strategy does not work for SP500 stocks since May 2010.

    Thanks.

    ReplyDelete
  57. You have amazing explanations of price actions. anyone who wants to learn forex trading should look at your stuff which has helped me tremendously with my trading.

    ReplyDelete
  58. Hi Ernie,

    What is the reasonable assumption of transaction costs for Russell 2000 stocks?

    Thanks.

    ReplyDelete
  59. Hi Ernie,

    Do you trade only intraday strategies?
    Are the capacity of intraday strategies limited?

    Thanks.

    ReplyDelete
  60. We trade mostly intraday strategies, because of their higher statistical significance and lower risk. Yes, they do have lower capacity, but then we don't have billions to manage at this point. We are, however, working on strategies with longer holding period and higher capacity, and will be able to launch soon.

    Ernie

    ReplyDelete
  61. Hi Ernie,

    Do you trade Russell 2000?

    Thanks.

    ReplyDelete
  62. Hi Ernie,

    Thank you for quick response.

    Is it because of ill-liquid for small cap stocks?

    Thanks.

    ReplyDelete
  63. Hi Ernie,
    If I understand it correctly from your second book p. 45, the Hurst Exponent can be between +1/-1. When 0.50 is random walk, it strikes be as bold to claim that H = 0.56 is STRONGLY trending as you do in the above article. I would have guessed that strongly trending would be H = 0.8 or thereabouts??

    Chis.

    ReplyDelete
  64. Hi Chris,
    Actually H is between 0 and 1. It isn't realistic to have negative H, because that would imply prices remain constant over the long term.

    Whether a price series is trending or not depends on whether it is statistically significantly greater than 0.5. Some of the price series I noted in the article passed this significance test by a good margin. However, I failed to find the adjective "strongly" mentioned in my article above. Can you please point out the sentence?

    Ernie

    ReplyDelete
  65. Sorry, between 0 and 1, my bad.

    I was referring to the following you wrote: "We can do the same analysis for USO (the WTI crude oil futures ETF). The intraday H is 0.515±0.001, indicating significant trending behavior. The daily H is 0.56±0.02, even more significantly trending."

    Chris

    ReplyDelete
  66. Hi Chris,
    Yes, I use the word "significant" in a specific sense. It means that it is more than 2 standard deviations away from the mean.
    Ernie

    ReplyDelete
  67. Chris,
    The significance testing in this specific context is to see if the Hurst exponent for random data of the same size will have the same value as what we obtained. The conclusion is that if we assume Gaussian distribution of such values, the chance that this happens is less than 2.5%. Hence with better than 97.5% probability this is a trending price series.

    See also p. 16 of my second book, section on Statistical Significance of Backtesting: Hypothesis Testing.

    Ernie

    ReplyDelete
  68. Hi Ernie,

    I was trying to replicate your results and noticed that because of you are taking logarithm from time based on 2, i.e. log2(Time), you should also take log2 from variance, i.e. log2(Var). Please correct me if I'm wrong. You didn't make this explicit, so I was straggling a bit. Thank you for the great topic.

    Pavel

    ReplyDelete
  69. Hi Pavel,
    The log2(t) on the x-axis is for display purposes only. In my actual linear regression, I have taken the natural log of both variance and timescale.
    Hope this helps.
    Ernie

    ReplyDelete
  70. Hi Ernie

    I have a question regarding this formula. I am aiming to replicate your study:

    volatility(τ)=√(Var(z(t)-z(t-τ)))

    z = log price
    τ = time interval?
    t = what plugs into t ?

    if i understand the process is below:
    Take variance over many sample times, in this case... mid price of 1 minute to 2^10 (1024 minutes or 17 hours) for intraday

    Once we do the above calculation, we need to multiply the daily volatility by √252? to obtain annualized volatility? or is that not needed?

    for the plot:

    plot the log(Var(τ)) against log(τ)

    is this a rolling window across a start date to end date in your example:
    2013-01-06 to 2016-01-15

    for τ

    would i take the width of the window, lets say for intraday.. 1 minute to 2^10. And i slide that width of window up to the end date. So a rolling variance you could say?


    Or do i just take to and from and work out plot the log(Var(τ)) against log(τ)?

    If that is the case, then how would i plug in the time periods into τ??

    It would be log variance of mid price from 1 minute to 2^10 2013-01-06 to 2016-01-15 to enter in log(Var(τ))... then against log(τ).. what would I enter in here?

    Let me know if you can perhaps make it in simple terms for someone like me :)

    Thanks a lot!
    Andrew



    ReplyDelete
  71. Hi Andrew,
    No, you do not annualize the volatility in this study. The whole point of the exercise is that we should not assume a Gaussian diffusion process for the log prices. I.e. Hurst exponent is not necessarily 0.5.

    Yes, the window for computation is entire data set.

    For any given time t taken from the data set, the time bar for the computation of log return is tau. I don't want to call it a "window", since it is just a bar (1 minute? 1 hour?) So it is just log(price(t+tau))-log(price(t)). You will have as many data points as the number of t in your data set. You will compute the variance of these data points (log returns). For different tau, you get different variances. These different variances vs tau form the plot.

    Ernie

    ReplyDelete
  72. Ok so for each day I would , lets say mid prices of 1 minutes to 2^10 minutes (1024 minutes)
    Lets say start date of: 2013-01-06

    1. Compute log returns from 1 minute to 1024 minutes ( so 1 minute bar for each incremental step to 1024 minutes)
    2. Compute the variance of those returns
    3. Subtract the variance of those returns - the log returns?

    How does one form the plot especially over date range: 2013-01-06 to 2016-01-15

    We are plotting the variance of log returns against the log returns right?

    Just struggling with how to structure it and how 1 minute to 1024 minutes relate over the 'n' range period.


    OR

    for the first 1 minute bar to the last 1024 1 minute bar...
    we do the log returns and the variance of the log returns...

    From date range: 2013-01-06 to 2016-01-15

    log(Var(1min bar)) - log(1min bar)
    We do that for every bar... first 1 minute bar up to the last 1024 minute bar....

    and the result of that... is what we plot?

    I can get this, just need a lil more 'dumbing down' again!

    I do see the value in it, if i could view the markets nature in this way, it means could 'fit' a model suited to exploit the characteristics of the that market. At least that's the initial thinking


    ReplyDelete
  73. Andrew,
    Your first scheme is closer to the way.

    You don't just compute the returns on one day. You should, for e.g., compute the 1-min log returns on all days, and then compute the variance of them. There is also no need for 3): no need to subtract anything from the variance.

    You have now a set of tau (1-min, 1-hour, 1-day, etc.), and a corresponding set of variances. Plot the log of those variances against the log of tau.

    Ernie

    ReplyDelete
  74. Hi Ernie,

    Great blog - I'm a fan of your books as well. I was wondering for some clarification on this part of your post:

    If we annualize the volatility of a mean-reverting price series, it will end up having a lower annualized volatility than that of a geometric random walk, even if both have exactly the same volatility measured at, say, 5-min bars. The opposite is true for a trending price series.

    Are you saying that when returns are negatively autocorrelated (mean reverting time series), if you take the standard deviation as a measure of volatility then you will underestimate the "true" standard deviation of the process? And vice versa for positively autocorrelated (trending) returns?

    This makes sense intuitively, but I ran some simulations in R and am not finding results that match with my intuition. We can simulate an AR(1) model in r with positive and negative autocorrelation:

    > sd(arima.sim(model=list(ar=.5),sd=1,n=1000))
    > sd(arima.sim(model=list(ar=-.5),sd=1,n=1000))

    and it turns out both of these actually overestimate the true standard deviation (which is 1). Does this match with your intuition? Or am I examining a different phenomenon

    Also, if you look at the wikipedia article here

    https://en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation#Effect_of_autocorrelation_.28serial_correlation.29

    it seems like from the equation stated at the top - having a negative autocorrelation parameter would lead to sample variance overestimating the true variance and vice versa - which I think is opposite to your statement about mean reverting/trend following variance.

    ReplyDelete
  75. Hi Cherkassky,
    Thank you for your kind words.

    There is no "true" volatility. Any volatility measurement is a function of the time scale it is measured. You should use the volatility for the time scale of your trading strategy.

    In particular, I was not comparing sample (unconditional) variance vs the conditional variance (1 in your example) of an ARIMA process. I was measuring the unconditional variance at various time scales.

    Hope this clarifies.

    Ernie

    ReplyDelete
  76. Hi Ernie

    I feel I was able to create the hurst exponent by your procedure:

    https://flare9xblog.wordpress.com/2017/09/24/modelling-the-hurst-exponent/

    ReplyDelete
  77. Hi Ernie!

    The literature says rescaled range (R/S) (and therefore Hurst…) is a measure for “long-term dependence” or long memory. Lo (1991) then modified the test to be robust to short-term dependence.

    Do you know if there is an argument against using R/S or Hurst on a short-horizon (i.e. why must Hurst describe long-term and not short-term dependence?)

    Thank you!

    Felix

    ReplyDelete
  78. Hi Felix,
    Such diffusive processes typically exhibit power-low dependence only when diffusion is a bit far away from the original (long-horizon). In the short term, there are other complicated dynamics at work requiring complicated functions to describe the time dependence of their distance from the origin.
    Ernie

    ReplyDelete
  79. I would greatly appreciate if you comment upon following strategy--
    THIS METHOD INVOLVES TRADING WITH THE TREND WITH ENTRY ON INTRADAY MEAN REVERSION REVERSALS ONLY WHEN THERE IS INTRADAY SPIKES IN VOLATILITY.TRADES LAST NO MORE THAN FEW DAYS.HIGH TURNOVER HELPS WITH STATISTICAL EDGE WITH IN BROWNIAN MOVEMENTS. Daily charts are used to determine market direction.
    There are days when volatility increases intraday— but we just do not know when. Those are the days when mean reversion has higher chance of making $$.
    This method works better in cross currencies which have better & longer trends compared to pairs which have U.S. Dollar in the pair. To control draw downs we only trade in the direction of the trend. Let us say for discussion that british pound cross currencies are trading up (GBP/JPY, GBP/AUD, GBP/CHF ect)
    Daily pip range for GBP/AUD presently is 140 pips(as of April 14,2018)
    Over thousands of runs one can state that GBP/AUD has daily range of 140 pips—70 pips up & 70 pips down from opening price. But on any given day anything can happen. Since trend is up. We only trade long.
    Everyday around 5PM EST ,we put order to buy GBP/AUD as limit order 100 pips below opening price with take profit 50 pips & stop loss of 50 pips on the same ticket. We let the market close the position either at profit or at a loss—no human intervention. Only one open order in each cross currency at any given time. This distance of 100 pips can be adjusted up/down based on volatility, balance in the account & number of open positions already in account.
    Next day again new orders are entered based on the then current prices in different cross pairs. Unfilled orders get cancelled at the end of every day. Similar orders are put in daily in several cross pairs to spread the risk.
    Since daily pip range for GBP/Aud is 70 pips up or down, volatility has to increase before buy order gets executed—and that is the day when mean reversion has higher chance of making money.
    Since buy limit order & stop loss are on the same side of market price, chances are higher for buy order to get executed compared to stop loss getting hit. Every thing is on the same ticket—so when take profit is hit, stop loss order no longer exists.
    How much more often market would travel distance of 100 pips (D1)compared to distance of 150 pips(D2) in the same time interval(t) , can be roughly computed as follows based on random motion( distance travelled is proportional to square root of time intervals— option formulas)
    150 X 150/100 X 100 which is—
    3x3/2x2 =2.25 . This means out of 3.25 runs we win 2.25 times & lose one time & THAT IS THE MATHEMAICAL EDGE. Trading with the trend gives additional mathematical edge above & beyond this number which cannot be computed but is common sense..
    We already know that in up market, corrective moves are shallow & that controls draw downs during that day—risk control is there on every ticket order with stop loss. No playing around with any ticket—win or lose. Trading with the trend gives additional mathematical edge above & beyond this number. Each position lasts only couple of days.There is also built in stability if trader mixes up good number of cross currencies. If Australian dollar goes up, pairs where Australian dollar is second entity(GBP/AUD,EUR/AUD) go down but pairs where Australian dollar is first Item go up(AUD/CAD,AUD/CHF,AUD/NZD).This balances fluctuations in equity/draw downs.
    Similar other orders are entered in other cross currencies in the direction of the trend of that pair. If trend is not clear, no order in that pair & select another pair with trend. No orders in pairs with U.S. dollar in the pair.
    I thank you for all your help.
    Respectfully Submitted,
    Prem Nath M.D.
    Email INDUS68@GMAIL.COM

    ReplyDelete
  80. Hi Dr. !
    How did you chose your "lag" parameter (as mentioned in your books "ALGORITHMIC TRADING") ? I found that if the lag is too large , the plot will not give us a "good line". Does it remains relevant ?. (Usually lag ~ 20 ? but why ?).

    Moreover, I try to simulate, Random Walk and Trending process but I can't find it throught the Hurst Exponent Test. It seems working only for mean-reverting process..

    Thank you!
    Antoine

    ReplyDelete
  81. Hi Dr. !
    How did you chose your "lag" parameter (as mentioned in your books "ALGORITHMIC TRADING") ? I found that if the lag is too large , the plot will not give us a "good line". Does it remains relevant ?. (Usually lag ~ 20 ? but why ?).

    Moreover, I try to simulate, Random Walk and Trending process but I can't find it throught the Hurst Exponent Test. It seems working only for mean-reverting process..

    Thank you!
    Antoine

    ReplyDelete
  82. Hi Antoine,
    Can you specify the lag in which strategy (which page in my 2nd book)? Generally speaking, I pick lag not by optimization, but by "common sense". 20 days ~ 1 month for a daily strategy. Others may pick 3 months. But anything shorter than 1 month or longer than 3 months are not as reasonable.

    If you simulate a geometric random walk, you should find that the Hurst exponent is not statistically different from 1/2. Is that what you meant by "can't find"?
    Ernie

    ReplyDelete
  83. "Example 2.2: Computing the Hurst Exponent, p.45" , you take the USD.CAD price series and compute the Hurst Exponent by using "genhurst" from MATHLAB (default time lag = 20).

    But in this post, In the daily SPY plot, the xaxis goes up to 2^8 (256 days).

    When I'm trying to find the same result as yours (H of 0.49 for the USD.CAD price series), the value of the Hurst Exponent can differ from 0.53 (Trending) to 0.48 (MR) for a time lag ranging from 15 to 25.

    My first question was : Is there a minimum or maximum "time lag" according to daily prices series ? (you have answered it before) but why a large time lag (ex. +100) could be not significant ?


    I apologize in advance for my poor English
    Thank you!
    Antoine

    ReplyDelete
  84. My second question is :

    When I'm simulating a Random Walk process ( X(t) = X(t-1) + N(0,1), t=1...1000), the Hurst Exponent calculation can give me value from H=0.47 to H=0.55 (max time lag = 20).

    - Is there an interval in which we can considerate that we have a random walk (ex: H=0.48 - 0.52) ?
    - A large max time lag >500 (with 1000 points) no longer give us "a line" in the plot (log(var);log2(T)), the slope of the curve flattened or reversed to a certain time lag value. Does this value make sense ?


    Thank you!
    Antoine

    ReplyDelete
  85. Hi Antoine,
    The method I used to calculate Hurst in the book differs from the method here. In the book method, I assumed that the Hurst is time-scale-independent. Here on this blog, I explicitly assume that it depends on time scale. Hence we need to compute it at different frequencies (lag). Using the blog method has the additional advantage that we can estimate the error bars associated with the Hurst estimation. So whether 0.48 or 0.53 for USDCAD is really the same or not can be answered by finding out whether they lie within statistical error.
    Ernie

    ReplyDelete
  86. Hi Ernie
    I'm trying to find how you compute the confidence interval (it looks very small) .. Do you use the bootstrapping method ?

    Tank you
    Antoine

    ReplyDelete
  87. Hi Antoine,
    The confidence interval in the slope was found using Matlab's least square fit function.
    Ernie

    ReplyDelete
  88. Hi Ernie,

    Thanks for sharing your knowledge and insights, I am new to Quant domain and doing a career switch from Human Resource to Quant ( drastic :) ) ,

    I am confused and puzzled about a problem I am facing. I ran ADF test and Hurst test for the same price series of a commodity Futures.

    The ADF test result is clearly showing mean reversion and the Hurst test showing it is momentum driven. I am using daily close price and test for 10 years.

    Would you or anyone kind enough to explain in what situation where one can have ADF test and Hurst test say different things?

    Thanks in advance!

    ReplyDelete
  89. Hi Swimtwitter,
    Did you apply the variance ratio test on your Hurst exponent to check that it is statistically significant?
    Ernie

    ReplyDelete
  90. Wow, that was quick and didnt really expect a reply that quick from the giant of quant :)

    ~~~
    Augmented DF test for unit root variable: variable 1
    ADF t-statistic # of lags AR(1) estimate
    -4.178511 1 0.996409

    1% Crit Value 5% Crit Value 10% Crit Value
    -3.458 -2.871 -2.594
    ~~~

    Hurst test
    H = 0.5351

    variance ratio test
    h =
    1
    pValue = 0.0073

    pretty certain this statistically significant for variance ratio test to say that this is not random. Do you think i may have messed up with the lag somewhere?

    thanks again!

    ReplyDelete
  91. Indeed they all seem highly significant!

    So the only explanation I can think of is a bug in the implementation.

    Did you use log price for Hurst?

    ReplyDelete
  92. hey Ernie, thanks for your time again !

    I did use log price for Hurst (if the below code is what you meant )

    H=genhurst(log(y), 2);

    From intuition perspective, I been observing that particular commodity price series/chart for a while now (6 mths) and know it is a momentum driven price series, so intuitively I think the Hurst test is correct. (again I am using daily close price of 10 years, about 2000 records for the above stats results)

    However what throws me off is the ADF test which signifies it is mean reverting, and it is a very convincing stats (t stats -4.178511). The only solace is the half life is 190 days.

    I have to admit my grasp of ADF is not really strong (I just know that you don't want a unit root to exist), and I am still trying to visualise the subtlety between Mean Reversion (ADF) and Stationarity(Hurst), I re-read your Algo book on adf/hurst section multiple times and have to revisit my high school maths knowledge decades ago, lol .

    Hence my query whether there exist a scenario where one can really have a Hurst test that is Trend driven but with an ADF test that is Mean Reverting. If mathematically such scenario cannot exist, then I do think there could a bug; however, if there exist one scenario, then I think it will be interesting to understand it. :)

    Apologies for my long winded brain dump! :)

    ReplyDelete
  93. Hi Swimtwitter,
    I think it is extremely unlikely that you will find a future price series mean reverting with such high degree of significance.
    I strongly suspect there is a bug.
    Best,
    Ernie

    ReplyDelete

  94. ahh ok, that make sense, let me look into it, thanks for the pointers! :)

    ReplyDelete