tag:blogger.com,1999:blog-35364652.post709974332562319000..comments2024-04-22T13:29:20.051-04:00Comments on Quantitative Trading: Mean reversion, momentum, and volatility term structure Ernie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comBlogger111125tag:blogger.com,1999:blog-35364652.post-35483116069434756282020-11-20T10:05:35.575-05:002020-11-20T10:05:35.575-05:00
ahh ok, that make sense, let me look into it, th...<br />ahh ok, that make sense, let me look into it, thanks for the pointers! :)Swimtwitterhttps://www.blogger.com/profile/00944750757272278566noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-69165908576871025442020-11-19T14:07:43.139-05:002020-11-19T14:07:43.139-05:00Hi Swimtwitter,
I think it is extremely unlikely t...Hi Swimtwitter,<br />I think it is extremely unlikely that you will find a future price series mean reverting with such high degree of significance.<br />I strongly suspect there is a bug.<br />Best,<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-41427813062701235112020-11-19T11:25:41.212-05:002020-11-19T11:25:41.212-05:00hey Ernie, thanks for your time again !
I did us...hey Ernie, thanks for your time again ! <br /><br />I did use log price for Hurst (if the below code is what you meant ) <br /><br />H=genhurst(log(y), 2);<br /><br />From intuition perspective, I been observing that particular commodity price series/chart for a while now (6 mths) and know it is a momentum driven price series, so intuitively I think the Hurst test is correct. (again I am using daily close price of 10 years, about 2000 records for the above stats results)<br /><br />However what throws me off is the ADF test which signifies it is mean reverting, and it is a very convincing stats (t stats -4.178511). The only solace is the half life is 190 days. <br /><br />I have to admit my grasp of ADF is not really strong (I just know that you don't want a unit root to exist), and I am still trying to visualise the subtlety between Mean Reversion (ADF) and Stationarity(Hurst), I re-read your Algo book on adf/hurst section multiple times and have to revisit my high school maths knowledge decades ago, lol . <br /><br />Hence my query whether there exist a scenario where one can really have a Hurst test that is Trend driven but with an ADF test that is Mean Reverting. If mathematically such scenario cannot exist, then I do think there could a bug; however, if there exist one scenario, then I think it will be interesting to understand it. :)<br /><br />Apologies for my long winded brain dump! :)<br />Swimtwitterhttps://www.blogger.com/profile/00944750757272278566noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-63289967016326682392020-11-18T15:56:11.018-05:002020-11-18T15:56:11.018-05:00Indeed they all seem highly significant!
So the o...Indeed they all seem highly significant!<br /><br />So the only explanation I can think of is a bug in the implementation. <br /><br />Did you use log price for Hurst?<br /><br />Ernie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-59942658365240672242020-11-18T10:46:11.320-05:002020-11-18T10:46:11.320-05:00Wow, that was quick and didnt really expect a repl...Wow, that was quick and didnt really expect a reply that quick from the giant of quant :) <br /><br />~~~<br />Augmented DF test for unit root variable: variable 1 <br /> ADF t-statistic # of lags AR(1) estimate <br /> -4.178511 1 0.996409 <br /><br /> 1% Crit Value 5% Crit Value 10% Crit Value <br /> -3.458 -2.871 -2.594 <br />~~~<br /><br />Hurst test<br />H = 0.5351<br /><br />variance ratio test<br />h = <br /> 1<br />pValue = 0.0073<br /><br />pretty certain this statistically significant for variance ratio test to say that this is not random. Do you think i may have messed up with the lag somewhere?<br /><br />thanks again!Swimtwitterhttps://www.blogger.com/profile/00944750757272278566noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-7473180494454990212020-11-18T08:59:05.394-05:002020-11-18T08:59:05.394-05:00Hi Swimtwitter,
Did you apply the variance ratio t...Hi Swimtwitter,<br />Did you apply the variance ratio test on your Hurst exponent to check that it is statistically significant?<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-58690866753049151062020-11-18T08:39:11.910-05:002020-11-18T08:39:11.910-05:00Hi Ernie,
Thanks for sharing your knowledge and...Hi Ernie, <br /><br /> Thanks for sharing your knowledge and insights, I am new to Quant domain and doing a career switch from Human Resource to Quant ( drastic :) ) , <br /><br /> I am confused and puzzled about a problem I am facing. I ran ADF test and Hurst test for the same price series of a commodity Futures. <br /><br />The ADF test result is clearly showing mean reversion and the Hurst test showing it is momentum driven. I am using daily close price and test for 10 years. <br /><br /> Would you or anyone kind enough to explain in what situation where one can have ADF test and Hurst test say different things? <br /><br />Thanks in advance!<br /><br />Swimtwitterhttps://www.blogger.com/profile/00944750757272278566noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-48943216660709634592019-06-20T14:16:25.798-04:002019-06-20T14:16:25.798-04:00Hi Antoine,
The confidence interval in the slope w...Hi Antoine,<br />The confidence interval in the slope was found using Matlab's least square fit function.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-37429745759248635682019-06-20T11:03:42.436-04:002019-06-20T11:03:42.436-04:00Hi Ernie
I'm trying to find how you compute t...Hi Ernie <br />I'm trying to find how you compute the confidence interval (it looks very small) .. Do you use the bootstrapping method ? <br /><br />Tank you <br />AntoineAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-91520121941179150342019-06-18T09:55:17.275-04:002019-06-18T09:55:17.275-04:00Hi Antoine,
The method I used to calculate Hurst i...Hi Antoine,<br />The method I used to calculate Hurst in the book differs from the method here. In the book method, I assumed that the Hurst is time-scale-independent. Here on this blog, I explicitly assume that it depends on time scale. Hence we need to compute it at different frequencies (lag). Using the blog method has the additional advantage that we can estimate the error bars associated with the Hurst estimation. So whether 0.48 or 0.53 for USDCAD is really the same or not can be answered by finding out whether they lie within statistical error.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-44641671300003204892019-06-18T09:36:20.929-04:002019-06-18T09:36:20.929-04:00My second question is :
When I'm simulating ...My second question is : <br /><br />When I'm simulating a Random Walk process ( X(t) = X(t-1) + N(0,1), t=1...1000), the Hurst Exponent calculation can give me value from H=0.47 to H=0.55 (max time lag = 20). <br /><br /> - Is there an interval in which we can considerate that we have a random walk (ex: H=0.48 - 0.52) ? <br /> - A large max time lag >500 (with 1000 points) no longer give us "a line" in the plot (log(var);log2(T)), the slope of the curve flattened or reversed to a certain time lag value. Does this value make sense ?<br /><br /><br />Thank you!<br />AntoineAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-1590160778694506102019-06-18T09:05:23.993-04:002019-06-18T09:05:23.993-04:00"Example 2.2: Computing the Hurst Exponent, p..."Example 2.2: Computing the Hurst Exponent, p.45" , you take the USD.CAD price series and compute the Hurst Exponent by using "genhurst" from MATHLAB (default time lag = 20). <br /><br />But in this post, In the daily SPY plot, the xaxis goes up to 2^8 (256 days). <br /><br />When I'm trying to find the same result as yours (H of 0.49 for the USD.CAD price series), the value of the Hurst Exponent can differ from 0.53 (Trending) to 0.48 (MR) for a time lag ranging from 15 to 25.<br /><br />My first question was : Is there a minimum or maximum "time lag" according to daily prices series ? (you have answered it before) but why a large time lag (ex. +100) could be not significant ? <br /><br /><br />I apologize in advance for my poor English<br />Thank you!<br />Antoine Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-83267396653345898082019-06-18T07:50:08.222-04:002019-06-18T07:50:08.222-04:00Hi Antoine,
Can you specify the lag in which strat...Hi Antoine,<br />Can you specify the lag in which strategy (which page in my 2nd book)? Generally speaking, I pick lag not by optimization, but by "common sense". 20 days ~ 1 month for a daily strategy. Others may pick 3 months. But anything shorter than 1 month or longer than 3 months are not as reasonable.<br /><br />If you simulate a geometric random walk, you should find that the Hurst exponent is not statistically different from 1/2. Is that what you meant by "can't find"? <br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-83985930509546994852019-06-18T06:18:02.596-04:002019-06-18T06:18:02.596-04:00Hi Dr. !
How did you chose your "lag" pa...Hi Dr. !<br />How did you chose your "lag" parameter (as mentioned in your books "ALGORITHMIC TRADING") ? I found that if the lag is too large , the plot will not give us a "good line". Does it remains relevant ?. (Usually lag ~ 20 ? but why ?). <br /><br />Moreover, I try to simulate, Random Walk and Trending process but I can't find it throught the Hurst Exponent Test. It seems working only for mean-reverting process..<br /><br />Thank you!<br />AntoineAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-12641086756134706152019-06-18T06:17:22.455-04:002019-06-18T06:17:22.455-04:00Hi Dr. !
How did you chose your "lag" pa...Hi Dr. !<br />How did you chose your "lag" parameter (as mentioned in your books "ALGORITHMIC TRADING") ? I found that if the lag is too large , the plot will not give us a "good line". Does it remains relevant ?. (Usually lag ~ 20 ? but why ?). <br /><br />Moreover, I try to simulate, Random Walk and Trending process but I can't find it throught the Hurst Exponent Test. It seems working only for mean-reverting process..<br /><br />Thank you!<br />AntoineAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-72747548779570760622018-04-14T08:51:45.896-04:002018-04-14T08:51:45.896-04:00I would greatly appreciate if you comment upon fol...I would greatly appreciate if you comment upon following strategy--<br />THIS METHOD INVOLVES TRADING WITH THE TREND WITH ENTRY ON INTRADAY MEAN REVERSION REVERSALS ONLY WHEN THERE IS INTRADAY SPIKES IN VOLATILITY.TRADES LAST NO MORE THAN FEW DAYS.HIGH TURNOVER HELPS WITH STATISTICAL EDGE WITH IN BROWNIAN MOVEMENTS. Daily charts are used to determine market direction.<br />There are days when volatility increases intraday— but we just do not know when. Those are the days when mean reversion has higher chance of making $$.<br />This method works better in cross currencies which have better & longer trends compared to pairs which have U.S. Dollar in the pair. To control draw downs we only trade in the direction of the trend. Let us say for discussion that british pound cross currencies are trading up (GBP/JPY, GBP/AUD, GBP/CHF ect)<br />Daily pip range for GBP/AUD presently is 140 pips(as of April 14,2018)<br />Over thousands of runs one can state that GBP/AUD has daily range of 140 pips—70 pips up & 70 pips down from opening price. But on any given day anything can happen. Since trend is up. We only trade long.<br />Everyday around 5PM EST ,we put order to buy GBP/AUD as limit order 100 pips below opening price with take profit 50 pips & stop loss of 50 pips on the same ticket. We let the market close the position either at profit or at a loss—no human intervention. Only one open order in each cross currency at any given time. This distance of 100 pips can be adjusted up/down based on volatility, balance in the account & number of open positions already in account.<br />Next day again new orders are entered based on the then current prices in different cross pairs. Unfilled orders get cancelled at the end of every day. Similar orders are put in daily in several cross pairs to spread the risk.<br />Since daily pip range for GBP/Aud is 70 pips up or down, volatility has to increase before buy order gets executed—and that is the day when mean reversion has higher chance of making money.<br />Since buy limit order & stop loss are on the same side of market price, chances are higher for buy order to get executed compared to stop loss getting hit. Every thing is on the same ticket—so when take profit is hit, stop loss order no longer exists.<br />How much more often market would travel distance of 100 pips (D1)compared to distance of 150 pips(D2) in the same time interval(t) , can be roughly computed as follows based on random motion( distance travelled is proportional to square root of time intervals— option formulas)<br />150 X 150/100 X 100 which is—<br />3x3/2x2 =2.25 . This means out of 3.25 runs we win 2.25 times & lose one time & THAT IS THE MATHEMAICAL EDGE. Trading with the trend gives additional mathematical edge above & beyond this number which cannot be computed but is common sense..<br /> We already know that in up market, corrective moves are shallow & that controls draw downs during that day—risk control is there on every ticket order with stop loss. No playing around with any ticket—win or lose. Trading with the trend gives additional mathematical edge above & beyond this number. Each position lasts only couple of days.There is also built in stability if trader mixes up good number of cross currencies. If Australian dollar goes up, pairs where Australian dollar is second entity(GBP/AUD,EUR/AUD) go down but pairs where Australian dollar is first Item go up(AUD/CAD,AUD/CHF,AUD/NZD).This balances fluctuations in equity/draw downs.<br />Similar other orders are entered in other cross currencies in the direction of the trend of that pair. If trend is not clear, no order in that pair & select another pair with trend. No orders in pairs with U.S. dollar in the pair.<br />I thank you for all your help.<br />Respectfully Submitted,<br />Prem Nath M.D.<br />Email INDUS68@GMAIL.COM<br />pnath01https://www.blogger.com/profile/02702242379200371670noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-26465655577878134192018-03-18T16:41:21.630-04:002018-03-18T16:41:21.630-04:00Hi Felix,
Such diffusive processes typically exhib...Hi Felix,<br />Such diffusive processes typically exhibit power-low dependence only when diffusion is a bit far away from the original (long-horizon). In the short term, there are other complicated dynamics at work requiring complicated functions to describe the time dependence of their distance from the origin.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-58005662371836437692018-03-18T14:09:28.423-04:002018-03-18T14:09:28.423-04:00Hi Ernie!
The literature says rescaled range (R/S...Hi Ernie!<br /><br />The literature says rescaled range (R/S) (and therefore Hurst…) is a measure for “long-term dependence” or long memory. Lo (1991) then modified the test to be robust to short-term dependence.<br /><br />Do you know if there is an argument against using R/S or Hurst on a short-horizon (i.e. why must Hurst describe long-term and not short-term dependence?)<br /><br />Thank you!<br /><br />FelixFelixnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-7012624022470389402017-09-25T07:17:54.588-04:002017-09-25T07:17:54.588-04:00Hi Andrew,
Thanks for letting us know!
ErnieHi Andrew,<br />Thanks for letting us know!<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-74065743207523070612017-09-24T20:48:47.037-04:002017-09-24T20:48:47.037-04:00Hi Ernie
I feel I was able to create the hurst e...Hi Ernie <br /><br />I feel I was able to create the hurst exponent by your procedure: <br /><br />https://flare9xblog.wordpress.com/2017/09/24/modelling-the-hurst-exponent/<br /><br />Anonymoushttps://www.blogger.com/profile/01681759786853960574noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-31461915563035476502017-08-25T15:20:08.427-04:002017-08-25T15:20:08.427-04:00Hi Cherkassky,
Thank you for your kind words.
The...Hi Cherkassky,<br />Thank you for your kind words.<br /><br />There is no "true" volatility. Any volatility measurement is a function of the time scale it is measured. You should use the volatility for the time scale of your trading strategy.<br /><br />In particular, I was not comparing sample (unconditional) variance vs the conditional variance (1 in your example) of an ARIMA process. I was measuring the unconditional variance at various time scales.<br /><br />Hope this clarifies.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-12413174956539383302017-08-25T14:54:22.231-04:002017-08-25T14:54:22.231-04:00Hi Ernie,
Great blog - I'm a fan of your book...Hi Ernie,<br /><br />Great blog - I'm a fan of your books as well. I was wondering for some clarification on this part of your post:<br /><br />If we annualize the volatility of a mean-reverting price series, it will end up having a lower annualized volatility than that of a geometric random walk, even if both have exactly the same volatility measured at, say, 5-min bars. The opposite is true for a trending price series.<br /><br />Are you saying that when returns are negatively autocorrelated (mean reverting time series), if you take the standard deviation as a measure of volatility then you will underestimate the "true" standard deviation of the process? And vice versa for positively autocorrelated (trending) returns?<br /><br />This makes sense intuitively, but I ran some simulations in R and am not finding results that match with my intuition. We can simulate an AR(1) model in r with positive and negative autocorrelation:<br /><br />> sd(arima.sim(model=list(ar=.5),sd=1,n=1000))<br />> sd(arima.sim(model=list(ar=-.5),sd=1,n=1000))<br /><br />and it turns out both of these actually overestimate the true standard deviation (which is 1). Does this match with your intuition? Or am I examining a different phenomenon<br /><br />Also, if you look at the wikipedia article here <br /><br />https://en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation#Effect_of_autocorrelation_.28serial_correlation.29<br /><br />it seems like from the equation stated at the top - having a negative autocorrelation parameter would lead to sample variance overestimating the true variance and vice versa - which I think is opposite to your statement about mean reverting/trend following variance. <br /><br />Cherkasskyhttps://www.blogger.com/profile/02641659632312867760noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-69712921786779614792017-08-03T19:33:31.811-04:002017-08-03T19:33:31.811-04:00Andrew,
Your first scheme is closer to the way.
Y...Andrew,<br />Your first scheme is closer to the way.<br /><br />You don't just compute the returns on one day. You should, for e.g., compute the 1-min log returns on all days, and then compute the variance of them. There is also no need for 3): no need to subtract anything from the variance.<br /><br />You have now a set of tau (1-min, 1-hour, 1-day, etc.), and a corresponding set of variances. Plot the log of those variances against the log of tau. <br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-72777286058278805772017-08-03T18:39:16.151-04:002017-08-03T18:39:16.151-04:00Ok so for each day I would , lets say mid prices o...Ok so for each day I would , lets say mid prices of 1 minutes to 2^10 minutes (1024 minutes)<br />Lets say start date of: 2013-01-06<br /><br />1. Compute log returns from 1 minute to 1024 minutes ( so 1 minute bar for each incremental step to 1024 minutes)<br />2. Compute the variance of those returns <br />3. Subtract the variance of those returns - the log returns? <br /><br />How does one form the plot especially over date range: 2013-01-06 to 2016-01-15 <br /><br />We are plotting the variance of log returns against the log returns right? <br /><br />Just struggling with how to structure it and how 1 minute to 1024 minutes relate over the 'n' range period.<br /><br /><br />OR<br /><br />for the first 1 minute bar to the last 1024 1 minute bar... <br />we do the log returns and the variance of the log returns... <br /><br />From date range: 2013-01-06 to 2016-01-15 <br /><br />log(Var(1min bar)) - log(1min bar)<br />We do that for every bar... first 1 minute bar up to the last 1024 minute bar.... <br /><br />and the result of that... is what we plot? <br /><br />I can get this, just need a lil more 'dumbing down' again!<br /><br />I do see the value in it, if i could view the markets nature in this way, it means could 'fit' a model suited to exploit the characteristics of the that market. At least that's the initial thinking<br /><br /><br />Anonymoushttps://www.blogger.com/profile/01681759786853960574noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-74924881594838369902017-08-03T16:56:06.357-04:002017-08-03T16:56:06.357-04:00Hi Andrew,
No, you do not annualize the volatility...Hi Andrew,<br />No, you do not annualize the volatility in this study. The whole point of the exercise is that we should not assume a Gaussian diffusion process for the log prices. I.e. Hurst exponent is not necessarily 0.5.<br /><br />Yes, the window for computation is entire data set.<br /><br />For any given time t taken from the data set, the time bar for the computation of log return is tau. I don't want to call it a "window", since it is just a bar (1 minute? 1 hour?) So it is just log(price(t+tau))-log(price(t)). You will have as many data points as the number of t in your data set. You will compute the variance of these data points (log returns). For different tau, you get different variances. These different variances vs tau form the plot.<br /><br />Ernie<br /><br />Ernie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.com