tag:blogger.com,1999:blog-35364652.post2087571512274908786..comments2019-07-12T08:45:25.942-04:00Comments on Quantitative Trading: Beware of Low Frequency DataErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comBlogger132125tag:blogger.com,1999:blog-35364652.post-84051021925179259762018-08-21T10:05:35.101-04:002018-08-21T10:05:35.101-04:00Hi Ben,
Yes, prices on Yahoo and Google *may* diff...Hi Ben,<br />Yes, prices on Yahoo and Google *may* differ from that on Bloomberg IF you select the primary exchange prices on Bloomberg. Not all prices on all stocks differ every day, but they do occasionally for some stocks.<br /><br />This is applicable only to US exchanges. I cannot speak for other exchanges.<br /><br />5 bps is a reasonable cost estimates for S&P 500 stocks. It is difficult to measure market impact on open and close prices. I haven't backtested this strategy recently, so I don't know if it continued to perform well.<br />Ernie<br />Ernie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-71692259874720050952018-08-21T09:14:19.347-04:002018-08-21T09:14:19.347-04:00Hi Ernie,
Are you saying that open/close prices ar...Hi Ernie,<br />Are you saying that open/close prices are different on yahoo or google than they are on Bloomberg HP page? Is this just the case in the US? In your second example is 5bps an accurate assumption of trading costs, did you find there was any market impact, has it continued to perform well 3 years later? <br />Many thanks<br />BenBen Humanhttps://www.blogger.com/profile/10589814609924411774noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-65383565842047866852018-01-19T13:34:54.424-05:002018-01-19T13:34:54.424-05:00Beppe,
Yes, my total returns are an arithmetic sum...Beppe,<br />Yes, my total returns are an arithmetic sum, not a compound return. This is useful for Sharpe ratio calculation, or if the trader wants to bet a fixed amount of capital each day.<br /><br />However, even if you were to compute compound return, it isn't correct to multiply the price with the returns. They are totally different animals. Compound return formula would be just cumret=cumprod(1+retO2C)-1.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-48485781335441800252018-01-19T10:32:59.624-05:002018-01-19T10:32:59.624-05:00Believe it or not, I made the same changes before ...Believe it or not, I made the same changes before your reply!!!<br />Well...to be honest, "almost" the same: I forgot the "minus" sign. Ironically I got a far better APR (more than 20%) while Sharpe almost equal. Unfortunately I still have values too far from yours. It's clear that depends on functions and by some changes I made. No problem, I think for the moment it's more important understand the "meaning" of a strategy. So now I can go on with the book. <br /><br />Now my very last question: <br /> "retO2C=(cl-op)./op;<br /> pnl=sum(positionTable.*(retO2C), 2);<br /> ret=pnl/topN;"<br />You sum returns for every day in order to have a final column vector pnl. It's all ok but...summing percentages from different prices shouldn't require us to multiply these percentages for the opening prices? I mean, that way we have the absolute value we're really gaining/loosing. Or maybe I'm doing confusion because I don't even understand why dividing for 10.<br />Anyway, I abused of your patience so thanks again for your help. Beppenoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-30193391121704540862018-01-19T08:52:49.666-05:002018-01-19T08:52:49.666-05:00Hi Beppe,
I believe you can substitute op(t, :) &l...Hi Beppe,<br />I believe you can substitute op(t, :) < buyPrice(t, :) with retgap < -stdretC2C90d, but note the minus sign. <br /><br />The function lag is in spatial-econometrics.com which you can download for free. But lag(x) can be substituted by backshift(1, x), and backshift is a function from epchan.com/book2.<br /><br />Matlab is mostly backward compatible. If you find a piece of code doesn't work, most likely it is because you need to download some utility functions from my website or from spatial-econometrics.com.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-39918775180665052372018-01-19T08:33:01.184-05:002018-01-19T08:33:01.184-05:00Thanks for your reply!
Yes you're right about ...Thanks for your reply!<br />Yes you're right about the point 2): the moving average is computed over closing prices and not return (sorry yesterday I was clearly done with my mind!!).<br />The point "1" instead...I'm still not sure about it: Shouldn't it be: retgap < stdretC2C90d? Anyway, I'll work on it.<br /><br />The real problem is (for the previous and future strategies I'll read on the book) working with different versions of Matlab (as I can't use any longer function as "lag" etc.) but I thought I managed to change functions in a way to get close to your results. I was clearly wrong (my Sharpe Ratio is only 0,63 compared to your 1,5 and my APR 0,0048 instead of 8,7%!!!). I don't know where, but I've failed into coding. So, is there by any chance, an update version of this book's Matlab code?<br />Thank again for your attention and your patience!<br />BeppeBeppenoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-44702658945473956192018-01-18T21:04:43.244-05:002018-01-18T21:04:43.244-05:00Hi anon,
1) The buyPrice was indeed set to backshi...Hi anon,<br />1) The buyPrice was indeed set to backshift(1, lo).*(1-entryZscore*stdretC2C90d); <br /><br />But note the condition <br /> hasData=find(isfinite(retGap(t, :)) & op(t, :) < buyPrice(t, :) & op(t, :) > ma(t, :));<br /><br />So only those days with op < buyPrice will be used.<br /><br />2) Indeed "open prices to be higher than the 20-day moving average of the closing prices. "<br />and ma is computed as ma=backshift(1, smartMovingAvg(cl, lookback));<br /><br />Note ma is NOT computed as moving average of returns. It is computed as moving average of close prices.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-24456403646567047912018-01-18T11:42:26.270-05:002018-01-18T11:42:26.270-05:00Hi, I'm following your book codes (strategy &q...Hi, I'm following your book codes (strategy "Buy on Gap" pg.93-96) on Matlab but as I'm using the 2016 version I had to rearrange some of your old functions (i.e "lagmatrix" instead of "lag" and "tsmovavg" + "movstd" instead of your personal functions for moving averages and moving standard deviations").<br />Anyway...I honestly find very different results from yours even if using the very same data!<br />Another thing I noticed is it seems you're not following your own conditions: for example:<br />1) You calculate 90 days moving average for daily returns (close to close) and say that we have to consider only (today open/yesterday low) returns < than the 1 standard deviation from 90 days movaverages just computed. But...in your code you make the previous day low - 1 standard deviation. Isn't it wrong??<br /><br />2)in the second condition you say that today low must be < than 20 days mov.average of closing prices. So, why in the code I read ma= moving average of close to close returns????<br /><br />Please, I hope you can help me as I'm just a newbie matlab user and I really love to follow your work. Thanks<br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-66967012994137287222016-07-14T08:01:29.295-04:002016-07-14T08:01:29.295-04:00Hi Slav,
Yes, your idea for using a volume bar at ...Hi Slav,<br />Yes, your idea for using a volume bar at the open is certainly a good alternative of using the official open price, which often has limited size. However, even though that auction size is limited, it isn't quite limited to 100 shares. You can typically trade at least $200K to $500K order depending on the market cap of the stock without moving the auction price.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-91183058194846006122016-07-14T00:08:41.144-04:002016-07-14T00:08:41.144-04:00Ernie, I see your point on figuring out the correc...Ernie, I see your point on figuring out the correct open and closing prices in order to conduct a proper backtest. This is where I wanna make a suggestion of forgetting ohlc data, I find liquidity way more important of an issue unless you only trade 100 share or 1 contract positions. Volume based bars or tick bars which are constructed off 1min may be a little better.<br /><br />As an example you would trade 500 share bars, and if you need to execute at open you would take the worst price of the range or median price of 1000share bar range as your price of execution.(latter is better because in reality even if you tried to execute this 500 share trade you wouldn't have been able to within first bars range because that would assume you were only buyer).My point is all these back tests are pointless without tick data - period. <br /><br />slavSlavhttps://www.blogger.com/profile/00993561158019019612noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-56249956249680012942016-06-19T19:33:45.987-04:002016-06-19T19:33:45.987-04:00Hi Sqrt Alpha,
I don't exactly know what you m...Hi Sqrt Alpha,<br />I don't exactly know what you meant by order or deal matching. My instinct tells me that my article isn't about what you are thinking.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-63815066851845134432016-06-19T13:49:24.473-04:002016-06-19T13:49:24.473-04:00Hi Ernie,
It seems like a discussion on mechanism ...Hi Ernie,<br />It seems like a discussion on mechanism of order matching. Though low-frequency data like day bar or month bar we backtest, matching a deal through tick data is a necessity. Correct?Sqrt Alphahttps://www.blogger.com/profile/14496960785936577254noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-20318263012880626382016-01-14T16:30:32.105-05:002016-01-14T16:30:32.105-05:00Ernie,
thanks so much for your quick reply! Now t...Ernie, <br />thanks so much for your quick reply! Now this makes sense and the book chapter helped. <br />Nielsnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-9817621216664127742016-01-14T07:37:54.220-05:002016-01-14T07:37:54.220-05:00Hi Niels,
Any "white" data noise always ...Hi Niels,<br />Any "white" data noise always inflates the returns of a mean-reversion strategy. (See explanation on p. 10 of my book Algorithmic Trading.) In fact, white noise is the best kind of noise for mean reversion. Consolidated stock/ETF/CEF data suffers from white noise, while primary exchange data does not.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-19293331222817888472016-01-13T21:25:55.475-05:002016-01-13T21:25:55.475-05:00Ernie,
thanks for the blog and information. About...Ernie, <br />thanks for the blog and information. About the 'CEF Premium Reversion' strategy I wonder if you have an explanation why the different price changes the outcome significantly? I understand that the price is different but why does it result in a disadvantage, in particular when you trade long and short positions? <br />Thanks, NielsNielsnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-9629410108032392522015-06-16T20:17:00.280-04:002015-06-16T20:17:00.280-04:00Hi Karen,
If you have generated a vector of tradin...Hi Karen,<br />If you have generated a vector of trading signals (buys or sells), you should use actual contract prices to determine the returns based on these trading signals.<br /><br />But in generating a trading signal at time t, you can back-adjust futures for all time t' <= t to create a continuous contract. In particular, if you want to run a Johansen test on spreads of different futures, the backadjustment should involve only additions. If your signal is generated based on ratio of prices, the backadjustment should involve only multiplications.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-65716970345046702632015-06-16T16:10:07.192-04:002015-06-16T16:10:07.192-04:00Hello Erine
I'm reading your book, at Page 14...Hello Erine<br /><br />I'm reading your book, at Page 14, you mentioned of continuous future contracts for arb trading.<br /><br />You said : If your strategy generates trading signals based on the<br />price difference between two contracts, then you must choose the price<br />back-adjustment method; otherwise, the price difference may be wrong and<br />generate a wrong trading signal..... However, if your strategy generates trading signals based on the ratio of prices between two contracts, then you must choose the return<br />back-adjustment method"<br /><br />But, after reading what you replied above, saying that professional futures traders stick to actual contract to backtest, and manage the roll over to the next contract by buying/selling on the rollover day.<br /><br />I'm a little bit confused of what should be the correct method to backtest, Let say I have 3 years data of future contract of commodity A and commodity B, and the data shows that Feb and August are the active contracts for both of A and B usually. Therefore I pick 2012 Feb, 2012 Aug, 2013 Feb, 2013 Aug, and 2014 Feb, 2014 Aug for commodity A , same for commodity B for backtesting<br /><br />Based on what you mentioned above, I shouldn't do any back-adjustment of the return series from A and B before applying any ADF, johansen Test to it? Please kindly advise. Thank You<br /><br />Karen<br /><br /><br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-73536164244231195502015-06-08T15:38:25.497-04:002015-06-08T15:38:25.497-04:00If you back-adjust futures prices by eliminating t...If you back-adjust futures prices by eliminating the return gap during rollover, you will create a price gap, and vice versa. There is no way to avoid both, and so both cases involve look-ahead bias. Hence professional futures traders stick to actual contract prices.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-10843099494063849412015-06-08T15:37:09.233-04:002015-06-08T15:37:09.233-04:00Hi David,
NN requires a large amount of data to av...Hi David,<br />NN requires a large amount of data to avoid overfitting financial data. Hence it may be useful in high frequency trading.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-90240953860025816722015-06-08T14:15:05.812-04:002015-06-08T14:15:05.812-04:00Hi Ernie
I have been following your blog. I dont ...Hi Ernie<br /><br />I have been following your blog. I dont think there is a look ahead bias when using for example backwards ratio adjustment of futures. You just shift the level of the price up and down which does not matter if you work with returns.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-50573765867129895092015-06-08T12:53:37.918-04:002015-06-08T12:53:37.918-04:00Hi Ernie -
Apologies my question is not related t...Hi Ernie -<br /><br />Apologies my question is not related to your post.<br /><br />I want to hear your thoughts on neural networks. I know the quant-trading community has traditional dismissed NN as too black-box. But with recent developments in DNN, the field has furthered its understanding around network topology, alternative response functions, efficient ways to train large networks, etc.<br /><br />Do you think this could potentially be a powerful tool for recognizing patterns from the financial market? And yes what type of frequenties/asset classes would it be most suitable?<br /><br />Thanks very much,<br />Daviddavid8373https://www.blogger.com/profile/01849859512905900834noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-89576624088633435292015-06-07T17:07:27.559-04:002015-06-07T17:07:27.559-04:00Hi Sam,
Backadjustment in futures means using the ...Hi Sam,<br />Backadjustment in futures means using the future price to adjust past prices: clearly there is a look-ahead bias. The way to remove this bias is to use the actual contracts to backtest, and manage the roll over to the next contract by buying/selling on the rollover day.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-70114178600230693602015-06-05T14:06:13.637-04:002015-06-05T14:06:13.637-04:00Interesting. Could you please expand on possible s...Interesting. Could you please expand on possible sources of bias and how professionals mitigate them?<br /><br />Thanks,<br />SamuelAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-17299167341136513282015-06-05T12:20:45.401-04:002015-06-05T12:20:45.401-04:00Hi Samuel,
There is a possibility of minor look-ah...Hi Samuel,<br />There is a possibility of minor look-ahead bias with back-adjustment, which is why many professionals prefer not to use continuous adjusted futures series for backtesting. <br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-10029520980248496392015-06-05T12:07:13.675-04:002015-06-05T12:07:13.675-04:00Hi again Ernie,
Sorry for bothering you with yet ...Hi again Ernie,<br /><br />Sorry for bothering you with yet another question but I find this stuff interesting :)<br /><br />When constructing a continuous futures series, I know a popular method is backwards ratio adjustment. Just want make sure: There is no look-ahead bias arising from this right? I guess you only change the level of the entire series so returns should be unaffected. Is this correct reasoning?<br /><br />Thanks,<br />SamuelAnonymousnoreply@blogger.com