tag:blogger.com,1999:blog-35364652.post8413621832481923148..comments2018-08-19T22:01:17.519-04:00Comments on Quantitative Trading: Optimizing trading strategies without overfittingErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comBlogger23125tag:blogger.com,1999:blog-35364652.post-88493655457256544302018-07-23T09:24:10.745-04:002018-07-23T09:24:10.745-04:00Hi Alex,
Even when markets are not correlated (i.e...Hi Alex,<br />Even when markets are not correlated (i.e. no first order correlation), they can still have higher order dependence (often called tail dependence). It is very hard to find contemporaneous markets that do not have tail dependence.<br /><br />I would not accept contemporaneous data as true OOS testing. However, they are OK for a quick and dirty backtest.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-28461019972019552982018-07-23T09:18:00.681-04:002018-07-23T09:18:00.681-04:00Thank you for your answer.
If we would have mark...Thank you for your answer. <br /><br />If we would have markets which are non-correlated, then you would accept them as OOS? <br /><br />Let's say for same EUR USD example I would take 10y Note and Copper. <br /><br />Idea came from simple public domain trend following models which are painfully simple, but work on many markets for decades. They just treat every market as OOS data....Alex Beehttps://www.blogger.com/profile/13809158830995204638noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-4876914685760039422018-07-23T08:00:55.317-04:002018-07-23T08:00:55.317-04:00Hi Alex,
To some extent, they are OOS. However, ma...Hi Alex,<br />To some extent, they are OOS. However, many currencies are correlated to some degree, so it isn't ideal OOS testing if they are contemporaneous.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-89837712604973456512018-07-23T07:29:55.291-04:002018-07-23T07:29:55.291-04:00Hi Ernest,
Thank you for the informative post.
C...Hi Ernest,<br /><br />Thank you for the informative post.<br /><br />Can we treat price data from other markets as out of sample data?<br /><br />For example if I am back testing EUR USD, can I use data from GBP JPY or XAU USD as a out of sample?Alex Beehttps://www.blogger.com/profile/13809158830995204638noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-27808206097929374872018-01-23T06:53:26.488-05:002018-01-23T06:53:26.488-05:00Hi Aym,
I have in fact discussed a bootstrapping p...Hi Aym,<br />I have in fact discussed a bootstrapping procedure similar to what you described in my QuantCon 2017 talk: https://twitter.com/quantopian/status/955545871348895746. That is for a multiasset strategy, and certainly improved results.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-18092746162008566142018-01-23T05:24:06.693-05:002018-01-23T05:24:06.693-05:00Hi Ernie,
Thanks for your usefull readings and as...Hi Ernie,<br /><br />Thanks for your usefull readings and as SR I'm a big fan of your work. Your method is very interesting for a mono-asset strategy. In my case, I only build and use multi-assets strategy. To avoid overfitting, I already use walkforward optimization, and bootstrap methods for resampling and generating "new time series" with the closest properties as possible in order to keep relationship between the time series. But the work is hard and often the time series properties are altered.<br /><br />Have you ever use your method for a multi-asset strategy?<br /><br />Thank you Ernie. Aymnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-56057020495505636022017-12-28T20:40:27.873-05:002017-12-28T20:40:27.873-05:00@Ernie - thanks for your comments. Above all, I a...@Ernie - thanks for your comments. Above all, I am very surprised by the simplicity of some of the strategies implemented (most are momentum based, with modifications of your examples) - and how they are managing to stay consistently in the green even in ranging markets. This is what made me question the actual economics of the trade and where the money comes from. I trade spot FX mainly, and am curious to see how long I'll be able to hold this edge. Guess I'll know/report back in a year or so!SRnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-26034796406680544622017-12-28T18:37:37.931-05:002017-12-28T18:37:37.931-05:00Hi SR,
Thanks for your kind words on my books, and...Hi SR,<br />Thanks for your kind words on my books, and great to hear you have found some good strategies!<br /><br />The market forces at work are typically either temporary liquidity demands leading to mean reversion, or event / fundamental events leading to trends.<br /><br />Yes, often arbitrage strategies are making money from less sophisticated (or slower, or less informed, or more emotional) traders. But in other times, it is merely the rewards of providing liquidity.<br /><br />If the capacity of your strategies are small, hedge funds are not interested. Capacity is proportional to the total amount of money you can make from this opportunity. Individuals may not have your sophistication to find these opportunities. Even if they have found them, they may not have enough capital to arbitrage away all profits.<br /><br />Without knowing more about your strategies, these are of course just general observations.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-65172383942645500462017-12-28T14:25:59.406-05:002017-12-28T14:25:59.406-05:00Ernie, thanks for your writing. Big fan of your bo...Ernie, thanks for your writing. Big fan of your books. Some background: I am no quant PhD, and certainly no hedge fund guy. I am just an engineer with decent programming skill.<br /><br />I've found a few profitable strategies inspired by your books and confirmed using paper trading and backtesting. Can you help me intuit how your systems work?<br /><br />- What are the market forces making these systems work?<br />- Is this effectively taking money from less sophisticated traders? Or, am I riding the wave with smart money who just don't mind taking smaller losses which are actually quite meaningful for retail traders?<br />- Why haven't hedge funds/smarter people arbitraged or sucked the alpha out of my system?<br />- From your books, it seems it might be related to 'capacity' but I am still not sure why it is actually working.<br /><br />Thanks in advance.SRnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-68480398110746731622017-12-17T11:55:44.313-05:002017-12-17T11:55:44.313-05:00Hi Laurent,
Interesting suggestion - yes, Bayesian...Hi Laurent,<br />Interesting suggestion - yes, Bayesian approach makes sense. <br />Thanks,<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-70137464776310409902017-12-17T06:54:22.954-05:002017-12-17T06:54:22.954-05:00Hi Ernie,
As usual, very interesting article, tha...Hi Ernie,<br /><br />As usual, very interesting article, thank you. Did you think about a Bayesian approach in this context? Given the highly uncertain nature of financial markets, I think Bayesian is really the way to go, except maybe if you have tons of data.<br /><br />First without talking about the underlying model, instead of picking the mode of K what would happen if you pick the expectation of K? This a nice way to avoid parameter optimization and express the result as a pseudo-Bayesian. I believe that it could add some robustness. In a full Bayesian framework, you could add parameter uncertainty in the model, or even use model combination. <br /><br />L<br /><br /><br /><br />laurent Kellerhttps://www.blogger.com/profile/05242020671887099299noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-53728146047543900992017-12-11T07:34:40.528-05:002017-12-11T07:34:40.528-05:00Good to know - thanks Andrew!Good to know - thanks Andrew!Ernie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-34923461067492551132017-12-11T00:19:29.958-05:002017-12-11T00:19:29.958-05:00Ernie
Good stuff. I have been reviewing many vol...Ernie <br /><br />Good stuff. I have been reviewing many volatility strategies recently and over fitting has been a common theme in the blogosphere. For that reason, I am very conscious moving forward with regards to optimization! Thanks for sharing the papers and your thoughts. Andrew Bannermanhttps://www.blogger.com/profile/01681759786853960574noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-18866682272809297672017-11-18T08:19:13.425-05:002017-11-18T08:19:13.425-05:00Hi Michael,
1) Theoretically, that is possible, bu...Hi Michael,<br />1) Theoretically, that is possible, but that often indicates that the time series model isn't a good fit to the underlying prices. If we ascertain that it is a good fit and it still gives bad backtest performance, we would reject the strategy. In practice, it hasn't happened yet.<br /><br />2) The simulated price series are sampled according to their likelihood of occurring in reality. Hence a unweighted average of their performance is a good estimate of their actual performance in reality. <br /><br />3) Yes, you can certainly choose to reduce DD instead of maximize Sharpe. A better metric may be the Calmar ratio which maximizes ratio of returns vs. DD.<br /><br />4) Thanks for the link! I also agree that simple strategies that work are the best, and parameter optimization is best avoided if possible.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-31787261019522331302017-11-18T04:17:01.278-05:002017-11-18T04:17:01.278-05:00Very interesting article Ernie. I have a few quest...Very interesting article Ernie. I have a few questions and comments.<br /><br />1. Will there be cases where the optimal parameters will result in negative performance in actual price series? Would you then trade the strategy regardless or reject it?<br /><br />2. Are all generated price series equally probable in the real world? If not, will that result in high Type-I error? . <br /><br />3. The max DD in your example is about 50% or even more. I realize this is an example. Will this method be effective if minimization of DD is desired?<br /><br />4. I have become skeptical of any generalizations against over-fitted strategies in recent years after discovering some simple ones that worked for several decades very well. If you find a chance, I have examples in my paper. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2810170<br /><br />Thanks Michael Harrishttp://www.priceactionlab.com/Blog/noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-60087613043614339692017-11-17T14:18:49.017-05:002017-11-17T14:18:49.017-05:00@John, I can see the similarity now - thanks.
Ern...@John, I can see the similarity now - thanks.<br /><br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-83380563090087217042017-11-17T14:02:06.191-05:002017-11-17T14:02:06.191-05:00@Ernie Perhaps it's more similar than you thin...@Ernie Perhaps it's more similar than you think. In Michaud resampling, you are estimating a model. Implicitly you are assuming the assets follow random walks with multivariate normal error (parameters mu and sigma as mean and covariance). Then you resample more mus and sigmas, optimize a portfolio for each, and then average the final portfolio weights. <br /><br />So if you took the model, resampled to get new parameters to the model, then sample a path of asset prices, you could calculate the mean and covariance at the end and input this into the optimizer. The optimizer is like whatever you would use to set up a trading strategy. <br /><br />I suppose the difference is if you are using one version of the model, whereas the resampling is like many versions of the model. So it's like you have some posterior distribution over mu and sigma.Johnhttps://www.blogger.com/profile/01457388998903348000noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-74198927348460686032017-11-17T13:47:47.064-05:002017-11-17T13:47:47.064-05:00Hi Jonathan,
Yes, you pointed out some very valid ...Hi Jonathan,<br />Yes, you pointed out some very valid limitations on this approach.<br />Thanks for describing how you approached the problem with equities strategies! It does make sense in that context.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-69662165255136116042017-11-17T13:37:19.278-05:002017-11-17T13:37:19.278-05:00Very nice idea. Overfitting is indeed a big probl...Very nice idea. Overfitting is indeed a big problem for strategy development. A potential issue in using this is how well one can model he underlying price / volume processes. Depending on what one's signal dependent on, the process may either not express the pattern or may have a different outcome then realized on average in the market.<br /><br />One thing I have done in creating more data for optimizing equity strategies has been to do the following:<br /><br />- normalize equity bars (by mean/ sd)<br />- cluster equities into groups by some similarity measures<br />- within each group, evaluate signal on the combined histories of equities in the group<br /><br />The MC / model-based approach is very appealing however, as one can generate even larger amounts of data. I'll have to give this approach a shot with equities and see how it does.Jonathan Shorehttps://www.blogger.com/profile/08858453641392256663noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-32648422742888796252017-11-17T09:10:54.732-05:002017-11-17T09:10:54.732-05:00John,
It isn't really resampling, since resamp...John,<br />It isn't really resampling, since resampling means we use real historical data to generate more historical data. Here, we merely use the model that describes the historical data to generate more historical data.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-8602393632525843722017-11-17T09:09:54.799-05:002017-11-17T09:09:54.799-05:00Anon,
You are welcome to email me at ernest@epchan...Anon,<br />You are welcome to email me at ernest@epchan.com for source codes.<br />ErnieErnie Chanhttps://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-12656760060552593872017-11-17T08:47:13.584-05:002017-11-17T08:47:13.584-05:00Interesting post. I see this as basically the same...Interesting post. I see this as basically the same thing as portfolio resampling, but applied to trading instead of portfolio optimization.Johnhttps://www.blogger.com/profile/01457388998903348000noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-86263905826235399452017-11-17T08:46:45.289-05:002017-11-17T08:46:45.289-05:00Could you include the Matlab for this post?Could you include the Matlab for this post?Anonymousnoreply@blogger.com