tag:blogger.com,1999:blog-35364652.post1725655651673460641..comments2017-03-28T12:04:11.392-04:00Comments on Quantitative Trading: More Data or Fewer Predictors: Which is a Better Cure for Overfitting?Ernie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comBlogger21125tag:blogger.com,1999:blog-35364652.post-30844016349499597252017-03-11T16:41:10.278-05:002017-03-11T16:41:10.278-05:00Thanks for the clarification. Thanks for the clarification. Ever Garciahttp://www.blogger.com/profile/01357178781354564617noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-53998867995554906242017-03-11T15:12:45.148-05:002017-03-11T15:12:45.148-05:00Even though each stock has new data 4 times a year...Even though each stock has new data 4 times a year, the stocks all have different earning release dates. Hence we have to compare all the stocks' fundamental numbers every day to make investment decision. <br /><br />Please see Example 2.2 in my new book for details and codes.<br /><br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-14203536660185369462017-03-11T15:10:25.711-05:002017-03-11T15:10:25.711-05:00Hi Ever,
I typically just take 5bps as transaction...Hi Ever,<br />I typically just take 5bps as transaction costs for S&P 500 stocks. It doesn't depend on your account NAV. <br />I haven't, however, deducted transaction costs in my results discussed above.<br /><br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-20555980227270993492017-03-11T13:16:39.187-05:002017-03-11T13:16:39.187-05:00Interesting post. I did not understand your answer...Interesting post. I did not understand your answer to dashiell on the number of training rows available, the post refers to using factor loadings from quarterly reports as the only independent factors in the regression model, how can you use it for daily returns? It would seem you have 4 different points per year so a total of only 5x4=20 points per stock for predicting the next quarterly return, unless I misunderstood the regression setup. Can you clarify please?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-24915294936470786512017-03-11T11:08:51.164-05:002017-03-11T11:08:51.164-05:00Does the transaction cost adjust based on the CAGR...Does the transaction cost adjust based on the CAGR for say $100K?Ever Garciahttp://www.blogger.com/profile/01357178781354564617noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-59761836980209480272017-03-07T09:33:11.238-05:002017-03-07T09:33:11.238-05:00In stepwise regression, we try *every* variable at...In stepwise regression, we try *every* variable at each iteration, with replacement, and pick the one that has the lowest BIC, either to be added or removed.<br /><br />So no randomness involved.<br /><br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-58185282100459757152017-03-07T09:27:28.641-05:002017-03-07T09:27:28.641-05:00So you try one variable and check BIC. Then add an...So you try one variable and check BIC. Then add another (at random?) and check for decreased BIC? Is it as simple as an iterative search for lowest BIC?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-3249227045714435302017-03-06T08:16:28.934-05:002017-03-06T08:16:28.934-05:00Hi Thomas,
Thank you for mentioning L1/L2 regulari...Hi Thomas,<br />Thank you for mentioning L1/L2 regularization as yet another method to reduce overfitting.<br /><br />I am not claiming that the methods I outlined above are superior to every other ML method out there such as L1/L2 regularization. If you are able to present the L1/L2 results here, you would be most welcome. Otherwise, I will try it and present my findings at QuantCon 2017.<br /><br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-58270016713542827772017-03-06T08:10:18.321-05:002017-03-06T08:10:18.321-05:00Hi dashiell,
5 years * 252 trading days per year ...Hi dashiell,<br /><br />5 years * 252 trading days per year = 1260 rows.<br /><br />Daily returns are used.<br /><br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-67190121383240015442017-03-05T22:30:34.225-05:002017-03-05T22:30:34.225-05:00Why not just do L1/L2 regularization? Why not just do L1/L2 regularization? Thomashttp://www.blogger.com/profile/07581607263422875506noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-70322999636664197222017-03-05T20:46:59.741-05:002017-03-05T20:46:59.741-05:00Great post.
Sorry, this may be a very silly ques...Great post. <br /><br />Sorry, this may be a very silly question, but I'm having trouble following what each row of data represents. How do you get 1260 rows for each stock? Based on the number, it seems like that represents daily returns for 5 years, but it didn't seem like daily returns were being used here. Thanks.dashiellhttp://www.blogger.com/profile/09985973563053624665noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-53786253801953283832017-03-05T10:26:49.097-05:002017-03-05T10:26:49.097-05:00Thanks for the caveat, Dr! Thanks for the caveat, Dr! Eduardo Gonzattihttp://www.blogger.com/profile/07499890051981038396noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-21758578533507542172017-03-05T10:18:48.343-05:002017-03-05T10:18:48.343-05:00Hi Eduardo,
I don't know how much the net expo...Hi Eduardo,<br />I don't know how much the net exposure may have inflated the result. My guess is that there is a long bias in the portfolio, because most stocks have positive returns during both the training and OOS periods.<br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-23278762423832464502017-03-05T08:37:05.035-05:002017-03-05T08:37:05.035-05:00Sorry Dr, I misread it!
Do you think that the lon...Sorry Dr, I misread it! <br />Do you think that the long boas from the market during the OOS window could have inflated the results by this much? <br /><br />Best Regards Eduardo Gonzattihttp://www.blogger.com/profile/07499890051981038396noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-85711567703677916382017-03-05T08:34:42.593-05:002017-03-05T08:34:42.593-05:00Thanks, Eduardo!
I have not tested the market neu...Thanks, Eduardo!<br /><br />I have not tested the market neutral version. However, as presented, it is already LS, though the 2 sides may not net to 0.<br /><br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-63953049806842097152017-03-05T08:19:09.512-05:002017-03-05T08:19:09.512-05:00Dr Chan, nice post, as usual!
Did you try to run ...Dr Chan, nice post, as usual! <br />Did you try to run a LS version of this simple strategy? Shorting those that seemed overpriced, or, hedging with the SP500 as you said? Could you post the rough numbers, if you did? <br /><br />Thanks! <br /><br />Best Regards Eduardo Gonzattihttp://www.blogger.com/profile/07499890051981038396noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-47093526444716515242017-03-04T06:33:56.403-05:002017-03-04T06:33:56.403-05:00Thank you, David!
Yes, AIC and BIC are both used ...Thank you, David!<br /><br />Yes, AIC and BIC are both used often.<br /><br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-720566439592893782017-03-04T06:33:34.682-05:002017-03-04T06:33:34.682-05:00Hi Honey,
That 2 variables worked for the entire i...Hi Honey,<br />That 2 variables worked for the entire in and out of sample period. So I am not sure what you meant by "last 40 days".<br /><br />Yes, I have always been a big fan of KISS!<br /><br />ErnieErnie Chanhttp://www.blogger.com/profile/02747099358519893177noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-18367311476701587342017-03-04T05:49:21.371-05:002017-03-04T05:49:21.371-05:00Great article. Could also use Akaike information c...Great article. Could also use Akaike information criterion .David Bryantnoreply@blogger.comtag:blogger.com,1999:blog-35364652.post-75694620824390472202017-03-04T03:06:52.711-05:002017-03-04T03:06:52.711-05:00BTW your results also convey that KISS principle R...BTW your results also convey that KISS principle Rocks! https://en.wikipedia.org/wiki/KISS_principleHoneyhttp://www.blogger.com/profile/09413821661660561472noreply@blogger.comtag:blogger.com,1999:blog-35364652.post-32648503203885279692017-03-04T03:04:56.825-05:002017-03-04T03:04:56.825-05:00Wow! only two Variables! :D But how we could estim...Wow! only two Variables! :D But how we could estimate that those two variables are sufficient for last 40 days but not for 400 days? Honeyhttp://www.blogger.com/profile/09413821661660561472noreply@blogger.com