Quantitative investment and trading ideas, research, and analysis.
Friday, November 07, 2008
My book on Quantitative Trading is published
My book on Quantitative Trading has been published and is now available from Amazon.com. Many thanks to all of you for your ideas, comments, and support!
I looked into it a bit on Amazon and noticed that you said you didn't know of any Matlab API offered by a broker. Here are two solutions I've been looking at: Matlab2IB and MatlabTrader. If you try them out I would very much like to know what you think. Thanks.
Most of my work experience has come from Network and Unix Engineering.
I have very recently developed an interest in quantitative algorithmic trading. Very excited to have come across your new book. Looking forward to receiving it next week.
I have also found "QuantLib: a free/open-source library for quantitative finance". For anyone interested in APIs.
Can you corroborate results 10%+ from stat arb books during this crisis? It seems that many stocks had little to no reversion and scanning some charts it looks like many pairs significantly diverged. Thanks!
NN, Mean-reversion strategies are not limited to pair trading (see examples 3.7 and 3.8 in my book). The 10% number is from informal chatting with fellow traders and portfolio managers in the statarb space, and also from my own trading. Almost everyone I spoke to in the mean-reversion business has been in positive territory this year. Ernie
I purchased and started reading your book. It is exactly the subject I wanted a book on for years! Thank you so much for writing it.
I have been having trouble with Yahoo real time quotes which I download every second. Often in the day, updating stops, or oscillates around a false price.
What is a good source of real time price data which allows downloading multiple symbols? I use a mac, so html query is best.
Dear spritrig, I am glad the book is useful to you. For real-time quotes, I use my broker's (Interactive Brokers) real-time feed. It should be free (unless you are a "professional trader"). As a backup, we also have Bloomberg, but that's much more expensive. Ernie
On page 7 of your book you mention when you earn more money you will devote more software resources to further automate, having programs automatically start themselves up. If you use a UNIX computer like an Apple Mac, you can use the UNIX/Linux "crontab" command to set up a schedule to execute commands at specific times. http://en.wikipedia.org/wiki/Cron There may be a clone for Windows. http://cronw.sourceforge.net/
Thanks for the tip on Interactive Brokers. I am starting an account with them.
Hi, I finished reading your book over the weekend. Well done! It is very practical and very good. It gives steps to follow, and ideas or references to research. Thank you for sharing your experience. Regards, Alexandre
Hi Max, Yes, I did look at the website that provided the Matlab/IB api. It looks reasonable. However, I have not tried this package as I have already, at considerable cost, have my programmers build an ATS using C#. Maybe for a future trading strategy I will explore this in more seriousness! Ernie
I live in Singapore. I just got your book and started reading it; I am curious to know if (some of) your algorithmic strategies are suitable for Forex Trading? Which ones would be better or how would you recommend to start? Best regards, Javier
Hi Javier, I haven't done any (profitable) Forex trading so far. However, if you have a good high frequency trading system, you can try to apply the Bollinger band strategy I discussed on page 23 (which was applied to ES), and see if you can reduce slippage to make it profitable. Good luck! Ernie
Does anyone know the Kindle equivalent of page 34 in the book? Kindle doesn't use page numbers so it's hard to find a certain page. But don't give me the password!!!!!!!!!! I want to find it myself.
Here's one more question tonight. Hope I'm not overloading you.
I ran Example 7.5 on p. 141 (Calculation of the Half-Life) in Matlab but received an error: "backshift not found."
Backshift doesn't appear to be in the Econometrics Toolbox by James P. LeSage. Are you aware if it is in one of the Matlab tooboxes (e.g., their Econometrics toolbox)?
Jacques, AR(1) is an estimate of the hedge ratio between the 2 stocks. It is not the mean value of the spread, which should always be close to zero.
You can always google VWAP Matlab to find freeware for this calculation. Like you said, it is a one-line calculation so you might as well do it yourself!
backshift.m is included in epchan.com/book for you to download.
I liked your book, but it is really annoying that the premium section of your website (http://epchan.com/book/) does not allow directory listings (it gives a 403 error). There is also no index file, despite what you say on http://www.epchan.com/subscription/spread.htm. So there is no way to know what files there are other than trying all the filenames mentioned in your book (some of which are case sensitive, others not). This also means that if you post a new file, we will never be able to find it.
John, That was a temporary glitch on the part of the web hosting company. It has been fixed now, and you can see the index of all the programs at epchan.com/book
Anonymous, You may have the same lag function in different packages higher up in your Matlab path. Type "which lag" to find out which function you are using and adjust your path precedence accordingly. Call Matlab customer support if you have further difficulties. Ernie
I just bought and devoured your book - really on point to my situation. Especially relevant was the conclusion section where you described the importance of knowing a strategy's capacity. I've got a nice strategy, Sharpe ratio of ~3.0, and really good consistency. I'm trading it with non-trivial amounts of money (>$500K equity) and want to better understand if its capacity is 2x, 20x or 200x the current size. The answer to this question is key for me in knowing whether I've got a profitable source of incremental income or an opportunity to build a fund partly with institutional money.
I can think of a few ratios to look at (e.g., pos size relative to ADV) but haven't seen any good literature on rules of thumb of estimating.
I define capacity as the point where marginal increase in equity will not produce a marginal increase in P&L.
Others, however, may define capacity as the point where marginal increase in equity will start to decrease return.
Obviously capacity under my definition is much larger! The common sense definition of capacity is probably somewhere in between.
If you are using the second definition, the first step is to look at the typical bid-ask size of the instruments you trade. If your order size has reached the typical bid-ask size, then you have reached capacity already, unless you find a way to slice up the order and enter in small chunks at different times without diminishing the return.
If you are using the in-between common-sense definition, you have to estimate the market impact of your orders. Many, many academic papers have discussed this topic, but you can start with this blog post: http://www.maxdama.com/?p=397
Thanks Ernie, I'll look at that blog post and the sources it references.
My strategies have hold periods of about 5 days, and I've backtested them based on getting closing price on the day following the signal (signal available after close of market on t=1 trades at close on t=2), so I'm pretty confident that as long as my order can get done within one full trading day without major impact I'll be OK.
I currently measure every execution relative to the daily close price (i.e, what was the execution price vs. the end of day close price) and consider this to be the "slippage", at least relative to the close price (I'm assuming here that I'm not a big enough buyer/seller to actually move the close, just to nudge around the intraday prices a bit). My "slippage" defined in this way is on the order of one basis point.
The journal article listed in Max Dama's blog seemed to focus on impact from size of order relative to average size of inside quote (if I'm reading this right). I'm specifically interested in size of position that can be taken over some period of time (like an hour or day) and what the impact of that would be. In some ways this is analogous, but not totally.
I'll keep going through the literature looking for a market impact equation based on TradeSize / ADV ratio
Gray, You might want to google the paper by Doyne Farmer (of Los Alamos National Lab/Prediction Company/Santa Fe Institute fame) "Liquidity, long-memory and market impact".
Ernie, I really like your book, but there is a slight issue I hope you adress in future editions.
Often you present code and some output, say "p-value 0.035". However, I may use other tools than Matlab and therefore would like to be able to calibrate my tool to yours. I would appreciate if you specified the input, too. Not only the output. Say "XLD/GM for dates 2008-01-01 up to 2010-01-01". Now I can calibrate my tool with your output. So please specify input AND output!
Actually, I have provided Excel input files for most of the examples in the book. Furthermore, the entirety of the source codes were displayed, so you can see the exact parameters as well.
Let me know if you have a question about a specific example where such inputs are not available. Ernie
Great book - inspired me very much. By the way, the function "prt" to show the data result table for the cointegration test in example 7.2 doesn't work for me.
Error msg.: Undefined function 'prt' for input arguments of type 'struct'.
Hi Ernie I am a master student in economics and I am doing my thesis on algorithm trading for high frequency firms. I was amazed by your book and the way it is presented. Even if I have no professional background, you took me through the mathematical struggle and I have a much better overview of the sector. I was about to start to chapter with the interday trading and I have some troubles. In fact, the premium link (http://www.epchan.com/subscription/spread.htm) is not working anymore and I have been trying for 2 days now. It is very annoying because my thesis deadline is coming and I really need those codes to be able to finish. Please Ernie could you help me?
Hi Ernie, I read your book Algorithmic Trading but I don't get grasp of the OU. OU- E( y(t)) = y0exp(λt) − μ/λ(1 − exp(λt)) (2.6) Remembering that λ is negative for a mean-reverting process, this tells us that the expected value of the price decays exponentially to the value −μ/λ with the half-life of decay equals to −log(2)/λ. How does log(2) get into the picture?
I had pre-ordered the book, so they shipped it yesterday. It is scheduled to arrive on Monday.
ReplyDeleteGood luck.
Ernie,
ReplyDeleteI ordered the book too.
I looked into it a bit on Amazon and noticed that you said you didn't know of any Matlab API offered by a broker. Here are two solutions I've been looking at: Matlab2IB and MatlabTrader. If you try them out I would very much like to know what you think. Thanks.
Regards,
Max
Thanks, folks.
ReplyDeleteMax: this is a great suggestion. I will check them out.
Ernie
My copy was waiting for me when I got home from work today..
ReplyDeleteMine should be here tomorrow. I can't wait!
ReplyDeleteMost of my work experience has come from Network and Unix Engineering.
ReplyDeleteI have very recently developed an interest in quantitative algorithmic trading. Very excited to have come across your new book. Looking forward to receiving it next week.
I have also found "QuantLib: a free/open-source library for quantitative finance". For anyone interested in APIs.
Thanks, Chuck, for your tip about QuantLib. Hope you enjoy my book.
ReplyDeleteErnie
I just got your book. Congrats on its publication.
ReplyDeleteErnie,
ReplyDeleteCan you corroborate results 10%+ from stat arb books during this crisis? It seems that many stocks had little to no reversion and scanning some charts it looks like many pairs significantly diverged. Thanks!
NN,
ReplyDeleteMean-reversion strategies are not limited to pair trading (see examples 3.7 and 3.8 in my book). The 10% number is from informal chatting with fellow traders and portfolio managers in the statarb space, and also from my own trading. Almost everyone I spoke to in the mean-reversion business has been in positive territory this year.
Ernie
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteI purchased and started reading your book. It is exactly the subject I wanted a book on for years! Thank you so much for writing it.
ReplyDeleteI have been having trouble with Yahoo real time quotes which I download every second. Often in the day, updating stops, or oscillates around a false price.
What is a good source of real time price data which allows downloading multiple symbols? I use a mac, so html query is best.
Dear spritrig,
ReplyDeleteI am glad the book is useful to you. For real-time quotes, I use my broker's (Interactive Brokers) real-time feed. It should be free (unless you are a "professional trader"). As a backup, we also have Bloomberg, but that's much more expensive.
Ernie
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteHi Ernie,
ReplyDeleteOn page 7 of your book you mention when you earn more money you will devote more software resources to further automate, having programs automatically start themselves up. If you use a UNIX computer like an Apple Mac, you can use the UNIX/Linux "crontab" command to set up a schedule to execute commands at specific times. http://en.wikipedia.org/wiki/Cron There may be a clone for Windows. http://cronw.sourceforge.net/
Thanks for the tip on Interactive Brokers. I am starting an account with them.
Dear Spritrig,
ReplyDeleteGood point about crontab. Now if I can just get my programmer to implement the Windows version...
Ernie
Ernie, it looks like crontab already exists for windows. No need to implement it.
ReplyDeletehttp://cronw.sourceforge.net/
Hi,
ReplyDeleteI finished reading your book over the weekend. Well done! It is very practical and very good. It gives steps to follow, and ideas or references to research.
Thank you for sharing your experience.
Regards,
Alexandre
Ernie,
ReplyDeleteHave you had a chance to check out those Matlab/IB APIs? I'd like to see a post on them.
Regards,
Max
Hi Alexandre,
ReplyDeleteThanks for your compliments!
Ernie
Hi Max,
ReplyDeleteYes, I did look at the website that provided the Matlab/IB api. It looks reasonable. However, I have not tried this package as I have already, at considerable cost, have my programmers build an ATS using C#. Maybe for a future trading strategy I will explore this in more seriousness!
Ernie
Great read, I am no trader, but still found it fun to read...
ReplyDeleteGreat book.....
greetings from Germany
Greetings Ernie,
ReplyDeleteI live in Singapore. I just got your book and started reading it; I am curious to know if (some of) your algorithmic strategies are suitable for Forex Trading? Which ones would be better or how would you recommend to start? Best regards, Javier
Hi Javier,
ReplyDeleteI haven't done any (profitable) Forex trading so far. However, if you have a good high frequency trading system, you can try to apply the Bollinger band strategy I discussed on page 23 (which was applied to ES), and see if you can reduce slippage to make it profitable.
Good luck!
Ernie
Does anyone know the Kindle equivalent of page 34 in the book? Kindle doesn't use page numbers so it's hard to find a certain page. But don't give me the password!!!!!!!!!! I want to find it myself.
ReplyDeleteThanks,
John
PS Great book!
Hello Ernie,
ReplyDeleteThanks again for an awesome book.
Example 3.1 on p. 34 provides MATLAB code to scrape Yahoo Finance for historical stock price data.
Can you offer any suggestions for Web sites from which historical *futures* price data can be scraped?
Thanks!
Ernie,
ReplyDeleteFor those of us with limited budgets, you mentioned alternatives to Matlab on p.33.
Do you know of any available consultants that can help configure Scilab or Octave in order to perform financial modeling and analysis?
For example: connecting to data sources/feeds and coding a few basic strategies.
Any input is greatly appreciated.
Thanks!
Hi Jacques,
ReplyDeleteI am not sure whether there are consultants for Octave/Scilab. But there are 2 ways to find out:
1) google "programmers Octave"
2) Ask the customer service at Octave.
Alternatively, you can try the programming language "R". I think more programmers are available for this freeware.
Ernie
Jacques,
ReplyDeleteFor futures historical data, try Quotes-plus.com. It is inexpensive and you don't have to web-scrape.
Ernie
Hello Ernie,
ReplyDeleteAre you aware of any errors in example7_2.m located at http://www.epchan.com/book/?
A colleague of mine tried to run it in Matlab but had problems.
Thanks!
Jacques,
ReplyDeleteSome readers forget to fully install the jplv7 package, and added all the subfolders to their Matlab path.
If this doesn't work, please tell me what the error message is.
Thanks,
Ernie
Thanks Ernie!
ReplyDeleteWe had cadf.m installed but not all files in the jplv7 package.
Best regards,
Steve
Dear Ernest,
ReplyDeleteI've been reading your blog since 2007, and I recently purchased your book.
Thanks for all the insights in your book and blog!
I'll try to comment here more often, maybe sharing some experiences.
Best Regards,
Newton Linchen.
Hello Ernie!
ReplyDeleteExample 7.2 on pages 128-130 runs the Augmented DF test.
Could the "AR(1) estimate" be considered the mean value for the spread's time series?
Thanks!
Hello again,
ReplyDeleteAre you aware of any pre-existing Matlab code that calculates the Volume Weighted Adjusted Price (VWAP) over a given time series?
It doesn't seem to be complicated code to write from scratch, but if someone has already written it... :)
Cheers!
Hello Ernie,
ReplyDeleteHere's one more question tonight. Hope I'm not overloading you.
I ran Example 7.5 on p. 141 (Calculation of the Half-Life) in Matlab but received an error: "backshift not found."
Backshift doesn't appear to be in the Econometrics Toolbox by James P. LeSage. Are you aware if it is in one of the Matlab tooboxes (e.g., their Econometrics toolbox)?
Thanks a lot for your input! :)
Jacques,
ReplyDeleteAR(1) is an estimate of the hedge ratio between the 2 stocks. It is not the mean value of the spread, which should always be close to zero.
You can always google VWAP Matlab to find freeware for this calculation. Like you said, it is a one-line calculation so you might as well do it yourself!
backshift.m is included in epchan.com/book for you to download.
Ernie
I liked your book, but it is really annoying that the premium section of your website (http://epchan.com/book/) does not allow directory listings (it gives a 403 error). There is also no index file, despite what you say on http://www.epchan.com/subscription/spread.htm. So there is no way to know what files there are other than trying all the filenames mentioned in your book (some of which are case sensitive, others not). This also means that if you post a new file, we will never be able to find it.
ReplyDeleteJohn,
ReplyDeleteThat was a temporary glitch on the part of the web hosting company. It has been fixed now, and you can see the index of all the programs at epchan.com/book
Sorry for the inconvenience!
Ernie
I can see the index for the book files now and everything is there, thanks!
ReplyDeleteHi,
ReplyDeleteI tried running example7.2, here is my error msg (I intalled jplv7 and did set the path):
?? Error using ==> lag
Too many input arguments.
Error in ==> cadf at 57
z = trimr(lag(r,1),1,0) ;
Error in ==> example7_2 at 53
res=cadf(adjcls(:, 1), adjcls(:, 2), 0, 1); % run cointegration check using augmented Dickey-Fuller test
Any comments?
Thank you
Anonymous,
ReplyDeleteYou may have the same lag function in different packages higher up in your Matlab path. Type "which lag" to find out which function you are using and adjust your path precedence accordingly. Call Matlab customer support if you have further difficulties.
Ernie
Hi Ernie,
ReplyDeleteI just bought and devoured your book - really on point to my situation. Especially relevant was the conclusion section where you described the importance of knowing a strategy's capacity. I've got a nice strategy, Sharpe ratio of ~3.0, and really good consistency. I'm trading it with non-trivial amounts of money (>$500K equity) and want to better understand if its capacity is 2x, 20x or 200x the current size. The answer to this question is key for me in knowing whether I've got a profitable source of incremental income or an opportunity to build a fund partly with institutional money.
I can think of a few ratios to look at (e.g., pos size relative to ADV) but haven't seen any good literature on rules of thumb of estimating.
Any thoughts or links would be appreciated.
Hi gray,
ReplyDeleteCapacity estimation is a complex topic.
I define capacity as the point where marginal increase in equity will not produce a marginal increase in P&L.
Others, however, may define capacity as the point where marginal increase in equity will start to decrease return.
Obviously capacity under my definition is much larger! The common sense definition of capacity is probably somewhere in between.
If you are using the second definition, the first step is to look at the typical bid-ask size of the instruments you trade. If your order size has reached the typical bid-ask size, then you have reached capacity already, unless you find a way to slice up the order and enter in small chunks at different times without diminishing the return.
If you are using the in-between common-sense definition, you have to estimate the market impact of your orders. Many, many academic papers have discussed this topic, but you can start with this blog post: http://www.maxdama.com/?p=397
Ernie
Thanks Ernie, I'll look at that blog post and the sources it references.
ReplyDeleteMy strategies have hold periods of about 5 days, and I've backtested them based on getting closing price on the day following the signal (signal available after close of market on t=1 trades at close on t=2), so I'm pretty confident that as long as my order can get done within one full trading day without major impact I'll be OK.
I currently measure every execution relative to the daily close price (i.e, what was the execution price vs. the end of day close price) and consider this to be the "slippage", at least relative to the close price (I'm assuming here that I'm not a big enough buyer/seller to actually move the close, just to nudge around the intraday prices a bit). My "slippage" defined in this way is on the order of one basis point.
The journal article listed in Max Dama's blog seemed to focus on impact from size of order relative to average size of inside quote (if I'm reading this right). I'm specifically interested in size of position that can be taken over some period of time (like an hour or day) and what the impact of that would be. In some ways this is analogous, but not totally.
I'll keep going through the literature looking for a market impact equation based on TradeSize / ADV ratio
Gray,
ReplyDeleteYou might want to google the paper by Doyne Farmer (of Los Alamos National Lab/Prediction Company/Santa Fe Institute fame) "Liquidity, long-memory and market impact".
Ernie
Hi Ernie,
ReplyDeleteShall we use log (price) for Ornstein-Uhlenbeck to model spread for mean reversion?
ShinjiOno
Hi Ernie,
ReplyDeleteShall we use log price to model spread for ornstein-uhlenbeck?
Z= log p1- log p2
Z OU process?
Shinji
Hi ShinjiOno,
ReplyDeleteYou can just use raw prices for OU.
Ernie
Ernie, I really like your book, but there is a slight issue I hope you adress in future editions.
ReplyDeleteOften you present code and some output, say "p-value 0.035". However, I may use other tools than Matlab and therefore would like to
be able to calibrate my tool to yours. I would appreciate if you specified the input, too. Not only the output. Say "XLD/GM for dates 2008-01-01 up to 2010-01-01". Now I can calibrate my tool with your output. So please specify input AND output!
Great book, I am recommending it to others!
Hi Anon,
ReplyDeleteThanks for the kind words on your book.
Actually, I have provided Excel input files for most of the examples in the book. Furthermore, the entirety of the source codes were displayed, so you can see the exact parameters as well.
Let me know if you have a question about a specific example where such inputs are not available.
Ernie
Hi Ernie,
ReplyDeleteGreat book - inspired me very much. By the way, the function "prt" to show the data result table for the cointegration test in example 7.2 doesn't work for me.
Error msg.: Undefined function 'prt' for input arguments of type 'struct'.
Do you have any idea what might be wrong?
Hi John,
ReplyDeleteThanks for your kind words.
Please make sure you added all the sub-folders of the jplv7 package to your Matlab path.
Ernie
Hi Ernie
ReplyDeleteI am a master student in economics and I am doing my thesis on algorithm trading for high frequency firms. I was amazed by your book and the way it is presented. Even if I have no professional background, you took me through the mathematical struggle and I have a much better overview of the sector. I was about to start to chapter with the interday trading and I have some troubles. In fact, the premium link (http://www.epchan.com/subscription/spread.htm) is not working anymore and I have been trying for 2 days now. It is very annoying because my thesis deadline is coming and I really need those codes to be able to finish. Please Ernie could you help me?
Hi Alberto,
ReplyDeleteThank you for your kind words on my book.
The link is http://www.epchan.com/subscription/spread.html.
The last "l" seems important.
Ernie
Hi Ernie,
ReplyDeleteI read your book Algorithmic Trading but I don't get grasp of the OU.
OU-
E( y(t)) = y0exp(λt) − μ/λ(1 − exp(λt)) (2.6)
Remembering that λ is negative for a mean-reverting process, this tells us
that the expected value of the price decays exponentially to the value −μ/λ
with the half-life of decay equals to −log(2)/λ.
How does log(2) get into the picture?
Hi Wynton,
ReplyDeleteThe 2 in log(2) just means we are interested in the half life, when the value dropped to 1/2 of the initial value.
Ernie