Friday, May 29, 2009

MATLAB as an Automated Execution System

I just published an article "MATLAB as an Automated Execution System". (It is available to readers of my book and subscribers to my Premium Content website.) It comes with example MATLAB codes executing a simple Bollinger-band high-frequency E-mini trading strategy.

As I mentioned before, I now find MATLAB to be a good platform not just for backtesting, but for automated execution as well. Of course, not all brokerages have API's that connect to MATLAB. My example codes are for submitting orders automatically to an Interactive Brokers account.

In general, I find that writing execution programs in MATLAB is a breeze compared to C++, Java or even C#. It takes about 1/5 the development time of a C++ program. Any performance limitations will probably not be due to MATLAB, but to the latency of your brokerage in updating positions and order status.

83 comments:

G-Fav said...

Hi there -

I've been a "lurker" for a while and enjoy your blog.

Quick question: have you tried Mathematica? (It has a powerful and quite elegant set of functions based on a "list" methodology that might map well to trading strategies. Or, random things like synthetic aperture photography...)

Best,
Gregg

Anonymous said...

Hi Ernie

Thanks again for another terrific article. Your book just keeps giving via the "embedded" password for your premium site !

I know your Bollinger code is designed more to show us how to implement MATLAB2IB than as a trading system, but since you are obviously really good at Matlab, how would you add a trailing stop exit condition.

I understand the logic - what is the highest open profit, what is current open profit, if different by X percent then exit, but I am having trouble with the Matlab syntax.

Any chance you could provide a simple example in Matlab code for either the Bollinger example or as a standalone?

Thanks

N N said...

Ernie,

You are a scholar and a gentleman, I have been looking for an example like this for awhile. Very much appreciated!

Ernie Chan said...

Hi G-Fav,
I have programmed in Mathematica before. However, I feel that MATLAB's array processing capability is more suited to statistical arbitrage research. Furthermore, I am not sure there is a Mathematica API for connection to brokerages.
Ernie

Ernie Chan said...

Hi Anonymous,
I will look into providing sample code for trailing stop at some point in the future. But you can always keep track of the maximum price of your stock since your entry in a Matlab variable. So any time the Last price generates a return (drawdown) that is below a certain minimum, send a market order to exit your position.
Ernie

Jimmie Goode said...

Loved the book Ernie, several of your helper functions lie on my MATLAB path.

If anyone is left out in the cold with IB, the MB Trading SDK also integrates well with MATLAB. It has
prefab ActiveX controls that are a snap to implement in the GUIDE for creating custom trading interfaces.

Anonymous said...

Ernie

This code is gold - thanks for being so generous with your knowledge.

I'd like to trade FX but with IB you don't get a last price in the FX data feed, you get the last close from the previous session, but not the last trade price in your current session.

What's a good way to approach this this problem? As a price taker in FX you often have to take the spread so averaging (take the midpoint) of the bid/ask isn't such a viable solution. Any recommendations?

Matt

Anonymous said...

You should also consider the R/Python/Perl combo. It's free yet powerful.

Anonymous said...

A question and a comment,
What is the advantage to use the IB2MATLAB software over the free version using ACTIVEX. Here is a sample code how to connect directly via COM.(http://www.matlabtrader.com/code.php?project=InteractiveBrokers&file=IBexamples)

I believe using a matlab timer object will make the code more friendly.

Thanks for the code. I found it very usful.

N N said...

N N

Upon further review, MatLab2IB API demo does not include all functions so it cannot be tested. I have contacted them to see if a full version can be trialed.

Additionally I was not able to get the code from MatLabtrader.com using ActiveX controls to work. I am running API 9.51. Anyone else have better luck??

Ernie Chan said...

Matt,
If IB does not provide last price for currencies, you may have to subscribe to Bloomberg and use Matlab's Datafeed toolbox to get Bloomberg data. This is of course a much more expensive proposition, but well worth it if you can generate revenue from your model.
Ernie

Ernie Chan said...

Anonymous,
Yes, I also mentioned in a post before that there is a free open-source R API that connects to Interactive Brokers. I haven't tried it myself, but did any reader here try?
Ernie

Ernie Chan said...

Anonymous,
There may not be an advantage in using matlab2ibapi over using the free version from matlabtrader. However, the cost of matlab2ibapi is so low, and the customer service so friendly, that I don't consider "free" an intrinsic advantage. The greatest cost in trading is the loss from bad execution or bad models.
Ernie

N N said...

ExchangeAPI refused my request for a fully functional trial. I cannot test this API for reliability, accuracy, and latency against my current systems so I will have to find a solution using the free MatLabTrader version.

I just can't sink $300 into another software that sounds good unless there is a distinct advantage over my current infrastructure.

Anonymous said...

i have your book.. how to get to this article

username ?

password?

Ernie Chan said...

Anonymous,
You can find the userid/password to the Matlab codes and premium content on epchan.com in the last paragraph of page 34.
Ernie

Smith said...
This comment has been removed by a blog administrator.
Nikolaus said...

I have been using the matlabtrader.com version for a long time now and it works perfectly. Of course you have to give it some tweaks here and there, but it provides you with full functionality. There are quite a few examples included, so it's really easy to get started. I would highly recommend it to anyone who wants to trade using the API, but doesnt want to spend money ;)

Anonymous said...

Dr. Chang,
this is somewhat unrelated to this post, but related to your book. Since you mention Jim Simon. I thought you would appreciate this:
http://www.economist.com/finance/displaystory.cfm?story_id=13751628

as well as this particular comment attached to the story above:
"Henry Leeds wrote:
May 29, 2009 2:49
Jim Simons is a Pool Operator with the ability to raise large amounts of money from investors. He lacks the mathematical skills and knowledge to develop a really superior Trading System. His claim to fame rests on his years assisting Shiing Shen Chern, a brilliant mathematician who generously allowed Simons to add his name to the 1974 paper Chern wrote. The Medallion Fund was created by Elwyn Berlekamp, a brilliant Professor of Electrical Engineering and Mathematics at Berkeley. Berlekamp utilized his knowledge of Claude Shannon's revolutionary Information Theory to create this marvel. Shannon was Berlekamp's PhD Advisor at MIT. Berlekamp developed his Medallion Fund in a matter of months and its incredible performance is described on Berlekamp's website. Simons wanted Berlekamp to re-locate to Long Island and to continue developing the algorithms comprising his Medallion masterpiece. Berlekamp did not want to leave Berkeley and as a result he made a huge mistake. He sold the rights to his invention to Simons for six times what it had cost him - a relatively small amount. He now states on his website that the Medallion Fund is worth many thousands of times the dollar amount that he received for his achievement. The Renaissance RIEF hedgefund is an example of Simon's creative abilities. Its performance is pitiful and has resulted in investor withdrawals of 18 billion dollars. Renaissance investors have complained bitterly how their investment has faired so poorly while the Medallion Fund, reserved exclusively for Renaissance employees, has done so well. One can certainly understand their vexation."

I've been in the trading industry for a little while and everybody and his mother things of Jim Simon as being a god. Mr. Leeds seem to offer another reading of Pr. Simon's succes which came as a shock to me.

Anonymous said...

More related to this post... What do you think
of Tradestation as an alternative to Matlab?
Do you cover that in your book?
Charles

Ernie Chan said...

Charles,
I actually have not used TradeStation myself. However, I have heard that the historical data that TradeStation provides is not of the highest quality. Furthermore, the kind of strategies you can construct with their proprietary language are more limited than the ones you can construct with Matlab.
Ernie

Ernie Chan said...

Dear Anonymous,
Thank you for highlighting the comments on the James Simons article.
Actually, it has been well-known that Renaissance Technologies has not been very expert in equities strategies.
But this is not unusual, few funds are excellent in both futures and equities trading. Apparently they require quite different mindset and perhaps skillset.
Ernie

Anonymous said...

how to get login?

Ernie Chan said...

Login is available to readers of my book and subscribers to my Premium Content website.

Anonymous said...

Hi Ernie,
Your book is extremely good. No hype. Just a systematic process. It has helped me quite a bit. I was going through your cointegration posts. Is there a sample MATLAB program where this has been implemented ?

Thanks again
Ram

Ernie Chan said...

Ram,
Thanks for your kind words.
Example 7.2 in my book contains Matlab codes for cointegration.
Ernie

Anonymous said...

Hi Ernie

I'm not so sure about MatLab2IB's "customer service so friendly" after N N's comments I emailed them asking if they can confirm that the program will definately work with IB's FX, particularly routing with IDEALPRO - I never got a reply.

So I am now playing around with http://www.matlabtrader.com, I am going to try and code your example with matlabtrader.com, if I get it working well maybe you might like to post the comparision code.

Love that site - thanks

Matt

MATLAB2IB said...

MATT:

I am one of the authors of the MATLAB2IB API. We really apologize. We are not sure how we missed your email. Can you be kind enough to email us again. We are being inundated with Trial Requests and other emails and so we might have missed it. You can email me directly. YES, MATLAB2IB will work with IB FX. As long as you have permissions with IB, MATLAb2IB will have no problem putting your order through. Infact, SOme of our license owners do use it speciifcally for FX.


Regarding NN: He asked us if we can provide him a Trial Version with Full api functionality. As a Policy we decided not to provide it. It is as simple as that. There are many reasons which I can discuss separately.

Ernie Chan said...

Matt,
I think if you want to route your FX orders to IDEALPRO, you just need to specify IDEALPRO as the exchange when you use the placeOrder function.
Ernie

Anonymous said...

Hi Ernie

I had written previously about some trailing stop loss code.

Over at Mathworks I see Aly Kassam has updated his excellent 'Algorithmic Trading with MATLAB' webinar and the associated code so that it now includes some stop loss code.

The updated webinar code is at:
http://www.mathworks.com.au/matlabcentral/fileexchange/24320

Your readers might be interested in watching both of Aly's webinars to see the benefits of MATLAB.

Search mathworks for
'Algorithmic Trading with MATLAB for Financial Applications' and
'Algorithmic Trading with MATLAB - Update for 2009 (United Kingdom)'

Anonymous said...

Hmm, the link for Aly's files broke.
try this:
http://www.mathworks.com/matlabcentral/fileexchange/24320

Ernie Chan said...

Anonymous,
Thanks for the link to the stop loss code!
Ernie

Anonymous said...

Hi Ernie

I see in the factsheet that MATLAB Datafeed Toolbox 3.3 supports Interactive Brokers.
Have you got this toolbox or ever used it?
If so does it allow order placement as well as live quotes?

Curious

Anonymous said...

Anonymous:

Are you confusing Interactive Data with Interactive Brokers?? I dont see Interactive Brokers being mentioned in the fact sheet

Anonymous said...

Ah yes, sorry it says Interactive Data...got a bit excited

Ernie Chan said...

Hi Anonymous,
Yes, I have purchased the Datafeed toolbox for use with Bloomberg. It works very well. Have not tried Interactive Data -- you can always request a free trial though.
Ernie

Anonymous said...

Hi Ernie

The MATLAB2IB program looks good.

May I ask a question about deploying a MATLAB based trading system?

If I run your example code won't that "lock up" my MATLAB all day between 9.30 and 16:14:59? If so that's not too good as I need to do other things with MATLAB during the day.

Does that mean any production algo MATLAB solution really needs a person to have two copies of MATLAB - or can you run your example code in the background whilst still using the MATLAB workspace?

I realise the other possibility would be to create a compiled version, but the MATLAB compiler is about twice the cost of MATLAB - so in that case you'd probably get another copy.

Are my assumptions correct?

Thanks

David

Anonymous said...

Hi david,

In such a case, you can always open TWO or more matlab sessions on the same computer using the same license. So you can use 1 session for algo trading and another one for your use.
Alternatively, you can use compiled versions.

Ernie Chan said...

Yes, that's right. You can open multiple instances of Matlab on the same computer, provided your machine's memory and CPU allow.
Ernie

Rob Ryley said...

Has anyone here looked at some open source alternatives including the R stat package? I believe interactive brokers enables you to write a trading model in R, and then interface with their back end for your execution system.

If you really need computing speed, LUSH is an alternative, as it has easy integration with C/C++ code, ample numerical, statistical, and machine learning libraries, and a simplified version of lisp.

The LUSH project can be found at:
http://lush.sourceforge.net/

Of course, R can be found at:
http://www.r-project.org/

Ulf said...

have you tried timer objects?

Unknown said...

hi,

any way to start on quant trading without a programing background? are there tools out there a non programmer can leverage? i'm planning to buy your book and can't wait to read it. thx

Ernie Chan said...

T,
If you are good with Excel, you can backtest strategies and even automate them to a large extent. There are also programmers you can hire to help you.
Ernie

Unknown said...

Ernie,

Is there a good book on leveraging excel for Quant trading? thx

Ernie Chan said...

T,
I haven't come across any books using Excel for backtesting. But since Excel is such a easy tool to use, you can probably figure out yourself how to code it based on some of the examples in my book.
(Caveat: it is not easy to run regression fits in Excel on multiple stocks simultaneously.)
Ernie

Anonymous said...

i am currently using Amibroker. I am using esignal as the data feed and am trying to get an automated program to work to send orders to IB.
I am having trouble with the code.

I am thinking of changing to Matlab. Is it easier? Is it possible to receive data from esignal and send orders to IB?Is it all difficult to program?

Ernie Chan said...

anonymous,
I don't use Amibroker, so I don't know whether Matlab is easier.
Matlab is easy if you know how to program in e.g. Visual Basic.

Currently, the Matlab datafeed toolbox cannot receive data from esignal. Reuters, Bloomberg and Yahoo feeds are fine, as is native IB feed.
Ernie

Peter Harrington said...

HI perhaps I'm late to the party, but I just stumbled upon this topic.
I noticed someone complaining about the price of MATLAB2IB, it's $600/year. If you wanted to repeat what the author of MATLAB2IB (now called QUANT2IB) you can take the IB examples written in C++ and use the information in this link http://www.mathworks.com/support/tech-notes/1600/1622.html to compile them as a Matlab subroutine. If you wanted to use another Retail firm than IB, and that firm has C++ APIs then this method will work there as well.
If you don't want to get your hands dirty doing this then you probably can hire a contractor to do it for under $600.

FasTechs.com, Inc. said...

Hi Ernie

I very much enjoy your book. With Matlab how do you have Matlab constantly monitor the stock price? Do you have it continously loop or what is the best way for this?

Ernie Chan said...

cbucks,
Yes, you can run a continuous loop to monitor the price, or you can use an event-driven API such as the one described by maxdama.com: http://www.maxdama.com/?p=127
(or simply search for "Matlab API" on his website. This way, any time a price changes, a callback function that you provide will be called.
Ernie

Anonymous said...

Ernie, I understand this is an ancient thread, but I am getting errors when trying to run bollinger_5min.m (backtest).
"
Error using smartsum (line 17)
Not enough input arguments.

Error in calculateAvgHoldday (line 13)
avgNumHoldDays=smartsum(smartsum((capital+capital1), 2))/smartsum(numtrades);

Error in bollinger_5min (line 57)
avgNumHoldBars=calculateAvgHoldday(position)
"

I am using Matlab 2012b, do I just need to tweak something or does the code need to be updated? The file was directly from your site.

Thanks

Ernie Chan said...

Anon,
Try using smartsum2 instead.
Ernie

Anonymous said...

Hello,


Is this code designed for second-by-second trading or can it work at a lower frequency?


Thank You.

Ernie Chan said...

Hi Anon,
The matlab2ib program works by retrieving the last price every time you request one, and is not event-driven. So you can request it once every 250ms, which is the maximum update rate of IB.
Ernie

Anonymous said...

So, how would the sample file specifically need to be modified or does the sample file already request the data from IB at that "250ms" update rate?

I see an 'update every 60 seconds' line in the code..

and also a command that retrieves 1-min bar data. Would that have to be changed to a command to retrieve the snapshot data instead?

Would I change it to 1 or 0.25 or how?

Would the system time retrieval code have to be modified as well?


By the way, I should tell you that I enjoyed your book. There is nothing more straight-forward and concise on the market.

Ernie Chan said...

Anon,
To retrieve every second, just change elapsed >= 60 to elapsed >=1.

Also, change the argument of reqHistData to '1 sec' instead '1 min'.

1 sec is the minimum bar size for IB historical data.
Ernie

Anonymous said...

And then matlab should be able to automatically trade at the '250ms' update rate?

Anything else needed to be done?

The same lookback period, etc. applies? Just at a lower frequency?

Ernie Chan said...

For this model, we can't go to 250ms because IB historical data has a minimum bar length of 1s.

The example program is a toy model, not designed to be really profitable. And IB is certainly not suitable for trading at high frequency.

Ernie

Anonymous said...

So, in order to effectively trade at under 1 second with IB, one would need to use an external data feed with a smaller minimum bar length?

Ernie Chan said...

Yes, external data feed is needed.
But even then, IB order confirmations have 250ms latency, so you can't trade at any higher frequency.

Anonymous said...

Hi Ernie,

So, if someone wanted to create a pairs trading program with this sample, they would create a "localsymbol2", then request the market data for that particular security and have the program execute buy or sell orders opposite to the primary security with accessory if loops?

And for calculating the deviation and lookback periods, one would be able to adjust it by modifying the zscore/'lookback'/'entryz'/'exitz' to represent say linear regression instead of bollinger bands?

Anonymous said...

Hi Ernie,

So, if someone wanted to create a pairs trading program with this sample, they would create a "localsymbol2", then request the market data for that particular security and have the program execute buy or sell orders opposite to the primary security with accessory if loops?

And for calculating the deviation and lookback periods, one would be able to adjust it by modifying the zscore/'lookback'/'entryz'/'exitz' to represent say linear regression instead of bollinger bands?

Ernie Chan said...

Hi Anon,
Yes, you can create a separate symbol2 and localsymbol2 for the other leg of the pair.

As for Zscore, lookback, etc., once you have determined a hedge ratio, you have reduced the time series to 1 dimension ("the spread"), so Bollinger band will work just as in a 1-instrument case.
Ernie

Anonymous said...

So, in the buy order for security 'localsymbol' part of a loop, right below it, one would place a sell order for security 'localsymbol2' and make it all part of a single loop to make it a tandem trade?

And Linear Regression or EMA or any other indicator should work as well provided the proper zscore parameters are given?

What about volume metrics?

Ernie Chan said...

Yes, if you are using MKT orders for both legs, that's how you would implement a pair trade.

I did not consider volume in my signal generation.

Anonymous said...

And for limit orders?

World that involve calling generated data based on a particular indicator (like a Kalman Filter) from a mysql database?

Or would you recommend MS SQL or something else based on your experience?

Ernie Chan said...

If you want to send limit orders, the program will be very complicated, because you have to figure out if the order is filled, partially filled, or not at all.

I don't see what this has to do with databases. I have never used any databases in my trading.

Anonymous said...

Hi Ernie,

How does all of this information tie in to the following post?

http://epchan.blogspot.com/2010/07/pair-trading-technologies-update.html

Can the Quant2ib API and the native IB java api, now, automatically handle pairs trades with limit orders without needing to worry about anything else?

Is that what you have been doing since the api update?

Anonymous said...

I notice that the native IB API has a "combolegs" command and that the "undocumentedmatlab" api also has a "comboactions" command.

http://undocumentedmatlab.com/files/IB-Matlab_User_Guide.pdf

Ernie Chan said...

Hi Anon,
Yes, as that previous article mentioned, you can indeed send combo (spread) orders through Quant2IB's API using limit orders.

I believe you can also do this through IB-Matlab, though I haven't tried it myself.
Ernie

Anonymous said...

Hi Ernie,

I'd like to know what parameters I would need to change in the sample code in order to use another indicator like williams %r, Parabolic SAR or Relative Strength Index.

What about genetic algorithms? Combining RSI with MACD?



Did you see that the new version of matlab now has a trading toolbox?

www.mathworks.com/products/trading/

It's only compatible with Bloomberg and X-Trader, for now.

Ernie Chan said...

Hi Anon,
Yes, I learned recently that you can send orders from Matlab's Trading Toolbox.

You can certainly add various indicators to the sample program. But I doubt that it would make much difference in profitability. The main improvement will be in execution strategy, such as using limit orders.

I have never found GA to work either.

Ernie

Anonymous said...

So, what lines of code would I need to make modifications to in order to change the strategy from bollinger bands to say RSI or to another indicator?

Is it easier to use quickfix/fix and banzai to conduct limit orders?

Anonymous said...

Are you going to include the code in your upcoming book?

Ernie Chan said...

Hi Anon,
Many lines of codes need to be changed if you are going use a different method than Bollinger band.

For e.g. 132-133, 143-144.

Whether you use QuickFix or not, limit orders are always more complex to manage. It has really nothing do to with the specific API.
Ernie

Ernie Chan said...

Anon,
Yes, I will be including codes with my new book.
Ernie

Jeff koh said...

Hi Ernie,

I am new to quantitative trading. I am thinking to get a student version for Matlab. I wonder if it is sufficient to do backtest on stock, ETF and FX?
I am currently reading your book and was inspired to be a quant. Currently, I am a Teacher in Singapore.

Ernie Chan said...

Hi Jeff,
Yes, I believe a student version of Matlab will suffice for most strategies. You can also consider learning R instead, which is free.
Ernie

Anonymous said...

Hi Ernie

I am looking for a stop loss and take profit strategy on MATLAB. I work with MA strategies so I need to have a command that compute take profit and stop loss strategy. Could you help me have it? Do you have the codes on MATLAB?

Thanks

Ernie Chan said...

Hi Sarkhan,
I assume you are trading a mean-reversal strategy?
If you use Bollinger bands, both profit cap and stoploss are automatically built-in.
Please see my book Algorithmic Trading, Example 3.2 and the codes therein.
Ernie

Anonymous said...

Hello Ernie,

Thanks for this insight, and sorry to revive this post after so long.

I am wondering: do you actually need to have subscription to market data through IB (hence pay the subscription and exchange fee) to get the data, or can you get delayed data and FX data in any way with this API?

Thanks

Ernie Chan said...

Anon,
If you are not a "professional" trader, the market data fee for stocks through IB is de minimus. FX market data is always free.

Ernie

Jan Afridi said...

Thanks for sharing a nice and well-written post. That's really helpful. keep posting ...