Sunday, September 18, 2011

More on automated trading platforms

The ideal software platform for automating backtesting and executing your algorithmic trading strategies depends mainly on your level of programming expertise and your budget. If you are a competent programmer in, say, Java or C#, there is nothing to prevent you from utilizing the API offered (usually for free) by many brokerages to automate execution. And of course, it is also easy for you to write a separate backtesting program utilizing historical data. However, even for programmer-traders, there are a couple of inconveniences in developing these programs from scratch:

A) Every time we change brokerages, we have to re-write parts of the low-level functions that utilize the brokerage's API;

B) The automated trading program cannot be used to backtest unless a simulator is built to feed the historical data into the program as if they were live. To reduce bugs, it is better to have the same code that both backtests and trades live.

This is where a number of open-source algorithmic trading development platforms come in. These platforms all assume that the user is a Java programmer. But they eliminate the hassles A) and B) above as they serve as the layer that shield you from the details of the brokerage's API, and let you go from backtesting to live trading mode with a figurative turn of a key. I have taken a tour of one such platforms Marketcetera, and will highlight some features here:

1) It has a trading GUI with features similar to that of IB's TWS. This will be useful if your own brokerage's GUI is dysfunctional.

2) Complex Event Processing (CEP) is available as a module. CEP is essentially a way for you to easily specify what kind of market/pricing events should trigger a trading action. For e.g., "BUY if ask price is below 20-min moving average." Of course, you could have written this trading rule in a callback function, but to retrieve the 20-min MA on-demand could be quite messy. CEP solves that data retrieval problem for you by storing only those data that is needed by your registered trading rules.

3) It can use either FIX or a brokerage's API for connection. Available brokerage connectors include Interactive Brokers and Lime Brokerage.

4) It offers a news feed, which can be used by your trading algorithms to trigger trading actions if you use Java's string processing utilities to parse the stories properly.

5) The monthly cost ranges from $3,500 - $4,500.

If Marketcera is beyond your budget, you can check out AlgoTrader. It has advantages 1)-3) but not 4) listed above, and is completely free. I invite readers who have tried these or other similar automated trading platforms to comment their user experience here.

P.S. For those of us who use Matlab to automate our executions, a reader pointed out there is a new product MATTICK that allows you to send order via the FIX protocol which should let us trade with a great variety of brokerages.

37 comments:

vedaranyam said...

marketcetra, if it's open source, why is it costing so much ? It looks like they are moving to proprietary.
-balu raman

Doug said...

Tradelink is a free program that has similar functionality.
http://code.google.com/p/tradelink/

Ken said...

Hi,
I am using IBRokers in R which has callback function. You talk about these callback function. Could you explain what is a callback function and what is the proper way to implement an algotrading software ?

Thanks,

olafhel said...

For a Java programmer numericalmethod AlgoQuant is also a good start.
I found Marketcetra pretty hard to costumize not to mention the lake of a native support for IB API. Frankly IMHO it's doesn't deserve 3/4k per month.AlgoTrader or the very professionnal TradeLink (C# however) are way better options. You can also refer to a related post at http://numericalmethod.com/blog/2011/06/01/open-source-trading-software-or-not/

ulrich said...

also consider activequant ... we are the guys from which many of these guys copied and partly still copy ...and no, AQ is not for beginners.

I have had many many consulting phone calls with proprietary people from all over the globe over the years ...

cheers.
ulrich

Alpha said...

hi Ernie,

I have checked few platforms; Marketcetera, AlgoTrader, TradingBlox and MetaTrader4; and they are unique and offer stuff that other packages don't offer.

However; I could not get any of the above platforms; to do what I want to do; i.e. not flexible enough to write strategies efficiently.

i stress the word efficiently; as they all allow you to write the same strategies; but will take much longer to get the same results; not to mention messy code.

the most important feature that I look for; is debugging, modularity and speed of running backtests. and sadly i could not find anything better than Java; so i went back to the roots and written my own platform; and ma much more happier now.

as you indicated; writing code that allow you to backtest and trade at the same time; is very important. Eliminating look ahead bias is a challenging task in programming.

if you can afford the time and flexibility; write your own. but remember; don't reinvent the wheel; use other platforms first to learn from their pros/cons; and go for the ultimate system. IMHO; the best platofrms to mix features from are (in the order of goodness): MetaTrader4, TradingBlox, and MarketCetera.

hope this help.

issy

Jim Hunt said...

Hi Ernie,

Thanks for mentioning AlgoTrader. One small correction though. Whilst a GUI is on our roadmap, it has yet to be implemented.

If you or any of your readers have any other specific enhancements that you would like to see incorporated into the platform please feel free to add them to our list. For example, the progress of one recent such request that might well be of interest to news watchers can be viewed here:

http://code.google.com/p/algo-trader/issues/detail?id=23

Cheers,

Jim

olafhel said...

I agree with Ulrich to consider ActivantQuant.
Pros : very professional.
Cons : no documentation .
You have to read/test the codebase by yourself which is a uge task.
If it was documented => bigger user community => easier to adopt ...
That's a shame.

Ernie Chan said...

Thanks, Doug. Correct me if I am wrong: I see that tradelink is a similar platform for C# programmers. It is missing a CEP module, but it does allow non-programmers to use it via a query-response app.

Ernie

Ernie Chan said...

Ken,
A callback function is a function that is called whenever a specific event happens. Examples of these events can be a change in the bid/ask price of a symbol, an execution of an existing order, or a change in the position of a symbol in your account, etc. You can specify in the callback function what action to take when this event happen. For example, you can specify that when your buy order is filled, immediately send out a sell order at $1 above the execution price.

Ernie

Ernie Chan said...

Soom,
Thanks for mentioning AlgoQuant, your endorsement of tradelink, and the numericalmethod.com article.
Ernie

Anonymous said...

Hi,

We are a small team that's using Marketcetera to develop quantitiative trading strategies. We fully customized the system to meet our needs and also developed the marketdata adapter utilizing market data vendor's API. We did it without incurring any costs beside the time and resources we have to invest. In doing this, we find that the system has a good architecture to extend to meet your trading needs.

We are just curious about the monthly cost of $3,500 - $4,500 for using Marketcetera. Is it from on-going development, support...etc, or something else. If you have specific development needs using Marketcetera, please let us know and we can help with that.

Thanks

Ernie Chan said...

Anon,
I will let representatives from Marketcera, if they are reading these comments, to clarify exactly what the 3-4K monthly fee covers.
Ernie

Andy Flury said...

Ernie

Indeed MarketCetera has a very good architecture and is very well designed. The only problem is that there is a commercial company behind it. So even though the software is Open Source available, it is not really possible to modify it to your needs, nor to actually contribute any code to it. It also seems that the company is moving away from providing Open Source Software, their latest version is indeed not available freely anymore, and the only version you can currently download is already over 2 years old.

Our goal at AlgoTrader is to provide something truly Open Source, without any commercial interests behind it. Since many quant traders with a Java-background are facing the same issue (start something from scratch or use an expensive commercial trading software), we try to bundle developer resources in order to come up with a trading platform that provides the same type of functionality yet is available truly open source.

We are still at an early stage and it might take a while until we reach the quality of MarketCetera, but we are in no rush.

One more point I'd like to make: Marketcetera does comes bundled with Esper, but the platform does not seam to actually benefit much from using CEP. CEP (and Esper) is indeed a very powerful technology that allows you to implement new types of strategies that otherwise would be very difficult (or even impossible) to implement. If you only want to compute a moving average, there are probably better solutions. But if your strategy involves rules like "during, between, afterwards, parallel with, finishes with, begins with..." then you should take a look at CEP, Esper and of course AlgoTreader (http://code.google.com/p/algo-trader/)

andy flury

jf said...

TradeLink is open source and .net compliant.... it's been around for 3+ years and has ~17k download loads. http://tradelink.org

TradeLink is broker neutral, lets you build strategies and full applications across currently 17+ different brokers and feeds. http://docs.tradelink.org

It's low latency (.05ms/50us) and has very fast historical simulator (300k-1million ticks/second, portfolio realistic). It's fully .net compliant so you can use any language which includes .net implementation (eg c#, f#, vb, c++, also python and java also have .net bridges). http://tutorials.tradelink.org

There are 300+ developers on the user list, we push edge releases at least twice a month. We're fully LGPL and will remain so. http://community.tradelink.org

Incidentally the vc firm sold marketcetera's assets to a brokerage company based in europe. That is why they have had no press release updates in 2 years :

http://www.marketcetera.com/site/press-releases/list

Anonymous said...

Can someone list some brokerages that provide free API? Oanda asks for $600/mo for API-enabled account.

Ernie Chan said...

Anon,
Interactive Brokers certainly has free API.
Ernie

Ken said...

@Anon : IB has free API (you can even use directly it from R and maybe Matlab)
@Ernie Chan : Thanks for your answer on callback function. Do you know any website/book on programming for API/Quant system ?
I am a big R user with a strong statistics/econometrics background and I feel that I miss a lot of time and efficiency without sufficient computer science knowledge.
Good code is difficult !

Jim Hunt said...

@Anon - Here's an FX centric list of broker APIs:

https://docs.google.com/spreadsheet/ccc?key=0AjjmEFspMqvadFhWOE94MTkzQXZWNDZvVUpRbzIyc1E

As you can see, lots of them are "free" to use, although there's a wide variation in minimum deposits.

I've no idea why Issy is so keen on the platform, because I'm not! However OANDA do now provide a "free" API courtesy of MetaTrader 4.

Cheers,

Jim

Mike said...

I've been quant trading for about a year now. So here is some info that may be helpful for a newbie like myself. In searching for an automated trading platform I've written and executed [in a production environment] some pretty complex strategies using a few different platforms over the last year or so. They are SmartQuant, RightEdge, FXSpyder, TradeStation, MT4, directly to Oanda API and NinjaTrader. I'm a c# guy. And I chose Ninja Trader as the platform I now use and run 24/7. I feel it's hands down the best one I've used. Some reasons I chose NT over the others are connecting to brokers & data feeds is a piece of cake, never a connection error and their provider list is huge. Their entire library of indicators (over a 100) are open source. it can run side by side with visual studio. You have to compile in VS, then recompile in NT and that 's it. I love their API. The community 'code sharing' and NT support are unparalleled. Good luck on your quest.

olafhel said...

Well guys I think there's no right answer.
Like in any software project a requirement specification should be established.
After all quant trading is just a software project ...
Just my 2 cents

Chuck R said...

The GOOD:

AlgoTrader looks very promising for Java developers.

So far I personally have enjoyed using NinjaTrader though there are some limitations (with some workarounds). I've also enjoyed programming in C# and look forward to eventually trying F# with NinjaTrader. Everything is free to use until you have the need for live trading and then its available for a very small fee. Their IQFeed branded data feed, Kinetic, is also very affordable. Over all this platform and data feed has the lowest barrier to entry, tons of features, and lots of support. I see a very bright future for this platform. Not affiliated just a happy user.

The BAD:

Marketcetera looks like a dead project from the open source community perspective. I don't recommend it.

When I evaluated TradeLink I came across too many bugs and no production quality releases. Only constant updates to a development branch.

jf said...

tradelink presently has two release tags, featured and edge.

featured installers are those with no bug reports on them. edge installers are the latest development branch. If somebody experienced problems with either type of release, and didn't report them, it's hard to help.

Based on feedback we're looking at adding a third level of release tagging and validation. This will most likely be called 'stable', and be more appropriate for new users.

Tony C said...

Hi Ernie and others,

This may be a bit off the topic. We are seeking a prime brokerage which allows a $5m capital fund to open an account and strongly supports high frequency trading (aggressive orders, not market making). Better the have a SOR (Smart Order Routing) available, and be able to access adtional dark pools. Surely, low mils per share for commission.

I shopped around, large investment banks like Barcalys, JPM, etc usually don't accept clients with this size ($5mm). Lime is so far one which may meet our requirements. Can anyone help mention some others, please?

Thanks a lot!

Ernie Chan said...

Hi Tony,
If you email me privately, I can connect you with someone who may be able to help.
Ernie

Anonymous said...

Hi Ernie
Thanks for a great website. We are keen to test out alphacet although noticed news was last updated in jan. Are they still doing ok?

Ernie Chan said...

Hi Anon,
Sorry, I haven't been in touch with Alphacet for a while, so I don't have any updates for you either.
Ernie

Anonymous said...

How come nobody ever talks about Tradestation and easy language?

Swiss_Dragon said...

Thanks Dr Chan for pointing to Algo-Trader. That will make my Quantitative Options Trading business possible - http://jmsdb.blogspot.com/

Anonymous said...

Our company has done a bit research on open source algo trading software. Cyan Spring ATS has come out on top. It works like a completed commercial product and the strategy framework looks really impressive.

Ernie Chan said...

Hi Anon,
Thanks for the info about Cyan Spring. Do you know if
1) it only has a Java API
2) can trade stocks, FX, futures, and/or options
3) has Complex Event Processing for fairly complicated events besides just a tick, for e.g. news events?

Ernie

Anonymous said...

Hi Ernie,

Why there is no C++ trading platform any more? I think C++ is considered for better performance in HFT, isn't it? But it looks like most of C++ open source projects are dead or inactive. Current trading softwares, either commercial or free, are based on Java or C#, like AlgoTrader, Tradelink, NinjaTrader, etc. Did C++ die out of this industry, or it is only used for proprietary software of big bank/fund?

Ernie Chan said...

Hi Eric,
Indeed C++ is the favored language for HFT, and is very much in use.

I believe TradeLink still uses C++ (among other languages.)

Other platforms dropped it probably because it requires much more time and skills to program in it.

Ernie

Andy Flury said...

Hi Guys

It has been a while since I talked about AlgoTrader. In the meantime we have put a lot of development work into the system. AlgoTrader is being actively used in quite a few hedge fund accounts (some VIX based Volatility strategies, a few FX strategies and also an Equity based Pair Trading Strategy).

In recent months a lot of companies approached us, if we could make AlgoTrader available as a commercial product (with Support, Professional Services, etc.)

So we did…. Have a look: www.algotrader.ch

The AlgoTrader Enterprise Edition now has many new Features, including three different GUI's, different Broker Interfaces (Native and Fix), support for custom Derivative Spreads, several built-in Execution Algorithms, a Option Pricing Engine, and much more.

The system is still heavily based on Open Source Technologies like Hibernate, Spring, ActiveMQ & AndroMDA. Full Documentation is available:

Also, AlgoTrader is still based on Complex Event Processing (CEP) using Esper and therefore accommodates strategies that cannot be programed with procedural programming languages.

However AlgoTrader is not a Chart based Trading Platform (like Tradestation, MetaTrader or NinjaTrader). It does not have drag-and-drop functionality and hundreds of indicators. It is our belief, that successful strategies are based on economic facts and not on extensive back-testing or chart pattern analysis based on many different technical indicators.

@Eric: Even though AlgoTrader is using Java (as opposed to C++) it can process up to 500'000 Events per second, thanks to Esper. This sort of throughput can definitely compete with C++ applications. So in our opinion your network topology (co-location etc.) will have a much bigger influence on latency than the choice between Java and C++.

Cheers
Andy

 

Unknown said...

Hi,

Question on AlgoTrader.

How did you guys get around GPL for Esper? That sort of makes also your Enterprise version GPL...

Regards
Peter

Rasheed said...

Hi,

"TradeSharp" (http://www.tradesharp.se/) is an other C# based data feed & broker neutral Algorithmic Trading Platform that lets trading firms or individuals automate any rules based trading strategies in stocks, forex and ETFs.

TradeSharp has ton's of http://www.tradesharp.se/#features and quite user friendly interface as well http://www.tradesharp.se/#gallery

Everyone is welcome to schedule a demo and see it in action!

Cheers,
/Rasheed

Anonymous said...

Consider tbg-Quant, very similar to Marketcetera using CEP provider. it's close-source but you can implement over it, it's a framework, java based.
Here you can find some simple documentation: https://github.com/tbg-quant/tbg-quant/blob/v1.1beta5/documentation/tbgQuant-v1.1beta5-Documentation.pdf