Monday, October 08, 2012

Order flow as a predictor of return

Order flow is signed transaction volume: if an order is executed at the ask price, the incremental order flow is +(order size); if executed at the bid price, it is -(order size). In certain markets where traders can only buy and sell from market makers but not from each other, a positive order flow means that traders are net buyers of a security. But even in markets where everyone can place and fill orders on a common order book, a positive order flow indicates that informed traders (those willing to aggressively get into a position) are eagerly acquiring a security.

The neat thing about order flow is that it has proven to be a good momentum indicator. That is to say, a positive flow predicts a positive future return. This might seem trivially obvious, but you have remember that generally speaking, a positive past return by no means predicts a positive future return. That FX order flow possesses this predictive power was shown by Evans and Lyons in a series of papers, but this indicator is useful in many other markets, and at many different time scales. For example, in a paper by Coval and Stafford, it was shown that if you can tease out the order flow of a stock due to mutual funds' trading alone, you can also predict its future return up to, say, a quarter. This paper not only shows that order flow is predictive, but that sometimes a specific kind of order flow (in this case, that of mutual funds only) is sometimes more predictive than general order flow. In many cases, traders find that by counting only order flow due to institutional traders, or order flow due to large orders, they can better predict future returns. (No wonder institutional traders are trying their darnedest to break up their orders into small chunks, or to trade in dark pools!) I recently also heard that order flow into sector ETFs can be predictive of that sector's return. If any reader has read papers or has experience with this type of sector rotation model, please leave a comment!

Despite the proven usefulness of order flow, not too many retail traders utilize it. The reason is simple: it can be hard to measure. In FX in particular, many markets do not report trade information, or they report with a sufficient delay such that the information has no predictive utility. Even for markets that report instantaneous trade information, you would need a good piece of software to capture every bid, ask, trade, and trade size, and store them in an array, in order to compute order flow, an operation that most retail trading software cannot accomplish. However, this barrier to entry may just mean that there are still decent alpha to be extracted from this indicator.

Now, a bunch of public service announcements ...

===

A new algorithmic trading platform called Rizm designed for retail traders is now available. You can sign up for their beta trial here.

===

Quantopian has created an event-driven version of my gold/gold-miners arb strategy with source codes and analysis available. I find that the performance metrics clear and useful: better than the output from my own backtest programs! (Quantopian is a platform where you can share backtest results and codes with other traders.)

===

Arbmaker is a platform for pair traders, and it incorporates software for cointegration tests, has integrated data feed from many vendors, and allows automated order submission to Interactive Brokers. Neural networks and Kalman filter are also included.

===

Finally, I will be giving a talk titled "Backtesting and Its Pitfalls" at the World MoneyShow at the Metro Toronto Convention Centre on Saturday, October 20. Interested readers can register here.

20 comments:

Dan Dunn said...

Hi Ernie - Glad you like our backtester. We worked pretty hard to get that backtest UI right, and it sounds like it paid off! Next up for us is live trading. Soon, you'll be able to trade that gold/goldminer strategy live on www.quantopian.com

Bernd said...

Hi Ernie!

Are you aware of any way then to have a look or get information from order flows from big players? Or is it simply a matter of guessing? I am pretty confident that the relevant players put a lot of resources obfuscating their orders ( See TWAP,VWAP as a very basic technique for this). All I can think of is gathering tick information,price,timing and volume and try to crunch those numbers in search for patterns.

Ernie Chan said...

Hi Bernd,
There are some ways in HFT to detect whether a series of small order executions is part of a large institutional order. For example, the ITCH data feed contains a lot more info than is available in a consolidated data feed.
See the books by Sal Arnuk and Michael Durbin at the top of my Recommended Books list.
Ernie

Anonymous said...

Hi Ernie,

Thanks for your post!

I'm actually working on this subject: for this reason I would like to share with you my working paper (written with other two researchers), based on order flow in high frequency environment http://www.eea-esem.com/files/papers/eea-esem/2011/885/Mosconi_Carlini_Manzoni_Oslo.pdf

Ernie Chan said...

Hi Anon,
Thanks for the preprint!
Ernie

Anonymous said...

Really interesting... just look at this novel "The Fear Index", just for fun, but is all about that..
My work is on:

http://backtestingvix.wordpress.com/

http://nightlypatterns.wordpress.com/

Anonymous said...

Hi Ernie, just wondering when your new book will come out? can't wait to read it! I'll be the first one to buy,lol

Ernie Chan said...

Hi Anon,
My editor is currently reviewing it. Hopefully by Q1 or Q2 2013!
Ernie

Anonymous said...

Hi Erine

I have a liquidity provider (broker) who is using FIX 4.2 gateway, they don't have any FIX API , do you know any free retail platform( or low subscription cost) which allows user to connect any FIX location and build automation strategy to do trading directly?

Thanks

Ernie Chan said...

Hi Anon,
Have you looked into quickfixengine.org?
ERnie

vincegata said...

Dr. Chan,

You mentioned you're increasingly trading currencies, can you recommend some books on trading currencies. Any other tips and links how to start in Forex would be appreciated. Will your new book cover Forex?

I've developed ATS and now at the stage to start developing / back testing strategies.

Thank you.

Ernie Chan said...

Hi Vincegata,
The only FX book that I have read is by Richard Lyons (see my Recommended Books list). Otherwise I got ideas for my strategies from blogs, academic papers, and sometimes spontaneously.

I do cover some FX strategies in my new book. But more so on futures.

Ernie

vincegata said...

Dr. Chan,

Do you think microstructure approach is not rendered useless by HFT traders? It was a legitimate strategy few years ago but nowadays?

Ernie Chan said...

vincegata,
If HFT are able to exploit microstructure to their benefits, why can't we do the same and become HFT ourselves?
Ernie

vincegata said...

Excuse for the late response.

Because (at my understanding) many of orders sittings at the exchange are coming from the HFT shops and so essentially bogus. That's why it's hard to gauge market direction.

Ernie Chan said...

vincegata,
I should emphasize that "order flow" is not actually computed based on orders. They are computed based on actual executions.
Ernie

jimgreen said...

Hi, Ernie, Is the order flow algorithm you mentioned the same as lee-ready trade classification algorithm?

Ernie Chan said...

hi jimgreen,
You are correct.
Ernie

vincegata said...

Right, thank you. I guess the main task would be to correctly identify the orders that come from the institutions and not from the market makers. I'll look at the Amuk & Durbin book you mentioned earlier.

Curtis said...

I use order flow quite a lot in my discretionary trading, and have developed a powerful platform for seeing the order flow in the eminis and futures.

This software is designed for discretionary futures trader and is not suitable for quants.

However in general as a tape reader I find that order flow can be mean reverting or trending. During range markets, liquidity providers get payed for flipping order flow. Of course, when the order flow trends then that can be very useful as well for finding momentum based trades.

But, we don't just look at the market flow but also the depth which we find makes it much more valuable.

We offer a free trial at
http://orderflowdashpro.com