Sunday, August 22, 2010

Phantom quotes

Have you ever got the feeling that your market orders are often filled at prices worse than the NBBO displayed on your trading screen? Apparently, this may be the result of deliberate manipulation of the market by high frequency traders. These HF traders submit thousands of quotes per second to the NYSE ("quote stuffing") and then cancel them within 50 ms. This slows down the exchange data queue so much that by the time a quote is transmitted to you, it is stale already, even if your trading server is collocated at the exchange. (Checking the time stamp of the quote is of no help: the time stamp is based on the time the quote enters the queue, not when it exits the queue.)

If you can no longer believe in the quotes, is there any integrity left in the market? Much as I think that HFT may be useful liquidity providers, I can't see how this specific practice could be good for anyone over the long term.

(Hat tip: Jim Liew of Alpha Quant Club.)

31 comments:

Erik Rigtorp said...

This is mostly a result of Reg NMS. Since the SIAC feed is 5-8 ms slower than the other feeds traders with their own consolidated feed built from the separate exchange feeds can see where the NBBO will be after the next update and if they are broker dealers they can post orders on the future NBBO in advance by raising the inter market sweep flag. So broker dealers always has an advantage over retail. This advantage is created on purpose by Reg NMS. The quote stuffing is most likely caused by order routers that can't send inter market sweep orders. In order to try to reduce the impact caused by NYSE and SIAC feeds being inferior to all other exchanges they have to "ping" the market by continously sending orders that are rejected due to Reg NMS locked market rule until it finally hits the book but still before the new NBBO have shown up on the SIAC feed. Solution: Remove the concept of a NBBO, arbitrage traders will take care of that anyway. NYSE will then be forced to upgrade their systems to stay competitive. Why does Reg NMS say that a zero bid-ask spread is illegal? Well it does and they call it a locked market.

Anonymous said...

Unfortunately, there's no order cancellation charge for institutional traders. Had such charge applied, all phantom quotes couldn't have existed.

Anonymous said...

surprised this is news to anyone on this feed. This has gone on for at least 8-10 years on the futures markets (particularly, during low volatility periods in the market, and with the pro-rata matching engines). You will also see large orders just behind the BBO flash in and out to influence the market place as well.

Anonymous said...

I find this whole quote stuffing accusation absolute nonsense. ALL the exchanges have message rate limits in place and round robin among order placers to ensure fairness. In fact, sending out many messages has the opposite effect of slowing yourself down! Nanex and ZeroHedge really have no clue.
erkki is right, the only truly unfair rule right now is RegNMS prohibition of locked quotes. The large BDs definitely have a huge edge here.

jesse slingerland said...

Hey Ernie,
Could you tell me if my thinking is right?
I noticed when using your Max Drawdown duration code, that if the last cumulative return of the last open position is not the highest cumulative return then a drawdown length calculation begins. So, even if this position is exited on the very next day (which seems like one day drawdown duration) the drawdown duration will be calculated for the remainder of the data set. This happens because the cumulative return will be a constant for the remainder of the series and therefore, so is the drawdown(t) calculation. Since drawdown is a constant number for the remainder of the time series, the if statement of the calculateMaxDD code “if(drawdown(t)==0)” is not activated.
A quick remedy for this is to pass pnl along in the calculateMaxDD(cumret) function, and to make the for mentioned if statement
if (drawdown(t)==0 || pnl(t)==0)
drawdownduration(t)=0;
else
drawdownduration(t)=drawdownduration(t-1)+1;
end

Does this make sense? I would like to know if there is a reason my thinking is wrong or if there is some kind of max drawdown duration calculation standard for reporting purposes that I am missing.

Jesse S.

Ernie Chan said...

Hi Jesse,
If you have entered a position on day t, lose some money and exited the position on day t+1, and never enter any position again, then clearly your drawdown duration will extend to infinity (i.e. the current date) since you have never regained the high watermark. The code that you reproduced here reflects this calculation as well.
Best,
Ernie

Jesse said...

ok, Thank you.

henn.de said...

Hey Ernie
I have a question concerning the timing of pair trades. Generally I have the criteria to enter into trades with no less than 2 std. from the mean, but I wonder whether you have experience with using technical indicators on the spread curve that helps one to better define at what point to enter and exit a pair trade.

Ernie Chan said...

Hi Henn,
No, I don't have any other technical indicator to determine entry. In fact, I advocate using parameterless entry, where you scale your market value (position size) proportional to the zscore.
Ernie

Anonymous said...

Exchange should start charging small fees for cancelling and changing orders.

For example, if an order over certain size from the same account is changed or cancelled within 50ms, then charge some extra fees.

I think it is unfair for HFT to trick the market like that.

henn.de said...

I have been thinking about using elements of momentum theory to increase my chances on getting the timing better with my trades and I am particulary interested in this paper http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1663266
What do you think about the idea ? Joined the alpha quant club by the way.

Ernie Chan said...

Hi Henn,
Thanks for the link to the article -- but I wish these academic researchers will start running their tests on intraday data!

I believe that short-term momentum is very hard to capture in stocks. Kudos to anyone who can get the timing right and take profit in time!

Ernie

henn.de said...

Indeed. At the moment my horizon for my pair trades mainly in ETF's is an average of 16 days. While I think there is value in trying to add more theory and math to some extent, I have experienced that having some gut feeling, nonquantitive decision making that is, helped me make better trades and prevent me from entering those where numbers look too good to be true.
So far my "timing" has been to enter after a strong widening of spread in the trading day, for example for GDX/SlV on friday, but that could be put into a more quantitive model very easily I guess. I am just starting with making my own codes and so far use quite crude tools.

Puzzled said...

Hi Ernie,

I am new to trading and have been trying to develop a first simple trading strategy. Reading your book and blogs, I thought to give "mean reversion" a try. However, I am a bit confused, as all my results have been poor so far. I am trying to implement mean reversion in a framework of day trading (window ranging from a couple of seconds to a couple of minutes) on a single stock (*not* market neutral).

I was wondering if these are the settings that you have on mind whenever you talk about "mean reversion", or whether I totally misunderstood you, and you think more of pair trading in this context?

Ernie Chan said...

Henn,
Yes, you can potentially add some rules to turn your pair trading model into a day trading model, which would vastly increase the returns via the day trading leverage.
Ernie

Ernie Chan said...

Hi Puzzled,
Certainly there are mean-reversion on stocks both on an intraday timeframe, as well as pairs on an overnight horizon.

However, mean-reversion on an intraday timeframe is more difficult to research, because the entry and exit times need to be optimized quite delicately. Hence I think for a beginner, pair trading ETF's are the easiest and least risky.

Ernie

Anonymous said...

Hello, I have been reading your blog and this interesting quote from you caught my eye..
"I advocate using parameterless entry, where you scale your market value (position size) proportional to the zscore."
Does this mean that for example if you have a zscore of 2 you might put up 2% of porfolio and for a zscore of 3 you might put up 3%? Could you expand on your comment Ernie?

Ernie Chan said...

Hi Anon,
Yes, your interpretation is correct.
Ernie

Puzzled said...

Hi Ernie,

thanks for your tip.

I tried in the last couple of days to play with the GLD/GDX pair, downloading the data from Yahoo, and using R as in Paul Teetor guest blog to test for co-integration and calculate the hedge ratio (Date range is 2006-05-23 to 2010-09-28). I got that the ADF p-value is 0.2184305, and the hedge ratio is 2.066777. Plotting the spread, it seems that it had a big jump around 2 years ago - probably during the financial crises. Given the p-value, is this still a valid pair? Would it make sense to strict the analysis to a sub-period of the above dates - say only for the last 1.5 years? I also was wondering regarding the hedge ratio, which differs from the one available on your subscription web-page. How to decide which one to use?

Ernie Chan said...

Hi Puzzled,
Yes, GLD-GDX lost cointegration around 2008. So it may be best to run any backtest or cointegration tests on 2009-10 data.

I have not updated the hedge ratio on my website for a deliberate reason: I would like to demonstrate that some pairs remain stationary, while others do not. I.e. I wish to avoid survivorship bias. I recommend recalculating the hedge ratio regularly, maybe even daily for actual trading.

Ernie

Anonymous said...

Hello Ernie,
I thank you in advance for your time.
1. When trading a particular pair do you find the optimal zscores to enter and exit for each pair or do trade all of your pairs using the same enter and exit zscores?

2. Do you enter into your pairs the first time they hit a particular z-score threshold or do you wait for the second time the threshold in touched, in hopes that the spread is now converging.

Ernie Chan said...

Hi Anon,
1) I would advise using the same entry/exit Zscores for all pairs to reduce data snooping bias. In fact, you can try the "parameterless trading" method which sets market value to be proportional to the Zscore.
2) I enter whenever it hits the entry Zscore without waiting.
Ernie

Puzzled said...

Hi Ernie,

thought to share my findings and impressions after back-testing the GLD/GDX pair, as I got really impressed from the profitability of the strategy. I have been struggling for months in vain with other strategies, and now I feel like the fish in the Zen story that was asking "where is the ocean?"

Moving to applying the strategy in real-time, I was wondering what is the best time to send an order? Is it early in the day based on the price data available till yesterday (in which case the limit combo order would not necessarily be filled as the spread might have got a totally different value than yesterday), or maybe 1 minute before the end of the trading day, while sampling the prices at that time and taking them as the close price of the day? In the latter case, one would need though to adjust the prices based on dividends and splits, which I am not sure how to do.

Thanks.

Ernie Chan said...

Hi Puzzled,
I recommend trading just a minute before market close.

However, no matter when you enter a trade, you would need to use split/dividend adjusted historical prices. You can use either Yahoo Finance or csidata for such historical prices.

Ernie

Anonymous said...

Ernie,
You mentioned calculating beta for hedge ratio every day. I have been doing this in my back test. One day out of the training set I calculate a new beta using the most recent 252 days. Then for every day I calc a new standard deviation and mean (using that rolling 252 day window) and then make a daily z score. My results so far, are not very superior to the boilerplate way you showed us in your book (thanks). Wonder if im doing something wrong. Should I continue down this path? I expected better results from making my code more dynamic? I could use some direction.

Ernie Chan said...

Anon,
You can try a much shorter lookback for your hedge ratio and half-life calculations.

Ernie

Anonymous said...

hi ernie,It seems that you would recommend strategies that don’t involve holding over night, for pairs trading strategies. I happen to have access to intra-day bid, ask data and want to try this out. I have mostly been using cointegration for my multi-day strategies. For intra-day would I use strictly correlation? Or could I use both correlation and cointegration. Would testing time period sample be intra-day over last couple days? Are there any sources I could read about intra-day pairs trading to answer these types of questions? thanks

Ernie Chan said...

Hi Anon,
Sure, you can try pair trading intraday, but you don't need correlation nor cointegration for that. You can just backtest your strategy with your intraday data, assuming you have a lot of trades to gain sufficient statistical significance. But you would still need to backtest this for at least 6 months. Sorry, I am unaware of any articles on this particular topic.
Ernie

Anonymous said...

This Trader lost over $100k

http://stockmarketloss.wordpress.com

Anonymous said...

Thank you, I have another question on the intraday pairs trading? I can get consolidated trades or consolidated quotes for the day. Do you have any advice as to what data I would be better off using? The quotes seem more complete since they will provide me with multiple bids and asks for every second. If I use the trades I won’t have data every second, but the data will be much less complex to sort. In intraday back-testing is it essential to use bid and ask prices as opposed to actual trade data?

Ernie Chan said...

Hi Anon,
Why not use bid-ask-last to backtest? If not possible, then last is good enough.
Ernie