Friday, May 09, 2014

Short Interest as a Factor

Readers of zerohedge.com will no doubt be impressed by this chart and the accompanying article:

Cumulative Returns of Most Shorted Stocks in 2013

Indeed, short interest (expressed as the number of shares shorted divided by the total number of shares outstanding) has long been thought to be a useful factor. To me, the counter-intuitive wisdom is that the more a stock is shorted, the better is its performance. You might explain that by saying this is a result of the "short squeeze", when there is jump in price perhaps due to news and stock lenders are eager to sell the stock they own. If you have borrowed this stock to short, your borrowed stock may be recalled and you will be forced to buy cover at this most inopportune time. But this is an unsatisfactory explanation, as this will result only in a short term (upward) momentum in price, not the sustained out-performance of the most shorted stocks. This long-term out-performance seems to suggest that short sellers are less informed than the average trader, which is odd.

Whatever the explanation, I am intrigued to find out if short interest really is a good factor to incorporate into a comprehensive factor model over the long term.

The result? Not particularly impressive. It turns out that 2013 was one of the best years for this factor (hence the impressive chart above). For that year, a daily-rebalanced long-short portfolio (long 50 most shorted stocks and short 50 least shorted stocks in the SPX) returned 6.9%, with a Sharpe ratio of 2 and a Calmar ratio of 2.9. However, if we extend our backtest to 2007, the APR is only 2.8%, with a Sharpe ratio of 0.5 and a Calmar ratio of 0.3. This backtest was done using survivorship-bias-free data from CRSP, with short interest data provided by Compustat.

Here is the cumulative returns chart from 2007-2013:

Cumulative Returns of LS Portfolio based on Short Interest: 2007-2013


Interesting, trying this on the SP600 small-cap universe yielded negative returns, possibly meaning that short-sellers of small caps do have superior information.

I promise, this will be the last time I talk about factors in a while!

===
Tech Update:

I was shocked to learn that Matlab now offers licenses for just $149 - the so-called Matlab Home  (h/t: Ken H.) In addition, its Trading Toolbox now offers API connection to Interactive Brokers, in addition to a few other brokerages. I am familiar with both Matlab and R, and while I am impressed by the large number of free, sophisticated statistical packages in R, I still stand by Matlab as the most productive platform for developing our own strategies. The Matlab development (debugging) environment is just that much more polished and easy-to-use. The difference is bigger than Microsoft Word vs. Google Docs.
A reader Ravi B. told me that there is a website called www.seasonalgo.com if you want to try out different seasonal futures strategies.
Finally, a startup at inovancetech.com offers machine learning algorithms to help you find the best combination of technical indicators for trading FX.
===
Workshops Update:

I am now offering the Millisecond Frequency Trading (MFT) Workshop as an online course on June 26- 27. Previously, I have only offered it live in London and to a few institutional investors. It has two main parts:

Part 1: introducing techniques for traders who want to avoid HFT predators.

Part 2: how to backtest a strategy that requires tick data with millisecond resolution using Matlab.

The example strategy used is based on order flow. For more details, please visit epchan.com/my-workshops.

Additionally, I will be teaching the Mean Reversion and Momentum (but not MFT) workshops in Hong Kong on June 17-20.

146 comments:

experquisite said...

How do you deploy/colocate your MFT MATLAB systems? Do you compile them with the MATLAB compiler, or literally have a Windows VM running MATLAB on the desktop of a colocated machine?

Ernie Chan said...

Hi experquisite:

I use MATLAB only to backtest such MFT systems. Deployment requires re-coding as C# program.

But yes, you can also use Matlab C compiler to compile it into C or C++ executables before deployment.

In any case, all executables should be colocated.

Ernie

aaron said...

Hi,
Are there cheaper alternatives to Lime Strategy Studio? Or maybe a combination of different solutions for deployment of strategies without backtesting capability? I'm willing to pay someone with expertise in this area to help me find the best solution for me.

My background is discretionary trading and algorithm development but I don't know much about the infrastructure needed for deployment. I already have a backtesting environment but am looking for a cheaper avenue for colocated deployment than I currently have.
Thanks
A F R 6 2 6 2 @ G M A I L . C O M

Ernie Chan said...

Aaron,
If you are only looking for automated execution system, there are many options. E.g Ninjatrader, MetaTrader, TradeStation, etc. Please see p. 28 of my book Algorithmic Trading for a complete discussion.

The unique feature that Lime Strategy Studio offers is that it comes with a fill simulator. There is no other way to backtest a strategy that uses limit orders without a fill simulator.

Feel free to email me for further discussions if necessary.

Ernie

Anonymous said...

FYI -- https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2216919
shows that for low IO stocks (generally small caps), higher (than expected) short interest results in lower than expected returns.

It's not clear how performance has changed over time, however.

Ernie Chan said...

Thanks for the link!

Ernie

Anonymous said...

Hi Ernie,

What kind of simulator does IB paper account use? Is it good?

Ernie Chan said...

There is a difference between fill simulator and a paper trading account. IB has a paper trading account, and it is decent, though not very sophisticated. IB doesn't have a fill simulator: you cannot backtest a strategy with limit orders over historical period, though you can do walk-forward testing. But it can take many months of forward-testing to gain enough statistical evidence.
Ernie

Thomas said...

Where can you get access to CRSP and Compustat data? I thought their licensing fees were huge...?

Anonymous said...

Hi Ernie,

I trade pairs in IB paper account.
I enter each trade in small amount (15,000) to avoid cost impact.
I plan to test for several months.
Can I trust the results?

Btw, is there cheap and good fill simulator?

Thanks.

Ernie Chan said...

Thomas,
If you are affiliated with a university, or collaborate with a professor in a business school, you can get for free through WRDS.
Ernie

Ernie Chan said...

Anon,
Several months of walk-forward testing may be sufficient to see if your strategy has data snooping bias, but is not sufficient to find out what the max drawdown is.

The only fill simulator I know of is Lime Strategy Studio. We are thinking of building one ourselves.
Ernie

Anonymous said...

"Finally, a startup at inovancetech.com offers machine learning algorithms to help you find the best combination of technical indicators for trading FX."

Enrie, I am surpised to see you are promoting naive data-mining schemes. "Finding combination of indicators" means data-mining bias necessarily. Neverthless, there are so many softwares already in the market for doing exactly this.

E said...

Hi Ernie,
While it is true that the debugging environment of Matlab is better than that of R, but it is becoming less true by the day with Rstudio which, in the last version, has very similar debugging features. Minus the hefty price tag mind you.
Eran

Anonymous said...

Ernie,

Have you used Matlab's Trading toolbox and its connectivity to IB? Is it robust and reliable?

Ernie Chan said...

Anon,
Just because I mentioned a new service or startup on my blog does not necessarily mean I endorse it, unless I explicitly said so. The Tech Update is a public service to the community, and readers are invited to exercise their own judgment to see if a service is useful.
Ernie

Ernie Chan said...

Hi Eran,
I believe I have the latest version of RStudio, but I still find a big productivity gap with MATLAB's IDE.
I think $149 is not a big price to pay for this gap.
Ernie

Ernie Chan said...

Anon,
I haven't used the Trading Toolbox myself, but you can request a 1 month free trial to see for yourself.
Ernie

Anonymous said...

HI Ernie
I am very interested in your MFT courses. I am a college student in China. I tis not practical for me to fly hours to take your class. IS their any other way to do it?
Xun

Ernie Chan said...

Hi Xun,
Thank you for your interest. The MFT course is offered online: so you can take it anywhere. For registration details, please email me at ernest@epchan.com.
Ernie

trice said...

Hi Ernie,
I have been experimenting with trading via the Matlab toolbox and IB. One place that it seems to fall short is gaining real time (or near real time) access to last price snapshots of the entire NYSE/Nasdaq market. For me, this ends up being ~6000 stocks. Real time IB feed only allows 100 stocks at a time, and the getdata function takes ~1s per stock, which is not practical for market scanning. Do you have any suggestions for a cost effective data feed with Matlab that can scan the market quickly? Thanks,
Tyler

Ernie Chan said...

Hi Tyler,
I heard IQFeed provides the best real-time tick-based data feed. It is accessible through Matlab's Datafeed toolbox.
Ernie

Anonymous said...

Hi Erine

The MFT trading strategy that you are going to teach online has to be colocated to the trading server at the broker side?

In this case, the trading strategy needs to be programmed in web based which allows the user to change settings or stop the trading program if necessary?

thks

Ernie Chan said...

Any strategy that is sensitive to latency within 20ms or so should be colocated.

You will be able to access the colocated server just like you can access your desktop computer. For e.g. if it is a Windows server, you can use Remote Desktop Connection. So the trading programs can be identical to your desktop programs.

Ernie

Anonymous said...

Hi Ernie,

Could we install Matlab Home in cloud server, such as Amazon EC2?

Is the cost still the same?

Ernie Chan said...

Sure, you can install Matlab Home on any computer or server, including EC2. In fact, the advantage of installing it on EC2 is that you can run many instances of the same license simultaneously.
Ernie

Anonymous said...

Hi Ernie,

But if I terminate that EC2 instance someday, could I still install that Matlab Home into other server or computer?

Ernie Chan said...

Sure, the license is portable.
Ernie

Anonymous said...

Hi Ernie,

What is the difference between prop trading firm and hedge fund?

Anonymous said...

Hi Ernie,

I read ad posted by a prop trading firm. i.e.,

"My company can offer you 80% payout with no software fees 5$ per 1000 shares, min volume of 30k shares a day/sterling platform/ unlimited buying power/no licensing/$500 deposit/no overnights."

What does payout mean here?
Do traders trade their own money or company's money?

Ernie Chan said...

A hedge fund manages money for outside passive investors. A prop trading firm does not (it only manages money for its own principals and their employees or contractors.)

Ernie

Ernie Chan said...

If the payout is 80%, it is very likely that the prop trading firm requires you to invest your own money, but will allow you to trade with high leverage (i.e. buying power).

I never understood why anyone would join a prop trading firm that requires one to risk one's own money yet not pay out 100% profits (minus some fees and expenses). It is easy to have enormous buying power at a regular brokerage (which of course pays out 100% profits) if one trades futures or FX.

Ernie

Anonymous said...

Hi Ernie,

Have you heard about a prop trading firm, Bright Trading?
Could we trade this firm's capital?
It seems their trading platform is Redi-Plus. Does it have Java/C++ API, which we can use to do algo trading?
Do we really have to pass series 7 to become prop trader?

Anonymous said...

Hi Ernie,

For trading stocks in US, what is sufficient market capitalization for each individual stock ?

Ernie Chan said...

At Bright Trading, you also have to invest your own capital and take first loss. They will provide you with high leverage, just like other prop trading firms. Besides, you also need Series 7.

Rediplus does come with C++ and Excel API.

Ernie

Ernie Chan said...

There is no universal definition of sufficient market cap for a stock. It depends on your holding period: longer holding period allows you to trade smaller cap stocks because you can enter/exit in small chunks.

Ernie

Anonymous said...

Hi Ernie,

How is the default risk in US stocks market?
During your trading life, are there ever stocks you hold and trade default?
Or roughly, how many companies default every year in US market?

Anonymous said...

Hi Ernie,

What is the remote trading for a prop trading firm?

Could I trade for a US prop when I live in the UK?

Could I just install TWS(or other platform) on my desktop in my room or VPS and trade?

Ernie Chan said...

I haven't researched the default risk of US stocks. Generally, we trade only S&P 500 stocks intraday, so by and large we avoid that risk.

Ernie

Ernie Chan said...

Remote trading in a prop shop means you can trade from your own home desktop. You don't even need a VPS.

Whether you can do that in the UK depends on UK regulations.

Ernie

Anonymous said...

Hi Ernie,

I have a question and I am looking for some advise.

I started trying the API for my Forex CFD broker, while trying to see how the API commands work, iby building a simplistic market making strategy, with sell and buy orders below and above the market bid and ask (a wider spread than the market, by mistake I used market order types with specified prices. ( e.g.: market bid 1.4000, market ask 1.4005, and my sell "matket order" at 1.4007). As weird as that is, the order are executed, and they are profitble as soon as they are opened.

My questions: Is that legal? Should I inform my broker? Should I use real money and just profit?

Regards,

Anonymous said...

Hi Erine

1. Have you ever tried CQG Time and Sales tick data? How much difference if I backtest with 1M OHLC data?

2. For your MFT online course, will you teach any MFT trading strategy? it is not easy to find reliable FX tick data, and different broker quotes differently, does it matter? Thx

Kimi

Ernie Chan said...

The ability to execute market buy order below the bid may just be an artifact of the paper trading system - please confirm if this is true in live trading first.

There is no regulation concerning executions regarding FX markets which can happen on private platforms, unlike the stock market.

Ernie

Ernie Chan said...

Hi Kimi,
1) I have bought data from CQG tick data before, but to my horror, I have found missing ticks compared to the exchange feed. Also, their tick data has 1-min time stamps, making them useless for spread or arbitrage trading.

2) In general, sales tick data is far less useful than quotes tick data, whether at ms or min bars.

3) My MFT course does cover a millisecond frequency strategy in futures, but not in FX.

4) Indeed, if you trade FX, you should get tick data from your own broker (usually free). There may be significant differences between data from different brokers, and those differences are very real.

Ernie

Anonymous said...

Hi Erine

In your opinion, any tick data provider is good for backtesting commodity futures strategy and the cost is affordable.

Thank you

Kimi

Anonymous said...

Hi Ernie,

Can US based prop. firms (like Bright Trading) hire traders from foreign countries?

Anonymous said...

Hi Ernie,

How to pass series 7 exam?
Do we need a sponsor?
Would you recommend some reading material?

Btw, there are Friends&Family account, advisor account, prop firm account in IB. If we want to trade other people's money, which is the easy way to do?

Ernie Chan said...

Hi Kimi,
The most affordable tick data is from tinyurl.com/oxv4d6k that runs only in Amazon EC2 based on a data rental (not ownership) model. They have 1ms data on stocks, futures, and options.

Ernie

Ernie Chan said...

Yes, I believe Bright Trading will "hire" traders from outside the US, but only if those countries allow this business model. E.g. Canada won't allow it.

Ernie

Ernie Chan said...

To pass Series 7, you do need to be affiliated with a broker-dealer. I believe Bright Trading will facilitate that.

There are many Series 7 practice exam sites around - just google them. I have not had to study for a long time.

To manage someone else account via Friends and Family is very easy if that person belongs to an exempt category (usually means a family member or friend or a high networth individual). Otherwise, you need to register as a RIA with the SEC, or as a CTA with the NFA.

Ernie

Anonymous said...

Hi Erine

Thank you of your suggestion.

Have you tried the tick data from this company

http://www.tickdatamarket.com/tarifs.php?famille=futures&type=individuel

Do you think its price for " Trade & quotes + millisecond" is reasonable

Thank you

Kimi

Ernie Chan said...

Hi Kimi,
Their price seems OK, but I won't know whether the data quality is good (i.e. whether there are missing ticks).
Ernie

Anonymous said...

Hi Ernie,

I find that a prop firm, Maverick Trading, uses IB TWS.

Do you know any other prop firm uses IB TWS?

It seems that each prop firm uses different trading platform. Do we have to use their platform? Or we can ask to use IB TWS.

Thanks.

Anonymous said...

Hi Ernie,

Could we install RediPlus C++ API as a demo? Where could we find its manual?

Is it as easy as IB Java API or better?

Ernie Chan said...

I don't know of any prop firms that use TWS.

In general, you are required to use the platform provided by the broker that has a Joint Back Office agreement with the prop firm.

Ernie

Ernie Chan said...

You can request the prop firm to let you download the Rediplus C++ API demo and documentation, but I don't think Goldman Sachs will just let everybody download it from the website.

It is a pretty professional API with lots of spread trading and order routing features, though maybe not as user friendly as IB API.

Ernie

Ken H. said...

> I don't know of any prop firms that use TWS.

That made me pause. Do you mean just not use TWS itself, or do not use IB either. I can see not using the TWS GUI (yuk), but would some still be using an in-house trading engine through the gateway to IB?

Guess my point is, I thought IB had a decent mix of retail and pro, but I could be quite mistaken of course. Curious to your understanding of this!

Ernie Chan said...

Ken,
I am not aware of any prop trading firm that has Joint Back Office Agreement with IB, but of course I haven't done a complete market survey on that.

There are of course numerous prop trading firms that use IB as brokerage, but in those cases the traders are strictly employees of the firm, not LLC members.

Most prop firms do not use in-house trading software. For e.g. Bright Trading uses Goldman Sachs' Rediplus platform for trading, either manually or through API.

Ernie

Ken H, said...

Thanks Ernie for the clarifications!

Anonymous said...

Hi Ernie,

We usually use trade prices ( last prices) to backtest our pairs trading strategy. However, when we trade, we need to use middle prices (average of bid and ask prices), right? They are the latest quotes, where we can really trade.

Thanks.

Ernie Chan said...

It is not a good practice to use last prices to backtest pairs strategy. Bid-ask quotes should be used: otherwise backtest results will be inflated. You should assume that only MKT orders can be filled, thus paying the bid-ask spread every trade.

When you trade live, there is no guarantee that you can get filled at mid point.

Ernie

Anonymous said...

Hi Ernie,

Do you mean that we need to use Bid, Ask, and Middle prices (average of Bid and Ask) to backtest pairs strategy, assume that only MKT orders can be filled, thus paying the bid-ask spread every trade?

Ernie Chan said...

If you use bid-ask, there is no need to use mid.
Ernie

Anonymous said...

Hi Ernie,

I mean, when we run regression, compute z-score, and build buy/sell signals, we still need prices (last or middle) from both legs of a pair. Then we only need bid-ask prices to compute P&L.
Shall we choose last or middle prices?
Btw, where can we get bid/ask one min bars data?

Ernie Chan said...

Mid is ok for regression purposes. You can get historical quotes from IB's API.
Ernie

Anonymous said...

Hi Ernie,

May I ask where we can find split/dividends adjusted bid/ask one min bars historical data?

Is it serious a problem if we use unadjusted data to backtest stock pairs strategy?

Thanks.

Ernie Chan said...

You would have to adjust the bid/ask yourself by using the adjustment factor for the daily data.
Yes, it can be a serious problem if you don't make this adjustment.
Ernie

Anonymous said...

Hi Ernie,

I quickly try QuantGo data you mentioned before as a cloud data service.

I find we can download data from instance to our desktop, just as we can do with Amazon EC2 instance.

Do they have bid/ask one min bars historical data?

In demo, I only successfully download EOD data, so I have not seen the sample of intraday data.

Ernie Chan said...

If you download significant quantity of data from QuantGo to your desktop, you will trigger an alarm. (I have tried that myself.)

QuantGo is designed specifically for 1ms time-stamped tick data. For daily data there are much cheaper sources.

Ernie

Anonymous said...

Hi Ernie,

We can only download one year data from IB.

May I ask where we can get longer history of one min Bid/Ask bars for US equities? say, 2 years.

Thanks.

Ernie Chan said...

You would have to rent from Quantgo or buy from Nanex.net.

Ernie

Anonymous said...

Hi Ernie,
I ask QuantGo about their service.

Besides data fees, to use QuantGo, we need to pay normal Amazon instance fee + 20%.

For Instance Security, they say they have NO access to instance as only we have the private key.

Do you think their instance security is ok?

Ernie Chan said...

Please note QuantGo is for backtesting, not live trading. I don't think security is a big issue for backtesting and I am sure that they have a good enough security for that.
Ernie

Anonymous said...

Hi Ernie,

I mean, backtesting code is good enough to know other people's trading strategy.

Basically, my backtesting code is very similar with my live trading code. I just re-write it in Java code for IB.

However, I think it is worth trying QuantGo since we do need data. Especially, I do use Amazon EC2 myself. We just need to do everything on EC2 for using their data.

Anonymous said...

Hi Ernie,

As you mentioned in your book,
we can do dividends adjustment by subtracting $d instead of multiplier, and then the historical daily changes in prices will be the same.

I may prefer subtracting $d because it is easier. Is there any drawback for this method?
Btw, I calculate Sharpe ratio based on P&L instead of returns.

Thanks.

Ernie Chan said...

The only drawback of adjusting dividends by subtraction is that daily returns will be distorted. But since you measure Sharpe by PL instead of returns, this actually is the preferred method.

Ernie

Anonymous said...

Hi Ernie,

So we can do dividends adjustment by subtraction, and split adjustment by a multiplier 1/N together?

Thanks.

Ernie Chan said...

Yes. It isn't ideal, but we can tolerate the small errors.
Ernie

Anonymous said...

Hi Ernie,

I find in IB one min BID/ASK bars, there are some extreme values.

May I ask how you deal with outliers?

What kind of filters do you use?

Thanks.

Ernie Chan said...

Those outliers are also present in live trading, which means that you should make sure your MKT orders are not sent when the bid is too low, and vice versa.

Ernie

Anonymous said...

Hi Ernie,

Have you read this paper on Internal Bar Strength for ETFs? Would be interested in your thoughts?

http://qusma.com/wp-content/uploads/2013/09/The-IBS-Effect-Mean-Reversion-in-Equity-ETFs1.pdf

Many thanks

Ernie Chan said...

Yes, I have read the ETF mean reversion paper.

I have found that most intraday mean-reverting ETF strategies would look profitable in backtest, unless we use bid-ask data in which case most of them will collapse.

Ernie

Anonymous said...

Hi Ernie,

Where do Bid/Ask one min bars outliers really come from?

Is it better to send orders to primary exchanges than "SMART" through IB?

Ernie Chan said...

Outlier quotes may occur because of temporary liquidity withdrawal, i.e. a mini-flash-crash. The cause is complex but can be due to HFT activities. This is one of the topics covered in my MFT course.

In my opinion Smart is usually better.

Ernie

Anonymous said...

Hi Ernie,

Is it better to send "MKT" oders for both legs of a US stock pair at the same time when we trade pairs?

Ernie Chan said...

If your strategy can afford to pay the bid-ask spread for every trade, you can do that. Otherwise, you need to use LMT for at least one leg.

Ernie

Anonymous said...

Hi Erine

I would like to know the MFT trading program online is focus on which type of future contract?

How high is the trading frequency per month usually?

The strategy needs to be colocated at the broker side?

Thanks

Jay

Ernie Chan said...

Hi Jay,
The example strategy is on E-mini SP500 contract, but the methodology is adaptable to any type of future or even FX.

The trading frequency is about one round-trip every 5 min.

All MFT strategies should be collocated with broker/exchange servers. The cost is usually minimal.

Ernie

Anonymous said...

Hi Ernie,

I wonder which brokers provide colocation service for retail traders.

Thanks.

Ernie Chan said...

I believe FXCM (and various other FX brokers) allow colo.

In futures, Mirus and Lightspeed both do.
Ernie

Anonymous said...

Hi Erine

THE MFT online course is $1,450 USD or 1,450 pound?

Any Slot still available?

Thx

Ernie Chan said...

It is USD 1,450. Yes, slots still available.

Best,
Ernie

Anonymous said...

Hi Erine

Based on your information, the MFT strategy can generate few thousand trades per month if I apply it to FX market ?

Is it easy to find reliable FX Bid/Ask quote to test the MFT strategy?

Jay

Ernie Chan said...

Hi Jay,
Some versions of the example strategy requires not only bid-ask quotes, but trades as well.

HotspotFX will give you historical bid-ask ticks if you are a client.

You can also choose to trade FX futures instead of spot FX.

Ernie

Anonymous said...

Hi Ernie,

Do you currently trade a MFT strategy yourself?

Thanks

Anonymous said...

Hi Ernie,

A bit off topic but I was wondering what your thoughts are on fading gap strategies? Is that something you've traded yourself?

Do you you know of a good source to study gaps (i.e. OHLC) for futures. I guess for gap analysis I need non adjusted prices.

Many thanks

Anonymous said...

Sorry by source I mean data provider for daily futures data

Ernie Chan said...

The short answer is "yes". The long answer is that we have to define what exactly is a MFT strategy: it doesn't mean a strategy that only holds for 1-20 ms. It means a strategy whos profitability is impacted by latency of the order of 1-20 ms.

Ernie

Ernie Chan said...

Yes, I have traded gap strategies like the one I described in page 92 of Algorithmic Trading.

csidata.com provides good unadjusted futures OHLC data.

Ernie

Anonymous said...

Hi Ernie,

Say I would like to incorporate a stop loss (not trailing) in my backtest and I only have daily closing/high/low prices available.

Would you agree that a simple way of doing that is to say if todays low<X where X is the stop loss level, assume you closed at X.

This of course ignores the volume you could have realistically traded at that level but might be a good proxy for a highly liquid instrument and where you as a small guy is an epsilon of daily volume.

What do you think?

Thanks for a great blog.

Ernie Chan said...

Thanks for your kind comment.

Yes, you can use the low of the day to determine whether a sell stop has occurred at your stop price.

Ernie

Anonymous said...

Hi Erine

I saw you have replied someone saying that the trading frequency of your MFT is about one round-trip every 5 min."

Does it mean that I will have over 10 trades in an hour and close to 100 trades daily?

And the strategy works for different asset class?

Thanks

michael

Ernie Chan said...

Hi Michael,
Yes, that's the FX strategy that we used to run for a while.

We now run a different one that holds for about an hour, but still sensitive to latency around 20ms.

No, this isn't a universal strategy that runs on every asset class. Few such strategies exist.

Ernie

Anonymous said...

Hi Erine

Thank you of your reply!

1. You are trading the MFT on FX spot or FX futures?

2. Which FX broker you are using? and may I know the reason of switching the holding time from 5 mins to an hour?

3. If I sign up the online course, which one I will be teaching , 5 mins or an hour holding time

Thanks

Michael

Ernie Chan said...

Michael,
1) It was spot FX.
2) We were test-trading on HotspotFX. Due to the market structure there (an issue that will be discussed in the MFT course), we found slippage to be too large for our strategy. So we continue trading only the one-hour version.
3) The sample strategy in the course is quite similar to the one we traded at HotspotFX, but adapted to E-mini.
Ernie

Anonymous said...

Hi Erine

By switching the trade holding time to an hour, how many trades per day usually?

How much HotSpot FX charges for colocation?

Do I need to re-program the MFT strategy by using their development platform? or just use their API to build the connection?

Any minimum fund size requirement from HotSpot for colocation

Thank You

Michael

Ernie Chan said...

Hi Michael,
Please note the 5-min and 1-hour strategies are different. I won't be talking about the 1-hour strategy, which trades about 2 roundtrips a day. Also, please note that I do not provide execution codes in my class, which are specific to the API you are using. I will provide backtest codes, which you will have to adapt to execute the strategy for a specific API. In fact, I always recommend traders to modify and improve a basic strategy and therefore its backtest codes also.

HotspotFX does not provide colo. You can rent your own box at a few hundred dollars a month at their NY4 data center from a 3rd party vendor. I can provide further instructions in class for this process.

To trade on HotspotFX, you need a prime broker. The minimum account size is about $0.5M at the prime brokers that I know of. Again, I can guide you through this process in class.

Ernie

Anonymous said...

Hi Ernie,

About 'First Half-Hour Return Predicts Last Half-Hour?', I backtest the latest one year data for SPY and some other stocks including transaction costs. The equity curve looks bad, especially for stocks, very bad.

Ernie Chan said...

Thanks for reporting your "First Hour..." backtest. Yes, that corroborates with my finding too. Oh well...

Ernie

Anonymous said...

Hi Ernie,

For mean reverting fx strategies, have you analysed mean reverting behaviour based on time of day, i.e. London session vs asia etc?

What type of data does your MFT strategies require, i.e. price data only?

Many thanks

Anonymous said...

Hi Ernie,

I am wondering about this case:

close today
ETF A50 2823 8.360 -0.080 (-0.948%)
FTSE China A50 Index 6,675.46 -43.22 (-0.64%)
Hang Seng Index 22,804.81
-389.25 (-1.68%)

ETF A50 2823 in Hong Kong drops around 0.3% more than FTSE China A50 Index, and the main reason should be because Hang Seng Index drops -1.68% today.

How should I calculate the relationship between A50 2823 ETF and Hang Seng index? This ETF suppose to almost 100% following the A50 index but there is serious different when hang seng has a daily big movement.

Thanks,
-HK

Ernie Chan said...

Hi HK,
That sounds like a good arbitrage opportunity by buying the ETF and shorting the future, assuming that the profit is still there after accounting for the ETF bid-ask spread!
Ernie

HK said...

Hi Ernie,

Do you mean shorting the A50 future? There isn't A50 future in Hong Kong, but there is A50 future in SGX and it is very active.

Is that A50 ETF in Hong Kong supposes to be almost 100% move with A50 future in SGX? Even thought these two are in two different exchanges?

I think there isn't A50 ETF in SGX. The markets are very interesting.

-HK

Ernie Chan said...

Hi HK,
Yes, in principle you can arbitrage between the ETF and the future traded in a different country as long as it also tracks the same index. Of course, you need to be careful about currency conversion when backtesting/implementing such strategies.
Ernie

HK said...

Hi Ernie,

I just compare the exact same timing open and close pricing then there isn't a price different between ETF in HK and future in SGX. My mistake was that I didn't look at the A50 index open and close pricing.

The single trading unit of A50 in HK is kind of much bigger than the A50 future in SGX. It is around 2 units of A50 ETF in HK equal to 3 units of A50 future in SGX. So sometimes when A50 ETF in HK hasn't moved one unit yet, A50 future in SGX would move 1 unit already. This is almost no risk profitable arbitrage strategy if someone has very low commission, but definitely not IB retail brokerage fee rate. Is there is a name for this kind of "spread arbitrage"? Is this kind of arbitrage very popular in US before they made the trading unit much smaller to prevent ibanks kind of FI kept taking this kind of profit?

Thanks,
-HK

Anonymous said...

Hi Ernie,

I trade through IB. Do you know if anyone in the market can see my stop loss orders (US equities)?

How can one avoid HFTs and others to basically sniff out the stop losses and gun them?

Thanks

Ernie Chan said...

Hi HK,
What you described is a kind of index arbitrage, still being traded everyday by investment banks who seldom have to pay bid-ask.
Ernie

Ernie Chan said...

Hi Anon,
Good question.

I have asked various people if the ITCH feed of exchanges will disclose stop orders, but haven't received any answer.

But in practice, I have found my stop orders often get "hunted". Hence I no longer enter stop orders.

Ernie

Anonymous said...

Hi Ernie,

For mean reverting fx strategies, have you analysed mean reverting behaviour based on time of day, i.e. London session vs asia etc?

What type of data does your MFT strategies require, i.e. price data only?

Many thanks

Ernie Chan said...

Indeed, there is intraday seasonality in FX that affects many strategies.

One of my MFT strategies requires quotes data, the other requires trades and quotes (TAQ).

Ernie

Anonymous said...

Hi Ernie,

Do you know where to get long-term intraday FX spot data (minute or up to hourly bars are fine) for a large cross section of pairs?

Thanks

Ernie Chan said...

You can get intraday FX data from your own FX broker (e.g. Interactive Brokers). If you trade on an ECN, you can ask the ECN (e.g. HotspotFX) to provide for free.

Ernie

Anonymous said...

Hi Ernie,

Have you heard about Nevis Trading Group? It is a offshore prop trading firm.

Ernie Chan said...

No, I haven't heard of Nevis. I am always concerned about the financial stability of prop trading groups, especially offshore ones that are not regulated as broker-dealer.
Ernie

Anonymous said...

Hi Ernie, do you have any good papers or other sources for intraday seasonality in FX markets?

Ernie Chan said...

The only published intraday FX seasonal strategy I heard of is London Breakout. Just google the term.
Ernie

HK said...

Hi Ernie,

For the most popular ETF in Hong Kong, the transaction volume is high only in open and end time period most of the days, but I trade during the day. Do I have higher chance to execute trading buy/sell orders by decreasing the size, or have a more "standard" size like 10000 instead of 11300 something like that?

For example, 2823 A50 ETF is highest trading volume ETF in Hong Kong in most of the days. Its minimum trading unit is 100. I had hard time trying to buy/sell with 17700 size for few days. I tried 17000 but doesn't help. I tried 10000 but then I don't see it helps today. I had to wait for so many minutes even half hours to execute. Sometimes it execute because the price drops below my limited price order.

I check the volume records today, at least over 70% of the minutes the trading volume is higher than 17700 size. The interesting thing is that at the end of the day, 2823 follows hsi to raise but SGX A50 future doesn't move at all(I watch both at the same time). I had hard time to buy today may be because some insiders kept collecting 2823 for the whole days and push the whole 2823 up at the end, this is just a guess.

Thanks,
-HK

Ernie Chan said...

Hi HK,
Is there a dark pool in HK? For executions at large size intraday, you might have to do that in such pools, or else you have to break up into a series of small orders.
Ernie

Anonymous said...

Hi Ernie,

You said you tested the short interest data from Compustat, but do they have the data daily updated? I asked this because I knew Bloomberg only has bi-weekly data, which is obtained from exchange. I doubt that Compustat has daily short interest data. However, you mentioned your portfolio is "a daily-rebalanced long/short portfolio". If the data is bi-weekly, why you daily rebalance it?

In addition, did you test other universe, for example Russell 3000 as a whole, or Europe universe?

Thanks,
Vincent

Ernie Chan said...

Hi Vincent,
You are correct that Compustat short interest data is also updated only bi-weekly.

However, the portfolio does need to rebalanced daily because stocks are added or deleted from the SPX.

No, I have not tested this strategy on universes other than SPX and SML.

Ernie

Anonymous said...

Hi Ernie,

Are you able to share with us the type of strategies you trade and return YTD?

Regards

Ernie Chan said...

We mainly trade 2 mean reverting strategies for FX and stocks, a momentum strategy for futures, and a few other strategies with minor allocations for testing.

The stocks and futures strategies are essentially flat YTD, but the FX strategy gained about 20%.

Ernie

Anonymous said...

Many thanks! Is the stock strategy on ETF or single stocks?

I assume that is levered return?

Ernie Chan said...

The stock strategies is a stock selection model.

Yes, all returns reported are levered.
Ernie

Anonymous said...

Hi Ernie,

Do you mainly trade G10 currency pairs?

Do you mind me asking what sort of frequency your fx strategy is trading?

Regards

Ernie Chan said...

We trade the most liquid 13 FX pairs.

Frequency is intraday, with holding periods from minutes to hours.

Ernie

Anonymous said...

IB trading tool box leaves much to be desired. Just to download historical data for several stocks at one time I had to develop my own wrapper. In the process I developed efficient interface realtime data and orders placement with stoploss.

Unknown said...

Hi,

I wanted to get your thoughts on how to analyse an HFT model where you are regressing two irregular (temporally uneven) time series, for example if you were to regress 10yr Notes vs 30yr bond futures. The idea is to find a relationship between the two so if the price of the 10yr flips bid you would buy the 30yr if it is the model says the expected price of the 30yr is higher.(I am not saying this would actually work with a simple model like this but it would a start of a more complex system)

How would you construct the model?

My idea is to regress the 30 yr on the 10yr over a rolling x min period using y min or second intervals to generate a linear regression using the change in (bidsize*askPrice + askSize*bidPrice)/(bidSize +askSize).

This should give a set of returns that can be used for the regression.

We can then use an ADF test to check the validity of the model i.e. it cointergrates well.

Next using the model determine if the 30yr is over or underpriced when the price of the 10 yr changes.

Is this a plausible method or am I missing something? Is there a better way to deal with the fact the data is irregular i.e use each tick to be a point with time in between assumed to be regularly spaced. Also should you smooth the data first using an EMA or something similar?

Thanks

Ernie Chan said...

Hi akhil,
First, let me point out that you cannot run adf test on returns series. Returns series are always stationary. You can only run that on price series.

Second, the highest frequency historical data I have ever seen has at least 1 ms bars. A time series is irregular only if your time resolution is not small enough. I see no difficulty in analyzing any time series as evenly spaced bars if you use the shortest bars available.

Third, I don't see the benefit of analyzing the stationarity of a price series using size-weighted midpoints. One may ask: why only weigh that by the best bid and ask? Why not weigh the entire order book? In my opinion, a simple midpoint based on best bid-ask is good enough.

Ernie

Anonymous said...

Hi Ernie,

Regarding your fx strategies, how many roundtrips do you normally do and do you trade each pair every day? May I ask what sharpe ratio you've achieved with your fx strategies?

Will you be offering your MFT course online again?

Many thanks for a great blog!

Regards

Ernie Chan said...

Our FX strategy nowadays trades about 2 round trips a day due to low volatility. Sharpe ratio (net of incentive and management fees) is over 1.2 in the last 3.5 years.

The online Mean Reversion course will be offered Aug 26-28. Please email me for further details.

Ernie

Anonymous said...

Hi Ernie,

For mean reversion fx strategies I assume you have a profit target. Do you think it makes sense to also have a stop loss? Even though for mean reversion this is a bit counterintuitive, but when there is no true cointegration it might make sense?

What do you believe to be the main factors for the current low volatility environment and how do you think volatility will evolve over the next couple of years?

Many thanks

Ernie Chan said...

The short answer: yes, it makes sense to have a stop loss for mean reverting strategies.

The long answer: please see page 183 of my book Algorithmic Trading.

As for explanation of low volatility, please see my Tweet @chanep: "Excellent explanation of why volume collapsed across all asset classes: http://shar.es/MOgJJ"

Ernie

Anonymous said...

JP Morgan put out a piece early last year that showed that short interest traders are more informed. So, your conclusion agrees with theirs.

Anonymous said...

Hi Ernie,

You said the following toward the top of this comments section:

“I use MATLAB only to backtest such MFT systems. Deployment requires re-coding as C# program.”

Why do you re-code them in C# instead of using the MATLAB compiler to compile them directly into C or C++ ?
Does re-coding them in C# somehow give you lower latency when live trading then compiling them into C or C++ via MATLAB compiler?

Thanks (and love your books FYI)!

Ernie Chan said...

We need to recode all execution codes in C# so that they conform to our execution architecture with all the bells and whistles. Just compiling using Matlab compiler does not give us that flexibility.

However, you can certainly use the Matlab compiler if you have their Trading Toolbox, or is running a standalone strategy.

Ernie