Algorithmic Trading Technology Update

Lately a number of new (at least to me) technologies useful to the algorithmic trader came to my attention:

1) Matlab2IB API

I said in my book that it is difficult to use Matlab as an execution platform. As Max has pointed out, this is no longer true. This inexpensive API connects Matlab to your Interactive Brokers' account. It allows you to retrieve historical data, get real-time quotes, and send orders. In other words, all the basic functions you need to create your own execution engine.

2) R

Many people (hat tip: Steve H.) know that R is an open-source (i.e. free) alternative to Matlab. I find that there is also an API that connects R to Interactive Brokers, though I have not tried it myself.

3) Trade Ideas

Trade Ideas (hat tip: Russell M.) is a complete automated trading platform that provides connections to different brokerages (scottrade, IB, TD Ameritrade, etc.)

4) Amazon EC2 cloud computing platform

Running out of PC's to run your myriad strategies? Try Amazon's EC2 cloud computing platform. For a modest hourly fee, you get access to an instance of either Linux or Windows environment, and you can add as many instances as you want. The connection speed is supposed to be at least 10x T-1 line, well-suited to high frequency traders . Here is some other performance benchmarks.

Hedge funds move to "easy-to-understand liquid strategies"

See this interesting article (registration required) on FT on the state of the hedge fund industry. Paul Tudor Jones, Citadel, and Fortress Investment Group are all said to be moving to "easy-to-understand liquid strategies", otherwise known as "statistical arbitrage".

(By the way, I have been urging traders to do just that in my book.)

How is the job market for quants these days?

Felix Salmon claimed in this post (hat tip: J. Rigg) that the quant job market is alive and well. However, I haven't heard much from the usually diligent headhunters in the last few months, which doesn't bode well. Maybe some of our readers can comment on the current state of the quant job market?

In that same post, Felix wondered whether to incorporate the extraordinary period of 2008 as part of backtesting data. Actually, I don't see much of a problem here -- of course one should include 2008. The only reason a trading model would have performed poorly in 2008, as opposed to 2006, 2007 or 2009, would be that its parameters are fitted too tightly to historical data. If you try out some parameterless trading models like I advocated, 2008 is not that unusual except for its higher volatility.

The enduring profitability of mean-reversion strategies

Some readers have doubts about my assertion that mean-reversal models continue to be very profitable during this whole year of financial and economic disasters. So I backtested the mean-reversion strategy in Example 3.8 of my book with the most recent one-year SP1500 data. Without transaction cost, the Sharpe ratio is 4.8. Even after subtracting 10 b.p. round-trip transaction cost, it is still at 3.5.

Since the strategy was constructed over a year ago while I was writing the book, this most recent backtest is done on unseen data, with absolutely no look-ahead bias!

Josh Brolin on day trading

Actor Josh Brolin ("Milk", "W", "No Country for Old Man") said on Charlie Rose that his trading portfolio had a 57% return this year. His strategy is based on entering on reversal from a trend. Sounds pretty reasonable to me: maybe some of our readers can backtest this.

Below is the full interview, beginning with Sean Penn, then goes on to Gus Van Sant, then finally Josh Brolin mentioned his day-trading at the very end of the 1 hour show.



Interested? He is starting a multi-million dollar hedge fund to manage your money.

My book on Quantitative Trading is published

My book on Quantitative Trading has been published and is now available from Amazon.com. Many thanks to all of you for your ideas, comments, and support!

Some riskless profit, and why it exists

Numerous commentators have pointed out the enormous yield spread between agencies debt (Fannie/Freddie) and US Treasuries.

Here are some links kindly provided by a reader: 10 yr Fannie/Treasury, 5 yr Fannie/Treasury, 10 yr Freddie/Treasury, and 5 yr Freddie/Treasury.

Currently their spreads are above 150 bp. Since the US government has nationalized Fannie and Freddie, this 150 bp is a riskless profit. As the blog Accrued Interest has pointed out, one reason this riskless profit exists is hedge fund deleveraging: nobody has the risk appetite to arbitrage this spread at a meaningful scale.

Brad Setser, a blogger at the Council of Foreign Relations, suggests that the Chinese government, who does have a lot of cash to benefit from this high yield, should go ahead and buy up these agencies debt. However, if you read the Chinese blogs and online comments, there is enormous internal pressure for the government to spend some of this money on infrastructure projects, social security, health care, etc., so I doubt that the Chinese government will have stabilizing the US mortgage market at the top of its agenda. As a result, arbitrageurs out there should have no fear that this opportunity will disappear any time soon.

How does the financial crisis affect quantitative trading?

Now that we are reasonably sure the financial world is not coming to an end yet, it is reasonable to ask how quantitative strategies have been faring under this extreme market stress. Despite reports of massive hedge fund deleveraging and negative YTD returns, I believe quantitative strategies, especially statistical arbitrage, have survived the period relatively unscathed. But here are a few of my thoughts:

1) The paltry 10% annual returns that a mediocre statarb fund can deliver is suddenly looking pretty good when the risk-free rate is under 1% and a prolonged bear market is on the horizon.

2) Mean-reversal models continue to beat momentum models in this crisis environment, as in previous crisis environments. This is not surprising because market returns have completely dominated specific returns, and of course market returns have been highly mean-reverting lately.

3) Models involving shorts are under some tumoil because of regime-change induced by new and ever-changing short-sale regulations. (For a while, I even have difficulties locating SPY for hedging purposes!)

4) Models are generally trained on data with far lower volatility than is recently realized. (Even incorporting VIX in a model does not guarantee that it can match realized volatility any better.)
As a result, P&L's fluctuations are also much higher than usual, which induces deleveraging as a risk-management measure, which drains liquidity from the market, which in turn leads to still higher volatility. The usual viscious cycle.

5) Political risks in an election year have further reduced leverage and increased volatility. What if there is an assassination? What if the wrong party got elected? What if the paper-trailess electronic voting machines cause another dispute for a month? The nightmares will continue at least until the morning of Nov 5.

6) Normally, lack of liquidity in the market is good for statarb models since they profit from renting out temporary liquidity. However, this profitability assumes that there are buyers of last resort for the market: the long-term investors, the mutual funds, Warren Buffet, etc. When they are absent, statarb investors can be left holding the bag. Fortunately, Warren Buffet & Co. has indeed stepped in and we statarb traders can breathe a sigh of relief.

7) I have been paying particular attention to 3 websites since the crisis began in order to judge whether I should return to my normal leverage: the Ted spread (I am waiting for it to return to below 2), the Calculated Risk blog, and Paul Krugman's blog. This crisis is caused by panic in the credit market, so we should look for credit market returning to normal before declaring victory. The VIX? Not so much because I believe it is backward-looking in this environment.

8) Watching Fannie, Freddie, Lehman, AIG, WaMu, Wachovia, Iceland, and the initial bailout bill failed feels like reading Chapter 8 of Harry Potter and the Deathly Hallows: "The Ministry has fallen. Scrimgeour is dead. They are coming." The Dark Lord is taking over our economy.

Webinar on algorithmic trading system

Recently I participated in a webinar on using an algorithmic trading system called Alphacet Discovery. A link to the webinar can be found here. Some previous research using this system was described here.

Index change strategy

Some years ago, I traded a simple index change strategy: buying stocks to be added to the SP500 index at the market open right after the index change announcement and exiting the position at the close, and similarly shorting stocks to be deleted. The results were mediocre at best.

However, new research by University of Edinburgh Business School suggests that a similar strategy works well for FTSE350 stocks (Hat tip to J. Rigg for the link). The trick is to predict which stocks are to be added or deleted 30 days before the announcement ("review date"), buy/sell the stocks, and close out the positions just before the review date.

Since the criteria for inclusion in the FTSE index is well-defined (and primarily based on market capitalization), it should not be hard for the interested traders to make their own predictions and profit from this rebalancing.

Behavioral finance we can all use

In their new book "Nudge: Improving Decisions About Health, Wealth and Happiness", U of Chicago economist Richard Thaler (of behavioral finance fame) and Harvard law professor Cass Sunstein gave a few pieces of personal finance advice, one of which coincided with my point in a previous post: buy insurance with the largest deductible available. The others are: don't invest much in your employer's stock, don't pay points on mortages, and don't pay for extended warranties. The book is reviewed in the NYT Book Review.

Predicting SP500 futures using investor sentiment

Ronald Domingues, an economics graduate student, has done an interesting study of how well a group of qualified investors with superior skills can predict the movement of the SP500 index. This group of qualified investors are selected by their track records of making correct predictions, and as a result of submitting their predictions going forward, they are eligible to be notified of the average predictions of other qualified investors, thus enabling them to make a better informed investment decision.

In other words, the elite will benefit from the collective wisdom of other elites -- sort of like the real world, isn't it?

How well does it work in practice? Well, they correctly predicted whether SP500 index will go up, down, or flat, a whopping 65.2% of the time. The details can be found on his website, where you can also sign up to see if you can join the elite.

More on parameterless trading model

I have written before that my ideal trading model is one that has no parameters, and what ways there are to accomplish this. Actually, I forgot to mention that a trading strategy proposed by Dr. Andrew Lo discussed previously is in fact parameterless, and the technique is so general that it can be applied to any mean-reverting strategy.

The technique is simply this: maintain a long (or short) portfolio with capital proportional to the distance between a supposedly mean-reverting measure and its long-term mean value.

For e.g. if you are pair-trading PEP vs KO, and you believe that the spread between PEP and KO is mean-reverting, then this spread is the mean-reverting measure you should employ.

As the spread moves away from its mean, keep buying (or shorting) the spread in equal dollar amount. And as the spread reverts, keep selling (or buying) the spread in the same dollar amount. What this dollar amount should be depends on: a) the total buying power you possess, b) the expected maximum deviation of the spread from its mean, and c) how often you intend to buy/short. Note that point c is not a parameter: it is arbitrary and limited only by transaction costs, technology, and other operational issues. As for the expected maximum deviation, it can be obtained by observing the history of the spread since inception.

This scheme thus obviates the need for entry or exit thresholds, and with them, the possibility of data-snooping bias. (You may still want to impose an entry threshold based on transaction cost consideration - but that would not count as a free parameter.)

What are we hedging here?

I wrote a blog article last year on why hedging isn't always better. The more I try to practice what I preached, the more I am convinced that most of the time, we are hedging the wrong risks.

Hedging should not be about reducing volatility in our portfolio. If reducing overall volatility is our goal, we should simply reduce leverage, as I have argued in my previous article. If volatility in a particular industry group is too much for us, (banks? brokerages? energy stocks?), just reduce the capital allocation in that group.

Sure, if hedging does increase your overall Sharpe ratio, go ahead and hedge to your heart's content. Kelly's formula tells us that the higher the Sharpe ratio, the higher the compounded growth rate of your wealth. The problem is, many of us hedge even when doing so do not clearly increase Sharpe ratio. A further problem is that we can achieve this maximum growth rate only if we use the high leverage recommended by Kelly's formula, but this leverage often exceeds what our brokerage would allow us. It is not clear that it is beneficial to waste our buying power on the hedge if we can only operate at sub-optimal leverage.

To me, hedging should be about eliminating the risk of ruin (equity reduced to zero) due to unexpected, catastrophic events. (Many sophisticated hedge fund managers cannot even meet this simple survival criterion, giving lie to the whole notion of "hedge" funds.)

For instance, let's assume that the worst one-day drop in the market index can be 20%. Furthermore, let's assume that you are able to endure a 30% reduction in equity during one trading period. Then you should not be afraid to have a net long exposure of 150% of your equity. In other words, not only should you not hedge, but you should go ahead and leverage your long-only portfolio 1.5 times.

I believe this notion of hedging, or buying insurance, extends to all spheres of our lives. We should avoid ruin, not mere losses. Otherwise, you will be paying too much on the insurance policy over the long term. In other words, max out the deductible on your insurance policy!

Have oil stocks exhausted their run?

Floyd Norris, the chief financial correspondent of The New York Times, suggested in his blog today that we are looking at "The Beginning of the End for High Oil Prices". What is the basis of his optimism? He argued that oil stocks have been lagging oil prices lately, and therefore equity investors must believe that high oil prices are causing demand destruction which will eventually reduce oil prices and oil companies earnings.

I beg to differ.

Look at the long-standing spread relation between an oil stock ETF and an oil commodity ETF, e.g. XLE vs USO, which I have been commenting on and tracking since October 2006. At the moment, this spread is within 1.4 standard deviations of its historical value. See my table here (subscription required). In other words, oil prices and oil stock prices are at approximately their long-time historical average. I would hardly call that suggestive of the beginning of the end.

Futures markets have no effect on spot prices

NYTimes columnist and Princeton economist Paul Krugman has previously (see my link here) argued that the trading of oil futures should have no effect on spot oil prices, contrary to what many politicians and pundits think. Here is his latest elaboration of that argument.

Statistical electoral vote predictor: Update

For readers who have been tracking the Gott and Colley presidential electoral vote prediction, they will notice a sudden switch over to a predicted Obama victory in the last few days. That is because polls from OH, VA and MO are now available -- surprising because the 3 states are not hitherto known for their Democratic leanings.

It seems to me that, after all, the stability of prediction at this early date is quite questionable due to the paucity of state polls, a point already made by Dr. Colley.

Statistical model predicts a McCain victory?

There has been a lot of buzz lately about a simple statistical model proposed by astrophysicists Prof. Gott and Dr. Colley that uses the median polls of each state to predict the November electoral vote. (For our un-American readers, the electoral vote is what determines the outcome of a general election, not the popular vote, in case the nightmarish 2000 election has not already drilled this fact into the world's collective consciousness.)

Dr. Colley has set up a website to track daily such polls to gauge the mood of the states. The authors have tested this method on the 2004 election, as well as numerous sporting events outcomes, and found it to be highly predictive.

Right now, they are betting on a McCain victory.

But there is one caveat that many bloggers have pointed out, and it is the same caveat that I have previously applied to the predictive accuracy of political futures market such as intrade.com. The caveat is this: polls (and futures market) change with time. And at different times, they predict different election outcomes. So for example, at this point (June 2008), the polls predict a McCain victory, while the futures market at intrade.com predicts an Obama victory. Who is right?

The answer is: neither. As Dr. Colley has explained to me, no backtest as far back as the June of an election year has been conducted. (Their research was based on polls from September onwards.) So we do not know if the June polling prediction has any accuracy. Similarly, as I pointed out before, the futures market can swing violently even on Election Day, even in the last hours of an election.

One advantage of the Gott and Colley method though, is that the predictions resulting from median poll statistics are remarkably stable over time. In 2004, there was very little movement in the electoral tally from September through election day. Extrapolating this result, we can be somewhat more confident of their prediction vs. Intrade.com's, even at this early date.

And in any case, I have observed that the political futures markets are highly mean-reverting, implying that the current large 20 points spread between the Obama and McCain futures is destined to decrease in the coming months.

As an arbitrage trader, I have therefore proceeded to short the Obama future.

Pre-order my book from Amazon.com

A reader told me that he can now pre-order my book "Quantitative Trading: How to Build Your Own Algorithmic Trading Business" from Amazon.com. The scheduled publication date is November 24, 2008.

Parameterless trading models

A portfolio manager that I used to work for like to pronounce that his trading models have "no free parameters". As is customary in our secretive industry, he would not elaborate further on his technique.

Lately, I begin to understand what a trading model with no free parameter means. It doesn't mean that it does not contain any lookback period for calculating trends, or thresholds for entry or exit. I think that would be impossible. It just means that all such parameters are dynamically optimized in a moving lookback window. This way, if you ask: "Does the model have a fixed profit cap?", the trader can honestly reply: "No, profit cap is not an input parameter. It is determined by the model itself."

The advantage of a parameterless trading model is that it minimizes the danger of overfitting the model to multiple input parameters. (The so-called "data-snooping bias".) So the backtest performance should be much closer to the actual forward performance.

Now, it is quite computationally challenging to optimize all these parameters just-in-time for your next order, but it is often even more difficult to do that in a backtest, given that a multidimensional optimization need to be performed for each historical bar. As a result, I personally have seldom traded parameterless models, until I get to research my regime-switching model. That model is almost parameterless (I left out a few parameters from optimization because of a lack of time, not because of any technical difficulties).

The reason backtest optimization can now be done within a few minutes is due to my use of Alphacet Discovery's server-based optimization engine. There may be other optimization software out there that performs similar functions efficiently -- I welcome comments from the reader.