Quantitative Trading

What worked in 2011?

2012-01-30T12:34:00.000-05:00

We all know that 2011 was a bad year for many hedge funds, with the average fund down 5%. But what type of strategies did well, and what did particularly poorly? The numbers are out: Forex funds lose more than average, down 6%. In fact, 71 out of 77 Forex funds tracked by a Citigroup currency analyst were down in 2011. And the winners are? Statarb funds, with a 5% averge return.

This superior performance of statarb funds is quite a contrast from the last financial crisis 2007-9. Then, most of the big factor-driven statarb models failed miserably. What caused this difference? Is it because the risk management techniques of big funds have improved? Or maybe that's because in 2011, the deviation from factor returns mean-revert within a few days, so those statarb models that re-balance on a daily basis can benefit from the buying/selling opportunity at steep discount/premium?

To settle this question, let me report the 2011 backtest results (without transaction costs) of running Andrew Lo's prototype mean-reversion model : ranking stocks based on their previous day's returns, shorting the top decile and buying the bottom one, rebalancing only at the close. (Click on chart to make it larger.)

The APR in 2011 was 18.6%. Note in particular its performance since the crisis began officially on 20110808: despite a steep drawdown, the overall performance was spectacular! Clearly, high volatility benefited a prototypical statarb strategy, and the out-performance has not much to do with improved risk management.

You might wonder what would happen if we had used the intraday version of this strategy instead: enter all positions at the open, and exit them all at the close? I tried it: the performance is surprisingly similar to the interday strategy. So intraday vs. interday volatility or mean-reversion does not seem to play a part in last year's equities market. Contrasting this with the performance of Forex models, it is clear that high volatilities benefited statarb models while they hurt FX models.

In the next article or two, I will explore the 2011 performance of some other equities mean-reverting models that I used to trade. But what about your models? If you have some thoughts on what worked and what didn't in 2011, please share them with us in the comments section.

Risk indicators

2011-12-27T09:32:00.000-05:00

During the financial crisis of 2008, I wrote about how I watched some risk indicators such as the VIX or the TED spread to decide what leverage I should use for my trading strategies. It turns out that this procedure is just as critical for the current crisis that began in August 2011. In fact, more than leverage-determinants, they can be used as the all-important variable that determines whether a certain strategy should be run at all. (What's the point of running a model that you think will lose money with low leverage?)

There are now more than a few of these risk indicators to pick from. Besides the VIX and the TED, there are the VSTOXX (EURO STOXX 50 Volatility), the VXY (JPMorgan G7 Volatility Index), the EM-VXY (JPMorgan Emerging Market Volatility Index), the ETF's ONN and OFF, and probably many more that I haven't heard of yet.

A lot of academic research has been done on whether we can devise "regime switching" models based on some complicated pattern-recognition algorithms to decide whether a market is in a certain "regime" which favors this or that particular model or parameter set. And often, these regime switching models rely on the recognition of some complicated set of patterns in the historical price series. Sorry to say, I have not found any of these complex regime switching model to have any real out-of-sample predictive power. On the other hand, my research shows that some of the aforementioned simple risk indicators will indeed prevent some trading models from falling off the cliff.

But which of these indicators are applicable to which model? This is not so obvious. For example, you might think that the EM-VXY would be an ideal leading indicator for Forex trading models that involve emerging market currencies, but I have found that it is only a contemporaneous (and thus useless) indicator to mine. Another example, I said during the 2008 financial crisis that VIX seems to be a useless contemporaneous indicator for equities trading models, but strangely, it is a good leading indicator for FX models. In contrast, the TED spread that everyone were obsessed about in 2008 shot up to over 300 bps then, but never went beyond 100 bps this time around. So really only rigorous backtesting can guide us here.

What risk indicators do you use? And have you really backtested their efficacies? Your comments would be very welcome here.

Trading platform and EC2 revisited

2011-11-11T08:31:00.000-05:00

Recently I opened a discussion on the various software platforms which allow the programmers among us to build trading strategies easily. Here is one other addition: Quantopian. It is only in alpha stage, but I did get a preview of its features:

1) You can code in Python, which is an easier language to learn than Java, but no less powerful. In fact, I know of a superb programmer who uses Python to backtest HF strategies.

2) It is web-based, which means you can take advantage of collocation on a server much more stable than your own desktops. (For those who worry about the confidentiality of your strategies, the founder indicated to me that they can run an image of the software on an Amazon EC2 account that you owned so they won't have access to your codes. As for the confidentiality of codes residing on EC2 itself, please see below*.)

3) It is event-driven (or for those who like the latest jargon: CEP-enabled), like all the Java API's that I discussed in the previous article.

4) They have 1-min US equities data for backtesting. Tick-level data will be available soon.

5) Toolboxes for common technical indicators, mathematical algorithms, etc. will be available soon.

6) They will run a competition for trading models which makes it easier for independent traders to become trading advisers to others, or to raise money for their own funds.

Unfortunately, live walk-forward testing is not yet available.

* Some readers have wondered whether it is safe to run their trading models on Amazon's EC2. Won't Amazon's employees have access to their wildly profitable strategies? The answer is no: Amazon's security policy:

Guest Operating System: Virtual instances are completely controlled by the customer. Customers have full root access

or administrative control over accounts, services, and applications. AWS does not have any access rights to customer

instances and cannot log into the guest OS....

Thanks to a reader OL from France who provided me with this info. He also told me that:

"So, I finally deployed my momentum strategy on a Linux instance of EC2 (which is free btw).

I wrote it based on the java demo application provided by Interactive Brokers and some parts of Algoquant.

So far, I use a European instance of EC2 which alas doesn't have the best latency to IB US servers (90 ms) but still better than my bedroom connection.

A test ping from a US instance to IB US servers results in only 15 ms ..."

So there you go: Java+Algoquant+IB+EC2=profit.

Stop loss, profit cap, survivorship bias, and black swans

2011-09-30T12:22:00.000-04:00

I have long espoused the view that we should not impose stop-losses on mean-reverting strategies, nor profit caps on momentum strategies. My view on the latter has not changed, but it has evolved on the former.

My original reason for opposing stop-losses on mean-reverting strategy is this. Say you believe your specific price series is mean-reverting, and say you have entered into a long position when the price is low. Now, however, the price gets much lower, and you are suffering a large unrealized loss. Well, based on your mean-reverting belief, you should buy more instead of liquidating! Indeed, if you backtest the effect of stop-losses on mean-reverting strategies, you will almost inevitably find that they decrease the overall returns and even Sharpe ratios.

But what this simplistic view ignored is 1) survivorship bias, and 2) black swan events. (Hat tip: Ben, who prompted me to consider these two issues.)

1) We normally would only trade those price series with a mean-reverting strategy only if we see that the prices did eventually revert. No one would bother to trade those price series that used to mean-revert, but suddenly stopped doing so. But saying that stop-losses are harmful to mean-reverting strategies is ignoring the fact that some mean-reverting will stop working altogether and would not survive our strategies selection process.

2) Let's define black swan events as those that did not occur in your backtest period. For example, let's say you never had a loss of 20% in a single day. So if you backtest a stop-loss of 20%, it will have no effect whatsoever on your backtest performance. However, no one can say for sure that it won't occur in the future. So if you or your investors simply cannot tolerate a 20% loss, you should impose this as a stop-loss. (After all, your brokerage has already imposed a stop-loss of 100% on you whether you like it or not.)

We can in fact turn point 2) around when deciding what stop-loss to use: a stop-loss should be loose enough so that it should have no effect on the backtest performance, and of course tight enough so that it will not result in the demise of your trading career.

There is also the issue of whether to use stop-loss on the intraday drawdown, or to use it on the multiple-day drawdown. I would argue that only intraday stop-loss is important to prevent a black-swan loss. In practice, when a strategy has a string of non-catastrophic losses occurring over multiple days, resulting in a large, unprecedented, drawdown, the trader will typically re-examine the strategy, taking into account this most recent performance and tweak the strategy so that it could theoretically be avoided. This is almost like a multi-day stop-loss strategy, as we stop an old strategy and start a new, modified, one. (Though the modified strategy might still recommend that you keep holding the current position!)

Now why am I still holding dear to the principle that one should not impose profit-caps on momentum strategies? Why, the possibility of black swan events again! But this time, any black swan can only result in unprecedented one-day gain instead of loss, since we should always have stop-losses on momentum strategies. We certainly don't want to impose a profit-cap to rule out this possibility!

More on automated trading platforms

2011-09-18T07:37:00.000-04:00

The ideal software platform for automating backtesting and executing your algorithmic trading strategies depends mainly on your level of programming expertise and your budget. If you are a competent programmer in, say, Java or C#, there is nothing to prevent you from utilizing the API offered (usually for free) by many brokerages to automate execution. And of course, it is also easy for you to write a separate backtesting program utilizing historical data. However, even for programmer-traders, there are a couple of inconveniences in developing these programs from scratch:

A) Every time we change brokerages, we have to re-write parts of the low-level functions that utilize the brokerage's API;

B) The automated trading program cannot be used to backtest unless a simulator is built to feed the historical data into the program as if they were live. To reduce bugs, it is better to have the same code that both backtests and trades live.

This is where a number of open-source algorithmic trading development platforms come in. These platforms all assume that the user is a Java programmer. But they eliminate the hassles A) and B) above as they serve as the layer that shield you from the details of the brokerage's API, and let you go from backtesting to live trading mode with a figurative turn of a key. I have taken a tour of one such platforms Marketcetera, and will highlight some features here:

1) It has a trading GUI with features similar to that of IB's TWS. This will be useful if your own brokerage's GUI is dysfunctional.

2) Complex Event Processing (CEP) is available as a module. CEP is essentially a way for you to easily specify what kind of market/pricing events should trigger a trading action. For e.g., "BUY if ask price is below 20-min moving average." Of course, you could have written this trading rule in a callback function, but to retrieve the 20-min MA on-demand could be quite messy. CEP solves that data retrieval problem for you by storing only those data that is needed by your registered trading rules.

3) It can use either FIX or a brokerage's API for connection. Available brokerage connectors include Interactive Brokers and Lime Brokerage.

4) It offers a news feed, which can be used by your trading algorithms to trigger trading actions if you use Java's string processing utilities to parse the stories properly.

5) The monthly cost ranges from $3,500 - $4,500.

If Marketcera is beyond your budget, you can check out AlgoTrader. It has advantages 1)-3) but not 4) listed above, and is completely free. I invite readers who have tried these or other similar automated trading platforms to comment their user experience here.

P.S. For those of us who use Matlab to automate our executions, a reader pointed out there is a new product MATTICK that allows you to send order via the FIX protocol which should let us trade with a great variety of brokerages.

Sorry, your return is too high for us

2011-07-23T12:52:00.000-04:00

I enjoyed reading Richard Wilson's The Hedge Fund Book (Richard also runs the Hedge Fund Blogger site). To be clear: it is purely marketing-oriented. It doesn't tell you how to find a successful trading strategy, but its focus is to tell you how to market your fund to investors once you have a successful strategy. To that end, it does a pretty good job in conveying what might be conventional wisdom to seasoned fund managers. (For e.g., don't bother to market to institutional investors if your AUM is less than $100M.) The book is filled with quite engaging interviews with fund managers, fund marketers, and other fund service providers (including our very own administrator Fund Associates). If Scott Patterson's The Quants is about the gods of hedge funds, this book is for and about the mortals.

One paragraph in the book stood out: "I've worked closely on the third-party marketing and capital introduction/prime brokerage side of the business, and I often see both types of firms deny clients service [to funds with high returns and high risk] ... Nobody wants to be associated with a manager aiming at 30 percent a month returns."

Maybe not aiming at, but what's wrong with achieving a 30 percent a month returns? I have actually met institutional investors who don't want to look at a fund that actually achieved double-digit monthly returns. Presumably that's because they believe that a high return automatically implies high risk, and also presumably a high leverage as well. I would argue that there are 2 reasons not to completely dismiss such funds out-of-hand:

1) Leverage should not be determined arbitrarily, but should be based on the minimum of what's dictated by half-Kelly (see my extensive discussions of Kelly formula on this blog and in my book) and what's dictated by the maximum single-day drawdown seen historically or in VaR simulations. And if this minimum still turns out to be higher than what most institutional investors are comfortable with, one should be bold enough to adopt it in your fund.

2) As an investor, there is an easy way to control leverage and risk: just apply Constant Proportion Portfolio Insurance (a concept also discussed elsewhere on this blog). For example, if the fund manager tells you the fund employs a constant 10x leverage (as dictated by the risk analysis outlined in 1) and you are only comfortable with 5x leverage, just invest half your capital into the fund, and keep the other half as cash in your bank account! Going forward, if the fund loses money, your effective leverage would have decreased to below 5x. Say you invested $1M into the fund, and kept $1M in the bank. And say the fund lost $0.5M. Your total equity is now $1.5M, and the fund manager is supposed to trade a $0.5M*10=$5M portfolio. Your effective leverage is now only 3.33x, well within your tolerance. Now if instead, the fund made money, you can immediately withdraw some of the profits to keep your effective leverage at 5x. So, say the fund made $0.5M. Your equity is now $2.5M, and the fund manager is supposed to trade a $1.5M*10=$15M portfolio. If you don't withdraw, this would increase your effective leverage to 6x. But if you immediately withdraw $0.25M, then the fund manager will trade a $1.25M*10=$12.5M portfolio, giving you an effective leverage of the desired 5x.

If you are an investor in hedge funds, please let us know what you think of this scheme in the comments section!

The social utility of hedge funds

2011-07-18T12:35:00.000-04:00

There is an article in the New Yorker magazine profiling Bridgewater Associates, the world's biggest global macro hedge fund. Inevitably, we come to the awkward question: "If hedge-fund managers are playing a zero-sum game, what is their social utility?"

I thought about this question a lot in the past, and I used to agree with many others that the social utility of hedge funds, or trading in general, is to provide liquidity to the markets. And a good economic case can be made that the more liquid a market is, the higher the utility it is to all participants. However, based on recent experience of flash crash and other unfortunate mishaps, we find out that traders typically do not provide liquidity when it is needed most! So this answer becomes quite unsatisfactory.

In trying to come up with a better reply, I though it is curious that few people asked "What is the purpose of having a Department of Defence?" since wars between nations are typically also zero-sum games, yet we greatly honour those who serve in the armed forces (in contrast to our feelings for hedge fund managers).

To me, clearly the answer with the best moral justification is that, in both cases, there is great social utility in defending either your clients' comfortable retirement from financial meltdown (e.g. due to governmental or corporate mismanagement), or in defending your country from foreign aggression. More specifically, the purpose of hedge funds is to reduce long-term volatility in your clients' net worth. (I would like to say "reduce risks to your clients' net worth", but that would be a bit too optimistic!)

I emphasize long-term volatility, because of course trading generates a lot of daily or hourly volatility in your clients' equity. But I do not believe that such short-term volatility affects ones' life goals. On the other hand, a 3-or-more-year drawdown in a typical buy-and-hold portfolio can wreck havoc with many lives.

If one day, the markets become so quiescent that few hedge funds can generate higher Sharpe ratio than a buy-and-hold portfolio (as indeed seems to be the case with the US equities markets these days), then yes, most hedge fund managers should just quit, instead of hogging intellectual resources from our best universities.

Hedge fund transparency and "barometers"

2011-07-03T09:21:00.000-04:00

Jim Liew of Alpha Quant Club recently posted an interesting article about the increasing demand for transparency of hedge fund strategies by institutional investors, so much so that they are essentially willing to invest only in managed accounts with real-time trades and positions updates. This is, of course, bad for fund managers, since not only can the investor reverse-engineer the simpler strategies from such knowledge, they can also piggy-back on the trades, thus paying a much smaller portion of their profits as performance fee. One might be tempted to think that since the investors are going to reverse-engineer the product anyway, why not just make it as simple and as generic as possible, and charge a much lower fee than the usual 2-20 (which hopefully will attract a much larger investor base), so that the main value to the investor is just convenience and not the originality of the strategy?

In fact, Jim wants to do just that. He proposes to construct hedge fund "barometers", essentially prototypical hedge fund strategies running in managed accounts. This would work well if these barometers have large enough capacities such that the performance can hold up even when a large number of investors sign up. From the investors' point of view, this is a trade-off between investing in a truly outstanding, high-performance strategy while paying a large fee and losing "transparency", versus just investing in a generic strategy that may still outperform the broad market. For some institutional investors, this might just be the bargain they are looking for.

When cointegration of a pair breaks down

2011-06-17T09:53:00.005-04:00

I have written a lot in the past about the cointegration of ETF pairs, and how this condition can lead to profitable pairs trading. However, as every investment advisor could have told you, past cointegration is no guarantee of future cointegration. Often, cointegration for a pair breaks down for an extended period, maybe as long as a half a year or more. Naturally, trading this pair during this period is a losing proposition, but abandoning such a pair completely is also unsatisfactory, since cointegration often mysteriously returns after a while.

A case in point is the ETF pair GLD-GDX. When I first tested it in 2006, it was an excellent candidate for pair trading, and I not only traded it in my personal portfolio, but we traded it in our fund too. Unfortunately, it went haywire in 2008. We promptly abandoned it, only to see the strategy recovered sharply in 2007.

So the big question is: how do we know whether the loss of cointegration is temporary, and how do we know when to resume trading a pair?

To answer the first question, it is often necessary to go beyond the technicals, and delve into the fundamentals of pair. Take GLD-GDX as the example. When I taught my pairs trading workshop in South Africa, several portfolio managers in attendance told me that there are 2 reasons why gold spot price diverged from gold miners' stock prices. Firstly, due to the sharp increase in oil prices during the first half of 2008, it costs the gold miners a lot more in energy to extract the gold from the ground, hence the gold miners' income lags behind the rise in gold prices. Secondly, many gold miners hedge their exposure to fluctuating gold prices with derivatives. Hence when gold price rise beyond a certain limit, the gold miners cease to benefit from this rise. Recently, the Economist magazine published an article that essentially confirms this view. But further confirmation can be gained by introducing oil (future) price into the cointegration equation. If you do that, and if you trade this triplet of GLD-GDX-USO, you will find that it is profitable throughout the entire period from 2006-2010. If you find trading a triplet too complicated, you can at least backtest a trading filter such that you will cease to trade GLD-GDX whenever USO goes beyond (above, and maybe below too) a certain band. If you have done all these backtests, you will have a plan in place to tell you when to resume trading this pair. But even if you haven't done this backtest, and you find that you need to stop trading a pair because of cumulating losses, you should at least continue paper trading it to see when it is turning around!

(By the way, if you think trading ETF pairs offers too low returns due to the low leverage allowed, consider the single stock futures on ETF's trading on the OneChicago exchange. Certainly the future on GDX is available there, while you might just trade the futures GC and CL directly on CME. There is, of course, the usual caveat that applies to futures pairs trading: the switch from contango to backwardation and vice versa can ruin many a pairs-trading strategy, even if the spot prices remain cointegrating. But that's a story for another time.)

Even more on news driven trading

2011-06-02T11:05:00.000-04:00

News driven trading is even more in vogue today than when I last mentioned it, judging from the increasing number of vendors (e.g. Ravenpack, Sensobeat, Recorded Future, etc.) and researchers pitching their wares. Not only are traditional financial and economic news deemed important, but researchers have found even blog posts (at least those on Seeking Alpha) and Twitter (Hat tip: Satya and William) to be predictive of stock prices.

One key ingredient to success in this type of trading is of course the ability to gain access to breaking news ahead of other traders. On the macroeconomic news front, the MIT Billion Prices project has spun off a company called PriceStats to deliver daily consumer product price index to subscribers. PriceStats compiles this index by continuously scanning online retailers' websites, and hopefully provides a preview of the official CPI numbers. Whether this is useful for futures and currencies traders is of course subject to their rigorous backtests, though the chart displayed on their website does suggest that the daily price index is a leading indicator of the CPI.

There is an important caveat to using news trading: not all news are equal. So another key ingredient to success is to carefully differentiate between the different types of news and backtest their predictive abilities separately. For example, I recall some research has indicated that an analyst downgrade of a stock from a "hold" to a "sell" rating has more impact than from "buy" to "hold" rating.

My own experience with news driven trading is that for all this trouble, the trading opportunities are relatively few compared to pure price driven trading, the consistency of success is low, and finally the profitability lifespan is short. If you have better experience, do share it with us.

A platform, a shareware site, and some courses for quant traders

2011-05-17T08:11:00.000-04:00

I mentioned in various places that Alphacet Discovery is an industrial strength integrated platform for backtesting and implementing quantitative trading strategies. But of course, it has many competitors, one of which is a relatively new company called Deltix. Deltix has the distinction of offering a full Matlab interface, which is convenient if you are already a Matlab programmer. (Full disclosure: I previously have a consulting relationship with Alphacet, but have none with Deltix.)

There is also a new website for sharing trading strategy software called Quantonomics. In the words of its founder Joshua, the goal is to "connect programmers and stock traders". Joshua also told me that he will create a custom application on his site for any of you readers as a gift!

A colleague of mine in Singapore, Dr. Li Haksun, who was previously a quant with UBS and BNP Paribas, is offering a course on quantitative trading strategy in July. It covers more theoretical concepts than my own courses: e.g. hidden markov model, stochastic control, and Kalman filters are included.

And of course, my own workshops on Backtesting and Statistical Arbitrage will be offered again in London next week.

Time-of-day effects in FX trading

2011-05-10T15:32:00.000-04:00

As I mentioned in a previous post, one of the main ingredients of success in constructing a profitable momentum trading strategy in Forex (and futures) is to pay attention to the entry and exit times. I haven't seen any good momentum strategy that has "time-translation invariance", i.e. works without reference to a fixed time of the day. The fixed time can refer to a benchmark level of the market (e.g. the previous close), or it can be the entry or exit time. (This is in contrast to mean-reverting strategies where the reference price can often be just a moving average.) A recent research paper (Hat tip: William) points to another example of such time-of-day effects in FX markets: a currency typically depreciates during its local trading hours.

The many facets of linear regression

2011-04-23T10:14:00.000-04:00

Many years ago, a portfolio manager asked me in a phone interview: "Do you believe that linear or nonlinear models are more powerful in building trading models?" Being a babe-in-the-woods, I did not hesitate in answering "Nonlinear!" Little did I know that this is the question that separate the men from the boys in the realm of quantitative trading. Subsequent experiences showed me that nonlinear models have mostly been unmitigated disasters in terms of trading profits. As Max Dama said in a recent excellent article on linear regression: "...when the signal to noise ratio is .05:1, ... there’s not much point in worrying about [higher order effects]". One is almost certain to overfit a nonlinear model to non-recurring noise.

Until recently, I have used linear regression mainly in finding hedge ratios between two instruments in pair trading, or more generally in finding the weightings (in number of shares) of individual stocks in a basket in some form of index arbitrage. Of course, others have found linear algebra useful in principal component analysis and more generally factor analysis as well. But thanks to a number of commenters on this blog as well as various private correspondents, I have begun to apply linear regression more directly in trading models.

One way to directly apply linear regression to trading is to use it in place of moving averages. Using moving average implicitly assumes that there is no trend in a price series, that the mean of the prices will remain the same. This of course may not be true. So using linear regression to project the current equilibrium price is sometimes more accurate than just setting it equal to a moving average. I have found that in some cases, this equilibrium price results in better mean-reverting models: e.g. short an instrument when its current price is way above the equilibrium price. Of course, one can also use linear regression in a similar way in momentum models: e.g. if the current price is way above the equilibrium price, consider this a "breakout" and buy the instrument.

Max in his article referenced above also pointed out a more sophisticated version of linear regression, commonly called "weighted least squares regression" (WLS). WLS is to linear regression what exponential moving average (EMA) is to simple moving average (SMA): it gives more weights to recent data points. Indeed I have found that EMA often gives better results than SMA in trading. However, so far I have not found WLS to be better than simple least squares. Max also referenced an article which establishes the equivalence between weighted least squares and Kalman filter. Now Kalman filter is a linear model that is very popular among quantitative traders. The nice feature about Kalman filter is that there is very few free parameters: the model will adapt itself to the means and covariances of the input time series gradually. And furthermore, it can do so one-step at a time (or in technical jargon, using an "online" algorithm) : i.e., there is no need to separate the data into "training" and "test" sets, and no need to define a "lookback" period unlike moving averages. It makes use of "hidden states" much like Hidden Markov Models (HHM), but unlike HHM, Kalman filter is faithfully linear.

I haven't used Kalman filter much myself, but I would welcome any comments from our readers on its usage. Also, if you know of other ways to use linear regression in trading, do share with us here!

Momentum strategies in futures and forex

2011-03-11T14:21:00.001-05:00

I have long found that it is easier to find good (i.e. high Sharpe ratio) mean-reverting strategies than good momentum strategies. Partly, that is because I was mainly a stock trader instead of a futures/currencies trader, and individual stocks mean-revert most of the time. There are exceptions, such as after special corporate events such as earnings announcements, and I have tested momentum strategies based on these events. But the success of even these event-driven strategies has been uneven, especially since more traders become aware of them.

Now that I am focusing more on trading futures and currencies, I have gradually been introduced to the world of momentum investing. There is a good book in this area that deserves to be better known: Joe Duffy's The Ultimate Trading Robot, which is an almost step-by-step guide to constructing futures trending strategies that rely on prices alone. Another example would be the London Breakout strategy mentioned by our reader Bernd in the comments here. After studying these examples, I realized why my previous, rather desultory, search for momentum strategies in the futures and FX markets had been in vain: the overnight gap in these markets seems critical. For futures, the overnight gap is obvious, but in the case of the London Breakout strategy, for example, the trader has the task defining for herself what the optimal closing and opening times are in order to compute the gap. Intraday trend without an overnight breakout does not seem persistent enough to be traded profitably. I also wonder if there is a more elegant (i.e. mathematical) way to quantify such breakout phenomena without using the traditional technical indicators.

If you know of ideas for good momentum strategies, you are most welcome to share and discuss them here!

High frequency trading ideas

2011-01-25T14:31:00.000-05:00

I just started reading Larry Harris' book "Trading and Exchanges" (thanks to Max Dama's glowing book review) and already a couple of potential high frequency trading techniques stood out:

"Quote matching" - a technique whereby front-runners place a limit buy order just a cent (for stocks) higher than the best bid price. If the order is filled, they then place a limit sell order just a cent lower than the best ask. Assuming the best bid-ask quotes don't move, the worst they can do is to lose 1 cent by selling the share back to the best bidder, while the most profit they can make is the bid-ask spread plus rebates for providing liquidity minus 2 cents by having the sell long limit order filled. This could work out quite profitably if the bid-ask spread is wide. But of course, the best bid-ask do change constantly, so front-runners would need to cancel and correct their limit orders constantly, and the optimal algorithm for doing this could get quite complicated. Meanwhile, if you are a bona fide liquidity provider, you would have to avoid providing this free option to the front-runners by constantly monitoring who is in front of you. As usual, this chess game can quickly degenerate into an HFT arms race.

"Manipulation of stop orders" - a.k.a. "gunning the market", a technique whereby the market gunners buy aggressively so as to trigger large buy stop orders that they believe are in place at a higher price. When these buy stop orders are filled, the prices are driven higher still, and these manipulators then sell their position profitably.

One of my old momentum strategies was a victim of these market gunners, and after that sad experience I refused to use stop orders any more, at least for stocks. However, here is a question for our knowledgeable readers: can other traders actually see what stop orders there are on an order book (whether for stocks, futures, or Forex markets)? And if so, would a trading robot that simulates stop orders by sending out buy market orders when the stop price is touched work better than manually placing a buy stop order on the order book?

Shorting the VIX calendar spread

2011-01-05T11:57:00.001-05:00

Lately there were a few interesting discussions in the blogosphere on the profitability of shorting the VXX-VXZ spread. (See Quantum Blog and The Speculator's Ball.) For background, VXX is an ETN that tracks the first and second month of the VIX future, which in turn tracks the VIX volatility index, which in turn tracks the volatility of SPX. VXZ is the ETN that tracks the 4th - 7th months of the VIX future. During the period 2009-2010, there were 2 different reasons why shorting this "calendar spread" was profitable:

1) The VIX futures were/are in contango: i.e. the back months' futures are more expensive than the front months'.
2) The volatility of SPX was decreasing with time.

However, some traders seem to think that either one of these conditions is enough to ensure the profitability of shorting a calendar spread. It is not. (Otherwise, life as a futures trader would be too easy!)

To see this, let's resort to a simplistic linear approximation to a model of futures prices. From John Hull's book on derivatives, section 3.12, the price of a future which matures at time T is

F(t, T)=E(ST)exp(c(T-t)),

where E(ST) is the expected value of the spot price at maturity, c is a constant, and t is the current time. If the futures are in contango, then c > 0.

If we assume that abs(c) is small, and T-t is also small (i.e. not too far from maturity), and that the expected value of the spot price changes slowly, we can linearize this formula as

F=(a+b(T-t))*(1+c(T-t))

If the market expects the future spot price to increase, then b > 0.

After a few simple algebraic steps, you can verify that the calendar spread's price is proportional to

F(t, T1)-F(t, T2) ~ bct

where T1 < T2 (i.e. F(t, T1) is the front month's price, and F(t, T2) the back month's).

This is a satisfyingly illustrative result. It says that shorting this calendar spread will be profitable if

A) futures are in contango and the expected spot price in the future is decreasing; or else
B) futures are in backwardation and the expected spot price in the future is increasing.

So what is the situation today? Will it still be profitable to short this spread? As our fellow bloggers have pointed out, VIX futures are still in contango, but the market is expecting volatility to increase in the future over the last month or so. So this may no longer be a profitable trade anymore.

Columbia Workshop on Financial Engineering

2010-11-18T11:48:00.000-05:00

Our readers in New York may be interested in this finance workshop at Columbia University tomorrow.
I am particularly interested in the talk by Kent Daniel on "Characterizing Momentum", and by Doug Borden (of Knight Equity Markets) on "Stochastic Control Theory in High-Frequency Trading". Doug's talk can be downloaded here.

Data mining and artificial intelligence update

2010-10-10T11:33:00.000-04:00

Long time readers of this blog know that I haven't found data mining or artificial intelligence techniques to be very useful for my own trading, for they typically overfit to non-recurring past patterns. (Not surprisingly, they are much more useful for driverless cars.) Nevertheless, one must keep an open mind and continues to keep tabs on new developments in this field.

To this end, here is a new paper written by an engineering student at UC Berkeley which uses "support
vector machine" together with 10 simple technical indicators to predict the SPX index, purportedly with 60% accuracy. If one includes an additional indicator which measures the number of news articles on a stock in the previous day, then the accuracy supposedly goes up to 70%.

I did not have the chance to reproduce and verify this result yet, but I invite you to try it out and share your findings here. If you do so, you may find this new data mining product called 11Ants Analytics useful. It is an Excel-based software that includes 11 machine learning algorithms including the aforementioned support vector machines. It also includes decision trees which are sometimes quite useful in automatically generating a small set of trading rules from an input set of technical indicators. (Whether those rules remain profitable in the future is another question!) If you have tried this product, I would also appreciate your comments here.

(If you are a die-hard MATLAB fan, support vector machines are available in their Bioinformatics Toolbox, and classification and decision trees in their Statistics Toolbox.)

The main virtue of buying options

2010-10-02T14:06:00.002-04:00

I realized that I have omitted the most obvious virtue of trading options instead of stocks in my last post: the much more attractive reward-risk ratio for options.

Suppose your stock strategy generated a buy signal. You can either buy the stock now, or you can buy an ATM call. If you buy the stock, you are of course benefiting from 100% of the upside potential of the stock price movement, but you are similarly exposed to 100% of the downside risk. Indeed you can lose the entire market value of the stock. If you buy the call, you will benefit from > 50% of the upside potential of the stock price, assuming that your holding period is so short that the time value will not dissipate much. As the stock price rises, so does your delta. (It increases from 0.5 to 1.) But what about the downside risk? All you can lose is the option premium, usually << 50% of the market value of the stock.

In other words, while one may be tempted to hedge a large stock position with stock index futures, there is no need to hedge an equivalent call option position. This should simplify your strategy implementation and reduce risk management costs (i.e. the probable loss on your short futures position).

Given that I am a short-term trader anyway, I can't figure out why I have been trading stocks instead of options all these years! (Aside from the caveats detailed in the previous post.)

Implementing stock strategies using options

2010-09-25T11:26:00.000-04:00

There are many stock trading strategies that are quite attractive in terms of Sharpe ratios, but not very attractive in terms of returns. (Pairs trading comes to mind. But in general, any market neutral strategy suffers from this problem.) Certainly, one cannot feed a family with annualized returns in the single or low double digits, unless one already has millions of dollars of capital. One way to solve this dilemma is of course to join a proprietary trading group, where we would have access to perhaps x30 leverage. Another way is to implement a stock trading strategy using options instead, though there are a sizable number of issues to consider. (I recently brushed up on my options know-how by reading the popular "Options as a Strategic Investment".)

Using options will allow you to increase your leverage beyond the Reg T x2 leverage (or even the day trading x4 leverage) only if you buy options only, but not selling them. For example, to implement a pairs trading strategy on 2 different stocks, you would have to buy call options on the long side, and buy put options on the short side (but not sell call options). Otherwise the margin requirement for selling calls is as onerous as shorting the underlying stock itself.
The effective leverage is computed by multiplying the delta of the option by the underlying stock price divided by the option premium. If you buy an out-of-money (OTM) option, the delta will be small (smaller than 0.5), but the option premium is small also. Vice versa for an in-the-money (ITM) option. So you would have to find the optimal strike price so that the effective leverage is maximized. I personally choose to buy an at-the-money (ATM) call or slightly ITM call without actually computing the optimized strike, but perhaps you have reached a different conclusion?
Naturally, the shorter the time-to-expiration, the cheaper the option and higher the effective leverage. Additionally, for ITM options, their deltas increase as we get closer to expiration, which also contributes to higher effective leverage. However, the time-to-expiration must of course be longer than the expected holding period of your position, otherwise you would incur the transaction cost of rolling over to the further-month options.
The discussion of finding the right strike price based on its delta is moot if your brokerage's API does not provide you with delta for your automated trading system. In theory, Interactive Brokers's API provide deltas for whole options chains, and quant2ib's MATLAB API will pass these on to your MATLAB exeuction program too. However, I have not been successful in retrieving deltas using quant2ib's API. If you have encountered a similar problem, and perhaps have found the reason/cure for this, please let me know. For now, I am reduced to assuming that all my near ATM calls for different stocks have the same delta, and I increase this common value from 0.5 to close to 1 as time passes.
Options don't have MOO, LOO, MOC or LOC order types. If one uses market orders to buy at the open or close, one would incur significant transaction costs due to the much wider bid-ask spread compared to stocks. I try to use limit orders on options orders as much as possible.

If you have used options to implement stock trading strategies, and have experiences with these or other issues, please do share them here.

====

Reminder: my next pairs trading workshop will take place in New York on October 26-27th.

Phantom quotes

2010-08-22T11:57:00.000-04:00

Have you ever got the feeling that your market orders are often filled at prices worse than the NBBO displayed on your trading screen? Apparently, this may be the result of deliberate manipulation of the market by high frequency traders. These HF traders submit thousands of quotes per second to the NYSE ("quote stuffing") and then cancel them within 50 ms. This slows down the exchange data queue so much that by the time a quote is transmitted to you, it is stale already, even if your trading server is collocated at the exchange. (Checking the time stamp of the quote is of no help: the time stamp is based on the time the quote enters the queue, not when it exits the queue.)

If you can no longer believe in the quotes, is there any integrity left in the market? Much as I think that HFT may be useful liquidity providers, I can't see how this specific practice could be good for anyone over the long term.

(Hat tip: Jim Liew of Alpha Quant Club.)

What are we to do with Sharpe ratio?

2010-08-14T13:39:00.000-04:00

I wrote several times before how useless Sharpe ratio is for certain types of strategies: see here and here. Not only is a high Sharpe ratio quite useless in telling you what damage extreme events can do to your equity, a low Sharpe ratio is also quite useless in telling you what spectacular gain your strategy might enjoy in the event of a catastrophe. I came across another brilliant example of the latter category in the best-selling book "The Big Short", where the author tells of the story of the fund manager Mike Burry.

Mike Burry started buying credit default swaps in 2005, essentially an insurance policy on mortgage-backed securities, betting that there will be widespread defaults on mortgages. Of course, we now know how this story would turn out: Mike Burry made $750 million in 2007 alone. But there was nothing but pain for the fund manager and his investors in 2005-2006, since they had to pay an annual premium of 8% of the portfolio. Investors who measured the performance of this strategy using Sharpe ratio, without knowing the details of the strategy itself, would be quite justified to think that it was an utter disaster prior to 2007. And indeed, many of them lost no time in trying to pull out their investments.

So what are we to do with Sharpe ratio, with its inherent reliance on Gaussian distributions? Clearly, it is useful for measuring high frequency strategies which you can count on to generate consistent returns every day, but which has limited catastrophic risks. But it is less useful for measuring statistical arbitrage strategies that hold positions over multiple days, since there may well be substantial hidden catastrophic risks in these strategies that would not be revealed by their track record and standard deviation of returns alone. As for strategies that are designed to benefit from catastrophes, such as Mike Burry's CDS purchases or Nassim Taleb's options purchases, it is completely useless. If I were to allocate my assets over different hedge funds, I would be sure to include some funds in the first category to generate cash flows for my daily needs, as well as funds in the last category to benefit from the infrequent black-swan events. As for the funds in the middle category, I am increasingly losing my enthusiasm.

Pair trading technologies update

2010-07-30T12:17:00.001-04:00

Pair trading was invented two decades ago, but automating its implementation has only recently become fashionable with independent traders. But once the spotlight is on, innovations come fast and furious. Here are a number of recent developments that I find interesting:

1. I mentioned previously the software called quant2ib. It is an API which allows us to get market data and send orders from a Matlab program to Interactive Brokers (IB). I have used it extensively for our trading, and it is as reliable as IB's native API. Their latest version now includes functions for constructing a "combo" security. This combo security can be pairs of stocks, ETF's, futures, etc. (with the notable exception of currencies), and the API allows you to get market data as well as to submit orders on a combo. This is a huge improvement because you can now automatically trade a pair of securities as one unit by submitting limit orders on the combo. (Previously, you would have had to submit market order on at least one side of the pair, and this would have required your program to continuously monitor the market prices and send orders when appropriate. Or else you had to give up using the API and manually enter a "generic combo" limit order in IB's TWS.)

2. Alphacet Discovery also has the ability to send limit orders on pairs, due to its partnership with Knight Trading. Besides, based on a demo that I have recently seen, they also now have great pairs portfolio and execution reporting functionality. (Full disclosure: I used to consult for them.)

3. IB itself has released a "Scale Trader" algorithm that can be applied to combos (see 1. above. Hat tip: Mohamed.) I can't explain this better than their press release: "... ScaleTrader algorithm allows clients to create conditions under which a long position in one stock is built while simultaneously creating an offsetting short position in the other. The ScaleTrader is named because investors can 'scale-in' to market weakness by setting orders to buy as the market moves lower. Similarly, sell orders can be 'scaled' into when a market is rising. The ScaleTrader algorithm can be programmed to buy the spread and subsequently take profit by selling the spread if the difference reaches predetermined levels set by the user." In other words, it allows us to automatically implement the "parameterless trading" or the "averaging-in" strategy that I blogged about previously without any programming on our part!

Speaking of pair trading, I will be teaching my first New York workshop in October. (My editor inevitably picks touristy locations for these workshops. My London workshop takes place across the street from the Tower of London, my New York workshop is across from the new World Trade Center, and my Hong Kong workshop is in the "Golden Mile" shopping district of Tsim Sha Tsui.)

The Quants

2010-05-29T09:54:00.000-04:00

Once in a while, a book about trading written for the general public contains some useful nuggets even for professionals. Fortune's Formula was one. It introduced me to the world of Kelly's formula, Universal Portfolios, and the maximization of compounded growth rate. The Quants, by WSJ reporter Scott Patterson, is another. (Hat tip to my partner Steve for telling me about it.)

What is the most important take-away in The Quants? No, it is not that you should learn to become a master poker player or chess player before hoping to make it big, though you would think that given Patterson's exhaustive coverage of poker games played by the top quants. Among my own professional acquaintances, trader-poker-players are still a minority.

The most important take-away is what ex-employees said about Renaissance Technologies: "there is no secret formula for the fund's success, no magic code discovered decades ago by geniuses .... Rather, Madallion [Fund]'s team of ninety or so Ph.D.'s are constantly working to improve the fund's systems, ..."

In other words, though you may not have 90 Ph.D.'s at your disposal, you can still work on continuously improving/refining your strategies, improving the engineering of your trading environment, and increasing the diversity of your strategies. And though you may still not archive 60-70% annualized returns every year, you will nevertheless enjoy stable returns year after year.

By the way, it is good to see my ex-colleagues Lalit Bahl, Vincent and Stephen Della Pietra mentioned in the book, all of whom left IBM to join Renaissance many years ago, and who are extraordinarily nice and friendly guys, quite in contrast to the norm on Wall Street.

A HFT primer

2010-05-22T09:56:00.000-04:00

As a follow-up of my previous discussions on high frequency trading, I have invited guest blogger Jennifer Groton to share with us a quick survey of various common HFT strategies used by equities and FX traders.

==

High frequency trading strategies are under fire. The recent trading spike in our national exchanges was duly noted as a short-circuit waiting to happen and drew immediate industry criticism of auto-trading robots. Before a witch-hunt ensues, perhaps a review of the common HFT strategies in stocks and Forex is in order.

High-frequency firms employ a wide variety of low-margin trading strategies that are implemented by professional market intermediaries who have invested heavily in technology. These firms claim that they make markets more efficient by enhancing liquidity and transparent price discovery to the benefit of investors. The Forex market’s unique combination of high liquidity and low volatility make it an ideal environment for deploying HFT strategies, although many of the ideas and technology are from the equity markets. The basic strategies fall into three categories: market-making, trending or predictive, and classic arbitrage.

Market-making strategies tend to focus on a single stock or currency pair. Many firms in this area have been described as engaging in "rebate-capture trading", a reference to the credits that firms get for providing liquidity on most market centers.

The second group consists of mean-reversion and trending strategies. These utilize technical indicators for stocks or forex indicators for currencies, and seek to generate more return from individual trades.

The last group may involve a cross-section of trades from multiple markets. The classic arbitrage strategy is a form of the “carry trade” that uses the prices of a domestic bond, a bond denominated in a foreign currency, the spot price of the currency, and the price of a forward contract on the currency. If the market prices are sufficiently different from those implied in the model to cover transactions costs, then four transactions can be made to guarantee a risk-free profit.

High frequency trading is attributed with generating over 70% of the volume of trades on our equity markets. Similar statistics are not available for forex markets, but speculating disguised as commercially necessary trades have been reported to be over two-thirds of the volume. Liquidity and pricing transparency are the benefits offered by its advocates, but regulators and other market participants who disagree with this positive assessment are presently discounting these benefits. Transaction taxes and time limits on orders have been proposed to mitigate the perceived risk created by HFT firms, but the wheels of Washington move slowly, even in crisis. For the time being, there is no indication that their participation will be discontinued.

Are flash orders to be blamed for Dow's 1,000 points drop?

2010-05-08T07:52:00.000-04:00

Before the smoke is clear, fingers are already pointing at flash orders. See these two NYT pieces here and here. Our reader Madan has convinced me previously that flash orders can indeed be used to front-run other traders, but until more evidence comes in, I am yet to be convinced that they are the main culprit. Couldn't old-fashioned automated momentum programs accomplished the same thing after an initial erroneous transaction price and/or quote was reported? Perhaps you know of discussions elsewhere on the blogosphere that bring more light to the issue?

An additional ETF pair

2010-05-02T13:19:00.000-04:00

Many of you know that there are a number of dependable commodity-related ETF pairs that remain cointegrated ever since I mentioned them in 2006: IGE-EWC, IGE-EEM, IGE-EWA, EWA-EWC, etc. (Their latest zScores are available here to my book's readers and to Premium Content subscribers.) A recent visit to a client in South Africa prompted me to add a new one: EWA-EZA.

It is worth noting that for those country ETF pairs that cointegrate, their underlying currency cross-rates are often stationary as well. Now, there are several advantages in trading currency cross rates instead of ETF pairs. When trading a stationary cross rate, you can enter a limit order to enter and exit, but trading pairs of ETF's involve market orders on at least one side. Also, ETF's can sometimes be hard-to-borrow, and their margin requirements are much more onerous than that of currencies. However, the one major disadvantage in trading cross rates is that they are not always available on your brokerage. For example, based on the cointegration of EWA and EZA you would think that trading AUDZAR would be quite profitable. And you would be right, theoretically, except that AUDZAR is not available for trading on Interactive Brokers. If you know of a good Forex brokerage that have many emerging markets cross-rates for trading, especially those of Latin American countries, please let the rest of us know!

How do you limit drawdown using Kelly formula?

2010-04-17T09:51:00.000-04:00

As many of you know, I am a fan of Kelly formula because it allows us to maximize long-term growth of equity while minimizing the probability of ruin. However, what Kelly formula wont' prevent is a deep drawdown, though we are assured that the drawdown won't be as much as 100%! This is unsatisfactory to many traders and especially fund managers, since a deep drawdown is psychologically painful and may cause you to panic and shut down a strategy prematurely.

There is an easy way, though, that you can use Kelly formula to limit your drawdown to be much less than 100%. Suppose the optimal Kelly leverage of your strategy is determined to be K. And suppose you only allow a maximum drawdown (measured from the high watermark, as usual) to be D%. Then you can simply set aside D% of your initial total account equity for trading, and apply a leverage of K to this sub-account to determine your portfolio market value. The other 1-D% of the account will be sitting in cash. You can then be assured that you won't lose all of the equity of this sub-account, or equivalently, you won't suffer a drawdown of more than D% in your total account. If your trading strategy is profitable and the total account equity reaches a new high watermark, then you can reset your sub-account equity so that it is again D% of the total equity, moving some cash back to the "cash" account. Otherwise, you continue to keep the equity in the cash account separate from the equity of the trading sub-account.

Notice that because of this separation of accounts, this scheme is not equivalent to just using a leverage of L=K*D% on your total account equity. Indeed, some of you may be too nervous to use the full K as leverage, and prefer to use a leverage L smaller than K. (In fact, the common wisdom is that, due to estimation errors, it is never advisable to set L to be more than K/2, i.e. half-Kelly.) The problem with using a L that is too small is that, besides not achieving maximum growth, the portfolio market value will be unresponsive to gains or losses and will remain relatively constant. Using the scheme I suggested above will cure this problem as well, because you can apply a higher leverage L_sub to the sub-account (e.g. use L_sub = L/D%) as long as L_sub < K, so that the portfolio market value is much more sensitive to your P&L while still ensuring the drawdown will not exceed D%.

Has anyone tried this scheme in their actual trading? If so, I would be interested in hearing your experience and see if practice is as good as theory.

Conference on the sociology of quantitative finance

2010-02-27T17:44:00.000-05:00

A new conference called Psi-Q will be held in London this June, featuring luminaries in the academic quantitative finance world, as well as risk and fund managers from various banks and hedge funds. Example topics:

How did shared beliefs, practices, ways of calculating, and technical systems impact evaluation of asset-backed securities and CDOs before and during the credit crises?
Was that Lucky or Good? Creating a framework for skill attribution in finance, business management and other risky endeavors.
The “backing out” phenomena observed in options markets: how traders use models to imply independent variables consistent with market observed pricing, and where enough traders can be wrong about the expected results and the backed-out positions can send the wrong message.

Sounds like an interesting bird's eye view of quantitative finance.

Pairs Trading Workshop in Hong Kong

2010-02-18T14:48:00.001-05:00

For my readers in Asia, I will be conducting a pairs trading workshop in Hong Kong on March 10-11. This workshop is organized by the Technical Analyst magazine and is similar to the one I gave in London last year.
However, I have added a few useful insights based on audience feedback. As always, no prior knowledge of Matlab or advanced statistics is assumed. The numerous in-class exercises should be sufficient to bring your Matlab programming skills up to speed.

A method for optimizing parameters

2010-01-31T16:03:00.000-05:00

Most trading systems have a number of parameters embedded, parameters such as the lookback period, the entry and exit thresholds, and so on. Readers of my blog (for e.g., here and here) and my book would know my opinion on parameter optimization: I am no big fan of it. This is because I believe financial time series is too non-stationary to allow one to say what was optimal in the backtest is necessarily optimal in the future. Most traders I know would rather trade a strategy that is insensitive to small changes in parameters, or alternatively, a "parameterless" strategy that is effectively an average of models with different parameters.

That being said, if you can only trade one model with one specific set of parameters, it is rational to ask how one can pick the best (optimal) set of parameters. Many trading models have a good number of parameters, and it is quite onerous to find the optimal values of all these parameters simultaneously. Recently, Ron Schoenberg published an article in the Futures Magazine that details a way to accomplish this with just a tiny amount of computer power.

The key technique that Ron uses is cubic polynomial fit of the P&L surface as a function of the parameters. Ron uses the VIX RSI strategy in Larry Connors' book "Short Term Trading Strategies That Work" as an example. This strategy has 5 parameters to be optimized, but Ron only needs to compute the P&L for 62 different sets of parameters, and the whole procedure only takes 58 seconds.

Although Ron has confirmed that most of the parameters that Connors picked are close to optimal, he did find a few surprises: namely, that RSI of period 3 or 4 is significantly more profitable than the 2 that Connors used, at least in the backtest period.

Now, for a true test of this optimization, it would be helpful if Ron performed this optimization withholding some out-of-sample data, and see if these parameters are still optimal in that withheld data set. Since he didn't do that, we need to wait for another year to find out ourselves!

Excel ADF test

2010-01-19T13:02:00.000-05:00

Some readers have asked whether there is an Excel version of the ADF test for cointegration (mentioned in articles here or here.) You can download one such package here (Hat tip: Bruce H.).

And as always, you can download the Matlab version from spatial-econometrics.com.

Does Averaging-In Work?

2010-01-09T20:23:00.000-05:00

Ron Schoenberg and Al Corwin recently did some interesting research on the trading technique of "averaging-in". For e.g.: Let's say you have $4 to invest. If a future's price recently drops to $2, though you expect it to eventually revert to $3. Should you

A) buy 1 contract at $2, and wait for the price to possibly drop to $1 and then buy 2 more contracts (i.e. averaging-in); or
B) buy 2 contracts at $2 each; or
C) wait to possibly buy 4 contracts at $1 each?

Let's assume that the probability of the price dropping to $1 once you have reached $2 is p. It is easy to see that the average profits of the 3 options are the following:
A) p*(1*$1+2*$2) + (1-p)*(1*$1)=1+4p;
B) 2; and
C) p4*$2=8p.

Profit A is lower than C when p > 1/4, and profit A is lower than profit C when p > 1/4. Hence, whatever p is, either option B or C is more profitable than averaging in, and thus averaging-in can never be optimal.

From a backtest point of view, the Schoenberg-Corwin argument is impeccable, since we know what p is for the historical period. You might argue, however, that financial markets is not quite stationary, and in my example, if the historical value of p was less than 1/4, it is quite possible that the future value can be more than 1/4. This is why I never make too much effort to optimize parameters in general, and I can sympathize with traders who insist on averaging-in even in the face of this solid piece of research!

Selecting tradeable pairs: which measure to use?

2009-12-24T10:24:00.000-05:00

A guest blog by Paul Farrington

One of the most important factors in statistical arbitrage pairs trading is the selection of the paired instruments. We can use basic heuristics to guide us, such as grouping stocks by industry in the anticipation that stocks with similar fundamental characteristics will share factor risk and tend to exhibit co-movement. But this still leaves us with potentially thousands of combinations. There are some statistical techniques we can use to quantify the tradeability of a pair: one approach is to calculate the correlation coefficient of each pair's return series. Another is to consider cointegration measures on the ratio of the prices, to see if it remains stationary over time.

In this article I briefly summarise the alternative approaches and apply them to a universe of stock pairs in the oil and gas industry. To measure how effective each measure is in real world trading, I back test the pairs using a simple means reversion system, then regress the generated win rate against the statistical results. Some basic insights emerge as to the effectiveness of correlation and cointegration as tools for selecting candidate pairs.

Please visit http://www.paulfarrington.com/research/Selecting%20tradeable%20pairs.htm for details of my methodology and results.

Public service announcements for quants

2009-12-18T21:48:00.003-05:00

   1. Conference on 'Computational Topics in Finance', February 19/20, 2010, National University of Singapore. The topics will include using R/Rmetrics in finance, but the conference is by no means confined to R. See http://www.rmetrics.org/.

   2. Consulting position (6-month renewable contract) available at a major Canadian bank in Toronto: research in various mathematical algorithms used for pricing of interest rate derivative instruments like swaps, caps, swaptions, FRAs. Please contact their recruiter at http://www.linkedin.com/pub/kevin-p-w-wang/6/899/29a.

   3. A free copy of Chapter 8 of "High Probability ETF Trading" which I mentioned here is now available for download.

Are financial speculations really "harmful human activities"?

2009-12-06T16:37:00.003-05:00

It is worrisome when not one but two eminent economists denounced financial speculation as "harmful human activities" in the short space of 2 weeks. (See Paul Krugman's column here and Robert Frank's here.) It is more worrisome when their proposed cure to this evil is to apply a financial transaction tax to all financial transactions.

Granted, you can always find this or that situation when financial speculation did cause harm. Maybe speculation did cause the housing bubble. Maybe speculation did cause an energy price bubble. In the same vein, you can also argue that driving is a harmful human activity because cars did cause a few horrific traffic accidents.

No, we can't focus on a few catastrophes if we were to argue that financial speculation is harmful. We have to focus on whether it is harmful on average. And on this point, I haven't seen our eminent economists present any scientific evidence. On the other hand, as an ex-physicist and an Einstein-devotee, I can imagine some thought experiments (or gedankenexperiment as Einstein would call them), where I can illustrate how the absence of financial speculation can clearly be detrimental to the interests of the much-beloved long-term investors. To make a point, a gedankenexperiment is usually constructed so that the conditions are extreme and unrealistic. So here I will assume that the financial transaction tax is so onerous that no hedge funds and other short-term traders exist anymore.

Gedankenexperiment A: Ms. Smith just received a bonus from her job and would like to buy one of her favorite stocks in her retirement account. Unfortunately, on the day she placed her order, a major mutual fund was rebalancing its portfolio and had also decided to shift assets into that stock. In the absence of hedge funds and other speculators selling or even shorting this stock, the price of that stock went up 40% from the day before. Not knowing that the cause of this spike was a temporary liquidity squeeze, and afraid that she would have to pay even more in the future, Ms. Smith paid the ask price and bought the stock that day. A week later, the stock price fell 45% from the peak after the mutual fund buying subsided. Ms. Smith was mortified.

Gedankenexperiment B: Mr. Smith decided that the stock market is much too volatile (due to the lack of speculators!) and opted to invest his savings into mutual funds instead. He took a look at his favorite mutual fund's performance, and unfortunately, its recent performance seemed to be quite a few notches below its historical average. The fund manager explained on her website that since her fund derived its superior performance from rapidly liquidating holdings in companies that announced poor earnings, the absence of liquidity in the stock market often forced her to sell into an abyss. Disgusted, Mr. Smith opted to keep his savings in his savings account.

Of course, our economists will say that the tax is not so onerous that it will deprive the market of all speculators (only the bad ones!?). But has anyone studied if we impose 1 unit of tax, how many units of liquidity in the marketplace will be drained, and in turn, how many additional units of transaction costs (which include implicit costs due to the increased volatility of securities) would be borne by an average investor, who may not have the luxury of submitting a limit order and waiting for the order to be filled?

Picking up nickels in front of steamrollers

2009-11-27T15:02:00.000-05:00

When I was growing up in the trading world, high Sharpe ratio was the holy grail. People kept forgetting the possibility of "black swan" events, only recently popularized by Nassim Taleb, which can wipe out years of steady gains in one disastrous stroke. (For a fascinating interview of Taleb by the famous Malcolm Gladwell, see this old New Yorker article. It includes a contrast with Victor Niederhoffer's trading style, plus a rare close-up view of the painful daily operations of Taleb's hedge fund.)

Now, however, the pendulum seems to have swung a little too far in the other direction. Whenever I mention a high Sharpe-ratio strategy to some experienced investor, I am often confronted with dark musings of "picking up nickels in front of steamrollers", as if all high Sharpe-ratio strategies consist of shorting out-of-the-money call options.

But many high Sharpe-ratio strategies are not akin to shorting out-of-the-money calls. My favorite example is that of short-term mean-reverting strategies. These strategies not only provide consistent small gains under normal market conditions, but in contrast to shorting calls, they make out-size gains especially when disasters struck. Indeed, they give us the best of both worlds. (Proof? Just backtest any short-term mean-reverting strategies over 2008 data.) How can that be?

There are multiple reasons why short-term mean-reverting strategies have such delightful properties:

Typically, we enter into positions only after the disaster has struck, not before.
If you believe a certain market is mean-reverting, and your strategy buy low and sell high, then of course you will make much more money when the market is abnormally depressed.
Even in the rare occasion when the market does not mean-revert after a disaster, the market is unlikely to go down much further during the short time period when we are holding the position.

"Short-term" is indeed the key to the success of these strategies. In contrast to the LTCM debacle, where they would keep piling on to a losing position day after day hoping it would mean-revert some day, short-term traders liquidate their positions at the end of a fixed time period, whether they win or lose. This greatly limits the possibility of ruin and leaves our equity intact to fight another day in the statistical game.

So, call me old-fashioned, but I still love high Sharpe-ratio strategies.

In praise of ETF's

2009-11-04T21:54:00.000-05:00

I have learned some years ago that ETF's are strange and wonderful creatures. Simple, long-only mean-reverting strategies that work very well on ETF's, won't work on their component stocks. (Check out a nice collection of these strategies in Larry Connors' book "High Probability ETF Trading". He has also packaged these strategies into a single indicator, the ETF Power Ratings, on tradingmarkets.com.) Simple pair trading strategies like the one I discussed in my book, also work much more poorly on stocks than on ETF's. Why is that?

Well, one obvious reason is that, as Larry mentioned in his book, ETF's are not likely to go bankrupt (with the notable exception of the triple-leveraged ETF's, as I explained previously), because a whole sector or country is not likely to go bankrupt. So you can pretty much count on mean-reversion if you are on the long side.

Another obvious reason is that though there are news which will affect the valuation of a whole sector or country, these aren't as frequent or as devastating as news affecting individual stocks. And believe me, news is the biggest enemy of mean-reversion.

But finally, I believe that the capital weightings of the component stocks also play a part in promoting mean-reversion. Typically, weighting of a component stock increases with its market capitalization, though not necessarily linearly. Perhaps large-cap stocks are more prone to mean-reversion than small-cap stocks? But more intriguingly, can we not construct a basket of stocks, with custom-designed weightings, with the objective of optimizing its short-term mean-reversion property? I (and others before me) have done something similar in constructing a basket of stocks that cointegrate best with an index. Can we not construct a basket that is simply stationary (with perhaps a constant drift)?

Now, perhaps you will agree with me that ETF's are strange and wonderful creatures.

The best environment for quantitative trading

2009-10-11T09:00:00.004-04:00

Let me talk about a topic that is far more mundane than the usual high-brow theoretical discussions of strategies and algorithms, but that has no less long-term impact on the bottom line: what is the best office environment for research and execution of quantitative trading strategies?

I have worked in different office environments before, so I feel qualified to offer an informed opinion.

At Morgan Stanley, I huddled over a desk that is semi-partitioned from the rest of the office: nobody could see or bother me unless I or they stood up. At Credit Suisse, I shared an office with 2 other prop trading colleagues, one of whom was prone to freely sharing his opinion on various current affairs with his officemates. (On the other hand, he complained my biting an apple for lunch was too loud for him.) At Maple, a hedge fund in New Jersey, I shared an office with about 100 other colleagues on the trading floor, many of whom were prone to same opinion-sharing temptation.

Here at my own firm, I sit in solitude (except for my cat) in my basement office, my beloved classical FM streaming over the internet, my desktop electronically connected to my partner in our Chicago office, our trading servers at Amazon and elsewhere, and other clients and partners around the world, but unmolested throughout the day unless I voluntarily pick up the phone or answer an email or instant message.

Can you guess which environment is the one I find the most productive? Which one has the least stress? And which one contributes most to the bottom line of my employers/partners/clients?

(Hint 1: read Timothy Ferriss' book The 4-Hour Workweek. This guy checks his email only once a week.)

(Hint 2: my trading Sharpe ratio went from negative to >7.)

P.S. I look forward to meeting some of you at the Automated Trading Conference in London this Friday (my talk will start at 0900), and others at my pairs trading workshop on the preceding two days.

Are flash orders really so bad?

2009-09-20T11:27:00.002-04:00

I confess I don't know much about flash orders, not being one of the Big Boys on the Street, until I read that the SEC is banning them. (For a clear diagrammatic explanation of flash orders, see here. For a refutation of some of the myths and misunderstanding surrounding flash orders, see here.)

It seems to me that flash orders can be understood as "request for liquidity" issued to various potential market makers/liquidity providers, not unlike the usual "request for quotes" (RFQ) common in other industries. They are issued when there is not enough liquidity on a specific exchange to satisfy an investor's need, and they ultimately benefit investors by lowering their transaction costs. The fact that high frequency traders are able to make lots of money by providing this liquidity is besides the point. Liquidity providers are supposed to make money by providing liquidity!

Some people, including Senator Charles Schumer and this New York Times op-ed, believe that flash orders are akin to front-running, a clearly illegal trading activity. But they are wrong. Front-running means that if you know someone is going buy a stock, you step in front of them
and buy it cheaply first, hoping to sell it to this slower buyer at a higher price. In the case of flash orders, the high frequency traders are instead selling this stock to the original investor, often at a lower price than available elsewhere and thus benefiting this investor, hoping that the prices will come down in the future after this liquidity need subsides. This is manifestly not illegal. This is what a market is built for!

Another way to understand that flash orders are not at all front running is that anybody, including you and me, are free to put in limit orders at the same price as those of the high frequency traders, way ahead of time, in a specific exchange, and become liquidity providers ourselves. You don't have to wait for a "request for liquidity" before doing so. And presumably you will reap the same benefits as the high frequency traders. You are not taking any additional risks over the HF traders either, since if no requests for liquidity ultimately arrive, you are not any worse off for wear. You cannot begrudge the profits of the HF traders just because you didn't put the limit orders in place beforehand!

Maybe there are some other angles which I miss which can convince me that flash orders are evil. But until my kind readers convince me otherwise in the comments section, I will regard this piece of legislation as another SEC attempt at demagoguery.

Can a trader be a do-gooder?

2009-09-11T20:57:00.007-04:00

It occurs to me that the only way in which a trader can become more than a completely selfish, self-enriching, narcissistic person is to trade well enough so that you can manage other people's money and thus saving these investors from crooks and charlatans (provided you are convinced you are not a crook and charlatan yourself).

Other traders have advanced other arguments in favor of trading. But I am not convinced by them.

They say that we provide liquidity to other long-term investors who may need to liquidate their investments. But then, this applies only to mean-reversal strategies. Momentum strategies take away liquidity from the market, and in some cases exacerbating price bubbles. Certainly not something your grandma would approve.

Others argue that momentum strategies help disseminate information about companies through quick price movements. But can't we just watch Bloomberg or CNBC? Do we really need some devious insiders to convey that information to the rest of us through price movements?

No, I think that independent trading should serve only one purpose (besides short-term self-sustenance): as training and preparation to become a fund manager. Once you graduated from independent trading, you then enter into the grand contest among all fund managers to see who can best serve and protect investors' assets, (and be rewarded according to your standing in this contest.)

I know, this is the idealistic way to look at things. Serving and protecting seem to be what policemen should be doing, not traders. But as in quantitative trading, I think it helps one becomes more successful in one's activities by having a simple guiding principle or model. And it doesn't hurt that in this case, the principle would also be conscience-nourishing!

Have you traded 10,000 hours yet?

2009-09-02T21:35:00.003-04:00

Author Malcolm Gladwell, in his fascinating bestseller "Outliers: The Story of Success", cites neurological research showing that "10,000 hours of practice is required to achieve the level of mastery associated with being a world-class expert." This seems to apply across many different types of experts, whether they are "writers, ice skaters, concert pianists, chess players ... Even Mozart ... couldn't hit his stride until he had his ten thousand hours in".

Reflecting on my own experience, I have become consistently profitable only after 4 years of actual trading (research alone doesn't count -- real money need to be at risk.) So while the number of hours may not be exactly 10,000, the order of magnitude is about right.

So if your trading has not been profitable, ask yourself this: "Have I traded 10,000 hours yet?"

Using R to Test for Cointegration

2009-08-21T21:15:00.002-04:00

Paul Teetor, who guest-blogged here about seasonal spreads, recently wrote an article about how to test for cointegration using R. Readers who don't want to pay for a copy of Matlab should find this free alternative with similar syntax quite interesting.

Interview on backtesting

2009-08-14T14:22:00.003-04:00

I have given a 2-part interview (here and here) on the various nuances of backtesting on tradingmarkets.com. Most of the ideas have been covered in my book, but it does serve as a summary of what I consider to be the most important issues.

For those of you who are interested, I may be giving a workshop on general techniques in backtesting in London as well, in addition to my pairs trading workshop. Additional details will be available on epchan.com at a later date.

Are Triple Leveraged ETFs suitable for long-term holding?

2009-07-27T12:09:00.002-04:00

Triple leveraged ETFs marketed by Direxion have been all the rage lately. The fund management company says that they do not recommend buying and holding these ETFs. But is there any mathematical justification for this caution?

Before I answer this, it is interesting to note that these ETFs (e.g. BGU is 3x Russell 1000, TNA is 3x Russell 2000) are managed as constant rebalanced portfolios, a concept I discussed before. In other words, the fund manager has to sell stocks (or futures) when there is a loss, and buy stocks (or futures) when there is a gain in the market value of the portfolio, in order to maintain a constant leverage ratio of 3. This is also identical to what Kelly formula would prescribe, a methodology discussed extensively in my book, if the optimal leverage f were indeed 3.

However, the optimal f for such market indices are quite a bit lower than 3. Both Russell 1000 and 2000 have f at about 1.8. This means that since the funds are leveraged at 3, there is a real possibility that sustained losses could ruin the funds (i.e. NAV going to zero unless new capital is injected, which, er..., reminds me of a Ponzi scheme). So I would argue that not only should an investor not hold these funds for the long term, the funds themselves should not be leveraged at this level. Otherwise, it is a disaster waiting to happen.

A free Matlab-to-Interactive Brokers API

2009-07-17T12:18:00.005-04:00

For readers who do not want to pay for a commercial Matlab2IB API, Max Dama has put together a free alternative. Domenic has provided some additional sample Matlab codes for trading.

A user of the commercial product that I previously mentioned reports that "My problem with the matlab2ib product was that it did not have a function for all the Active X methods. ( for example the Market Scanners, Real time Bars and Fundamental Data methods are missing). I also had issues when I tried to steam in trades data(I'm not sure if the matlab2ib product allows you to even do this?)." Apparently Max's API has included these methods, though I have not personally tried them.

My interview, stop loss, and the Principle of Latest Information

2009-06-29T13:11:00.005-04:00

You can find an interview of me in the July 2009 issue of Technical Analysis of Stocks & Commodities magazine. I mentioned in that interview and also in my book that I believe stop loss should only be applied to momentum strategies but not to mean-reverting strategies. I explained my reasoning better in my book than in the interview, and so I will paraphrase the explanation here.

In algorithmic trading, it is reasonable and intuitive that we should always make use of the latest information in determining whether we should enter into a position, whether that information is price, news, or some analysis. Let's call this the Principle of Latest Information. (If someone can think of a better or sexier name, let me know!)

So let's say we have a stock model based on price momentum, and we entered into a long position based on a recent positive return on price. A few minutes later, the price went down instead of up, causing a big loss on our position. If we now ran this momentum model again, very likely it would tell us to short the stock instead because of the recent negative return on price. If we did that, we would be exiting the previously long position and became flat. This is in effect a stop loss, and it follows strictly from adhering to our model and our Principle of Latest Information.

In contrast, suppose we now have a stock model based on mean-reversion, and we entered into a long position based on a recent drop in price. A few minutes later, the price went down further instead of up, again causing a big loss on our position. If we now ran this mean-reversion model again, it would definitely tell us to buy the stock again because of the ever cheaper price. The model would not ask you to exit this position and take a loss. Hence, adhering to the model and the Principle of Latest Information will not lead to a stop loss for a mean-reverting model.

(Now, if we hold this losing long position long enough, the model will incorporate new historical prices into determining its long or short signals as it retrain itself, as the Principle of Latest Information says it should! At that time, it may indeed recommend that we exit the previously held long position at a loss. But this adjustment takes place at a much longer time scale, and therefore cannot really be considered a stop-loss in its usual sense.)

More generally, I find that at every turn, and not only in the realm of stock trading, applying the Principle of Latest Information always help me to be disciplined and not be afraid to enter into new positions, take loss or endure a drawdown as the case may be.

A job opening for quants

2009-06-25T10:35:00.003-04:00

Alphacet told me that they have a job opening for a quant who will be helping their clients backtest trading strategies, among other responsibilities. Given that Alphacet's clients include several major investment banks and hedge funds, this position should provide pretty good close-up view of how the major quantitative players operate.

A good book for quantitative traders

2009-06-15T11:14:00.005-04:00

Larry Connors and Cesar Alvarez (the guys behind tradingmarkets.com) recently published Short Term Trading Strategies That Work, a nice collection of simple technical trading strategies that you can easily backtest and verify.

As I have argued in my own book, simple strategies are often the ones that work best. As with any published strategies, you may find that their backtest performance may not be as high as advertised if you test them on a different time period or a different security, or with different transaction cost assumptions; but the main value of these strategies is that they serve as an inspiration to trigger your own imagination and motivate you to refine them further.

(For e.g., though the book mainly covers long-only strategies, you can easily imagine the accompanying short strategies.)

To be quite honest, this is one of the few books on trading strategies that I actually manage to finish reading from cover to cover.

MATLAB as an Automated Execution System

2009-05-29T16:10:00.003-04:00

I just published an article "MATLAB as an Automated Execution System". (It is available to readers of my book and subscribers to my Premium Content website.) It comes with example MATLAB codes executing a simple Bollinger-band high-frequency E-mini trading strategy.

As I mentioned before, I now find MATLAB to be a good platform not just for backtesting, but for automated execution as well. Of course, not all brokerages have API's that connect to MATLAB. My example codes are for submitting orders automatically to an Interactive Brokers account.

In general, I find that writing execution programs in MATLAB is a breeze compared to C++, Java or even C#. It takes about 1/5 the development time of a C++ program. Any performance limitations will probably not be due to MATLAB, but to the latency of your brokerage in updating positions and order status.

My pairs trading workshop in London

2009-05-07T13:02:00.004-04:00

I will be holding a 2-day, hands-on, pairs trading workshop in London, October 14-15. It will be held in conjunction with the Automated Trading 2009 conference organized by the Technical Analyst magazine. Please see details here.

Seasonal trades in natural gas and gasoline futures

2009-04-30T20:19:00.002-04:00

In my book, I mentioned 2 seasonal trades in natural gas and gasoline futures that have been consistently profitable for 14 years. (Mentioned here and here also.) And not only in backtest: I paper-traded them in 2006, and actually traded them in 2007-8, and all 3 years were profitable. How did they fare in this recession year? Quite poorly.

Depending on your exact entry and exit points, the gasoline trade lost about $2,500 per contract of RB. The natural gas trade lost about $7,700 per contract of NG.

You may have heard that natural gas price is at a 6-year low. In fact, we are not seeing any increase in industrial demand for natural gas. Apparently, somebody has forgotten to tell the nation's industrialists that an economic recovery is supposed to be under way.

Will I enter into these seasonal trades again next year? You bet I will.

Fios and EC2

2009-04-19T09:44:00.004-04:00

As an algorithmic trader, I am constantly in search of a better physical infrastructure where I can connect via the internet to my execution broker at the highest speed and with the least possibility of outage, and at a reasonable cost.

To that end, I would like to mention Fios, a fiber-optics service from Verizon with download speed of 50 Mpbs, upload speed is 20 Mbps, both faster than your typical T-1 line (1.5 Mbps). Furthermore, it costs only $45/month. Hey, even Paul Krugman has installed it at his home!

(I haven't tried it myself, and would like to hear from those of you who have and see if it is time to say goodbye to T-1.)

And as I have reported earlier, I am also constantly looking for a good cloud computing platform so that I can run more strategies without cluttering my office with computers. Finding one will obviate the need for any big investment in internet connectivity at the office.

To that end, I have been trying out Amazon's EC2 for several months. I use it to run one of our strateiges, and I have to report that my experience is mixed.

Firstly, if you are not an IT person, it does take a lot of time (8 person-hours?) to get set up and running, especially with their securities precautions. The learning curve is steep.

Secondly, and more annoyingly, the instances sometimes fail to start properly, or fail to bundle properly. (Bundling means saving the software configuration for future use.) I am using Windows instances. Maybe those who use Linux instances have better experiences?

Thirdly, and most annoyingly, when a new instance is started, Windows often cannot automatically synchronize its clock with time.windows.com or any other internet clock. As a result, the time is often wrong. Now, this may not be a big deal for usual office work. But when your automated trading strategy depends crucially on the time of the day, it can be quite fatal to your profit. If anyone has experienced a similar problem with Window's clock and know a fix, please let me know!

Despite all these hassles, I am still running strategies on EC2, hoping that once EC2 get past the beta release, things will be better.

The upcoming Black Swan of Black Swans?

2009-04-12T12:07:00.002-04:00

In a recent post (hat tip: Russell M.), Tyler Durden at Zero Hedge quoted a quant trader saying that "Anyone who is doing anything sensible right now is either losing money or is out of the market entirely", and that "liquidity deleveraging is approaching (if not already is at) critical levels", and finally the scariest part: "we have crossed into major statistically deviant territory, likely approaching a level that is 6 standard deviation away from the recent norms."

He pointed out that NYSE weekly volume is running about 9% below 52 wk average. But this may not necessarily be the result of deliberate hedge fund deleveraging or increasing risk-aversion by quant traders. From my personal experience, the usual opportunities for mean-reversion have just markedly decreased in the last few months, with much of the cash sitting on the sideline. I believe that quant traders are still ready jump in at any time to provide liquidity should the market demands it. I don't think that the recent market condition portends a 6-sigma event, but if one should occur, it may actually be a great profit opportunity for many short-term mean-reversion traders just as in those past 6-sigma events.

A reader comments on trading using Excel VBA and Factor Model

2009-03-27T14:10:00.003-04:00

Thoughtful comments from a reader John S. from the UK on his experience with trading technology and models:

"I have been developing my own personal automatic trading systems using Excel VBA and based on rules I have developed over the years as an active private trader investor using both technical and fundamental data analysis.

One of the key merits in adopting an automatic trading system approach that has helped me is to avoid the temptation for manual interference and thereby improving profitability by maintaining consistency. I have found the challenge of developing a successful system very rewarding from a personal perspective as I recognise that there are many that have tried and failed. However one problem I have encountered is my ongoing desire to regularly modify and improve the system which I have found can become counter productive as there is a real danger that system development becomes an end in itself! I just can't seem to stop tinkering as soon as I come up with a new idea or feature!

One advantage of using Excel VBA that I have found is that it is inherently flexible as it facilitates the processing of data which can be important especially when using fundamental data as part of the system. In this respect I recognise that every trader is trying to build in an edge that will make the system more profitable. I have noticed that many traders seem to only focus on price by trying to seek an edge by looking at special indicators or combination of indicators etc. Combining price data analysis with a Factor Model approach is a challenge which is ideally suited Excel VBA as it can be easily used to process both fundamental and macroeconomic data into a form that can be integrated with price data analysis.

I recognise from your book that Matlab is more powerful than Excel VBA and may be just as flexible in integrating fundamental and macroeconomic data but I just wanted to draw your attention to benefits I have found using Excel VBA which may suit those who like myself are more comfortable in using Excel VBA and are reluctant to change. Other features that can be exploited that I have found helpful when back testing are automatically producing Price Charts that incorporate Entry and Exit points which provides visual reassurance that the system is working as intended as well as generating automatic Word reports recording key output for future reference.

I am sorry if I sound too much like an advert for Microsoft!"

Mean-reversion is getting stronger

2009-03-13T11:44:00.003-04:00

As I mentioned in various previous blog posts, (e.g. see here), I believe mean-reversion strategies have been performing very well in the last year. Now here is an article (hat tip: Laurence) that provides concrete analysis to support this hypothesis. In fact, the author points out that most of the mean-reversion in recent years comes from the overnight close-to-open reversal.

A new service for retail investors

2009-02-26T17:34:00.004-05:00

Here is a new low-cost service called Alerts4All that offers technical trading signals for retail investors. You can, for example, have an alert sent to you every time a "Double bottom" pattern occurs.

A much more advanced version of the service will be rolled out soon -- I saw a demo today where you can backtest your strategies online, combining different fundamental and/or technical variables as entry or exit signals. They also have some built-in models for you to adapt (e.g. a model based on The Little Book that Beats the Market by Joel Greenblatt.) More interestingly, you can look at other people's trading models and their historical and/or real-time performance.

Matlab or Alphacet it is not, but I think it will be quite useful for many retail traders. It might even be useful to professional traders who want a quick-and-dirty way to test out ideas.

Trader tax proposal will be the death knell for statistical arbitrage

2009-02-22T11:22:00.002-05:00

U.S. Congressman Peter DeFazio, introduced H.R. 1068: “Let Wall Street Pay for Wall Street's Bailout Act of 2009”, which aims to impose a 0.25% transaction tax on the “sale and purchase of financial instruments such as stock, options, and futures.

Ladies and gentlemen, 0.25% is 50 basis points round-trip. Few if any statistical arbitrage strategies can survive this transaction tax.

And no, this is not "Wall Street paying for Wall Street's Bailout". This is small-time independent trader-entrepreneur like ourselves paying for Wall Street's Bailout.

Furthermore, this tax will drain the US market of liquidity, and ultimately will cost every investor, long or short term, a far greater transaction cost than 0.25%.

If you want to stop this insanity, please sign this online petition.

Finding seasonal spreads

2009-02-18T12:01:00.002-05:00

I am pleased to introduce guest blogger Paul Teetor for today's article.

----------

Finding Seasonal Spreads

By Paul Teetor

A seasonal spread is a spread which follows a regular pattern from year to year, such as generally falling in the Spring or generally rising in October. To find seasonal spreads, I've been using ANOVA, which stands for analysis of variance. ANOVA is a well-established statistical technique which, given several groups of data, will determine if the groups have different averages. Importantly, it determines if the differences are statistically significantly.

I start with several years of spread data, compute the spread's daily changes, then group the daily changes by their calendar month, giving me 12 groups. The ANOVA analysis tells me if the groups (months) have significantly different averages. If so, I know the spread is seasonal since it is consistently up in certain months and consistently down in others.

The beauty is that I can automate the process, scanning my entire database for seasonal spreads. A recent scan identified the spread between crude oil (CL) and gasoline (RB), for example. The initial ANOVA analysis indicated the CL/RB spread is very likely to be seasonal. This bar chart of each month's average daily change demonstrates the seasonality. (Click on the graph to enlarge it.)

The lines show the confidence interval for each month's average. Notice how May and June are definitely "up" months because their confidence interval is entirely positive (above the axis). Likewise, November and December are definitely "down" months. For all other months, we cannot be certain because the confidence interval crosses zero, so the true average change could be either negative or positive. The conclusion: Be long the spread during May and June; be short during November and December.

For more details, please see my on-line paper regarding ANOVA and seasonal spreads.

- Paul Teetor

The limitation of Sharpe ratio

2009-02-12T18:19:00.002-05:00

Just as one should not trust VaR completely, one should also beware of high Sharpe ratio strategies. As this Economist article pointed out, a strategy may have a high Sharpe ratio because it has so far been accumulating small gains quite consistently, but it could still be subject to a large loss when black-swan events strike.

Personally, I am more comfortable with strategies that do the opposite: those that seldom generate any returns, but always earn a large profit when financial catastrophes occur.

The peril of VaR

2009-02-06T18:53:00.002-05:00

This Quebec pension fund lost some $25 billion due to non-bank asset-backed commercial paper (ABCP). Their Value-at-Risk (VaR) model did not take into account liquidity risk. As usual, the quants got the blame. But can someone tell me a better way to value risk than to run historical simulations? Can we really build risk models on disasters we have not seen before and cannot imagine will happen?

(Hat tip: Ray)

Kelly formula revisited

2009-02-01T08:11:00.003-05:00

Some discussions on Kelly's formula with a reader Steven L:

Q:

"I am more than half way through your book and am stuck at a concept that I can't seem to find an answer in any other forum.

I have read Ralph Vince's "Portfolio Management Formulas," which uses Kelly's formula to calculate an optimal "fraction" of the bankroll to bet on each trial. So a trader can calculate a fraction of his total trading account value to risk on each trade. What I am referring to is the so-called "fixed-fractional" trading. There exists an optimal fraction that will maximize the geometric growth rate of the trading equity, in theory anyway.

However, in the money management chapter of your book, you use Kelly's formula to derive an optimal "leverage." This seems to be in conflict with what I learned from Ralph Vince, since leverage is usually great than unity and fraction is usually less than unity. I can't seem to make a connection between these two concepts. I have also seen the same optimal leverage formula in Lars Kestner's Quantitative Trading Strategies and asked the same question on some forums, but no one was able to give me a clear satisfactory answer. It would be greatly helpful if you can help me sort out the confusion."

A:

I don't have Ralph Vince's book with me, but if I recall correctly, his formulation is based on discrete bets (win or lose, no intermediate outcome), much like horse-betting or in a casino game. My approach, or rather, Professor Ed Thorp's approach, is based on continuous finance, assuming that every second, your P&L could fluctuatate in a Gaussian ("log-normal") fashion.

For discrete bets where you could have lost all of your equity in one bet, surely one should only bet a fraction of your total equity. For continuous finance, there is very little chance one could have lost all of the equity in one time period, due to the assumed log-normal distribution of prices. Hence one should bet more than your equity, i.e. use leverage.

Q:

In example 6.2 in your book, the portfolio consists of only long SPY, which has little chance of going to zero. So I can see how it is reasonable that you use the continuous finance approach and apply the optimal leverage to scale up the return.

But let's assume that the portfolio consists of a single strategy that buys options. Suppose this strategy will lose most of the time due to time decay but will make profit once in a while due to black-swan events. I don't think it's a good idea to bet the entire portfolio equity on each trade for this strategy. Can you still apply the continuous finance approach in this case, since in reality trading is like making discreet bets? Should we expect the mean and variance of this strategy automatically result in an Optimal Leverage that is less than one? So that we actually need to risk a fraction of the account equity per trade?

A:

The formula I depicted in the book is valid only if the P&L distributions are Gaussian. If one expects a fat-tailed distribution due to black-swan events, a different mathematical model needs to be used, though it can still be within the continuous finance framework. However, for simplicity's sake, if the distribution looks multinomial (e.g. high probability of "Win a lot" v "Lose a lot"), then you may model it with fractional betting just like a casino game.

Algorithmic Trading Technology Update

2009-01-16T19:15:00.002-05:00

Lately a number of new (at least to me) technologies useful to the algorithmic trader came to my attention:

1) Matlab2IB API

I said in my book that it is difficult to use Matlab as an execution platform. As Max has pointed out, this is no longer true. This inexpensive API connects Matlab to your Interactive Brokers' account. It allows you to retrieve historical data, get real-time quotes, and send orders. In other words, all the basic functions you need to create your own execution engine.

2) R

Many people (hat tip: Steve H.) know that R is an open-source (i.e. free) alternative to Matlab. I find that there is also an API that connects R to Interactive Brokers, though I have not tried it myself.

3) Trade Ideas

Trade Ideas (hat tip: Russell M.) is a complete automated trading platform that provides connections to different brokerages (scottrade, IB, TD Ameritrade, etc.)

4) Amazon EC2 cloud computing platform

Running out of PC's to run your myriad strategies? Try Amazon's EC2 cloud computing platform. For a modest hourly fee, you get access to an instance of either Linux or Windows environment, and you can add as many instances as you want. The connection speed is supposed to be at least 10x T-1 line, well-suited to high frequency traders . Here is some other performance benchmarks.

Hedge funds move to "easy-to-understand liquid strategies"

2009-01-12T13:58:00.002-05:00

See this interesting article (registration required) on FT on the state of the hedge fund industry. Paul Tudor Jones, Citadel, and Fortress Investment Group are all said to be moving to "easy-to-understand liquid strategies", otherwise known as "statistical arbitrage".

(By the way, I have been urging traders to do just that in my book.)

How is the job market for quants these days?

2009-01-09T12:18:00.002-05:00

Felix Salmon claimed in this post (hat tip: J. Rigg) that the quant job market is alive and well. However, I haven't heard much from the usually diligent headhunters in the last few months, which doesn't bode well. Maybe some of our readers can comment on the current state of the quant job market?

In that same post, Felix wondered whether to incorporate the extraordinary period of 2008 as part of backtesting data. Actually, I don't see much of a problem here -- of course one should include 2008. The only reason a trading model would have performed poorly in 2008, as opposed to 2006, 2007 or 2009, would be that its parameters are fitted too tightly to historical data. If you try out some parameterless trading models like I advocated, 2008 is not that unusual except for its higher volatility.

The enduring profitability of mean-reversion strategies

2008-12-09T15:14:00.000-05:00

Some readers have doubts about my assertion that mean-reversal models continue to be very profitable during this whole year of financial and economic disasters. So I backtested the mean-reversion strategy in Example 3.8 of my book with the most recent one-year SP1500 data. Without transaction cost, the Sharpe ratio is 4.8. Even after subtracting 10 b.p. round-trip transaction cost, it is still at 3.5.

Since the strategy was constructed over a year ago while I was writing the book, this most recent backtest is done on unseen data, with absolutely no look-ahead bias!

Josh Brolin on day trading

2008-12-02T10:27:00.001-05:00

Actor Josh Brolin ("Milk", "W", "No Country for Old Man") said on Charlie Rose that his trading portfolio had a 57% return this year. His strategy is based on entering on reversal from a trend. Sounds pretty reasonable to me: maybe some of our readers can backtest this.

Below is the full interview, beginning with Sean Penn, then goes on to Gus Van Sant, then finally Josh Brolin mentioned his day-trading at the very end of the 1 hour show.

Interested? He is starting a multi-million dollar hedge fund to manage your money.

My book on Quantitative Trading is published

2008-11-07T10:07:00.000-05:00

My book on Quantitative Trading has been published and is now available from Amazon.com. Many thanks to all of you for your ideas, comments, and support!

Some riskless profit, and why it exists

2008-10-28T10:58:00.003-04:00

Numerous commentators have pointed out the enormous yield spread between agencies debt (Fannie/Freddie) and US Treasuries.

Here are some links kindly provided by a reader: 10 yr Fannie/Treasury, 5 yr Fannie/Treasury, 10 yr Freddie/Treasury, and 5 yr Freddie/Treasury.

Currently their spreads are above 150 bp. Since the US government has nationalized Fannie and Freddie, this 150 bp is a riskless profit. As the blog Accrued Interest has pointed out, one reason this riskless profit exists is hedge fund deleveraging: nobody has the risk appetite to arbitrage this spread at a meaningful scale.

Brad Setser, a blogger at the Council of Foreign Relations, suggests that the Chinese government, who does have a lot of cash to benefit from this high yield, should go ahead and buy up these agencies debt. However, if you read the Chinese blogs and online comments, there is enormous internal pressure for the government to spend some of this money on infrastructure projects, social security, health care, etc., so I doubt that the Chinese government will have stabilizing the US mortgage market at the top of its agenda. As a result, arbitrageurs out there should have no fear that this opportunity will disappear any time soon.

How does the financial crisis affect quantitative trading?

2008-10-20T09:45:00.001-04:00

Now that we are reasonably sure the financial world is not coming to an end yet, it is reasonable to ask how quantitative strategies have been faring under this extreme market stress. Despite reports of massive hedge fund deleveraging and negative YTD returns, I believe quantitative strategies, especially statistical arbitrage, have survived the period relatively unscathed. But here are a few of my thoughts:

1) The paltry 10% annual returns that a mediocre statarb fund can deliver is suddenly looking pretty good when the risk-free rate is under 1% and a prolonged bear market is on the horizon.

2) Mean-reversal models continue to beat momentum models in this crisis environment, as in previous crisis environments. This is not surprising because market returns have completely dominated specific returns, and of course market returns have been highly mean-reverting lately.

3) Models involving shorts are under some tumoil because of regime-change induced by new and ever-changing short-sale regulations. (For a while, I even have difficulties locating SPY for hedging purposes!)

4) Models are generally trained on data with far lower volatility than is recently realized. (Even incorporting VIX in a model does not guarantee that it can match realized volatility any better.)
As a result, P&L's fluctuations are also much higher than usual, which induces deleveraging as a risk-management measure, which drains liquidity from the market, which in turn leads to still higher volatility. The usual viscious cycle.

5) Political risks in an election year have further reduced leverage and increased volatility. What if there is an assassination? What if the wrong party got elected? What if the paper-trailess electronic voting machines cause another dispute for a month? The nightmares will continue at least until the morning of Nov 5.

6) Normally, lack of liquidity in the market is good for statarb models since they profit from renting out temporary liquidity. However, this profitability assumes that there are buyers of last resort for the market: the long-term investors, the mutual funds, Warren Buffet, etc. When they are absent, statarb investors can be left holding the bag. Fortunately, Warren Buffet & Co. has indeed stepped in and we statarb traders can breathe a sigh of relief.

7) I have been paying particular attention to 3 websites since the crisis began in order to judge whether I should return to my normal leverage: the Ted spread (I am waiting for it to return to below 2), the Calculated Risk blog, and Paul Krugman's blog. This crisis is caused by panic in the credit market, so we should look for credit market returning to normal before declaring victory. The VIX? Not so much because I believe it is backward-looking in this environment.

8) Watching Fannie, Freddie, Lehman, AIG, WaMu, Wachovia, Iceland, and the initial bailout bill failed feels like reading Chapter 8 of Harry Potter and the Deathly Hallows: "The Ministry has fallen. Scrimgeour is dead. They are coming." The Dark Lord is taking over our economy.

Webinar on algorithmic trading system

2008-09-29T19:35:00.001-04:00

Recently I participated in a webinar on using an algorithmic trading system called Alphacet Discovery. A link to the webinar can be found here. Some previous research using this system was described here.

Index change strategy

2008-09-08T20:03:00.001-04:00

Some years ago, I traded a simple index change strategy: buying stocks to be added to the SP500 index at the market open right after the index change announcement and exiting the position at the close, and similarly shorting stocks to be deleted. The results were mediocre at best.

However, new research by University of Edinburgh Business School suggests that a similar strategy works well for FTSE350 stocks (Hat tip to J. Rigg for the link). The trick is to predict which stocks are to be added or deleted 30 days before the announcement ("review date"), buy/sell the stocks, and close out the positions just before the review date.

Since the criteria for inclusion in the FTSE index is well-defined (and primarily based on market capitalization), it should not be hard for the interested traders to make their own predictions and profit from this rebalancing.

Behavioral finance we can all use

2008-08-25T13:13:00.002-04:00

In their new book "Nudge: Improving Decisions About Health, Wealth and Happiness", U of Chicago economist Richard Thaler (of behavioral finance fame) and Harvard law professor Cass Sunstein gave a few pieces of personal finance advice, one of which coincided with my point in a previous post: buy insurance with the largest deductible available. The others are: don't invest much in your employer's stock, don't pay points on mortages, and don't pay for extended warranties. The book is reviewed in the NYT Book Review.

Predicting SP500 futures using investor sentiment

2008-08-22T12:41:00.001-04:00

Ronald Domingues, an economics graduate student, has done an interesting study of how well a group of qualified investors with superior skills can predict the movement of the SP500 index. This group of qualified investors are selected by their track records of making correct predictions, and as a result of submitting their predictions going forward, they are eligible to be notified of the average predictions of other qualified investors, thus enabling them to make a better informed investment decision.

In other words, the elite will benefit from the collective wisdom of other elites -- sort of like the real world, isn't it?

How well does it work in practice? Well, they correctly predicted whether SP500 index will go up, down, or flat, a whopping 65.2% of the time. The details can be found on his website, where you can also sign up to see if you can join the elite.

More on parameterless trading model

2008-08-16T09:36:00.003-04:00

I have written before that my ideal trading model is one that has no parameters, and what ways there are to accomplish this. Actually, I forgot to mention that a trading strategy proposed by Dr. Andrew Lo discussed previously is in fact parameterless, and the technique is so general that it can be applied to any mean-reverting strategy.

The technique is simply this: maintain a long (or short) portfolio with capital proportional to the distance between a supposedly mean-reverting measure and its long-term mean value.

For e.g. if you are pair-trading PEP vs KO, and you believe that the spread between PEP and KO is mean-reverting, then this spread is the mean-reverting measure you should employ.

As the spread moves away from its mean, keep buying (or shorting) the spread in equal dollar amount. And as the spread reverts, keep selling (or buying) the spread in the same dollar amount. What this dollar amount should be depends on: a) the total buying power you possess, b) the expected maximum deviation of the spread from its mean, and c) how often you intend to buy/short. Note that point c is not a parameter: it is arbitrary and limited only by transaction costs, technology, and other operational issues. As for the expected maximum deviation, it can be obtained by observing the history of the spread since inception.

This scheme thus obviates the need for entry or exit thresholds, and with them, the possibility of data-snooping bias. (You may still want to impose an entry threshold based on transaction cost consideration - but that would not count as a free parameter.)

What are we hedging here?

2008-07-18T14:51:00.000-04:00

I wrote a blog article last year on why hedging isn't always better. The more I try to practice what I preached, the more I am convinced that most of the time, we are hedging the wrong risks.

Hedging should not be about reducing volatility in our portfolio. If reducing overall volatility is our goal, we should simply reduce leverage, as I have argued in my previous article. If volatility in a particular industry group is too much for us, (banks? brokerages? energy stocks?), just reduce the capital allocation in that group.

Sure, if hedging does increase your overall Sharpe ratio, go ahead and hedge to your heart's content. Kelly's formula tells us that the higher the Sharpe ratio, the higher the compounded growth rate of your wealth. The problem is, many of us hedge even when doing so do not clearly increase Sharpe ratio. A further problem is that we can achieve this maximum growth rate only if we use the high leverage recommended by Kelly's formula, but this leverage often exceeds what our brokerage would allow us. It is not clear that it is beneficial to waste our buying power on the hedge if we can only operate at sub-optimal leverage.

To me, hedging should be about eliminating the risk of ruin (equity reduced to zero) due to unexpected, catastrophic events. (Many sophisticated hedge fund managers cannot even meet this simple survival criterion, giving lie to the whole notion of "hedge" funds.)

For instance, let's assume that the worst one-day drop in the market index can be 20%. Furthermore, let's assume that you are able to endure a 30% reduction in equity during one trading period. Then you should not be afraid to have a net long exposure of 150% of your equity. In other words, not only should you not hedge, but you should go ahead and leverage your long-only portfolio 1.5 times.

I believe this notion of hedging, or buying insurance, extends to all spheres of our lives. We should avoid ruin, not mere losses. Otherwise, you will be paying too much on the insurance policy over the long term. In other words, max out the deductible on your insurance policy!

Have oil stocks exhausted their run?

2008-06-26T15:40:00.001-04:00

Floyd Norris, the chief financial correspondent of The New York Times, suggested in his blog today that we are looking at "The Beginning of the End for High Oil Prices". What is the basis of his optimism? He argued that oil stocks have been lagging oil prices lately, and therefore equity investors must believe that high oil prices are causing demand destruction which will eventually reduce oil prices and oil companies earnings.

I beg to differ.

Look at the long-standing spread relation between an oil stock ETF and an oil commodity ETF, e.g. XLE vs USO, which I have been commenting on and tracking since October 2006. At the moment, this spread is within 1.4 standard deviations of its historical value. See my table here (subscription required). In other words, oil prices and oil stock prices are at approximately their long-time historical average. I would hardly call that suggestive of the beginning of the end.

Futures markets have no effect on spot prices

2008-06-23T06:28:00.000-04:00

NYTimes columnist and Princeton economist Paul Krugman has previously (see my link here) argued that the trading of oil futures should have no effect on spot oil prices, contrary to what many politicians and pundits think. Here is his latest elaboration of that argument.

Statistical electoral vote predictor: Update

2008-06-21T11:33:00.000-04:00

For readers who have been tracking the Gott and Colley presidential electoral vote prediction, they will notice a sudden switch over to a predicted Obama victory in the last few days. That is because polls from OH, VA and MO are now available -- surprising because the 3 states are not hitherto known for their Democratic leanings.

It seems to me that, after all, the stability of prediction at this early date is quite questionable due to the paucity of state polls, a point already made by Dr. Colley.

Statistical model predicts a McCain victory?

2008-06-12T12:46:00.001-04:00

There has been a lot of buzz lately about a simple statistical model proposed by astrophysicists Prof. Gott and Dr. Colley that uses the median polls of each state to predict the November electoral vote. (For our un-American readers, the electoral vote is what determines the outcome of a general election, not the popular vote, in case the nightmarish 2000 election has not already drilled this fact into the world's collective consciousness.)

Dr. Colley has set up a website to track daily such polls to gauge the mood of the states. The authors have tested this method on the 2004 election, as well as numerous sporting events outcomes, and found it to be highly predictive.

Right now, they are betting on a McCain victory.

But there is one caveat that many bloggers have pointed out, and it is the same caveat that I have previously applied to the predictive accuracy of political futures market such as intrade.com. The caveat is this: polls (and futures market) change with time. And at different times, they predict different election outcomes. So for example, at this point (June 2008), the polls predict a McCain victory, while the futures market at intrade.com predicts an Obama victory. Who is right?

The answer is: neither. As Dr. Colley has explained to me, no backtest as far back as the June of an election year has been conducted. (Their research was based on polls from September onwards.) So we do not know if the June polling prediction has any accuracy. Similarly, as I pointed out before, the futures market can swing violently even on Election Day, even in the last hours of an election.

One advantage of the Gott and Colley method though, is that the predictions resulting from median poll statistics are remarkably stable over time. In 2004, there was very little movement in the electoral tally from September through election day. Extrapolating this result, we can be somewhat more confident of their prediction vs. Intrade.com's, even at this early date.

And in any case, I have observed that the political futures markets are highly mean-reverting, implying that the current large 20 points spread between the Obama and McCain futures is destined to decrease in the coming months.

As an arbitrage trader, I have therefore proceeded to short the Obama future.

Pre-order my book from Amazon.com

2008-05-28T09:55:00.002-04:00

A reader told me that he can now pre-order my book "Quantitative Trading: How to Build Your Own Algorithmic Trading Business" from Amazon.com. The scheduled publication date is November 24, 2008.

Parameterless trading models

2008-05-27T10:08:00.004-04:00

A portfolio manager that I used to work for like to pronounce that his trading models have "no free parameters". As is customary in our secretive industry, he would not elaborate further on his technique.

Lately, I begin to understand what a trading model with no free parameter means. It doesn't mean that it does not contain any lookback period for calculating trends, or thresholds for entry or exit. I think that would be impossible. It just means that all such parameters are dynamically optimized in a moving lookback window. This way, if you ask: "Does the model have a fixed profit cap?", the trader can honestly reply: "No, profit cap is not an input parameter. It is determined by the model itself."

The advantage of a parameterless trading model is that it minimizes the danger of overfitting the model to multiple input parameters. (The so-called "data-snooping bias".) So the backtest performance should be much closer to the actual forward performance.

Now, it is quite computationally challenging to optimize all these parameters just-in-time for your next order, but it is often even more difficult to do that in a backtest, given that a multidimensional optimization need to be performed for each historical bar. As a result, I personally have seldom traded parameterless models, until I get to research my regime-switching model. That model is almost parameterless (I left out a few parameters from optimization because of a lack of time, not because of any technical difficulties).

The reason backtest optimization can now be done within a few minutes is due to my use of Alphacet Discovery's server-based optimization engine. There may be other optimization software out there that performs similar functions efficiently -- I welcome comments from the reader.

Regime switching paper

2008-05-26T08:36:00.003-04:00

A preprint version of my Regime Switching and Machine Learning article can be found on my premium content area.

Machine Learning + Regime Switching = Profitability?

2008-05-23T14:26:00.005-04:00

My article on a trading strategy based on regime switching and machine learning techniques is now available on Automated Trader magazine (subscription required). The software I used to research this model is Alphacet Discovery, an industrial-strength backtesting, optimization, and execution platform.

Are high oil prices due to hedge fund speculation?

2008-05-12T07:33:00.002-04:00

The economist Paul Krugman advances an interesting argument today in the New York Times against the idea that high oil prices are due to hedge fund speculation.

He believes that speculative buying can lead to persistent high prices (which has been the case for the last few years) only if there is physical hoarding. Yet oil inventory level has been normal for this period.

Indeed, I have been trying to find a mean-reverting strategy to trade oil and oil-related assets for some time now. So far, none have outperformed (even on a risk-adjusted basis) just buy-and-hold energy stocks for the long term!

5%: an important number for real estate investors

2008-05-10T12:57:00.003-04:00

Equity investors like to check out a company's price/earnings ratio before they invest in its stock. Likewise, real estate investors should do the same before buying a house. The equivalent of price/earnings ratio for real estate is the price/rent ratio, or inversely, the rent/price yield.

What is a reasonable rent/price yield for US residential real estate? According to Morris Davis of the University of Wisconsin-Madison, and Andreas Lehnert and Robert Martin of the Fed, the long-term average is 5% (i.e. the annual rent of a house should be about 5% of its market value). As the Economist magazine has reported, at the height of the US housing boom, this figure dropped to as low as 3.5%.

Currently, this ratio is at about 4.3%, which implies that average US housing price has to drop another 14% in order to return to its historical fair value.

Can quantitative traders profit from this prediction? Well, we can always short the S&P/Case-Shiller Home Price Indices futures at the Chicago Mercantile Exchange.

A combination momentum and mean reversal model based on earnings annoucements

2008-05-04T08:34:00.005-04:00

Mark Hulbert of the New York Times just discussed 2 momentum strategies investigated by professors David Aboody, Brett Trueman and Reuven Lehavy.

Strategy A: pick stocks in the top percentile of 12-month returns. Buy them (individually) 5 days before their earnings announcements and sell them just before the announcement.

Strategy B: pick stocks in the top percentile of 12-month returns. Buy them (individually) 5 days immediately after their earnings announcements and hold them for 5 days.

Strategy A is very profitable: the annualized excess return is 47% before costs. (To be taken with a grain of salt due to the large transaction costs associated with trading momentum strategies, especially if small-cap stocks are involved.) Strategy B is very unprofitable: the annualized excess return is -43% before costs.

So what are the ways we can make best use of this research?

Naturally, instead of buying the top percentile after the earnings announcements, we should have shorted the stocks, thus making Strategy B a reversal strategy instead.

Furthermore, what about the bottom percentile of stocks? Should we have shorted them prior to the announcements, and bought them after the announcements? If so, we would have a very nice dollar-strategy for you statistical arbitrageurs out there!

8 Recommended Sites for Economic Research

2008-04-20T17:30:00.003-04:00

I am happy to have my guest blogger Heather Johnson write about economics again. (I hope to emerge from my hiatus soon after finishing the final draft of my book on quantitative trading.)

=====================================

8 Recommended Sites for Economic Research

By Heather Johnson

Without the proper research, your trading strategies are just a shot in the dark. Don't rely on soundbites and headlines to tell you how the economy is doing. A wise investor will be following trends and analyzing his or her own collected data. Below are eight recommended sites for economic research that traders should find very useful.

AEI Research - The American Enterprise Institute (AEI) for Public Policy Research is a non-profit group that is dedicated to educating people on economics, as well as politics, government and social welfare. You can find economic policy reports here that may influence your trading.
BEA – The U.S. Bureau of Economic Analysis (BEA) provides economic data in a timely and unbiased manner. This service to the public helps people to gather a more accurate view of the U.S. economy. Reports are categorized by region and industry.
CIBC World Markets – This organization is the corporate banking department of CIBC, one of the largest North American financial institutions. The global economic data provided by CIBC World Markets is considered to be amongst the most reliable sources for economic indicators.
FedStats – This site offers the full range of economic statistics provided by the U.S. federal government. It also gathers data and trends from over 100 agency Websites.
Federal Reserve – A trader should always be interested in what is going on with the Fed. Here, the institution provides regularly updated bulletins and data.
The Financial Forecast Center – While this isn't a virtual crystal ball, it does offer third-party, objective economic data and forecasts. Compiled by artificial intelligence and available in a free subscription, everything found on this site is completely quantitative.
Free Lunch – Ah, and you thought there was no such thing. This source of economic data and analysis begs the question, "Why pay anything?"
Bloomberg.com Economic Calendar – This helpful calendar is brought to you by one of the most well-known names in finance. A day trader will find this calendar most useful when trying to determine how the market will move.

Although the list above is far from exhaustive, it should give you plenty of information to chew on for a while. Whether you are trying to forecast today's market or the market over the next six months, you will need to conduct some serious research beforehand.

=====================

Heather Johnson is a freelance finance and economics writer, as well as a regular contributor for CurrencyTrading.net, a site for currency trading and forex trading information. Heather welcomes comments and freelancing job inquiries at her email address heatherjohnson2323@gmail.com .

5 Steps to Managing Risk as a Microfinancier

2008-03-20T07:12:00.002-04:00

It might surprise some of you that lending money to middle-class American home owners to buy houses may be much riskier than lending money to Bangladeshi farmers to buy their first cellphones. (The beauty of diversification at work here?)

I have invited guest blogger Heather Johnson to explain microfinancing, and the quantitative risk management tools available if you want to do it yourself.

====

5 Steps to Managing Risk as a Microfinancier

By Heather Johnson

Microfinancing is a growing trend among investors, as it offers low-risk money opportunities and a way to bring social change to poverty-stricken communities. "Low-risk" doesn't equal "no risk," of course, so many potential microlenders are keen to learn the ins and outs of credit risk management in this arena. After all, most microborrowers have poor credit or no credit at all. A villager who needs $200 for starting a third-world chicken farm isn't going to fare well in that department, as you can imagine.

The good news is, even in the event of a loan delinquency, you won't be losing a substantial amount of money. Most microloans range from a few hundred to a few thousand dollars. Any losses are unfortunate, though, so you will want to manage your microlending risks and keep loan delinquencies to a minimum.

Here are five steps to managing your risk as a microfinancier:

Research Your Borrower – If you're lending through a site, such as Prosper, then you will have access to your borrower's profile and credit reports. However, don’t be afraid to ask more questions if you have any doubts about this person's ability to repay the loan. If you are lending the money through other channels, definitely start with the credit reports and interview the borrower.
Lend With a Group – Though this won't make your borrower any more likely to repay a loan, lending with a group will help to spread out the cost and share responsibility. In other words, you will be risking less money and will have other people with the same interests to consult with.
Use Analytical Tools – Third-party applications can help you determine what is working best with your microlending. Both seasoned microlenders and newcomers are highly encouraged to use such tools. Microfinance sites that come with excellent built-in tools include Trickle Up, Opportunity International and Heifer International.
Provide Incentives – Consider an incentive program for those who pay on time. A small, inexpensive gift will be very appreciated by those living in third-world countries. Lenders have used food, such as rice or corn meal, as a bonus.
Be Proactive in Collecting – This doesn't mean you should harass your borrowers. However, you should research your delinquent accounts as soon as payments are late, rather than letting them go into default. There could be a simple breakdown in communication or an emergency on the borrower's end.

One of the biggest draws of microfinancing is the relatively low risk involved. However, that doesn't mean that you will have a 100% success rate. The best way to get your feet wet is to start with a small loan. Something as low as $100 will let you learn the process and allow you to become more comfortable with the system. Microfinancing isn't for everyone, but you may just find your niche with this kind of investment.

====

Upcoming seminar on subprime mortgage crisis

2008-03-03T14:58:00.002-05:00

For readers who live in the New York area, here is an interesting upcoming seminar at Columbia University:

The subprime mortgage crisis of 2007: Anatomy of a market failure

Date: 03-10-2008
Start Time: 6:00pm
End Time: 7:30pm
Speaker: Kenneth A. Posner, Morgan Stanley
Location: 412 Schapiro CEPSR, Davis Auditorium

ABSTRACT

As home prices soared in 2004-5, consumers, realtors, mortgage lenders,
homebuilders, and investment banks all benefited. But few thought the good
times would last -- after all, everyone had learned to recognize a bubble
when they saw one. If that's the case, how did mortgage losses turn out so
large, and why do we find ourselves today confronting a major financial
crisis? This presentation will survey the damage resulting from the
subprime mortgage crash and provide a possible explanation for the magnitude
of the surprise which may be relevant to investors and risk managers in
other markets.

BIO

Kenneth Posner is a managing director and head of the mortgage finance and
specialty finance equity research team. Prior to joining the Equity Research
department in 1995, Ken worked in Morgan Stanley's investment banking group,
where he focused on commercial real estate transactions. He previously
served as a captain of infantry in the US Army, and was airborne and ranger
qualified. Ken earned a B.A. from Yale University in 1985 and an M.B.A. with
honors from the University of Chicago Graduate School of Business in 1991.
He is a Certified Public Accountant and holds the Chartered Financial
Analyst and Financial Risk Manager designations

Had it been really that bad?

2008-02-22T08:24:00.002-05:00

According to Eurekahedge Hedge Fund Index, hedge funds had the worst performance in eight years during this past January. And long-short equity funds had the poorest performance among them all.

Looking for momentum? Check outside the US

2008-02-19T08:24:00.004-05:00

Momentum vs. mean-reversion has been a perennial theme in investing, not least quantitative investing. My contention has always been that momentum strategies are generally less reliable than mean-reversal strategies. (See here or here.) My reader Mr. J. Rigg told me about a recent article in the Financial Times suggesting that momentum strategies are alive and well, according to the research by Prof. Elroy Dimson et al at the London Business School. The strategy is very simple: buy the stocks with the highest returns in, say, the last 12 months, short the ones with the lowest returns, and hold for, say, 1 month. If you run this strategy for the top 100 UK stocks from 1900 to 2007, the average annualized return before costs is about 10%.

There are, however, a number of caveats worth noting in this study:

First, it is very transaction-costly to implement momentum strategies for small or even mid-cap stocks. If you factor in costs, 10% can easily become 5% -- not an impressive number even for a dollar-neutral strategy. (Though one should note that the infrequent rebalancing renders transaction costs consideration less important.)

Second, the drawdown durations are quite lengthy -- sometimes exceeding 2 years. This is not acceptable performance for many hedge funds. Such lengthy drawdowns have been a common feature of many momentum strategies that I have personally studied and traded.

Third, and most interestingly, in the period 2001-2007, this momentum strategy has stopped working altogether for the US market, while continuing to deliver positive returns in other markets!

What may be the reason for this dichotomy between US and international markets? Momentum strategies generally derive their power from the slow diffusion and analysis of information: if all investors are simultaneously aware of all the relevant financial information about a company and can analyze the significance of the information instantaneously, they will have come to a consensus fair market value instantaneously and no momentum in the price will result. Hence perhaps the disappearance of momentum in the US equity market means what most people know already: that it is the most efficient equity market of all.

Are quant strategies in trouble yet again?

2008-01-27T12:17:00.000-05:00

There were reports that quant strategies have been suffering again in January, given the market turmoil generated partly by the Societe Generale scandal. Mr. Matthew Rothman of Lehman Brothers pinned the blame on momentum strategies (Hat tip: 1440 Wall Street). I partly agree with that assessment, but the full picture is more nuanced.

As I have written in my previous post, December has been a disastrous month for value (or mean-reverting) strategies, based on both public commentaries and personal experience. Yet, as always, mean-reverting strategies bounced back in January and all the pain is gone. In fact, the Societe Generale scandal and the subsequent 1/22 Fed bailout has been a huge bonanza to mean-reversion traders, just like the August disaster had been. (Remember: mean-reversion traders profit from providing liquidity during market panic.) Meanwhile, though December has been a good month for momentum strategies, January has become increasingly inhospitable to them. But one should not be surprised at all. As I have explained before, momentum strategies generally tend to be more unstable and have lower Sharpe ratios than reversal strategies. Any wise quantitative portfolio managers would always allocate a lower proportion of capital to momentum strategies than to reversal strategies. Hence it is no excuse at all to say that a quant portfolio has been hurt by losses in momentum trading -- they are to be expected quite frequently!

A sea of pain

2007-12-15T16:21:00.000-05:00

This Economist Magazine article confirms my personal experience that value investing is in a sea of pain at the moment. The reasons are quite different from the last time (during the dotcom era) when value investing was in the doldrums. This time around, people are not full of euphoria about the prospects of growth stocks -- they are just getting increasingly gloomy of value stocks which seem to be getting cheaper by the minute.

Seasonal trades in stocks

2007-11-23T07:45:00.000-05:00

Readers of this blog have seen my discussions of various seasonal trades in commodities futures (e.g. see this article). Recently, Mark Hulbert of the NYTimes drew our attention to a seasonal trade in stocks. The strategy is very simple: each month, buy a number of stocks that performed the best in the same month a year earlier, and short the same number of stocks that performed poorest in that month a year earlier. The average annual return is more than 13% before transaction costs, and since it is market neutral, this already considerable return can be leveraged to 2 or 3 times higher. Also, since it turns over the stocks only once a month, transaction costs should not be a major problem. The strategy was developed by Profs. Steven Heston and Ronnie Sadka, and details can be found online here. Besides its simplicity, the strategy is not as affected by survivorship bias in the data set as a mean-reverting strategy, since survivorship bias would tend to lower its backtest performance by excluding very poorly performing stocks that we would short. All in all, it seems to be a market neutral strategy made for retail trading!

Economist article on quant funds

2007-10-27T10:34:00.000-04:00

The media seems to have an endless fascination with quant funds. Here is the latest article from the Economist magazine, summarizing the postmortem published by several researchers. (Hat tip, once again, to reader Mr. J. Rigg.)

The key points are as follows:

1) Quant funds are now becoming the primary market makers in many securities, which normally would provide liquidity and decrease volatility.

2) Unlike ordinary market makers, however, quant funds are highly leveraged.

3) Because of the high leverage, in the face of large losses these market-making quant funds are forced to liquidate their assets instead of buying them, thus behaving in a way opposite to ordinary market makers just when the need for liquidity is direst.

4) Thus quant funds are actually contributing to instability of the market despite their apparent market-making function.

Fortunately, when all else has gone wrong, there is alway Mr. Bernanke to count on ...

Emerging markets stocks vs. natural resource stocks

2007-10-07T09:30:00.000-04:00

Emerging market stocks have been reaching new highs almost everyday (see this article in the Economist magazine), and the natural resource sector has been on a tear as well. Given the giddy valuations of both sectors, which one is a better relative buy at this point? For those of you who have been following the IGE-EEM spread that I proposed before, its value is at an all-time-low these days -- it was at -6.77 standard deviations. Given their historical cointegration, I wouldn't be surprised if it will revert to a more sane value in the near future.

How a mean-reversion strategy performed in August

2007-10-06T09:41:00.000-04:00

Prof. Andrew Lo and Mr. Amir Khandani at MIT recently wrote a paper on "What Happened To The Quants In August 2007?" (Hat tip to my reader Mr. J. Rigg for the article). Most of their conclusions confirm what many observers already suspected: that the loss is likely due to the simultaneous forced liquidation of portfolios holding similar positions by various quantitative funds. What is noteworthy, however, is that they constructed a mean-reversion strategy and observed what happened to it during August. This strategy is very simple: buy the stocks with the worst previous 1-day returns, and short the ones with the best previous 1-day returns. Despite its utter simplicity, this strategy has had great performance since 1995, ignoring transaction costs. The Sharpe ratio was an astounding 53.87 in 1995, gradually decreasing to 4.47 in 2006. However, the strategy also had a disastrous few days on August 7-9, suffering a cumulative (arithmetic) return of -6.85% in those 3 days. Then on August 10, it rebounded, like the rest of the quant funds, with a return of 5.92%, almost reversing all of its previous losses. For me, this experiment reveals three interesting points: 1) a simple price factor seems to capture most of the performance of the complex factor models run by the gigantic hedge funds; 2) even technical mean-reverting factors suffer losses, not just momentum (growth) factors based on fundamentals; and 3) if one wants to avoid disasters and enjoy spectacular returns, even a one-day holding period is too long. I haven't done the experiment myself yet, but I bet that if we were to liquidate the portfolio at market close each day, not only would we avoid the loss of -6.85% in those 3 days, but would probably end up with a positive return of a similar magnitude!

So how much did quantitative strategies actually lose last quarter?

2007-09-20T08:04:00.000-04:00

The numbers have started to come in: Morgan Stanley lost $480MM last quarter due to quantitative trading -- about 10% of operating profits.

Hedge fund replication

2007-09-19T14:07:00.001-04:00

I wrote about how hedge fund returns can be replicated with simple factor models. I just learn that IndexIQ, a company in Rye Brook, NY, has just launched such products available to retail investors as managed accounts.