The Quants

Once in a while, a book about trading written for the general public contains some useful nuggets even for professionals.  Fortune's Formula was one. It introduced me to the world of Kelly's formula, Universal Portfolios, and the maximization of compounded growth rate. The Quants, by WSJ reporter Scott Patterson, is another. (Hat tip to my partner Steve for telling me about it.)

What is the most important take-away in The Quants? No, it is not that you should learn to become a master poker player or chess player before hoping to make it big, though you would think that given Patterson's exhaustive coverage of poker games played by the top quants. Among my own professional acquaintances, trader-poker-players are still a minority.

The most important take-away is what ex-employees said about Renaissance Technologies: "there is no secret formula for the fund's success, no magic code discovered decades ago by geniuses .... Rather, Madallion [Fund]'s team of ninety or so Ph.D.'s are constantly working to improve the fund's systems, ..."

In other words, though you may not have 90 Ph.D.'s  at your disposal, you can still work on continuously improving/refining your strategies, improving the engineering of your trading environment, and increasing the diversity of your strategies. And though you may still not archive 60-70% annualized returns every year, you will nevertheless enjoy stable returns year after year.

By the way, it is good to see my ex-colleagues Lalit Bahl, Vincent and Stephen Della Pietra mentioned in the book, all of whom left IBM to join Renaissance many years ago, and who are extraordinarily nice and friendly guys, quite in contrast to the norm on Wall Street.

A HFT primer

As a follow-up of my previous discussions on high frequency trading, I have invited guest blogger Jennifer Groton to share with us a quick survey of various common HFT strategies used by equities and FX traders.

==

High frequency trading strategies are under fire.  The recent trading spike in our national exchanges was duly noted as a short-circuit waiting to happen and drew immediate industry criticism of auto-trading robots. Before a witch-hunt ensues, perhaps a review of the common HFT strategies in stocks and Forex is in order. 

High-frequency firms employ a wide variety of  low-margin trading strategies that are implemented by professional market intermediaries who have invested heavily in technology. These firms claim that they make markets more efficient by enhancing liquidity and transparent price discovery to the benefit of investors.  The Forex market’s unique combination of high liquidity and low volatility make it an ideal environment for deploying HFT strategies, although many of the ideas and technology are from the equity markets.  The basic strategies fall into three categories: market-making, trending or predictive, and classic arbitrage.

Market-making strategies tend to focus on a single stock or currency pair.  Many firms in this area have been described as engaging in "rebate-capture trading", a reference to the credits that firms get for providing liquidity on most market centers.

The second group consists of mean-reversion and trending strategies. These utilize technical indicators for stocks or forex indicators for currencies, and seek to generate more return from individual trades.

The last group may involve a cross-section of trades from multiple markets.  The classic arbitrage strategy is a form of the “carry trade” that uses the prices of a domestic bond, a bond denominated in a foreign currency, the spot price of the currency, and the price of a forward contract on the currency.  If the market prices are sufficiently different from those implied in the model to cover transactions costs, then four transactions can be made to guarantee a risk-free profit.

High frequency trading is attributed with generating over 70% of the volume of trades on our equity markets.  Similar statistics are not available for forex markets, but speculating disguised as commercially necessary trades have been reported to be over two-thirds of the volume.  Liquidity and pricing transparency are the benefits offered by its advocates, but regulators and other market participants who disagree with this positive assessment are presently discounting these benefits.  Transaction taxes and time limits on orders have been proposed to mitigate the perceived risk created by HFT firms, but the wheels of Washington move slowly, even in crisis.  For the time being, there is no indication that their participation will be discontinued.   

Are flash orders to be blamed for Dow's 1,000 points drop?

Before the smoke is clear, fingers are already pointing at flash orders. See these two NYT pieces here and here. Our reader Madan has convinced me previously that flash orders can indeed be used to  front-run other traders, but until more evidence comes in, I am yet to be convinced that they are the main culprit. Couldn't old-fashioned automated momentum programs accomplished the same thing after an initial erroneous transaction price and/or quote was reported? Perhaps you know of discussions elsewhere on the blogosphere that bring more light to the issue?

An additional ETF pair

Many of you know that there are a number of dependable commodity-related ETF pairs that remain cointegrated ever since I mentioned them in 2006: IGE-EWC, IGE-EEM, IGE-EWA, EWA-EWC, etc. (Their latest zScores are available here to my book's readers and to Premium Content subscribers.) A recent visit to a client in South Africa prompted me to add a new one: EWA-EZA.

It is worth noting that for those country ETF pairs that cointegrate, their underlying currency cross-rates are often stationary as well. Now, there are several advantages in trading currency cross rates instead of ETF pairs. When trading a stationary cross rate, you can enter a limit order to enter and exit, but trading pairs of ETF's involve market orders on at least one side. Also, ETF's can sometimes be hard-to-borrow, and their margin requirements are much more onerous than that of currencies. However, the one major disadvantage in trading cross rates is that they are not always available on your brokerage. For example, based on the cointegration of EWA and EZA you would think that trading AUDZAR would be quite profitable. And you would be right, theoretically, except that AUDZAR is not available for trading on Interactive Brokers. If you know of a good Forex brokerage that have many emerging markets cross-rates for trading, especially those of Latin American countries, please let the rest of us know!

How do you limit drawdown using Kelly formula?

As many of you know, I am a fan of Kelly formula because it allows us to maximize long-term growth of equity while minimizing the probability of ruin. However, what Kelly formula wont' prevent is a deep drawdown, though we are assured that the drawdown won't be as much as 100%! This is unsatisfactory to many traders and especially fund managers, since a deep drawdown is psychologically painful and may cause you to panic and shut down a strategy prematurely.

There is an easy way, though, that you can use Kelly formula to limit your drawdown to be much less than 100%. Suppose the optimal Kelly leverage of your strategy is determined to be K. And suppose you only allow a maximum drawdown (measured from the high watermark, as usual) to be D%. Then you can simply set aside D% of your initial total account equity for trading, and apply a leverage of K to this sub-account to determine your portfolio market value. The other 1-D% of the account will be sitting in cash. You can then be assured that you won't lose all of the equity of this sub-account, or equivalently, you won't suffer a drawdown of more than D% in your total account. If your trading strategy is profitable and the total account equity reaches a new high watermark, then you can reset your sub-account equity so that it is again D% of the total equity, moving some cash back to the "cash" account. Otherwise, you continue to keep the equity in the cash account separate from the equity of the trading sub-account.

Notice that because of this separation of accounts, this scheme is not equivalent to just using a leverage of L=K*D% on your total account equity. Indeed, some of you may be too nervous to use the full K as leverage, and prefer to use a leverage L smaller than K. (In fact, the common wisdom is that, due to estimation errors, it is never advisable to set L to be more than K/2, i.e. half-Kelly.) The problem with using a L that is too small is that, besides not achieving maximum growth, the portfolio market value will be unresponsive to gains or losses and will remain relatively constant. Using the scheme I suggested above will cure this problem as well, because you can apply a higher leverage L_sub to the sub-account (e.g. use L_sub = L/D%) as long as L_sub < K, so that the portfolio market value is much more sensitive to your P&L while still ensuring the drawdown will not exceed D%.

Has anyone tried this scheme in their actual trading? If so, I would be interested in hearing your experience and see if practice is as good as theory.

Conference on the sociology of quantitative finance

A new conference called Psi-Q will be held in London this June, featuring luminaries in the academic quantitative finance world, as well as risk and fund managers from various banks and hedge funds. Example topics:

• How did shared beliefs, practices, ways of calculating, and technical systems impact evaluation of asset-backed securities and CDOs before and during the credit crises?
• Was that Lucky or Good? Creating a framework for skill attribution in finance, business management and other risky endeavors.
• The “backing out” phenomena observed in options markets:  how traders use models to imply independent variables consistent with market observed pricing, and where enough traders can be wrong about the expected results and the backed-out positions can send the wrong message.

Sounds like an interesting bird's eye view of quantitative finance.

Pairs Trading Workshop in Hong Kong

For my readers in Asia, I will be conducting a pairs trading workshop in Hong Kong on March 10-11. This workshop is organized by the Technical Analyst magazine and is similar to the one I gave in London last year.
However, I have added a few useful insights based on audience feedback. As always, no prior knowledge of Matlab or advanced statistics is assumed. The numerous in-class exercises should be sufficient to bring your Matlab programming skills up to speed.

A method for optimizing parameters

Most trading systems have a number of parameters embedded, parameters such as the lookback period, the entry and exit thresholds, and so on. Readers of my blog (for e.g., here and here) and my book would know my opinion on parameter optimization: I am no big fan of it. This is because I believe financial time series is too non-stationary to allow one to say what was optimal in the backtest is necessarily optimal in the future. Most traders I know would rather trade a strategy that is insensitive to small changes in parameters, or alternatively, a "parameterless" strategy that is effectively an average of models with different parameters.

That being said, if you can only trade one model with one specific set of parameters, it is rational to ask how one can pick the best (optimal) set of parameters. Many trading models have a good number of parameters, and it is quite onerous to find the optimal values of all these parameters simultaneously. Recently, Ron Schoenberg published an article in the Futures Magazine that details a way to accomplish this with just a tiny amount of computer power.

The key technique that Ron uses is cubic polynomial fit of the P&L surface as a function of the parameters. Ron uses the VIX RSI strategy in Larry Connors' book "Short Term Trading Strategies That Work" as an example. This strategy has 5 parameters to be optimized, but Ron only needs to compute the P&L for 62 different sets of parameters, and the whole procedure only takes 58 seconds.

Although Ron has confirmed that most of the parameters that Connors picked are close to optimal, he did find a few surprises: namely, that RSI of period 3 or 4 is significantly more profitable than the 2 that Connors used, at least in the backtest period.

Now, for a true test of this optimization, it would be helpful if Ron performed this optimization withholding some out-of-sample data, and see if these parameters are still optimal in that withheld data set. Since he didn't do that, we need to wait for another year to find out ourselves!

Excel ADF test

Some readers have asked whether there is an Excel version of the ADF test for cointegration (mentioned in articles here or here.) You can download one such package here (Hat tip: Bruce H.).

And as always, you can download the Matlab version from spatial-econometrics.com.

Does Averaging-In Work?

Ron Schoenberg and Al Corwin recently did some interesting research on the trading technique of "averaging-in". For e.g.:  Let's say you have $4 to invest. If a future's price recently drops to $2, though you expect it to eventually revert to $3. Should you

A) buy 1 contract at $2, and wait for the price to possibly drop to $1 and then buy 2 more contracts (i.e. averaging-in); or
B) buy 2 contracts at $2 each;  or
C) wait to possibly buy 4 contracts at $1 each?

Let's assume that the probability of the price dropping to $1 once you have reached $2 is p. It is easy to see that the average profits of the 3 options are the following:
A) p*(1*$1+2*$2) + (1-p)*(1*$1)=1+4p;
B) 2; and
C) p4*$2=8p.

Profit A is lower than C when p > 1/4, and profit A is lower than profit C when p > 1/4. Hence, whatever p is,  either option B or C is more profitable than averaging in, and thus averaging-in can never be optimal.

From a backtest point of view, the Schoenberg-Corwin argument is impeccable, since we know what p is for the historical period. You might argue, however, that financial markets is not quite stationary, and in my example, if the historical value of p was less than 1/4, it is quite possible that the future value can be more than 1/4. This is why I never make too much effort to optimize parameters in general, and I can sympathize with traders who insist on averaging-in even in the face of this solid piece of research!

Selecting tradeable pairs: which measure to use?

A guest blog by Paul Farrington

One of the most important factors in statistical arbitrage pairs trading is the selection of the paired instruments.  We can use basic heuristics to guide us, such as grouping stocks by industry in the anticipation that stocks with similar fundamental characteristics will share factor risk and tend to exhibit co-movement.  But this still leaves us with potentially thousands of combinations.  There are some statistical techniques we can use to quantify the tradeability of a pair: one approach is to calculate the correlation coefficient of each pair's return series. Another is to consider cointegration measures on the ratio of the prices, to see if it remains stationary over time.

In this article I briefly summarise the alternative approaches and apply them to a universe of stock pairs in the oil and gas industry.  To measure how effective each measure is in real world trading, I back test the pairs using a simple means reversion system, then regress the generated win rate against the statistical results.  Some basic insights emerge as to the effectiveness of correlation and cointegration as tools for selecting candidate pairs.

Please visit http://www.paulfarrington.com/research/Selecting%20tradeable%20pairs.htm for details of my methodology and results.

Public service announcements for quants

   1.  Conference on 'Computational Topics in Finance', February 19/20, 2010, National University of Singapore. The topics will include using R/Rmetrics in finance, but the conference is by no means confined to R. See http://www.rmetrics.org/.

   2.  Consulting position (6-month renewable contract) available at a major Canadian bank in Toronto:  research in  various mathematical algorithms used for pricing of interest rate derivative instruments like swaps, caps, swaptions, FRAs. Please contact their recruiter at http://www.linkedin.com/pub/kevin-p-w-wang/6/899/29a.

   3.  A free copy of Chapter 8 of "High Probability ETF Trading" which I mentioned here is now available for download.

Are financial speculations really "harmful human activities"?

It is worrisome when not one but two eminent economists denounced financial speculation as "harmful human activities" in the short space of 2 weeks. (See Paul Krugman's column here and Robert Frank's here.) It is more worrisome when their proposed cure to this evil is to apply a financial transaction tax to all financial transactions.

Granted, you can always find this or that situation when financial speculation did cause harm. Maybe speculation did cause the housing bubble. Maybe speculation did cause an energy price bubble. In the same vein, you can also argue that driving is a harmful human activity because cars did cause a few horrific traffic accidents.

No, we can't focus on a few catastrophes if we were to argue that financial speculation is harmful. We have to focus on whether it is harmful on average. And on this point, I haven't seen our eminent economists present any scientific evidence. On the other hand, as an ex-physicist and an Einstein-devotee, I can imagine some  thought experiments (or gedankenexperiment as Einstein would call them), where I can illustrate how the absence of financial speculation can clearly be detrimental to the interests of the much-beloved long-term investors. To make a point, a gedankenexperiment is usually constructed so that the conditions are extreme and unrealistic. So here I will assume that the financial transaction tax is so onerous that no hedge funds and other short-term traders exist anymore.

Gedankenexperiment A: Ms. Smith just received a bonus from her job and would like to buy one of her favorite stocks in her retirement account. Unfortunately, on the day she placed her order, a major mutual fund was rebalancing its portfolio and had also decided to shift assets into that stock. In the absence of hedge funds and other speculators selling or even shorting this stock, the price of that stock went up 40% from the day before. Not knowing that the cause of this spike was a temporary liquidity squeeze, and afraid that she would have to pay even more in the future, Ms. Smith paid the ask price and bought the stock that day. A week later, the stock price fell 45% from the peak after the mutual fund buying subsided. Ms. Smith was mortified.

Gedankenexperiment B: Mr. Smith decided that the stock market is much too volatile (due to the lack of speculators!) and opted to invest his savings into mutual funds instead. He took a look at his favorite mutual fund's performance, and unfortunately, its recent performance seemed to be quite a few notches below its historical average. The fund manager explained on her website that since her fund derived its superior performance from rapidly liquidating holdings in companies that announced poor earnings, the absence of liquidity in the stock market often forced her to sell into an abyss. Disgusted, Mr. Smith opted to keep his savings in his savings account.

Of course, our economists will say that the tax is not so onerous that it will deprive the market of all speculators (only the bad ones!?). But has anyone studied if we impose 1 unit of tax, how many units of liquidity in the marketplace will be drained, and in turn, how many additional units of transaction costs (which include implicit costs due to the increased volatility of securities) would be borne by an average investor, who may not have the luxury of submitting a limit order and waiting for the order to be filled?

Picking up nickels in front of steamrollers

When I was growing up in the trading world, high Sharpe ratio was the holy grail. People kept forgetting the possibility of "black swan" events, only recently popularized by Nassim Taleb, which can wipe out years of steady gains in one disastrous stroke. (For a fascinating interview of Taleb by the famous Malcolm Gladwell, see this old New Yorker article. It includes a contrast with Victor Niederhoffer's trading style, plus a rare close-up view of the painful daily operations of Taleb's hedge fund.)

Now, however, the pendulum seems to have swung a little too far in the other direction. Whenever I mention a high Sharpe-ratio strategy to some experienced investor, I am often confronted with dark musings of "picking up nickels in front of steamrollers", as if all high Sharpe-ratio strategies consist of shorting out-of-the-money call options.

But many high Sharpe-ratio strategies are not akin to shorting out-of-the-money calls. My favorite example is that of short-term mean-reverting strategies. These strategies not only provide consistent small gains under normal market conditions, but in contrast to shorting calls, they make out-size gains especially when disasters struck. Indeed, they give us the best of both worlds. (Proof? Just backtest any short-term mean-reverting strategies over 2008 data.) How can that be?

There are multiple reasons why short-term mean-reverting strategies have such delightful properties:

1. Typically, we enter into positions only after the disaster has struck, not before.
   
2. If you believe a certain market is mean-reverting, and your strategy buy low and sell high, then of course you will make much more money when the market is abnormally depressed.
3. Even in the rare occasion when the market does not mean-revert after a disaster, the market is unlikely to go down much further during the short time period when we are holding the position.

"Short-term" is indeed the key to the success of these strategies. In contrast to the LTCM debacle, where they would keep piling on to a losing position day after day hoping it would mean-revert some day, short-term traders liquidate their positions at the end of a fixed time period, whether they win or lose. This greatly limits the possibility of ruin and leaves our equity intact to fight another day in the statistical game.

So, call me old-fashioned, but I still love high Sharpe-ratio strategies.

In praise of ETF's

I have learned some years ago that ETF's are strange and wonderful creatures. Simple, long-only mean-reverting strategies that work very well on ETF's, won't work on their component stocks. (Check out a nice collection of these strategies in Larry Connors' book "High Probability ETF Trading". He has also packaged these strategies into a single indicator, the ETF Power Ratings, on tradingmarkets.com.) Simple pair trading strategies like the one I discussed in my book, also work much more poorly on stocks than on ETF's. Why is that?

Well, one obvious reason is that, as Larry mentioned in his book, ETF's are not likely to go bankrupt (with the notable exception of the triple-leveraged ETF's, as I explained previously), because a whole sector or country is not likely to go bankrupt. So you can pretty much count on mean-reversion if you are on the long side.

Another obvious reason is that though there are news which will affect the valuation of a whole sector or country, these aren't as frequent or as devastating as news affecting individual stocks. And believe me, news is the biggest enemy of mean-reversion.

But finally, I believe that the capital weightings of the component stocks also play a part in promoting mean-reversion. Typically, weighting of a component stock increases with its market capitalization, though not necessarily linearly. Perhaps large-cap stocks are more prone to mean-reversion than small-cap stocks? But more intriguingly, can we not construct a basket of stocks, with custom-designed weightings, with the objective of optimizing its short-term mean-reversion property? I (and others before me) have done something similar in constructing a basket of stocks that cointegrate best with an index. Can we not construct a basket that is simply stationary (with perhaps a constant drift)?

Now, perhaps you will agree with me that ETF's are strange and wonderful creatures.

The best environment for quantitative trading

Let me talk about a topic that is far more mundane than the usual high-brow theoretical discussions of strategies and algorithms, but that has no less long-term impact on the bottom line: what is the best office environment for research and execution of quantitative trading strategies?

I have worked in different office environments before, so I feel qualified to offer an informed opinion.

At Morgan Stanley, I huddled over a desk that is semi-partitioned from the rest of the office: nobody could see or bother me unless I or they stood up. At Credit Suisse, I shared an office with 2 other prop trading colleagues, one of whom was prone to freely sharing his opinion on various current affairs with his officemates. (On the other hand, he complained my biting an apple for lunch was too loud for him.) At Maple, a hedge fund in New Jersey, I shared an office with about 100 other colleagues on the trading floor, many of whom were prone to same opinion-sharing temptation.

Here at my own firm, I sit in solitude (except for my cat) in my basement office, my beloved classical FM streaming over the internet, my desktop electronically connected to my partner in our Chicago office, our trading servers at Amazon and elsewhere, and other clients and partners around the world, but unmolested throughout the day unless I voluntarily pick up the phone or answer an email or instant message.

Can you guess which environment is the one I find the most productive? Which one has the least stress? And which one contributes most to the bottom line of my employers/partners/clients?

(Hint 1: read Timothy Ferriss' book The 4-Hour Workweek. This guy checks his email only once a week.)

(Hint 2: my trading Sharpe ratio went from negative to >7.)

P.S. I look forward to meeting some of you at the Automated Trading Conference in London this Friday (my talk will start at 0900), and others at my pairs trading workshop on the preceding two days.

Are flash orders really so bad?

I confess I don't know much about flash orders, not being one of the Big Boys on the Street, until I read that the SEC is banning them. (For a clear diagrammatic explanation of flash orders, see here. For a refutation of some of the myths and misunderstanding surrounding flash orders, see here.)

It seems to me that flash orders can be understood as "request for liquidity" issued to various potential market makers/liquidity providers, not unlike the usual "request for quotes" (RFQ) common in other industries. They are issued when there is not enough liquidity on a specific exchange to satisfy an investor's need, and they ultimately benefit investors by lowering their transaction costs. The fact that high frequency traders are able to make lots of money by providing this liquidity is besides the point. Liquidity providers are supposed to make money by providing liquidity!

Some people, including Senator Charles Schumer and this New York Times op-ed, believe that flash orders are akin to front-running, a clearly illegal trading activity. But they are wrong. Front-running means that if you know someone is going buy a stock, you step in front of them
and buy it cheaply first, hoping to sell it to this slower buyer at a higher price. In the case of flash orders, the high frequency traders are instead selling this stock to the original investor, often at a lower price than available elsewhere and thus benefiting this investor, hoping that the prices will come down in the future after this liquidity need subsides. This is manifestly not illegal. This is what a market is built for!

Another way to understand that flash orders are not at all front running is that anybody, including you and me, are free to put in limit orders at the same price as those of the high frequency traders, way ahead of time, in a specific exchange, and become liquidity providers ourselves. You don't have to wait for a "request for liquidity" before doing so. And presumably you will reap the same benefits as the high frequency traders. You are not taking any additional risks over the HF traders either, since if no requests for liquidity ultimately arrive, you are not any worse off for wear. You cannot begrudge the profits of the HF traders just because you didn't put the limit orders in place beforehand!

Maybe there are some other angles which I miss which can convince me that flash orders are evil. But until my kind readers convince me otherwise in the comments section, I will regard this piece of legislation as another SEC attempt at demagoguery.

Can a trader be a do-gooder?

It occurs to me that the only way in which a trader can become more than a completely selfish, self-enriching, narcissistic person is to trade well enough so that you can manage other people's money and thus saving these investors from crooks and charlatans (provided you are convinced you are not a crook and charlatan yourself).

Other traders have advanced other arguments in favor of trading. But I am not convinced by them.

They say that we provide liquidity to other long-term investors who may need to liquidate their investments. But then, this applies only to mean-reversal strategies. Momentum strategies take away liquidity from the market, and in some cases exacerbating price bubbles. Certainly not something your grandma would approve.

Others argue that momentum strategies help disseminate information about companies through quick price movements. But can't we just watch Bloomberg or CNBC? Do we really need some devious insiders to convey that information to the rest of us through price movements?

No, I think that independent trading should serve only one purpose (besides short-term self-sustenance): as training and preparation to become a fund manager. Once you graduated from independent trading, you then enter into the grand contest among all fund managers to see who can best serve and protect investors' assets, (and be rewarded according to your standing in this contest.)

I know, this is the idealistic way to look at things. Serving and protecting seem to be what policemen should be doing, not traders. But as in quantitative trading, I think it helps one becomes more successful in one's activities by having a simple guiding principle or model. And it doesn't hurt that in this case, the principle would also be conscience-nourishing!

Have you traded 10,000 hours yet?

Author Malcolm Gladwell, in his fascinating bestseller "Outliers: The Story of Success", cites neurological research showing that "10,000 hours of practice is required to achieve the level of mastery associated with being a world-class expert." This seems to apply across many different types of experts, whether they are "writers, ice skaters, concert pianists, chess players ... Even Mozart ... couldn't hit his stride until he had his ten thousand hours in".

Reflecting on my own experience, I have become consistently profitable only after 4 years of actual trading (research alone doesn't count -- real money need to be at risk.) So while the number of hours may not be exactly 10,000, the order of magnitude is about right.

So if your trading has not been profitable, ask yourself this: "Have I traded 10,000 hours yet?"

Using R to Test for Cointegration

Paul Teetor, who guest-blogged here about seasonal spreads, recently wrote an article about how to test for cointegration using R. Readers who don't want to pay for a copy of Matlab should find this free alternative with similar syntax quite interesting.