My new book on Algorithmic Trading is out

A reader (Hat tip: Ken) told me that my new book Algorithmic Trading: Winning Strategies and Their Rationale is now available for purchase at Amazon.com. The difference with my previous book? A lot more sample strategies with an emphasis on their "rationale", and more advanced techniques. It covers stocks, futures, and FX. A big thank-you to my editors, reviewers, and you, the reader, for your on-going support.

And when you are done with it, please post a review on Amazon whether you like it or hate it!

Also, I am now offering a live online course on Backtesting in June. It covers in excruciating details the various nuances of conducting a correct backtest and the numerous pitfalls one can encounter when backtesting different types of strategies and asset classes. For syllabus and registration details, please visit my website.

Nonlinear Trading Strategies

I have long been partial to linear strategies due to their simplicity and relative immunity to overfitting. They can be used quite easily to profit from mean-reversion. However, there is a serious problem: they are quite fragile, i.e. vulnerable to tail risks. As we move from mean-reverting strategies to momentum strategies, we immediately introduce a nonlinearity (stop losses), but simultaneously remove certain tail risks (except during times when markets are closed). But if we want to enjoy anti-fragility and are going to introduce nonlinearities anyway, we might as well go full-monty, and consider options strategies. (It is no surprise that Taleb was an options trader.)

It is easy to see that options strategies are nonlinear, since options payoff curves (value of an option as function of underlying stock price) are plainly nonlinear. I personally have resisted trading them because they all seem so complicated, and I abhor complexities. But recently a reader recommended a little book to me: Jeff Augen's "Day Trading Options" where the Black-Scholes equation (and indeed any equation) is mercifully absent from the entire treatise. At the same time, it is suffused with qualitative ideas. Among the juicy bits:

1) We can find distortions in the 2D implied volatility surface (implied volatility as z-axis, expiration months as x, and strike prices as y) which may mean revert to "smoothness", hence presenting arbitrage opportunities. These distortions are present for both stock and stock index options.

2) Options are underpriced intraday and overpriced overnight: hence it is often a good idea to buy them at the market open and sell them at market close (except on some special days! See 4 below.). In fact, there are certain days of the week where this distortion is the most drastic and thus favorable to this strategy.

3) Certain cash instruments have unusually high kurtosis, but their corresponding option prices consistently underprice such tail risks. Thus structures such as strangles or backspreads can often be profitable without incurring any left tail risks.

4) If there is a long weekend before expiration day (e.g. Easter weekend),  the time decay of the options value over 3 days is compressed into an intraday decline on the last trading day before the weekend.

Now, as quantitative traders, we have no need to take his word on any of these assertions. So, onward to backtesting!

(For those who may be stymied by the lack of affordable historical intraday options data, I recommend Nanex.net.)

===

There are still 2 slots available in my online Mean Reversion Strategies workshop in May. The workshop will be conducted live via Adobe Connect, and is limited to a total of 4 participants. Part of the workshop will focus on how to avoid getting hurt when a pair or a portfolio of instruments stop cointegrating.

An Integrated Development Environment for High Frequency Strategies

I have come across many software platforms that allow traders to first specify and backtest a strategy and then, with the push of a button, turn the backtest strategy into a live trading program that can automatically submit orders to their favorite broker. (See all my articles on this topic here.)  I called these platforms "Integrated Development Environment" (IDE) in my new book, and they range from the familiar and retail-oriented (e.g. MetaTrader, NinjaTrader, TradeStation), to the professional but skills-demanding (e.g. ActiveQuant, Marketcetera, TradeLink),  and finally to the comprehensive and industrial-strength (e.g. Deltix, Progress Apama, QuantHouse, RTD Tango). Some of these require no programming skills at all, allowing you to construct strategies by dragging-and-dropping, others use some simple scripting languages like Python, and yet others demand full-blown programming abilities in Java, C#, or C++. But which of these allow us to backtest and execute high frequency strategies?

To state the obvious: backtesting HFstrategies is quite hard. The volume of data is one issue. But in addition, the execution details are very important to such strategies: details such as the exact exchange/venue to which we are routing our orders, the precise state of the order book that triggers our orders, the order types we are using, and finally the probability of getting filled if we use non-marketable orders. Messing up one of these details and the backtest will be far from realistic. I often tell people that it is easier to paper trade a HF strategy than to backtest one. While many of the platforms I reported above do allow backtesting using tick data, I don't know that they enable backtesting using the full order book and choice of execution venue. With this background, I am happy to report I have recently come across just such a platform called Lime Strategy Studio.

First, the bad news. LimeTrader is useful only to traders who trade with Lime Brokerage, as it is configured to send live orders to Lime only. [UPDATE: I have since learned that there are adapters available for 3rd party brokers.] However, if you are going to trade HF stocks and futures strategies, why not go with Lime, since they provide you with a comprehensive API, direct ultra-low latency feeds from the exchanges, and allow (nay, insist on) colocation either at the exchanges or at their data center at a reasonable fee? (Full Disclosure: I have no current business relationship with Lime, though I was a customer.) Another piece of bad news: the specification of the strategy must be in C++.

But once you get over these two hurdles, the benefits are manifold. Every detail that you can specify for a live trading strategy can be specified for the backtest and paper trading. As I said, these details may include order type, trading venue, state of order book, and even statistics of the order book, not to mention fundamental data such as earnings, corporate actions, and other user-provided data such as news. A fill simulator is included for your non-marketable orders. As with other IDEs, once you backtested a strategy in its every detail and are satisfied with the performance metrics, you can go live (either for paper or production trading) with the push of a button.

If any reader know of other IDEs that have similar features and useful for backtesting HF strategies, please let us know!

===

Speaking of HF strategies, traders often lament the ultra-high secrecy around them and the difficulty of gathering knowledge in this field. A friend (hat tip: Dave) referred me to this paper by Prof. Dragos Bozdog et. al. that gives a flavor of what sort of modeling may be involved. I find it very readable and thought-provoking.

===

There are still 2 slots available in my online Mean Reversion Strategies workshop scheduled for May.

What Can Quant Traders Learn from Taleb's "Antifragile"?

It can seem a bit ironic that we should be discussing Nassim Taleb's best-seller "Antifragile" here, since most algorithmic trading strategies involve predictions and won't be met with approval from Taleb. Predictions, as Taleb would say, are "fragile" -- they are prone to various biases (e.g. data snooping bias) and the occasional Black Swan event will wipe out the small cumulative profits from many correct bets. Nevertheless, underneath the heap of diatribes against various luminaries ranging from Robert Merton to Paul Krugman, we can find a few gems. Let me start from the obvious to the subtle:

1) Momentum strategies are more antifragile than mean-reversion strategies.

Taleb didn't say that, but that's the first thought that came to my mind. As I argued in many places, mean reverting strategies have natural profit caps (exit when price has reverted to mean) but no natural stop losses (we should buy more of something if it gets cheaper), so it is very much subject to left tail risk, but cannot take advantage of the unexpected good fortune of the right tail. Very fragile indeed! On the contrary, momentum strategies have natural stop losses (exit when momentum reverses) and no natural profit caps (keep same position as long as momentum persists). Generally, very antifragile! Except: what if during a trading halt (due to the daily overnight gap, or circuit breakers), we can't exit a momentum position in time? Well, you can always buy an option to simulate a stop loss. Taleb would certainly approve of that.

2) High frequency strategies are more antifragile than low frequency strategies.

Taleb also didn't say that, and it has nothing to do with whether it is easier to predict short-term vs. long-term returns. Since HF strategies allow us to accumulate profits much faster than low frequency ones, we need not apply any leverage. So even when we are unlucky enough to be holding a position of the wrong sign when a Black Swan hits, the damage will be small compared to the cumulative profits. So while HF strategies do not exactly benefit from right tail risk, they are at least robust with respect to left tail risk.

3) Parameter estimation errors and vulnerability to them should be explicitly incorporated in a backtest performance measurement.

Suppose your trading model has a few parameters which you estimated/optimized using some historical data set. Based on these optimized parameters, you compute the Sharpe ratio of your model on this same data. No doubt this Sharpe ratio will be very good, due to the in-sample optimization. If you apply this model with those optimized the parameters on out-of-sample data, you would probably get a worse Sharpe ratio which is more predictive. But why stop at just two data sets? We can find N different data sets of the same size, calculate the optimized parameters on each of them, but compute the Sharpe ratios over the N-1 out-of-sample data sets. Finally, you can average over all these Sharpe ratios. If your trading model is fragile, you will find that this Sharpe ratio is quite low. But more important than Sharpe ratios, you should compute the maximum drawdown based on each set of parameters, and also the maximum of all these max drawdowns. If your trading model is fragile, this maximum of maximum drawdowns is likely to be quite scary.

The scheme I described above is called cross-validation and is well-known before Taleb, though his book reminds me of its importance.

4) Notwithstanding 3) above, a true estimate of the max drawdown is impossible because it depends on the estimate of the probability of rare events. As Taleb mentioned, even in case of a normal distribution, if the "true" standard deviation is higher than your estimate by a mere 5%, the probability of a 6-sigma event will be increased by 5 times over your estimate! So really the only way to ensure that our maximum drawdown will not exceed a certain  limit is through Constant Proportion Portfolio Insurance: trading risky assets with Kelly-leverage in a limited liability company, putting money that you never want to lose in a FDIC-insured bank, with regular withdrawals from the LLC to the bank (but not the other way around).

5) Correlations are impossible to estimate/predict. The only thing we can do is to short at +1 and buy at -1.

Taleb hates Markowitz portfolio optimization, and one of the reasons is that it relies on estimates of covariances of asset returns. As he said, a pair of assets that may have -0.2 correlation over a long period can have +0.8 correlation over another long period. This is especially true in times of financial stress. I quite agree on this point: I believe that manually assigning correlations with values of  +/-0.75, +/-0.5, +/-0.25, 0 to entries of the correlation matrix based on "intuition" (fundamental knowledge) can generate as good out-of-sample performance as any meticulously estimated numbers.The more fascinating question is whether there is indeed mean-reversion of correlations. And if so, what instruments can we use to profit from it? Perhaps this article will help.

6) Backtest can only be used to reject a strategy, not to predict its success.

This echoes the point made by commenter Michael Harris in a previous article. Since historical data will never be long enough to capture all the possible Black Swan events that can occur in the future, we can never know if a strategy will fail miserably. However, if a strategy already failed in a backtest, we can be pretty sure that it will fail again in the future.

===

The online "Quantitative Momentum Strategies” workshop that I mentioned in the previous article is now fully booked. Based on popular demand, I will offer a "Mean Reversion Strategies" workshop in May. Once again, it will be conducted in real-time through Skype, and the number of attendees will be similarly limited to 4. See here for more information.

A workshop, a webinar, and a question

There is a workshop on the 25th of February titled "Market turbulence; monetization; and universality" by Mike Lipkin at Columbia University that promises to be interesting to those traders who have a physics background. Mike is a former colleague of mine at Cornell's Laboratory of Atomic and Solid State Physics, and I fondly remember the good old days when we all hunched over the theory group's computers while day-dreaming of our future. Mike has since gone on to become an options market-maker at the American Stock Exchange and an Adjunct Associate Professor at Columbia. He published some very interesting research on the "stock pinning" phenomenon near options expirations, i.e. stock prices often converge to the nearest strike prices of their options just before expirations.

---

If we want to trade directly on various FX ECNs such as HotspotFX or EBS, perhaps because we want to run some HFT strategies, we will need to be sponsored by a prime broker. However, since the Dodd-Frank act has been in full force, no prime brokers that I know of are willing to take on customers with less than $10M assets. (I often feel that the CFTC's primary goal is to prevent small players like myself from ever competing with bigger institutions. Of course, their stated goal is to "protect" us from financial harm ....) The only exception may be CitiFX TradeStream ECN. Has any reader ever traded on this market? Any reviews or comments will be most welcome.

---

I am now offering an online workshop "Quantitative Momentum Strategies” to a select number of traders and portfolio managers. It will be conducted in real-time through Skype, and the number of attendees will be limited to 4. See here for more information.

A stock factor based on option volatility smirk

A reader pointed out an interesting paper that suggests using option volatility smirk as a factor to rank stocks. Volatility smirk is the difference between the implied volatilities of the OTM put option and the ATM call option. (Of course, there are numerous OTM and ATM put and call options. You can refer to the original paper for a precise definition.) The idea is that informed traders (i.e. those traders who have a superior ability in predicting the next earnings numbers for the stock) will predominately buy OTM puts when they think the future earnings reports will be bad, thus driving up the price of those puts and their corresponding implied volatilities relative to the more liquid ATM calls. If we use this volatility smirk as a factor to rank stocks, we can form a long portfolio consisting of stocks in the bottom quintile, and a short portfolio with stocks in the top quintile. If we update this long-short portfolio weekly with the latest volatility smirk numbers, it is reported that we will enjoy an annualized excess return of 9.2%.

As a standalone factor, this 9.2% return may not seem terribly exciting, especially since transaction costs have not been accounted for. However, the beauty of factor models is that you can combine an arbitrary number of factors, and though each factor may be weak, the combined model could be highly predictive. A search of the keyword "factor" on my blog will reveal that I have talked about many different factors applicable to different asset classes in the past. For stocks in particular, there is a short term factor as simple as the previous 1-day return that worked wonders. Joel Greenblatt's famous "Little Book that Beats the Market" used 2 factors to rank stocks (return-on-capital and earnings yield) and generated an APR of 30.8%.

The question, however, is how we should combine all these different factors. Some factor model aficionados will no doubt propose a linear regression fit, with future return as the dependent variable and all these factors as independent variables. However, my experience with this method has been unrelentingly poor: I have witnessed millions of dollars lost by various banks and funds using this method. In fact, I think the only sensible way to combine them is to simply add them together with equal weights. That is, if you have 10 factors, simply form 10 long-short portfolios each based on one factor, and combine these portfolios with equal capital. As Daniel Kahneman said, "Formulas that assign equal weights to all the predictors are often superior, because they are not affected by accidents of sampling".

The Pseudo-science of Hypothesis Testing

Backtesting trading strategies necessarily involves a very limited amount of historical data. For example, I seldom test strategies with data older than 2007. Gathering longer history may not improve predictive accuracy since the market structure may have changed substantially. Given such scant data, it is reasonable to question whether the good backtest results (e.g. a high annualized return R) we may have obtained is just due to luck. Many academic researchers try to address this issue by running their published strategies through  standard statistical hypothesis testing.

You know the drill: the researchers first come up with a supposedly excellent strategy. In a display of false modesty, they then suggest that perhaps a null hypothesis can produce the same good return R. The null hypothesis may be constructed by running the original strategy through some random simulated historical data, or by randomizing the trade entry dates. The researchers then proceed to show that such random constructions are highly unlikely to generate a return equal to or better than R. Thus the null hypothesis is rejected, and thereby impressing you that the strategy is somehow sound.

As statistical practitioners in fields outside of finance will tell you, this whole procedure is quite meaningless and often misleading.

The probabilistic syllogism of hypothesis testing has the same structure as the following simple example (devised by Jeff Gill in his paper "The Insignificance of Null Hypothesis Significance Testing"):

1) If a person is an American then it is highly unlikely she is a member of Congress.

2) The person is a member of Congress.

3) Therefore it is highly unlikely she is an American.

The absurdity of hypothesis testing should be clear. In mathematical terms, the probability we are really interested in is the conditional probability that the null hypothesis is true given an observed high return R: P(H0|R). But instead, the hypothesis test merely gives us the conditional probability of a return R given that the null hypothesis is true: P(R|H0). These two conditional probabilities are seldom equal.

But even if we can somehow compute P(H0|R), it is still of very little use, since there are an infinite number of potential H0. Just because you have knocked down one particular straw man doesn't say much about your original strategy.

If hypothesis testing is both meaningless and misleading, why do financial researchers continue to peddle it? Mainly because this is de rigueur to get published. But it does serve one useful purpose for our own private trading research. Even though a rejection of the null hypothesis in no way shows that the strategy is sound, a failure to reject the null hypothesis will be far more interesting.

(For other references on criticism of hypothesis testing, read Nate Silver's bestseller "The Signal and The Noise". Silver is of course the statistician who correctly predicted the winner of all 50 states + D.C. in the 2012 US presidential election. The book is highly relevant to anyone who makes a living predicting the future. In particular, it tells the story of one Bob Voulgaris who makes $1-4M per annum betting on NBA outcomes. It makes me wonder whether I should quit making bets on financial markets and move on to sports.)

The Importance of 2 (as Sharpe Ratio)

A reader ezbentley recently pointed out a little-noticed fact in the derivation of Kelly's formula: if we apply the optimal Kelly leverage, then the standard deviation of the annualized compounded growth rate of your equity is none other than the Sharpe ratio (Sdev=S). This fact is of mild interest in itself, but its implication has relevance to another interesting fact of behavioral finance, so I will reproduce our discussions here.

Suppose our strategy has an annualized Sharpe ratio of 2. According to the above result, Sdev=2 as well. This may startle some of us: a standard deviation of 200% of our compounded growth rate g - wouldn't ruin be very likely? But check out g itself: g=S^2/2, so g=2 when S=2, which means that g itself is exactly 200%. A Sdev of 200% here means that if the growth rate drops one standard deviation below its mean, we will still manage not to lose money for the year. Another way to put this is that there is a 84.1% chance that our annual return will be greater than 0, based on the Gaussian distribution.

It gets better if S goes above 2. For example, at S=3, g=4.5, but Sdev is just 3. So you can see that as S goes above 2, a 1 standard deviation fluctuation of g below the mean will still get you a positive number: profitable for the year.

This is a very interesting result: this means that S=2 is really an important threshold in more ways that I realized. From behavioral finance experiments, we already know that humans demands $2 profits for $1 risk. Given the universal desire of portfolio managers not to lose money on the year, it turns out that the demand of a Sharpe ratio of at least 2 is quite rational!

===

Now, time for a couple of public service announcements:

1) Those who are looking for a way to connect Matlab to Interactive Brokers should check out undocumentedmatlab.com. The creator of this product has an accompanying book, and the documentation for the product is excellent.

2) NAG sells high performance Matlab toolboxes for those who prefer alternatives to the native ones.

3) Here is the Twitter feed for FIXGlobal Online, the magazine from the creator of the FIX Protocol, an order submission standard. Interesting breaking news from the global finance scene.

A leveraged ETFs strategy

In a post some years ago, I argued that leveraged ETF (especially the triple leveraged ones) are unsuitable for long-term holdings. Today, I want to present research that suggests leveraged ETF can be very suitable for short-term trading.

The research in question was just published by Prof. Pauline Shum and her collaborators at York University. Here is the simplest version of the strategy: if a stock market index has experienced a return >= 2% since the previous day's close up to the current time at 2:15pm ET, then buy this index (via its futures, ETFs, or stock components) right away, and exit at the close with a market-on-close order. Vice versa if the return is <= -2%. The annualized average return from June 2006 to July 2011 was found to be higher than 100%.

Now this strategy is actually quite well-known among institutional traders, although this is the first time I see the backtest results published. The reason why it works is also quite well-known: it has to do with the fact that every leveraged ETF need to rebalance at the market close in order to keep its leverage constant (at x2 or x3, depending on the fund). If the market index goes up, the fund needs to buy the component stocks; otherwise, it needs to sell stocks. If there is major market movement (with absolute return >= 2%) since the previous close, then the amount of stocks that need to be bought or sold will be correspondingly larger, resulting in momentum in all those stocks near the close. This strategy aims to front-run this rebalancing to take advantage of the anticipated momentum.

It has been estimated that if the market moves by 1%, the rebalancing could account for up to 16.8% of the market-on-close volume, so the induced momentum can be substantial. Now who is paying for this profits for those momentum traders? Why, the buy-and-hold investors, of course. This loss for the ETFs shows up as their tracking errors, resulting in a cost of as much as 5% per annum for the buy-and-hold investors. Yet another reason we should not be one of those investors!

As Prof. Shum pointed out, if you trade this strategy live today, you will likely get a lower return, because of all those momentum traders who drove up the price way before the close. However, there may be an ameliorating factor at work here: this momentum is proportional to the NAVof the ETFs. As their NAV goes up with time (either due to additional subscriptions or positive market returns), the returns of this strategy should also increase.

===
Now for some public service announcements:

1) A company called Level 3 Data Corp sells proprietary data indicating buying and selling pressure on stocks. Their internal backtests show that adding these data to some common stock trading strategies essentially double their returns. An explanatory video is available, and I heard they are offering 3-month free trials.

2) The London Systematic Traders (LST) Club has asked me to say a few words about their new initiative to build a London centric collaborative community of traders, developers and researchers.

LST aims to be at the intersection of traders, developers and quants with a strong emphasis community building and on knowledge exchange, providing a trading networks with a very specific focus on systematic, algorithmic (i.e. automated) or quantitative trading.

Membership is free and open to everybody with an interest in the above topics.

http://www.meetup.com/London-Systematic-Traders/

On Friday, Nov 23, I expect to be hosting a Q&A session with members of the LST (see 2 above) at the Apex Hotel in London. All are welcome. Please visit their website for details.

3) I will be conducting my Backtesting and Statistical Arbitrage workshops in London, Nov 19-22, and look forward to seeing some of our readers there!

Order flow as a predictor of return

Order flow is signed transaction volume: if an order is executed at the ask price, the incremental order flow is +(order size); if executed at the bid price, it is -(order size). In certain markets where traders can only buy and sell from market makers but not from each other, a positive order flow means that traders are net buyers of a security. But even in markets where everyone can place and fill orders on a common order book, a positive order flow indicates that informed traders (those willing to aggressively get into a position) are eagerly acquiring a security.

The neat thing about order flow is that it has proven to be a good momentum indicator. That is to say, a positive flow predicts a positive future return. This might seem trivially obvious, but you have remember that generally speaking, a positive past return by no means predicts a positive future return. That FX order flow possesses this predictive power was shown by Evans and Lyons in a series of papers, but this indicator is useful in many other markets, and at many different time scales. For example, in a paper by Coval and Stafford, it was shown that if you can tease out the order flow of a stock due to mutual funds' trading alone, you can also predict its future return up to, say, a quarter. This paper not only shows that order flow is predictive, but that sometimes a specific kind of order flow (in this case, that of mutual funds only) is sometimes more predictive than general order flow. In many cases, traders find that by counting only order flow due to institutional traders, or order flow due to large orders, they can better predict future returns. (No wonder institutional traders are trying their darnedest to break up their orders into small chunks, or to trade in dark pools!) I recently also heard that order flow into sector ETFs can be predictive of that sector's return. If any reader has read papers or has experience with this type of sector rotation model, please leave a comment!

Despite the proven usefulness of order flow, not too many retail traders utilize it. The reason is simple: it can be hard to measure. In FX in particular, many markets do not report trade information, or they report with a sufficient delay such that the information has no predictive utility. Even for markets that report instantaneous trade information, you would need a good piece of software to capture every bid, ask, trade, and trade size, and store them in an array, in order to compute order flow, an operation that most retail trading software cannot accomplish. However, this barrier to entry may just mean that there are still decent alpha to be extracted from this indicator.

Now, a bunch of public service announcements ...

===

A new algorithmic trading platform called Rizm designed for retail traders is now available. You can sign up for their beta trial here.

===

Quantopian has created an event-driven version of my gold/gold-miners arb strategy with source codes and analysis available. I find that the performance metrics clear and useful: better than the output from my own backtest programs! (Quantopian is a platform where you can share backtest results and codes with other traders.)

===

Arbmaker is a platform for pair traders, and it incorporates software for cointegration tests, has integrated data feed from many vendors, and allows automated order submission to Interactive Brokers. Neural networks and Kalman filter are also included.

===

Finally, I will be giving a talk titled "Backtesting and Its Pitfalls" at the World MoneyShow at the Metro Toronto Convention Centre on Saturday, October 20. Interested readers can register here.

An options workshop and other miscellany

I confess I have always found it hard to trade options. This is despite having read some of the "bibles" of options trading, including Lawrence McMillan's Options as a Strategic Investment and Euan Sinclair's Option Trading: Pricing and Volatility Strategies and Techniques. Partly that is because I prefer simple strategies, and options strategies are rarely simple. Partly that is because I was brought up on stocks, but stock options are depressingly illiquid. Most successful options traders that I know of prefer to trade index options instead, an area that I unfortunately have no intuition at all. Papers and books written by options professionals on this topic tend to be dense with equations, and worse, they seldom focus on the practical side of trading.

That's why I am pleased to learn that Larry Connors, whose books I enjoy due to their simplicity of exposition, is presenting his first ever quantitative index options trading seminars. Interested traders can register for his free preview webinars on August 9 and 15 here, or a pre-recorded preview here.

===

Speaking of seminars, readers in Asia may be interested to know that my own workshops on Backtesting and Statistical Arbitrage will be held in Hong Kong on October 2-5. The same workshops will be held in London on November 19-22.

(I enjoy giving those workshops very much, because many of the participants are institutional traders whose knowledge and points of view are very much at the cutting edge. Past participants include quants and traders from, in no particular order, Goldman Sachs, Morgan Stanley, Royal Bank of Scotland, Bank of America, UBS, Societe Generale, Deutsche Bank, BNP Paribas, JP Morgan, Barclays, Citigroup, Blackrock, and various other Asian and European hedge funds, energy companies, banks, and asset managers.  I humbly submit that the in-class discussions are sometimes more interesting than my prepared materials.)

===

I wrote some time ago about those FX brokers or ECNs where algo-traders can colocate their trading programs to lower latency for a reasonable price. There are also similar options for futures algo-traders. For e.g. Optimus Trading Group provides a market data service called Rithmic which is colocated at the major futures exchanges, and traders can colocate with Rithmic to reduce latency. Of course, traders can also directly colocate at the new CME data center in Aurora, IL. I suspect, though, that the cost of the latter option will be considerable.

====

Finally, as a quant trader, I nevertheless read macroeconomic analyses occasionally, if only to figure out why some of my strategies suddenly start to fail. One website that provides interesting analysis of the energy markets is oilprice.com. In particularly, this interview with economic commentator Mike Shedlock is unusually detailed and thoughtful.

Extracting roll returns from futures

Futures returns consist of two components: the returns of the spot price and the "roll returns". This is kind of obvious if you think about it: suppose the spot price remains constant in time (and therefore has zero return). Futures with different maturities will still have different prices at any point in time, and yet they must all converge to the same spot price at expirations, which means they must have non-zero returns during their lifetimes. This roll return is in action every day, not just during the rollover to the next nearest contract. For some futures, the magnitude of this roll return can be very large: it averages about -50% annualized for VX, the volatility futures. Wouldn't it be nice if we can somehow extract this return?

In theory, extracting this return should be easy: if a future is in backwardation (positive roll return), just buy the future and short the underlying asset, and vice versa if it is in contango. Unfortunately, shorting, or even buying, an underlying asset is not easy. Except for precious metals, most commodity ETFs that hold "commodities" actually hold only their futures (e.g. USO, UNG, ...), so they are of no help at all in this arbitrage strategy. Meanwhile, it is also a bit inconvenient for us to go out and buy a few oil tankers ourselves.

But in arbitrage trading, we often do not need an exact arbitrage relationship: a statistical likely relationship is good enough. So instead of using a commodity ETF as a hedge against the future, we can use a commodity-producer ETF. For example, instead of using USO as a hedge, we can use XLE, the energy sector ETF that holds energy producing companies. These ETFs should have a higher degree of correlation with the spot price than do the futures, and therefore very suitable as hedges. In cases where the futures do not track commodities (as in the case of VX), however, we have to look harder to find the proper hedge.

Which brings me to this fresh-off-the-press paper by David Simon and Jim Campasano. (Hat tip: Simon T.) This paper suggests a trading strategy that tries to extract the very juicy roll returns of VX. The hedge they suggest is -- you guessed it! -- the ES future. In a nutshell: if VX is in contango (which is most of the time), just short both VX and ES, and vice versa if VX is in backwardation. 


Why does ES work as a good hedge? Of course, its very negative correlation with VX is the major factor. But one should not overlook the fact that ES also has a very small roll return (about +1.5% annualized). In other words, if you want to find a future to act as a hedge, look for ones that have an insignificant roll return. (Of course, if we can find a future that has high correlation with your original future but which has a high roll return of the opposite sign, that would be ideal. But we are seldom that lucky.)


P.S. The reader Simon who referred me to this paper also drew my attention to an apparent contradiction between its conclusion and my earlier blog post: Shorting the VIX Calendar Spread. This paper says that it is profitable to short VX when it is in contango and hedge with short ES, while I said it may not be profitable to short the front contract of VX when it is in contango and hedge with long back contract of VX. Both statements are true: hedging with the back contract of VX brings very little benefit because both the front and back contracts are suffering from very similar roll returns, so there is little return left when you take opposite positions in them!

Momentum strategies: a book review

As a devout mean-reversion trader, I find Mike Dever's new book "Jackass Investing" unexpectedly well-argued and readable.

You see, momentum and mean-reversion traders live in two separate universes, and they are often mutually incomprehensible to each other. Dever, as a CTA, inhabits the momentum universe. Example: my favorite performance measure, the Sharpe ratio, has been brusquely dispatched as a bad measurement of risk, and drawdown becomes king. But all for good reasons: Dever argues that Sharpe ratio measures only the daily volatility of returns, but disregarded the "black swan" events, which are much better captured by the maximum drawdown. I agree with the author on this point, but there are other uses of Sharpe ratio: a high Sharpe ratio strategy does indicate high statistical significance of the trading strategy, a claim that momentum strategies can seldom make. I often think of momentum strategies as being long options: you have to keep paying premium until one day, you make them all back with a home run. But when you are backtesting a strategy, how would you know that the rare, statistically insignificant, home run was not due to data snooping bias? Unless of course, like the author, you have fundamental insights into the traded instruments.

Fundamental insights are in fact one of the delicious highlights of this book. Dever describes his orange juice futures strategy using the "marginal cost of production" as a fundamental valuation tool. He argues that orange juice cannot be sold below this cost, since farmers would have no incentive for production otherwise. And he was right: orange juice futures started to rebound from the 27-year low of 55 cents/pound in May 2004, to almost 90 cents/pound in September (thanks partly to hurricanes hitting Florida). Dever went long at 70 cents. Oh, how we quantitative traders would love to have the confidence that such insights inspire!

Of course, I don't agree with everything written in the book. For example, though the author rightly pointed out that the distribution of returns often have a positive kurtosis, he uses that as evidence of trending behavior. While I agree that price trends can indeed produce positive kurtosis, we can certainly construct mean-reverting price series with occasional catastrophes that have the same kurtosis. To us mean-reversion traders, positive kurtosis is not an invitation to "follow-the-trend", but as a warning sign to find risk management measures that protect us from catastrophes. 

Even though momentum strategies in general are in a state of trauma right now (more on that later), Dever nevertheless makes a good case why we should include them as part of our portfolio of strategies. Comparing the S&P500 index (SPX) with the S&P Diversifed Trends Indicator (DTI, a simple trend-following strategies on 24 futures), he finds that the Sharpe ratio (though of course he refuses to use that hated term) of the DTI is more than double that of the SPX, with only about 1/3 of the maximum drawdown. But before you, the reader, decides to join the momentum bandwagon, I invite you to take a look at a plot of DTI's values since inception:

S&P DTI index

Since its high watermark in 2008/12/5, this representative momentum strategy has been in a relentless drawdown. Why? This is due to another well-studied and troubling property of momentum strategies: they always performed poorly for several years after a financial crisis.

A few announcements

First, an iPad version of this blog has been launched, so if you are reading this on an iPad, the look will be different. If you want to go back to the old look, just hit Page Turn in the bottom left corner and choose the option there. Any comments or suggestions on this new look are most welcome!

Second, and this is probably irrelevant to most of you reading this blog, a Chinese translation of my book Quantitative Trading is now available.

Third, and most interesting, Larry Connors will be hosting a webinar on "How to Trade High Probability Stock Gaps" on Tuesday, May 1, 2:00pm ET. (Click on link to register.) It is sheer coincidence that I was just writing about stock gaps in my previous post! I have always found Larry's strategies to be clear, concise, and simple - exactly the ingredients for out-of-sample as opposed to in-sample returns!

The life and death of a strategy

Sometimes it is instructive to look back at some strategies that used to thrive, and then quite suddenly contracted a chronic illness that ultimately led to its demise. It gives us a sense of the unreliability of backtests and curb our over-confidence, which is always useful when dealing with the financial markets.

One good example is a well-known strategy that we called "buy-on-gap". In its simplest version, just buy at the open 100 stocks within the S&P500 which have the lowest returns from their previous day's lows to the current day's open, provided that these returns are lower than one standard deviation. (The standard deviation is computed as the 90-day moving standard deviation of close-to-close returns of a stock.) Exit such  long positions at the day's close.

Many traders know of variants of this strategy, and I started trading it around the beginning of 2007, and in fact, it formed part of my first fund's portfolio of strategies. You can see the cumulative return chart below (click to enlarge) from 2007/01/03-2008/10/29. The APR is 19%, unlevered. The Sharpe ratio is 1.4 and the maximum drawdown is just 4%. Note that Lehman Brothers went bankrupt on 2008/09/15, and this is a long-only strategy, yet the performance was spectacular in September-October 2008. We were patting ourselves furiously on the back.

Now look what happened after this happy period.

 The APR was -6%. 2008/10/29 turned out to be the high watermark.

I have seen some strategies that have the opposite behavior: poor performance prior to 2009, and stellar performance since then. Was there a structural break in the market due to the financial crisis? Was this due to the advent of high frequency trading? The declining volume in the equities market? I will leave these deep questions to financial economists. The only lesson I have learned from this and other examples is that, once a strategy is in decline for some time, it seldom comes back to health, and the best course of action is to bury it swiftly.

High-frequency trading in the foreign exchange market

This is the title of a report published by the Bank of International Settlements (which serves central banks around the world) in September 2011. As a Forex trader myself, I of course peruse it with great interest hoping to glimpse whatever is the state-of-the-art. Here are a few interesting nuggets, together with my commentary:

1) FX HFT operate with a latency of less than 1 ms, while most of us mere algorithmic traders typically suffer a latency of at least 10ms.  For example, Interactive Brokers does not yet provide collocation facilities for its customers, so the best we can do is to place our trading servers on the internet backbone close to its Stamford, CT, location. The best round-trip ping time is 10ms. Those who trade with FXCM may have a better chance for lower latency, as they provide free collocation to their clients. Those who trade on the ECN FXall can collocate at their Equinix data center, while FCM360 provides collocation service to EBS traders. I cannot find any collocation service for Hotspot FX or Currenex. If you know of such services, or FX brokers who provide collocation, do leave a comment!

2) HFT typically operate in markets with high liquidity and low volatility. The former is not surprising, since markets with low liquidity has few counter-parties to take advantage of. The latter requires a bit of nuance. I think most HFT would benefit from high volatility in a mean-reverting market, but unfortunately high volatility is usually correlated with market in a free fall. So don't be surprised if you find that HFT-provided liquidity suddenly disappears when the market is in stress, though the BIS report stated that they are also quick to re-enter the market once the turmoil is over.

3) As a corollary of 2), HFT mostly trade in the major currency pairs. But increasingly, NZD and MXN have drawn many automated and HF traders.

4) Almost by definition, the bid/ask quotes placed by HFT tend to remain on the book for a very short time, measured in ms, unless forced by the exchange to stay longer. EBS and Reuters both has minimum quote life or minimum fill ratio. One exchange that does not have such minimums is Currenex, which is therefore particularly attractive to HF trading. Hence if you are not a HF player, and do not wish to be taken advantage of  by a HF player, be wary of Currenex!

5) Two of the favourite categories of HFT strategies: triangle arbitrage and liquidity-redistribution (taking advantage of pricing discrepancies across different trading platforms.) Despite the bad reputation HFTers have been acquiring in the last few years, I think they do provide a useful service to other algo traders like myself via these 2 strategies. It is a hassle to keep looking for a better broker/prices for your strategy!

Hidden Markov model applied to FX prediction

I read with interest an older paper "Can Markov Switching Models Predict Excess Foreign Exchange Returns?" by Dueker and Neely of the Federal Reserve Bank of St. Louis. I have a fondness for hidden Markov models because of its great success in speech recognition applications, but I confess that I have never been able to create a HMM model that outperforms simple technical indicators. I blame that both on my own lack of creativity as well as the fact that HMM tend to have too many parameters that need to be fitted to historical data, which makes it vulnerable to data snooping bias. Hence I approached this paper with the great hope that experts can teach me how to apply HMM properly to finance.

The objective of the model is simple: to predict the excess return of an exchange rate over an 8-day period. (Excess return in this context is measured by the % change in the exchange rate minus the interest rate differential  between the base and quote currencies of the currency pair.) If the expected excess return is higher than a threshold (called "filter" in the paper), then go long. If it is lower than another threshold, go short. Even though the prediction is on a 8-day return, the trading decision is made daily.

The excess return is assumed to have a 3-parameter student-t distribution. The 3 parameters are the mean, the degree of freedom, and the scale. The scale parameter (which controls the variance) can switch between a high and low value based on a Markov model. The degree of freedom (which controls the kurtosis, a.k.a. "thickness of the tails") can also switch between 2 values based on another Markov model. The mean is linearly dependent on the values assumed by the degree of freedom and the scale as well as another Markov variable that switches between 2 values. Hence the mean can assume 8 distinct values. The 3 Markov models are independent. The student-t distribution is more appropriate for the modelling financial returns than normal distribution because of the allowance for heavy tails. The authors also believe that this model captures the switch between periods of high and low volatility, with the consequent change of preference (=different mean returns) for "safe" versus "risky" currencies, a phenomenon well-demonstrated in the period between August 2011 to January 2012.

The parameters of the Markov models and the student-t distributions are estimated in the in-sample period (1974-1981) for each currency pair in order to minimize the cumulative deviation of the excess returns from zero. There are a total of 14 parameters to be so estimated. After these estimations, we have to also estimate the 2 trading thresholds by maximizing the in-sample return of the trading strategy, assuming a transaction costs of 10 basis point per trade.

With this large number (16 in total) of parameters, I dread to see the out-of-sample (1982-2005) results. Amazing, these are far better than I expected: the annualized returns range from 1.1% to 7.5% for 4 major currency pairs. The Sharpe ratios are not as impressive: they range from 0.11 to 0.71. Of course, when  researchers report out-of-sample results, one should take that with a grain of salt. If the out-of-sample results weren't good, they wouldn't be reporting them, and they would have kept changing the underlying model until good "out-of-sample" results are obtained! So it is really up to us to implement this model, apply it to data after 2005 and to more currency pairs, to find out if there is really something here. In fact, this is the reason why I prefer to read older papers - to allow for the possibility of true out-of-sample tests immediately.

What do you think can be done to improve this model? I suspect that as a first step, one can see whether the estimated Markov states correspond reasonably to what traders think of as risk-on vs risk-off regimes. If they do, then regardless of the usage of this model as a signal generator, it can at least generate good risk indicators. If not, then maybe the hidden Markov model need to be replaced with a Markov model that is conditioned on observable indicators.

Ideas from a psychologist

I have just finished reading Daniel Kahneman's bestseller "Thinking, Fast and Slow", and found it full of inspirations important for traders. This is no surprise, of course, since Kahneman won the 2002 Nobel prize in economics for his work on decision theory. Here are some of the notables:

1) Simple sum is often better than a linear regression fit. Remember my constant mantra that "simpler is better" when building trading models? I have always advocated linear regression over nonlinear models, but Kahneman went a step further. He said that in social science modeling (which of course includes financial markets modeling), assigning equal weights to the predictive factors is often superior to weighting them using multivariate linear regressors when applied to out-of-sample data.

2) Overconfidence in corporate acquisitions. Managers of acquiring companies often believe that they are better than the managers of acquirees. This overconfidence has several causes: there is an illusion of control which overemphasizes the role of skill and neglects the role of luck, and there is a focus on what one knows and a neglect of what one does not, etc. The market already knows this: the stock of the acquirer usually suffers a sell-off upon announcement of the acquisition, because the result of any acquisition is more often bad than good, but the question is whether it has sufficiently discounted this phenomenon. Would shorting the stock of an acquirer at the completion of an acquisition and holding the short position for, say, 5 years, hedging this position with SPY, be profitable?

3) Premortem. After designing a trading strategy, it is always useful to write a brief imaginary history of how it has become an unmitigated financial disaster for you a few years from now. This will likely reveal scenarios that you have not previously thought of, and triggering additional risk management measures.

4) Risk seeking in the face of losses. Suppose you are running a strategy that has a fixed holding period. Have you ever extended this holding period when the position is losing, in the hope that the position will recoup some of its losses? I have, and the result was double the loss I would have suffered had I exited on time. Apparently this is a very common suboptimal behavioral bias: this is why many defendants with a weak legal case often risk continued litigations instead of accepting an unfavorable settlement.

5) Why do we often demand Sharpe ratio  >=2? Psychological experiments have shown that people find the pain of losing $1 can only be compensated by the pleasure of winning $2. So if we equate standard deviation as the average drawdown of a strategy, then we need to have twice the average return!

Many businesses have profited from arbitraging the difference between rational decisions and biased decisions that people commonly made. (For e.g. lottery franchises benefit from people overweighting the probability of winning, sellers of extended warranties benefit from buyers' risk-aversion.) I wonder if there are still opportunities left for rational traders to take advantage of the biased decisions of irrational traders?

What worked in 2011?

We all know that 2011 was a bad year for many hedge funds, with the average fund down 5%. But what type of strategies did well, and what did particularly poorly? The numbers are out: Forex funds lose more than average, down 6%. In fact, 71 out of 77 Forex funds tracked by a Citigroup currency analyst were down in 2011. And the winners are? Statarb funds, with a 5% averge return.

This superior performance of statarb funds is quite a contrast from the last financial crisis 2007-9. Then, most of the big factor-driven statarb models failed miserably. What caused this difference? Is it because the risk management techniques of big funds have improved? Or maybe that's because in 2011, the deviation from factor returns mean-revert within a few days, so those statarb models that re-balance on a daily basis can benefit from the buying/selling opportunity at steep discount/premium?

To settle this question, let me report the 2011 backtest results (without transaction costs) of running Andrew Lo's prototype mean-reversion model : ranking stocks based on their previous day's returns, shorting the top decile and buying the bottom one, rebalancing only at the close. (Click on chart to make it larger.)

The APR in 2011 was 18.6%. Note in particular its performance since the crisis began officially on 20110808: despite a steep drawdown, the overall performance was spectacular! Clearly, high volatility benefited a prototypical statarb strategy, and the out-performance has not much to do with improved risk management.

You might wonder what would happen if we had used the intraday version of this strategy instead: enter all positions at the open, and exit them all at the close? I tried it: the performance is surprisingly similar to the interday strategy. So intraday vs. interday volatility or mean-reversion does not seem to play a part in last year's equities market. Contrasting this with the performance of Forex models, it is clear that high volatilities benefited statarb models while they hurt FX models.

In the next article or two, I will explore the 2011 performance of some other equities mean-reverting models that I used to trade. But what about your models? If you have some thoughts on what worked and what didn't in 2011, please share them with us in the comments section.

Risk indicators

During the financial crisis of 2008, I wrote about how I watched some risk indicators such as the VIX or the TED spread  to decide what leverage I should use for my trading strategies. It turns out that this procedure is just as critical for the current crisis that began in August 2011. In fact, more than leverage-determinants, they can be used as the all-important variable that determines whether a certain strategy should be run at all. (What's the point of running a model that you think will lose money with low leverage?)


There are now more than a few of these risk indicators to pick from. Besides the VIX and the TED, there are the VSTOXX (EURO STOXX 50 Volatility), the VXY (JPMorgan G7 Volatility Index), the EM-VXY (JPMorgan Emerging Market Volatility Index), the ETF's ONN and OFF, and probably many more that I haven't heard of yet. 


A lot of academic research has been done on whether we can devise "regime switching" models based on some complicated pattern-recognition algorithms to decide whether a market is in a certain "regime" which favors this or that particular model or parameter set. And often, these regime switching models rely on the recognition of some complicated set of patterns in the historical price series. Sorry to say, I have not found any of these complex regime switching model to have any real out-of-sample predictive power. On the other hand, my research shows that some of the aforementioned simple risk indicators will indeed prevent some trading models from falling off the cliff.


But which of these indicators are applicable to which model? This is not so obvious. For example, you might think that the EM-VXY would be an ideal leading indicator for Forex trading models that involve emerging market currencies, but I have found that it is only a contemporaneous (and thus useless) indicator to mine. Another example, I said during the 2008 financial crisis that VIX seems to be a useless contemporaneous indicator for equities trading models, but strangely, it is a good leading indicator for FX models. In contrast, the TED spread that everyone were obsessed about in 2008 shot up to over 300 bps then, but never went beyond 100 bps this time around. So really only rigorous backtesting can guide us here.


What risk indicators do you use? And have you really backtested their efficacies? Your comments would be very welcome here.