Friday, September 05, 2014

Moving Average Crossover = Triangle Filter on 1-Period Returns

Many traders who use technical analysis favor the Moving Average Crossover as a momentum indicator. They compute the short-term minus the long-term moving averages of prices, and go long if this indicator just turns positive, or go short if it turns negative. This seems intuitive enough. What isn't obvious, however, is that MA Crossover is nothing more than an estimate of the recent average compound return.

But just when you might be tempted to ditch this indicator in favor of the average compound return, it can be shown that the MA Crossover is also a triangle filter on the 1-period returns. (A triangle filter in signal processing is a set of weights imposed on a time series that increases linearly with time up to some point, and then decreases linearly with time up to the present time. See the diagram at the end of this article.) Why is this interpretation interesting? That's because it leads us to consider other, more sophisticated filters (such as the least square, Kalman, or wavelet filters) as possible momentum indicators. In collaboration with my former workshop participant Alex W. who was inspired by this paper by Bruder et. al., we present the derivations below.

===

First, note that we will compute the moving average of log prices y, not raw prices. There is of course no loss or gain in information going from prices to log prices, but it will make our analysis possible. (The exact time of the crossover, though, will depend on whether we use prices or log prices.) If we write MA(t, n1) to denote the moving average of n1 log prices ending at time t, then the moving average crossover is MA(t, n1)-MA(t, n2), assuming n1< n2.  By definition,

MA(t, n1)=(y(t)+y(t-1)+...+y(t-n1+1))/n1
MA(t, n2)=(y(t)+y(t-1)+...+y(t-n1+1)+y(t-n1)+...+y(t-n2+1)/n2

MA(t, n1)-MA(t, n2)
=[(n2-n1)/(n1*n2)] *[y(t)+y(t-1)+...+y(t-n1+1)] - (1/n2)*[y(t-n1)+...+y(t-n2+1)]    
=[(n2-n1)/n2] *MA(t, n1)-[(n2-n1)/n2]*MA(t-n1, n2-n1)
=[(n2-n1)/n2]*[MA(t, n1)-MA(t-n1, n2-n1)]

If we interpret MA(t, n1) as an approximation of the log price at the midpoint (n1-1)/2 of the time interval [t-n1+1, t], and MA(t-n1, n2-n1) as an approximation of the log price at the midpoint (n2-n1-1)/2 of the time interval [t-n1, t-(n2-n1)], then [MA(t, n1)-MA(t-n1, n2-n1)] is an approximation of the total return over a time period of n2/2. If we write this total return as an average compound growth rate r multiplied by the period n2/2, we get

MA(t, n1)-MA(t, n2)  ≈ [(n2-n1)/n2]*(n2/2)*r

r ≈ [2/(n2-n1)]*[MA(t, n1)-MA(t, n2)]

as shown in Equation 4 of the paper cited above. (Note the roles of n1 and n2 are reversed in that paper.)

===

Next, we will show why the MA crossover is also a triangle filter on 1-period returns. Simplifying notation by fixing t to be 0,

MA(t=0, n1)
=(y(0)+y(-1)+...+y(-n1+1))/n1
=(1/n1)*[(y(0)-y(-1))+2(y(-1)-y(-2))+...+n1*(y(-n1+1)-y(-n1))]+y(-n1)

Writing the returns from t-1 to t as R(t), this becomes

MA(t=0, n1)=(1/n1)*[R(0)+2*R(-1)+...+n1*R(-n1+1)]+y(-n1)

Similarly,

MA(t=0, n2)=(1/n2)*[R(0)+2*R(-1)+...+n2*R(-n2+1)]+y(-n2)

So MA(0, n1)-MA(0, n2)
=(1/n1-1/n2)*[R(0)+2*R(-1)+...+n1*R(-n1+1)]
 -(1/n2)*[(n1+1)*R(-n1)+(n1+2)*R(-n1-1)+...+n2*R(-n2+1)]
+y(-n1)-y(-n2)

Note that the last line above is just the total cumulative return from -n2 to -n1, which can be written as

y(-n1)-y(-n2)=R(-n1)+R(-n1-1)+...+R(-n2+1)

Hence we can absorb that into the expression prior to that

MA(0, n1)-MA(0, n2)
=(1/n1-1/n2)*[R(0)+2*R(-1)+...+n1*R(-n1+1)]
 -(1/n2)*[(n1+1-n2)*R(-n1)+(n1+2-n2)*R(-n1-1)+...+(-1)*R(-n2+2)]
=(1/n1-1/n2)*[R(0)+2*R(-1)+...+n1*R(-n1+1)]
 +(1/n2)*[(n2-n1-1)*R(-n1)+(n2-n1-2)*R(-n1-1)+...+R(-n2+2)]

We can see the coefficients of R's from t=-n2+2 to -n1 form the left side of an triangle with positive slope, and those from  t=-n1+1 to 0 form the right side of the triangle with negative slope. The plot (click to enlarge) below shows the coefficients as a function of time, with n2=10, n1=7, and current time as t=0. The right-most point is the weight for R(0): the return from t=-1 to 0.


Q.E.D. Now I hope you are ready to move on to a wavelet filter!

P.S. It is wonderful to be able to check the correctness of messy algebra like those above with a simple Matlab program!

===
New Service Announcement

Our firm QTS Capital Management has recently launched a FX Managed Accounts program. It uses one of the mean-reverting strategies we have been trading successfully in our fund for the last three years, and is still going strong despite the low volatility in the markets. The benefits of a managed account are that clients retain full ownership and control of their funds at all times, and they can decide what level of leverage they are comfortable with. Unlike certain offshore FX operators, QTS is a CPO/CTA regulated by the National Futures Association and the Commodity Futures Trading Commission.

===
Workshops Update

Readers may be interested in my next workshop series to be held in London, November 3-7. Please follow the link at the bottom of this page for information.

===
Follow me on Twitter: @chanep

145 comments:

Anonymous said...

Hi Ernie,

I wonder if moving average crossover strategy can still make profits in short-term time frame. I thought it is
at least a mid-term or even long-term trend following strategy.

Thanks.

Ernie Chan said...

That may well be the case: you can optimize n1 and n2 and find out what values work presently.

Ernie

Anonymous said...

Hi Ernie,

May I ask where we can get historical fundamental data for companies, such as
EBITDA, P/E ratio?

Ernie Chan said...

Quandl.com has free historical fundamental data. Quality is not assured.

Compustat sells them for a fee. Quality is industrial strength.

Ernie

Anonymous said...

Hi Erine

I would like to know how much is the fee that you charge on the FX managed Acct, How's the average yearly return of the past few years?
Investor must open an account with IB?

Thanks

Ernie Chan said...

Incentive fee of 20% subject to high watermark, annual management fee of 1%.

The annual returns are listed in the YTD column of the Monthly % Returns table on epchan.com/accounts. Please note all returns are calculated net of fees.

Ernie

Ernie Chan said...

IB account is not necessary, but would certainly simplify things.


Ernie

Anonymous said...

Hi Ernie,

What is high watermark?

Ernie Chan said...

High watermark means the highest cumulative return since inception up to the previous month's end. No incentive fee will be charged unless the cumulative return up to the current month's end is higher than the high watermark.

Ernie

Anonymous said...

Hi Erine

Which currency pairs that you FX fund is trading for this intraday mean revision? Any hard stoploss?

What is the minimum funding required for subscription?

Thx

Ernie Chan said...

This strategy trades G7 currencies plus the most liquid EM currency.

There is a hard stoploss of 15%.

Please email me to discuss minimum investment, as it depends on various factors.

Ernie

Anonymous said...

Hi Ernie,

To deal with TAQ data, shall we focus on primary exchange or all exchanges?

What is the difference?

Ernie Chan said...

It depends on what order types you are using, which broker you are using, and where you/they are routing your orders. If you are sending MOO, LOO, MOC, or LOC, then you have to use primary exchange data only.

Ernie

Anonymous said...

Hi Ernie,

In IB, the round trip transaction cost per share for US stocks is $0.01.

For US stock pairs trading in backtestiung, how much do we need to use for slippage besides tc?

Ernie Chan said...

Slippage depends on your order size, and the specific stock's bid-ask spread. In general, I assume 5bps all-in one-way tcost for the average SP500 stock.

Ernie

Anonymous said...

Hi Ernie,

My order size is small, less than 200 shares per trade, but I may trade small cap stocks. (300k daily volume)

I use Bid/Ask bars to compute P&L, and we know commissions in IB per share is $0.005. Therefore, I only need to estimate slippage. How about $0.01 one way for slippage?
Thanks.

Ernie Chan said...

The top of book size is typically just 100 shares. So yes, slippage is at least $0.01 per side, or $0.02 round trip, even if you use quotes for backtesting.

Ernie

Anonymous said...

Hi Ernie,

I've been live paper trading (using IB's simulated trading account) a mean reversion intra day FX strategy for some time now, with promising results. Could you comment on how realistic these results really are? Thanks.

-T

Ernie Chan said...

Hi T,
I think FX paper trading in IB is pretty realistic. On the other hand, paper trading in many FX ECN's won't be, due to an effect I explained in some details in my MFT course.
Ernie

Anonymous said...

Hi Ernie,

May I ask where we can get historical FX intraday bars?

Ernie Chan said...

Historical FX data is best obtained free from your own broker, since every broker has different prices.

Otherwise ratedata.gaincapital.com has free data.

Ernie

Anonymous said...

Hi Erine

You trade mean revision of the spread of 2 different currency pairs or just trading the mean revision of an outright position of a single currency pair?

Likes GBPUSD, it has fallen steeply in recent days under the worry of the potential implications of Scottish independence. In your opinion, besides putting a hard stoploss in the trading program, what other algorithms that can help when this kind of unexpected breakdown occurs? Thx

Ernie Chan said...

We trade spreads of currency pairs.

Generally speaking, it is not a good idea to enter into a mean-reverting trade when we anticipate significant news that may affect a country's currency.

Ernie

Anonymous said...

Hi Ernie,

For US stocks, small order size (less than 200 shares), is it realistic to set slippage $0.02 per share per side, or $0.04 round trip?

Besides slippage, I already use quotes to compute P&L, and $0.01 per share for round trip commissions.
So, tc = spread + slippage + commissions.

My execution is fully automated via IB TWS.

Ernie Chan said...

For 200 shares, a slippage of $0.01 per trade is sufficient.

Ernie

Anonymous said...

Hi Ernie,

According to your book, when we trade FX, we need to close positions before 17:00 ET to avoid rollover interests?

Is this lose or gain significant?

Thanks.

Ernie Chan said...

The loss or gain is not significant if the differential interest rate is small.

Ernie

Anonymous said...

Hi Ernie,

What's a typical allocation of equity in a startup hedge fund? Suppose it's a discretionary traded quant driven fund with revenues split threeways between a portfolio manager, a seed capital provider and a quant researcher/trader.

Does 60/35/5% seem too unfair?

I appreciate any input I can get!
/the quant

Ernie Chan said...

Hi quant,
Whether 5% is fair or not depends on how much of the revenue comes from your contribution. For e.g. if they are trading a model that you created, you could argue that you should deserve more share.
Ernie

Anonymous said...

Hi Ernie,

May I ask how to do divergence stock pairs trading ( the opposite of mean reverting) ?

It seems to be easy to set entry, but how do we exit the position? Do we have to do stoploss and profit taking?

Thanks

Ernie Chan said...

Divergence stock pairs trading is a momentum strategy, often triggered by a fundamental event on one or both stocks. E.g. M&A event. As with most momentum strategies, stoploss is recommended, but profit taking is not. Exit is usually triggered by the completion of the event (e.g. closing of M&A deal), or a fixed holding period.

Ernie

Unknown said...

Hi Ernie,

I have a question about example3_6.m in your first book.

The way I read the hedgeRatio is you calculate it based on the entire trainset data. Then, this hedgeRatio value is used to backtest that trainset data. Isn't this a lookahead bias?
I meant it seems it is using hedgeRatio value that is obtained from the entire data that we are backtesting from older data to newer ones. In real trading we can't determine the hedgeRatio using future data, since it is the future and not yet available.

Or, am I being confused here ? :)

Ernie Chan said...

Hi Sonny,
You are right. As I wrote in my introduction to my second book, many of my examples has this type of subtle look-ahead bias. The reason is for simplicity, and an implicit exercise for readers to figure out these subtleties, as you just did.

Ernie

Anonymous said...

Hi Ernie,

How do we calculate Sortino ratio?

How do we set target return for computing downside deviation?

Assume we use data of 1 hour returns , and daily returns.


Ernie Chan said...

You can calculate Sortino ratio the same way as Sharpe ratio, except that you should include only negative deviations from the mean in computing the standard deviation of returns in the denominator.

Ernie

Unknown said...

Thanks Ernie!
After trying to understand your code for that example 3.6, my understanding is...

- You are using hedgeRatio calculated from the trainingset and use it for testset. This is why you seperate the data to training and test set, because I thought there are no difference in both sets, but actually the test set is tested using hedgeratio, spreadmean, and spreadStd from trainingset.

- In real world trading, how do we adjust the hedgeratio, spreadmean and spreadstd? Every day we calculate the new numbers using updated daily data from yesterday,... or ?

- Here the exit is only determined if the zscore comes back below abs(1). Therefore, the trades are guaranteed winner even if we have to wait for a long time for zscore to drop below abs(1). Is this correct analysis?
In real trading, do you apply stop loss if zscore climbs above abs(3), for example, or any other methods you use?

Thanks for all the kind helps. You are very generous for retail people.

Ernie Chan said...

Hi Sonny,
True: whenever I separate training and testing data and used the parameters obtained solely from the training set, there is no look-ahead bias in the test set.

In live trading, you can certainly using a moving training set every day to retrain your data.

Also in live trading, we use Bollinger bands to determine exits, hence the average is a moving average, and exits do not guarantee profits. There is an implicit stoploss embedded in Bollinger bands.

Ernie

Anonymous said...

Hi Ernie,
I am reading your second book right now. It is a very interesting and inspiring book.

I am having some difficulties in reproducing your results in certain examples of the book. Since your book's website only provides data files in Matlab format, which I could not read or use. In your book the data for
EWC during the periods (04/04/2006-04/09/2012) had some highs very close to 35$, but the data for EWC which I downloaded from Yahoo had only reached $32 or so. I was wondering if there is any possibility that you could provide data EWC-EWA pair in the above periods in ASCI format so that I could use, I would greatly appreciate.

Thanks

Pete

Anonymous said...

Hi Ernie,

If the order size is not big, what is the slippage to trade G7 and commodities currencies?

Thanks.

Ernie Chan said...

Hi Pete,
The difference in data is due to the dividend adjustments to the prices after my book was published. You should get very similar results for the backtests even though the prices are different.
Ernie

Ernie Chan said...

The slippage in trading currencies if your order size is small is due solely to the latency in data feed and order submission, and is entirely specific to your network and broker's system. Generally, we can assume the slippage is zero and just take into account bid-ask spread and commissions in such backtests.

Ernie

Anonymous said...

Hi Ernie,

Would you please explain what happen to stocks prices in M&A event?

Ernie Chan said...

In an M&A event, it was found that the acquirer stock price will typically go up, while the acquiree's price will go down. (This was research done by Ravenpack, contrary to popular beliefs.)
This movement will continue until closing of the transaction.

Ernie

Anonymous said...

Hi Ernie,

Could we know when M&A starts and when it ends?
Do they do disclosure as public information?

Btw, do you trade parameters with highest Sortino ratio?

Ernie Chan said...

The announcement of an M&A is a closely guarded secret, but you can enter a trade after the announcement as a momentum trade. The expected closing of a M&A is announced, but there is no guarantee that it will proceed as announced.

I typically approximately optimize our parameters based on Sharpe ratio.

Ernie

Anonymous said...

Thanks, Ernie.
The cointegration test results are pretty similar. But I could not reproduce the KF test results as your book showed. I could not understand the jump (~100%) in the cumret plot.

Thanks again.
Pete

Ernie Chan said...

Hi Pete,
Please email me if you wish to compare results in more details.
Ernie

Anonymous said...

Hi Ernie,

It seems using Kelly formula is to maximize Sharpe ratio.
May I ask if you know any similar research or formula which is used to maximize Sortino ratio?

Thanks.

Ernie Chan said...

Kelly formula is not used for maximizing Sharpe ratio. It is used for optimizing the leverage in order to maximize the long term compounded growth. No matter what leverage one uses, the Sharpe ratio of a strategy will be the same. The only way to increase the Sharpe ratio is to change your strategy.

Ernie

Anonymous said...

Hi Ernie,

When we do parameters optimization,
usually how long is the historical data we use for Daily bars, 1 hour bars and 1 min bars?

Thanks

Ernie Chan said...

The lookback used for parameter optimization is itself a parameter to be optimized!

To avoid going in circles, you can choose to optimize the lookback in a fixed in-sample period.

Ernie

Anonymous said...

Hi Ernie,

What do you mean, "to avoid going in circles"?

Ernie Chan said...

If you consider the lookback for optimizing parameters as a parameter to be optimized, then one has to consider what lookback to optimize this lookback parameter. Thus the circle.

Ernie

Anonymous said...

Hi Ernie,

For Kalman filter, as you said before,
there is no distinction between in sample and out sample data, so there is no lookback window to choose.

Therefore, we just feed KF with all data to optimize its parameter(which control speed of beta change)?

Thanks.

Ernie Chan said...

For KF, the effective lookback is always the beginning of the data to the last bar.

Ernie

Anonymous said...

Hi Ernie,

To some extent, I find Sharpe ratio and Sortino ratio are negative correlated.

High Sharpe ratio makes you trade more (smooth equity curve) while high Sortino ratio makes you trade less (more positive jumps on less smooth equity curve). Calmer ratio is more correlated to Sortino ratio.

How do you choose parameters between different ratios?

Ernie Chan said...

Since I am increasingly cognizant of abnormal risks, I tend to optimize Calmer ratio of late. But there is a problem with Calmer ratio: over what period of max drawdown should we use? Evidently, the longer the period, the bigger the max drawdown. So any period is an arbitrary number. Sharpe ratio has much more sensible mathematical properties that do not depend on lookback.

Ernie

Unknown said...

Hi Ernie,

I have a quick follow-up question from your reply above
"Also in live trading, we use Bollinger bands to determine exits, hence the average is a moving average, and exits do not guarantee profits. There is an implicit stoploss embedded in Bollinger bands.
"

About the moving average, what is usual length or is it dynamically changed?

From the backtest code pnl, I assume that even if the pairs converge from std 2 into std 1, that pair still can lose money?

last question....
Using cadf conintegration test, is it valididea if I test 1 year of data, 6 months of pair data, and 3 months of pair data and use the 3 months cointegration result as the most weight to trade the pair ?
any suggestion?

thanks

Ernie Chan said...

Hi Sonny,
The lookback length is a parameter to be optimized in-sample.

Mean reversion to a Bollinger band does not guarantee profit.

We cannot determine cointegration using 3 months. 1 year is minimum, and 3 years is optimal.

Ernie

Chengxi Yang said...

Hi Ernie,

You imply that wavelet filter is a better choice. Could you please suggest several useful mother wavelet functions?

Thanks,

Ernie Chan said...

Hi Chengxi,
I don't know for a fact that wavelet filter will be better - I merely suggested it is worth trying.

I did try it many many years ago, but it didn't produce great results. But that maybe because I was naive and foolish back then.

Ernie

Anonymous said...

Hi Ernie,

Usually, how many years of data are used to backtesting?

Ernie Chan said...

One should always start from at least 2007 in a backtest in order to see how model performs under stress.

Ernie

Anonymous said...

Hi Ernie,

As you say, from 2007, could we use "walk forward" to backtesting?

For example, 2 years in sample (optimization parameters), 1 year out sample(trading), then all move forward 1 year, repeat until today.

Collect all statistics from all 1 year out sample.

Ernie Chan said...

You can use walk-forward, but the out-of-sample period must include 2007 and onwards.

Ernie

Anonymous said...

Hi Ernie,

May I ask what are the momentum strategies for US stocks market?

Ernie Chan said...

Please read my book Algorithmic Trading, chapter 6 and 7 on momentum strategies, mostly for US markets.

Ernie

Anonymous said...

Hi Ernie,

Is it possible to do intraday trading using market depth?

Is there any books or papers we could read about it?

Ernie Chan said...

Sure, please google Hellstrom and Simonsen, 2006.

Ernie

Anonymous said...

Hi Ernie,

Thank you for the information.

It seems they report their results at the level of 1 min, 2 mins etc..

If we buy an extra fast real-time data feed, maybe we can trade market depth on Interactive Brokers? Have you traded market depth on any markets (forex, stocks, futures) on IB? Or with other brokers.

Many thanks.

Ernie Chan said...

You should be able to improve results with a fast data feed. But with IB's order handling latency at >= 10ms, it is not the ideal broker for such HF strategies.

Ernie

Anonymous said...

Hi Ernie,

How do we choose "in sample period" for stock pairs trading? 1 year, 3 years or .... where we can optimize lookback period.

Ernie Chan said...

We typically use half of all the data for in-sample training.

Ernie

Anonymous said...

Hi Ernie,

For intraday strategies, I could get higher Sharpe ratio using the last year for in-sample training while I get lower Sharpe ratio using the last 3 years.

Which parameter set shall I use for real trading?

Thanks.

Ernie Chan said...

I suggest you can use 1 year rolling in-sample data for optimization.

Ernie

Anonymous said...

Hi Ernie,

So it is like walk forward optimization?

Maybe I could use 1 year rolling in-sample data, and half-year ( or 1 year) out of sample.
So I need to do re-optimization every half-year (1-year) in real trading.

Thanks.


Ernie Chan said...

I don't know how you can use out-of-sample data for optimization. Out-of-sample is your live trading record.

Ernie

Anonymous said...

Hi Ernie,

It seems that no parameter set could last forever for a stock pairs.

May I ask when we need to re-optimize to get a new parameter set or even stop trading the stock pairs?

Thanks.

Ernie Chan said...

One should discover a fundamental or economic reason why a stock pair stops working. That way, one may be able to introduce a new variable that will repair the cointegration relationship. Re-optimizing the parameters is not the best solution.
See p. 91 of my book Algorithmic Trading.

Ernie

Anonymous said...

Hi Ernie,

Thank you for quick response.

It seems to be easier to analyse fundamental or economic reasons between ETFs as you mention in your book.

Recently, I focus on intraday stock pairs trading. I just run program to search a whole sector (such as energy sector) to get "good" stock pairs based on Sharpe ratio among thousands of combinations.

It seems it is intraday trading, and there are hundreds of stocks in a sector, and I am not very good at industry analysis.

I wonder how we could trade intraday stock pairs properly. Or we need to trade interday stock pairs and ETFs.

Thanks.

Ernie Chan said...

It is for the exact reason you mentioned that I don't trade stock pairs!

Ernie

Anonymous said...

Hi Ernie,

Usually, what is the ratio between in-sample and out of sample data length?

Ernie Chan said...

1:1

Ernie

Anonymous said...

Hi Ernie,

It seems there are 2 approaches to systems parameters:

1. Decide on a set of parameters for the strategy and stick to it.

2. Keep changing the system parameters based on the latest market conditions (chameleon approach).

Which one do you prefer? And why?

Ernie Chan said...

You can indeed continuously adapt your parameters based on a rolling lookback period, but the lookback period should be long (> 1 year), since many parameters do mean-revert. Using such a long lookback means that the parameters typically do not change much. So in practice, I just keep them fixed.

Ernie

Anonymous said...

Hi Ernie,

Have you heard about walk forward test by Robert Pardo?

What is the pros and cons of this method?

Thanks.

Ernie Chan said...

If I understand correctly, Robert Pardo's walk forward analysis is just backtesting with parameters optimized in a rolling lookback period.

This is certainly reasonable, but does not eliminate certain data snooping biases that are not due to parameters, but to strategy logic. For example, if you find that your strategy doesn't work well in a walk forward test, won't you change a few rules and run the test again? Aha! There you have introduced data snooping bias.

Ernie

Anonymous said...

Hi Ernie,

Thanks for response.

Also, the walk forward window is a parameter itself. The rule of thumb is to choose 25% of optimization data set. Ideally, we need to know the "speed" of the market we trade.

Moreover, the other con is that it is computation intense, and it takes extra effort to build a walk forward trading system.

Anonymous said...

Hi Ernie,

Usually, different instruments have different margin requirement, which means we could get different leverage. For example, if we trade forex (40x), futures(30x), and intraday stocks (4x), how could we allocate our buying powers to different types of assets via Kelly formula?

It seems the buying powers is not fixed.

Many thanks.



Ernie Chan said...

Kelly formula tells you what leverage you should deploy for each instrument. It is independent of that instrument's margin requirement. Of course, your broker will impose different constraints on margin (or equivalently, maximum leverage) on different instruments, so you should scale down the overall leverage of your portfolio to meet those constraints. Please see page 98 of my book Quantitative Trading which discussed this precise issue in some depth.

Ernie

David said...

Interesting! Thanks Ernie.

Anonymous said...

Hi Ernie,

Are there slippage for trading US ETFs when we just send market orders?

It seems ETFs have bigger bid/ask size than US stocks.

Ernie Chan said...

No, you won't incur slippage trading ETF with MKT orders, only bid-ask spread, unless your order size is huge.

Ernie

Anonymous said...

Hi Ernie,

In your book, one of the basic strategies you discuss involves adjusting changing the hedge ratios daily. Ignoring transaction costs, should we expect the cash flows from these hedge ratio changes to be significant/material? How do we account for them when looking at the profit/loss from mean reverting strategies?

Thanks,
flapjackery

Ernie Chan said...

Hi flapjackery,
There shouldn't be significant cash flow from adjusting the hedge ratios and rebalancing. But in any case, I would have accounted for any such cash flow in the P&L the way I computed it.
Ernie

Anonymous said...

Hi Ernie,

For pairs trading, usually, how often do we need to evaluate pairs traded live and think about replacing bad performers of live portfolio by good performers of paper portfolio?

Thanks.

Ernie Chan said...

It depends on the max drawdown duration in your backtest. Any drawdown longer than that requires a re-examination.

Another measure is the half-life of mean-reversion. A drawdown that is an order of magnitude larger than halflife should trigger alarm bells.

Ernie

Anonymous said...

Hi Ernie,

Thank you for response.

"Another measure is the half-life of mean-reversion. A drawdown that is an order of magnitude larger than halflife should trigger alarm bells."

I do not understand the paragraph above.

For example, half-life is 5 days.
What do you mean?

Ernie Chan said...

You should be concerned if drawdown is 50 days.
Ernie

Anonymous said...

Hi Ernie,

Could we use stocks "prices" to compute correlations?

Ernie Chan said...

No, you have to turn stock prices into returns in order to compute correlations.

Ernie

Anonymous said...

Hi
If anyone knows...
How can I get primary exchange EOD data (NYSE)?

Anonymous said...

My strategy uses open and close prices, MOO and MOC orders.
Should I simulate bid-ask spread and slippage?

Ernie Chan said...

MOO and MOC orders do not suffer slippage nor bid-ask spread. They get the auction price.

Ernie

Ernie Chan said...

Primary exchange EOD data can be obtained from Bloomberg. Otherwise you have to buy tick data and find the trade tick with the largest volume at the close.
Ernie

Anonymous said...

Hi Ernie,

Could we send MOO and MOC in Interactive Brokers via API?

Ernie Chan said...

Sure you can!
Ernie

Anonymous said...

Hi Ernie,

Some people trade cointegrated stocks pairs with high correlations.

Does that make sense?

Ernie Chan said...

That sounds OK to me.
Ernie

Anonymous said...

Hi Ernie,

Thank you for response.

I guess that the advantage is that
even when cointegration is broken, if the correlation still holds, then the loss is limited.

Anonymous said...

I have a question regarding IB paper trading. Market On Open orders are not executed some times (I get message: "At the open order could not be executed") and finally orders are canceled. Does anyone know what might be causing this?

Ernie Chan said...

It is just a technical issue that
IB does not support MOO orders in paper trading. There is no deep explanation for it: after all this is just paper trading.
Ernie

Anonymous said...

Hi Ernie,

Any idea why historic closing prices often differ between IB and Google/Yahoo?

Thanks

Ernie Chan said...

What Yahoo, Google or IB report as closing price is just the last price captured on any one of the dozens of exchanges in the US. These prices tend to generate erroneous backtest performance, since we cannot be assured that we can execute at that price historically.

The only closing price that matters is the "official" closing price from the primary exchange, obtainable if you subscribe to Bloomberg, or if you have access to high frequency (1ms) historical data.

Ernie

Anonymous said...

Thanks for your reply Ernie.

Another thing:
Do you find the metric Max Drawdown/Volatility useful? Max Drawdown scales with vol (not linearly though) which means one need to standardize drawdowns somehow.

Thanks

Ernie Chan said...

The usefulness of maximum drawdown is to see what extreme volatility can do to your equity curve. So I won't advise dividing it by volatility.

Ernie

Anonymous said...

However when comparing different strategies with different volatilities one need to standardize drawdowns somehow.

Anonymous said...

Hi,

I just want to share I had a similar idea before on the crossover and I eventually ended up optimizing filter weights of Finite Impulse Filters (with N-weighted sum of N historical prices). Long/short decision is triggered when comparing the current price against the weighted sum. The weights are "calibrated" using raw historical closing prices with the objective function being optimizing sharp ratio. A sharp ratio of 5+ was easily achievable with couple thousand percent annual return even with harsh simulated commissions. Then I extended the idea to a multi-dimension model where I added more market info. Thus even more parameters to be calibrated. It was a 60taps by 8market info by 1000stocks “matrix”. And it resulted even higher sharp ratio, ONLY when looking at calibration results. :P

The calibrated model performance is same as a coin flipping decision process in backtesting. It was my first time experiencing over fitting first hand out side of class and got fooled by randomness. It was fun tho. I Just want to FYI of what i tried if anyone is thinking about this route and hope this is helpful.

This post is very refreshing. Will dig up my decade old codes and try the method in this post. Nice post. Thanks.

Irvin

Ernie Chan said...

The standard way to compare max drawdowns for strategies with different volatilities is to use Calmar ratio (google it).

Ernie

Ernie Chan said...

Hi Irvin,
Thanks for sharing your idea!

If I understand you correctly, your filter is actually something like a feed forward neural network?

When you mentioned overfitting, are you suggesting that the live trading results (or live walk-forward testing results) are much worse than the +5 Sharpe ratio that you found in-sample?

Ernie

Anonymous said...

Hi Ernie,

I suppose we can call it a NN, it is more of a single layer one with each "neuron/synapses" a impulse function. It is a feed-forward filter.

In sample performance was unrealistically good. Out of sample performance did not generate statistically significant return even without considering commission. So I didn't bother to try it in a live account.

I've also tried to use daily index data as far back as 30yrs to expand the sample size for calibration. However, it also failed to work as soon as it touches the out of sample data.

Irvin


Anonymous said...

Why don't people give up on these neural networks. They just don't work out of sample or are highly unstable. Simplicity is a good thing. Occam's razor always applies.

Anonymous said...

Hi Erine

What timeframe you prefer to use for calculating the spread for trading Mean Revision of FX usually?

trice said...

Hi Ernie,

Been following your blog for quite a while now and it has been very informative. Thanks for sharing all this great info with us.

I have recently finished writing software for automated intraday trading & analysis in Matlab using IB (with the standard 100 contracts real time data). Now I am trying to hone in on the most effective strategies.

You discuss the continuing effectiveness of mean reversion, despite what you say is "low volatility in the markets." My question is, if mean-reversion has a higher return with increased volatility, why not use extremely volatile leveraged etfs like junior gold miners or some such thing? Is it because of drawdown or black swan issues?

On a related note, can you give any generally effective ways of choosing stocks that work for mean reversion, or at least things to avoid?

For example, can historically high amounts of "reversion to the mean" be an indicator? A high standard dev but a small kurtosis? I see a lot of information about mean reversion as a trading concept, but little in the way of choosing the most effective time scales and contracts for performing it. Thanks for your help!

Tyler

Ernie Chan said...

Hi Tyler,
Sure, if you find mean reversion in leveraged ETF, you can certainly trade them. (I haven't).

There is strong cross-sectional mean reversion of stocks intraday. Please see the example strategy in the chapter on stock mean reversion in my book Algorithmic Trading.

To discover cross-sectional mean reversion, you don't need any indicator. Just backtest a simple long-short strategy will do.
Ernie

HK said...

Hi Ernie,

How Premiums and Discounts of ETF affect strategies? Is Premiums and Discounts arbitrage possible between two ETF of the same asset/index?

-HK

Ernie Chan said...

Hi HK,
There are unlikely to be much premium/discount in ETFs left for retail investors to arbitrage. Fund managers have Authorized Participants (typically big banks) that specialize in minimizing these premium/discount intraday to by buying and selling basket of stocks.
Ernie

Anonymous said...

Hi Ernie,

Could we assume that there is no individual default risk for intraday stocks trading?

Thanks.

Ernie Chan said...

Even though stocks typically default after market close, there is nothing to prevent bad news from leaking out during market hours and cause a stock to plunge 99%.

Ernie

Anonymous said...

Hi Ernie,

Do you have any comment about post–earnings-announcement drift, or PEAD ?

Thanks.

Ernie Chan said...

PEAD continues to work, provided some bells and whistles are added.

Ernie

Anonymous said...

Hi Ernie,

Thank you for response.

Shall we avoid stocks pairs trading after earnings announcement?

Ernie Chan said...

Yes, avoiding mean reversion trades immediately after any corporate event is a good idea.

Ernie

Anonymous said...

Hello Ernie, with regard to currency pairs like EUR/MXN, CHF/NOK, bid/ask + slippage costs are probably more significant than for EUR/USD. My question is, is there an estimate available such as 3 basis points in each direction - I understand this may depend on the time of day, but I was looking for an estimate.

Thanks.

Ernie Chan said...

The bid-ask spread for any FX pair is highly dependent on the broker you use. In fact, some brokers do not charge commission, and earn all their profits from the spreads. So you have to go to your broker and observe the actual spread.

Ernie

Anonymous said...

Thanks for the answer. But what about slippage? The broker is IB, but its just that I have limited myself to EUR/USD, EUR/CAD etc. and was concerned about the non-majors - the spread itself seems quite low after adjusting it for the value of the underlying currency.. Also the planned number of round trips can go > 300 in a year, so this can get tricky.. I am going to paper trade with them but that will take some time to tell how strong the strategy will be :-)

Ernie Chan said...

By slippage, I think you mean the difference in the price that triggered your order, and the execution price. This slippage of course depends on how long it takes for your order to travel from your computer to IB's server. It could be anywhere from 0ms to 100ms. And thus the slippage could be 0 to 2 pips.

Ernie

Anonymous said...

Hi Ernie,

What happen to ATHL in US stocks market today?

There are no quotes.

Anonymous said...

Go figure

Anonymous said...

Hi Ernie,

always good to read your articles and comments about others' strategies.

But in regards to your FX managed account, dont you think 30+% is quite much?

Ernie Chan said...

What is this 30% figure you are referring to?

Ernie

Unknown said...

Hi Ernie and congratulations for this post, it s very interesting. I don't know what is a wavelet filter, there is some paper or book which explain it? thank you

Ciro

Ernie Chan said...

Hi Ciro,
Thanks for your kind words!

There is no better intro than Wikipedia:
https://en.wikipedia.org/wiki/Wavelet

Matlab also has a wavelet toolbox with good documentation:
https://www.mathworks.com/help/wavelet/index.html

Ernie

Unknown said...

Thank you Ernie,

so if i understood well, this filter could be suitable for a more statistical cyclical analysis, could be?

Ernie Chan said...

Hi Ciro,
Yes, wavelets transform is similar to Fourier transform. Both can be used to filter out "noise" of various frequencies.
Ernie