Maximizing growth without risking bankruptcy
Many hedge fund disasters come not from making the wrong bets – that happen to the best of us – but from making too big a bet by overleveraging. On the other hand, without using leverage (i.e. borrowing on margin to buy stocks), we often cannot realize the full growth potential of our investment strategy. So how much leverage should you use?
Surprisingly, the answer is well-known, but little practiced. It is called the Kelly criterion, named after a mathematician at Bell Labs. The leverage f is defined as the ratio of the size of your portfolio to your equity. Kelly criterion says: f should equal the expected excess return of the strategy divided by the expected variance of the excess return, or
f = (m-r)/s2
(The excess return being the return m minus the risk-free rate r.)
This quantity f looks like the familiar Sharpe ratio, but it is not, since the denominator is s2, not s as in the Sharpe ratio. However, if you can estimate the Sharpe ratio, say, from some backtest results of a strategy, you can also estimate f just as easily. Suppose I have a strategy with expected return of 12% over a period with risk-free rate being 4%. Also, let’s say the expected Sharpe ratio is 1. It is easy to calculate f, which comes out to be 12.5.
This is a shocking number. This is telling you that for this strategy, you should be leveraging your equity 12.5 times! If you have $100,000 in cash to invest, and if you really believe the expected values of your returns and Sharpe ratio, you should borrow money to trade a $1.2 million portfolio!
Of course, estimates of expected returns and Sharpe ratio are notoriously over-optimistic, what with the inevitable data-snooping bias and other usual pitfalls in backtesting strategies. The common recommendation is that you should halve your expected returns estimated from backtests when calculating f. This is often called the half-Kelly criterion. Still, in our example, the recommended leverage comes to 6.25 after halving the expected returns.
Fixing the leverage of a portfolio is not as easy or intuitive as it sounds. Back to our $100,000 example. Say you followed the (half-) Kelly criterion and bought a portfolio worth $625,000 with some borrowed money. The next day, disaster struck, and you lost 5%, or $31,250, of the value of your portfolio. So now your portfolio is worth only $593,750, and your equity is now only $68,750. What should you do? Most people I know will just stick to their guns and do nothing, hoping that the strategy will “recover”. But that’s not what the Kelly criterion would prescribe. Kelly says, if you want to avoid eventual bankruptcy (i.e. your equity going to zero or negative), you should immediately further reduce the size of your portfolio to $429,688. Why? Because the recommended leverage, 6.25, times your current equity, $68,750, is about $429,688.
Thus Kelly criterion requires you to sell into a loss (assuming you have a long-only portfolio here), and buys into a profit – something that requires steely discipline to achieve. It also runs counter to the usual mean-reversion expectation. But even if you strongly believe in mean-reversion, as no doubt many of the ruined hedge funds did, you need to consider protecting you and your investors from the possibility of bankruptcy before the market reverts.
Besides helping you to avoid bankruptcy, the Kelly criterion has another important mathematically proven property: it is a “growth-optimal” strategy. I.e. if your goal is to maximize your wealth (which equals your initial equity times the maximum growth rate possible using your strategy), Kelly criterion is the way.
Notice this goal is not the same as many hedge managers’ or their investors’ goal. They often want to maximize their Sharpe ratio, not growth rate, for the reason that their investors want to be able to redeem their shares at any time and be reasonably sure that they will redeem at a profit. Kelly criterion is not for such investors. If you adopt the Kelly criterion, there may be long periods of drawdown, highly volatile returns, low Sharpe ratio, and so forth. The only thing that Kelly guarantees (to an exponentially high degree of certainty), is that you will maximize the growth potential of your strategy in the long run, and you will not be bankrupt in the interim because of the inevitable short-term market fluctuations.
For further reading:
Poundstone, William. (2005). Fortune’s Formula. New York: Hill and Wang.
Thorp, Edward O. (1997; revised 1998). The Kelly Criterion in Blackjack, Sports Betting, and the Stock Market. www.bjmath.com/bjmath/thorp/paper.htm
38 comments:
"This is telling you that for this strategy, you should be leveraging your equity 12.5 times! If you have $100,000 in cash to invest, and if you really believe the expected values of your returns and Sharpe ratio, you should borrow money to trade a $1.2 million portfolio!"
Hi, I'm not sure this is the correct interpretation of the Kelly Criterion... if f is 8% (the reciprocal of 12.5), and the trader's net worth is $100k, shouldn't he only trade a portfolio of $8k? Ie. if he has $8k in cash, he should buy $8k worth of stock but not borrow anything; if he has no cash (eg. $100k vested in properties), then he should borrow $8k to buy stock. In other words, one should only trade a fraction of one's net worth, and not borrow if he has enough cash on hand to take the trade. Or am I missing something here?
Dear Anonymous,
You should not be taking the reciprocal of f=12.5. f itself is the fraction of your wealth you should use for betting. This is intuitively clear since f is proportional to the excess returns of your strategy -- the higher the return, the higher the fraction. If the "fraction" turns out to be greater than 1, that means you should bet more than your net worth, i.e. use leverage.
Ernie
Is the Kelly fraction (f) in your formula the same f as in http://en.wikipedia.org/wiki/Kelly_criterion (f=p-q/b)? On the wiki page, the maximum for p is 100% and the minimum for q is 0%, hence f can never exceed unity and that is why I thought f was 8% instead of 12.5. This could be the cause of the confusion here...
Dear Anonymous,
The Kelly's formula you quoted is for betting with binary outcomes only. In the case of continuous finance, please refer to the formula I wrote in the main body of my article which comes from the paper by Prof. Ed Thorpe cited at the end of that article. In the continuous finance case, f can most definitely be greater than 1, otherwise it would be quite useless to the hedge fund community.
Ernie
This works, like much of financial theory, only in the case of continuous hedging without trading costs. In the real world, different from the lognormal world, there is a real possibility of bankruptcy because one is not able to hedge continuously and because firms can go bankrupt--geometric returns don't work for the possibility of a 100% loss. The optimal leverage with limited hedging (say once per month) and trading costs should be substantially smaller than the make-believe world of the Kelly criterion which holds for gambling (where one can adjust the wager before each bet and returns don't have fat tails) but not for real financial markets. Otherwise one might well run into the surprise of a "seven-sigma" event encountered by many a hedge fund.
Dear Anonymous,
I disagree with your statement that financial theory works only when there is no transaction cost or when we assume log-normality. In fact, most of the papers published in finance these days explicitly incorporate transaction costs in their modeling. Furthermore, if you read Dr. Ed Thorpe's paper on Kelly criterion, you will find that he has managed a well-known and highly successful hedge fund using this technique. On the other hand, many hedge fund managers that "blew up" have never heard of Kelly's formula.
As quantitative practitioners, just as if we were physicists or biologists, we are of course aware that every theory is an approximation, but that does not mean that theories are useless for practical implementation.
Ernie
Ernie: I didn't mean to say that nothing of financial theory works in practice, and I follow Dr. Thorpe religiously. My point merely was that--like unmodified Black-Scholes which rests on similar assumptions--applying the Kelly criterion to a lognormal random walk seriously underestimates risk and thus overestimates optimal leverage. The basic idea of the Kelly criterion, optimize geometric return of your portfolio, makes a lot of sense, only the optimal leverage will be substantially lower than a naive application of the Kelly criterion suggests. You can convince yourself of this if you do a Monte-Carlo simulation of differently leveraged portfolios with actual return distribution instead of Gaussian returns, periodic instead of continuous rehedging, and trading costs.
Dear Anonymous,
You are right in saying that a log-normal model of risk underestimates the true risk of a portfolio. That's why most practitioners use the half-Kelly formula to increase the margin of safety. I would regard the leverage that Kelly formula suggests (with log-normal assumptions) as an upper limit.
Ernie
why use 1/2 kelly and not some other fraction of kelly? where did the 1/2 come from?
Another interesting observation: If you believe that you don't have any special portfolio management skills and for the invested asset mu = rf + rp*sigma then the optimal leverage under GBM etc. works out to f=rp/sigma, i.e. the market price of risk divided by your investment's volatility.
Dear Anonymous,
The half in half-Kelly comes from the cumulative experience of traders using Kelly's formula to manage their investments. The consensus opinion is that taking half of the Kelly number gives a good enough safety margin.
I am not familiar with the acronym "GBM". Perhaps you can elaborate or give a reference to that formula?
Best,
Ernie
1/2-kelly is derived from "feedback of experienced traders" !? can u put some numbers or proof in your assertion?
recommending whatever fraction of kelly is the same as using your gut for leverage (w/ a max limit of kelly, which is an insane amount of leverage anyway).
dont quant traders hate guts? especially trader's guts?
Hi Ernie: There are several anonymous posters active now, which creates some confusion. I wasn't the one with the half-Kelly question, though I'd probably do some modelling under the more realistic assumptions I outlined to gain confidence before trading it. Anyhow, with GBM I meant geometric Brownian motion--I just put that in as a caveat because earlier I said that this model underestimates risk.
Dear Anonymous,
Quant traders rely on backtesting, not just their own, but other traders collective back (and live) testing. Most strategies do not have proofs in the mathematical sense. And yes, I regret to inform you that a majority of quant traders (based on my circle of acquaintances) do rely on gut instincts in a number of places, and the "1/2" in half-Kelly is one of those places. Since you follow Dr. Thorpe's work religiously, you will notice he use half-Kelly in his paper too, where he also did not give any "proof" that 1/2 is the best fraction to use. However, there are numerous simulations that show that, reducing leverage to half reduces the chance of ruin greatly without reducing the growth rate of wealth significant.
Ernie
There's the trader's backtested strategy and there's the "ideal" leverage for that given strategy.
GIVEN a strategy,
u assert that the ideal leverage is 1/2 kelly.
If so,
this question can be treated mathematically (or via monte carlo) given the outcome of that strategy; either simplistically by a wager and % success rate, or by your mentioned continued "finance case" stream of possible outcomes.
Indeed,
a strategy does not have any proof,
but the leverage for a particular strategy does.
I have not read any papers or simulations defending the use of 1/2 kelly and would love to read them.
Please post some links to them.
Congrats on your great thread and initiative.
1/2 is arbitrary. In the spirit of much Bayesian hackery (e.g. shrink all off-diagonal covariances 50% towards the mean...), half seems to be a happy medium :-)
The reason full kelly betting isn't used is not because it's "wrong" (i.e. lognormal vs. fat tailed, though that causes problems), but because we usually don't really know the expected return nor variance. We might be able to make good guesses, but they are still guesses. Kelly shows that overbetting leads to ruin, whereas underbetting is "merely" suboptimal.
won't different definitions of ruin yield different optimal leverages?
in the market full kelly is crazy and
"1/2 kelly" is a number madonna gave you while consulting her cabala.
guesstimate your odds,
define ruin and then set leverage.
Dear Anonymous,
That pretty much sums up the situation.
Numbers such as returns and volatility are all estimates. The exact leverage is indeed guesswork. What's important is that one must adjust the portfolio size in order to maintain a fairly constant leverage. (And even this is sometimes quite hard if you don't know ahead of time how many positions you are going to have.)
Ernie
i disagree w/ the constant leverage claim, it is as arbitrary as the 1/2 kelly statement.
Give me a
- wager
- % profitable and
- a horizon (times we will play),
and i'll show u how a moving leverage will beat a static in maximum total return, chance of ruin and % that loses money.
we will always be below kelly, so,
increases in leverage as gains accumulate r always welcome.
Dear Anonymous,
Given the probabilistic nature of returns, it is always possible, for a given historical returns series, to construct a "moving" leverage that generates higher total growth than Kelly's formula.
In fact, you can always set the leverage to zero just before a down day, and set it to infinity just before an up day!
So I am not exactly clear what your "moving leverage" strategy is.
What Kelly has proven mathematically, however, is that if your returns time series is stationary, in the sense of stationary returns, volatility, and therefore Sharpe ratio, then statistically the leverage given by Kelly's formula will generate the maximum growth rate. It is impossible to exceed this leverage and not risk ruin (i.e. equity goes to zero). However, as another reader pointed out before, due to the uncertainty in estimates of return and volatility, it is advisable to be on the safe side and halve the leverage.
Increasing leverage as gains accumulate is, according to Kelly's theorem, very likely to lead to ruin, unless you can justify an increase in expected Sharpe ratio going forward as well. Now, this is different from increasing the bet (capital) size as gains accumulate. We should of course engage in the latter without changing the leverage at all.
Ernie
your statement:
"Increasing leverage as gains accumulate is, according to Kelly's theorem, very likely to lead to ruin"
is true only if u increase leverage above the kelly's optimum.
if u r trading at your cabalistic 1/2 kelly,
and you've doubled your money during the 1st half of the period u r going to be judged,
why not lever up to 3/4 of Kelly?
The Kelly criterion is based on the assumption that you can borrow at the risk-free rate. In your example, if you borrow at 4% over the risk-free rate, then the optimal leverage works out to be 6.25 (half-Kelly in this case).
See the following post where I discuss the effect of higher borrowing rates: link
However, I don't know where you can find an investment that has a 12% return with a standard deviation of only 8%. If the standard deviation is 20% (roughly matching long-term volatility of the S&P 500), then the Sharpe ratio drops to 0.4, and the Kelly criterion drops to f=2. Toss in the assumption of borrowing at 4% above the risk-free rate, and the optimal leverage point drops to f=1 (i.e., no borrowing at all).
It's amazing how the case for leverage evaporates as you repair unrealistic assumptions.
Dear Michael,
You are right that if your funding cost increases, the Kelly leverage will be reduced. However, your assumption of 4% over risk-free rate is far too onerous. For e.g., if you trade through Interactive Brokers (popular choice for many independent traders and small hedge funds), the funding rate is only 1% or less above risk-free rate.
You are also correct in pointing out that if we own S&P500 index, we will not get returns of 12% vs. standard deviation of 8% (a Sharpe ratio of 1.5). However, many market-neutral strategies that quantitative traders employ have even better returns vs. risk ratio than what I used as an example here. In fact, Sharpe ratios (after cost) of over 2 are quite common. So I don't believe these are unrealistic assumptions -- and certainly high leverage is very appropriate for such strategies.
Ernie
Dear Ernie,
I can't say that I know much about brokers other than the one I use. I can borrow only a modest amount from my broker in the form of margin. After that, I woud need to get a loan elsewhere at higher rates. It is unlikely that I could borrow even 3 times the size of my portfolio at any interest rate.
Of course, hedge funds are very likely to have better access to borrowed funds than I have. However, given the number of hedge funds that go under, I'm guessing that many of them assess the volatility of their strategies incorrectly.
Suppose that a hedge fund has a strategy with expected excess return 20% and standard deviation of 10% (Sharpe Ratio 2). Borrowing at 1% excess leads to f=19. If such a fund had $10 million in assets and sought to borrow $180 million at 1% excess, the potential lender would be insane (in my opinion) to agree. The strategy seems too good to be true. The expected leveraged geometric return is 181.5%. But this drops to -44% if the standard deviation is actually 15% instead of 10%.
If crazy lenders exist out there then I can't blame hedge funds for trying their strategies. It amounts to a huge free roll. If things work out, then everybody gets rich. If it all goes south, then the lender loses a pile of money.
Michael
Michael,
If you have a dollar or market neutral portfolio, you can borrow from some brokers (such as Interactive Brokers) at higher than Reg-T margin. It is called risk-based margining or portfolio margining.
The Kelly formula I have displayed here is based on a Gaussian model of returns, which is clearly wrong in cases of extreme market events. That's why half-Kelly is the more prudent choice.
(As a commercial pitch for my forthcoming book, I have a long and detailed chapter on the application of Kelly formula to real-life trading.)
Ernie
Hey Ernie, Based on what I have read so far, Kelly Crtn talks about optimal bet size based on the key variable of probability of success, while optimal leverage is very different concept, which talks about how much to leverage based on portfolio/equity expected returns/variace. Please clarify the reason why for you to combine Kelly's concept with Optimal leverage?
Hi Vibhu,
Your question has been discussed elsewhere on this blog and in the comments. The quick answer is that I am using the continuous finance version of Kelly formula. See http://www.edwardothorp.com/sitebuildercontent/sitebuilderfiles/KellyCriterion2007.pdf
Ernie
Dear Ernie
I backtested the GLD-GDX strategy for the last 4 years and now wanted to employ the Kelly Betting on it.
The problem which arises, is that I need to define a window in which i estimate mean and var of the strategy, thus I would update every day f for the next day using the last n bars.
I tried it with n=200, the problem is that f results in beeing very volatile and finally it is a disaster in terms of performance.
If estimate f using all the data (1000 bars), then i don't have any data to test forward ..
So the big question, how to choose n ?
Thanks in advance for your help ...
Nicolas
Hi Nicolas,
Picking the right lookback is indeed a good question, and it is more art than science in my opinion. I like to pick n=3 years for a strategy like GLD-GDX that holds multiple days.
Best,
Ernie
Dear Ernie,
In example 6.2 in your book, you calculate that the compounded return on SPY using the recommended 2.528 Kelly leverage is 13.14%. Do you mean that the return to the investor on the full leveraged amount of $252,800 (using $100,00 in capital), would be 0.1314*252,800 = $33,218 (33% of capital)? Or, do you mean you would get 0.1314*100,000 = $13,140 back using the full $252,800 (using $100,000 in capital)? If you would only get $13,140, this seems very low considering how much money has been borrowed and the 9.8% return without borrowing.
Using the formula in the appendix for example 6.2, I got g = 0.04 + 2.528*(0.1123-0.04) + (.1691*2.528)^2/2 = 31.4%. But what initial investment does gain value this apply to ($100,000 or $252,80?)? Does this mean using $100,000 in capital, and 2.52 leverage I would make $31,400? Essentially, how much money would I have at the end of the year if I had $100,000 in capital, and used the Kelly leverage to borrow up to $252,800 to buy SPY in your example?
Thanks very much for your great book and for your help.
-E
E,
In example 6.2, if you have a $100K capital, and buy SPY with a leverage of 2.528, you should expect to earn $13,140 at the end of the first year.
It is not much higher than the $9,800 that you would earn with no leverage because of the low Sharpe ratio of SPY.
Ernie
Hello Ernie,
What do you recommend in the case of $100k capital trading futures contracts across 3 models (to figure out the # of contracts / model)? One could calculate f for each using e.g. half Kelly and then round down to the get the # contracts / model, however, this does not take into consideration the portfolio risk as a whole...
Grateful for your thoughts and / or links on where to research this.
best regards
Ed
PS presumably the formula values of m & r could be either annualized or summed (if longer or shorter period than 1 year) so long as they are both treated equally?
Hi Ed,
I am not sure why you think that portfolio risk is not taken into account using Kelly formula. As you can see from example 6.3 of my book, covariance between each futures are used to generate allocation across the 3 contracts.
Means and variances can be of any time frame: Kelly formula is time-scale invariant, unlike the Sharpe ratio.
Ernie
Dear Ernie,
I'm looking at implementing some of the ideas in your book and have some questions. What I'm specifically looking at is trading a basket of models that all trade different types of futures. The Kelly formula I'm using is to solve the linear system
CF* = M
for F*, the Kelly fraction vector
where C is the covariance of the various models and M is the return vector. All averaged over some recent period.
Now, cranking the numbers I run into two or three practical problems, that mostly boil down to the question "how important are the correlations":
1. I sometimes see negative Kelly fractions for some models, which opens up a few questions..
a. I am assuming I can get negative fractions for even a profitable model because it may be highly correlated with another profitable model but less profitable, so therefore I can use it to hedge.
b. But what do I do with the numbers if I determine that it doesn't in fact make sense to short a model? Maybe I just don't want to do it for some reason (the gains might be small from doing so and it does seem very weird to be shorting a profitable strategy) or maybe you just can't figure out what it "means" to short a complicated model... Specifically if my Kelly fractions are, let's say +2, +2, -1, and I decide I am just going to not trade the -1, what do I do with the other 2s? I am clearly abusing the math since the +2s are calculated under the assumption I'm going to diversify with the -1. Can I just clamp the -1 to 0 and not mess things up?
2. On a similar issue, since these are futures models with different margin requirements, if I have, let's say now +5, +5, +5 that suggests I should put in 15 times my capital, but what if the margin limits stop me from doing so? You suggest that we should then roll back the models equally, to shall we say +2, +2, +2. But if one of the models has much higher margin requirements than the other, it may be that we could instead trade +4, +4, +1.5 ..... again I realize I'm not following what the covariances are telling me, but it seems I should be getting a higher return.
Anon,
If you cannot short one of the strategies, why not just exclude it when computing C and M?
If you want to limit the leverage of certain strategies but not others and still want to find the optimal growth rate, you can numerically maximize (with constraints) the compounded growth rate of the portfolio, with the leverage on each strategy as the parameters to be optimized. This constrained optimization can use the backtest returns as input.
(More specifically, you can assume a certain set of leverages for each strategy, compute the compounded growth rate of the entire portfolio using your backtest daily returns, and then vary these leverages while maintaining your constraint until the growth rate is maximized.)
Ernie
I am doing some calculations similar to the last few posts on this blog i.e. calculating F* for different
strategies ... I have quite a lot of them in a large covariance matrix and am looking back around 1/2 year.
One of the things I don't understand (having checked my calculations several times) is that I am getting positive Kelly fractions for some strategies despite them have negative M (geometric returns) over the last 1/2 year or more. I believe that this may be due to the covariance matrix which averages covariances over different periods as a way to deal with strategies that do not trade all the time.
I can understand that the overall equity curve may benefit if these 'recent loser strategies' perform well while others do not. Nonetheless they do make losses.
Can you see a rationale for trading these strategies as described here, or would you exclude any that had negative returns over the calculation window?
thank you
PS calculations that lead to counter-intuitive results can be quite tough to swallow
Anon,
Since Kelly formula only cares about overall growth rate, if your losing strategy is anti-correlated with your other strategies, Kelly will recommend that you include it, because it will increase the Sharpe ratio. This is assuming that you can adopt the Kelly optimal leverage. If you can't use that leverage, then it may be optimal to just keep the strategies with the highest returns. I have an example in my new book to illustrate this.
Ernie
Post a Comment