Sunday, January 07, 2007

Universal Portfolios

Let me describe a portfolio optimization scheme that, over the long run, is supposedly guaranteed to outperform the best stock in the portfolio.

Before we begin, let’s agree that we will rebalance our portfolio every day so that each stock has a fixed percent allocation of capital, just as your favorite financial consultant would have advised you. What this means is that if you own IBM and MSFT, and IBM went up after one day whereas MSFT went down, you should sell some IBM and use the capital to buy some more MSFT. There is a technical term for such portfolios: they are called “constant rebalanced portfolios”. Notice also the similarity with the Kelly criterion which I wrote about before: Kelly criterion asks you to maintain a constant leverage, which is like maintaining a fixed percent allocation between cash (debt) and stock.

But what should the fixed percent allocation be? Here is where the scheme gets interesting. Suppose we start with an equal capital allocation, for lack of any better choice. At the end of the day, your portfolio has a certain net worth. But then you can calculate what the net worth would have turned out if you had started with a different allocation. Indeed, we can run this simulation: try all possible initial allocations, and calculate the hypothetical net worth of the resulting portfolio. Use these hypothetical net worth as weights (after normalizing them by the sum of all net worth), and compute a weighted-average percent allocation. Finally, adopt this weighted average allocation as the new desired allocation and rebalance the portfolio accordingly. So actually the “fixed” percent allocation is not fixed after-all: it gets adjusted daily, but probably not by much. Repeat this process everyday, always calculating a new weighted allocation by simulating various initial allocations since day 1.

This scheme of portfolio optimization can be proven to produce a net worth greater than just holding the best stock, given long enough time. If this sounds like a miracle, it is partly because this is in fact an ingenious result of information theory, and partly because there are various caveats that actually limit its practical application. The proof that it works (at least in theory) is rather technical and I will let the interested reader peruse the original paper published by Prof. Thomas Cover, a noted information theorist from Stanford University. He coined the term “Universal Portfolios” for portfolios rebalanced/optimized with this scheme. Without understanding the mathematical intuition, this scheme may appeal to those who believe in long-term trending behavior of stocks, because if a stock performs very well in the past, we will end up allocating more capital to it in the long run. It may also appeal to those who believe in short-term mean reversal behavior, since in the short-term, we are performing daily rebalancing of the stock positions based on an approximately constant allocation. However, this seeming confirmation of either trending or mean-reverting characteristics of stock prices is illusory – this scheme is supposed to work even if the stock prices are totally random! How can we manage to squeeze out a gain even with random price series? Remember that we have done the opposite before (see my earlier articles): we manage to lose money even when a price series exhibits a geometric random walk. So it is not too surprising that we can also make money using similar information theoretic juggling.

Now for the caveats. Every time an information theorist start saying “In the long run, …”, you will be well-advised to ask: How long? In my geometric random walk example where the volatility (standard deviation) of returns every period is 1%, we find that the compounded rate of return is an agonizingly small -0.005% per period. In the case of the universal portfolio scheme, the out-performance over the best stock in the portfolio is similarly dependent on the volatilities of the stocks: the higher the volatility, the faster the out-performance. Let me run a simulation with a portfolio consisting of two ETF’s RTH and OIH. If we were to run the Universal Portfolio scheme from 2001/5/17 – 2006/12/29, I find that the cumulative return is 32% (without transaction cost). Contrast that with just buying-and-holding the best ETF (namely OIH here): the cumulative return is 54%. The Universal Portfolio loses. Does this mean the theory is wrong? Not really: RTH and OIH may just have too low volatility. Herein lies the first practical caveat with the Universal Portfolio scheme: it can take too long to realize its benefit if the volatility is low.

How do we find ETF’s that have high enough volatility to realize the out-performance of Universal Portfolio? Actually, we can simply boost the volatility of RTH and OIH artificially by increasing their leverage. So let’s say we leverage both of them 2x. This means their daily returns and volatilities are both doubled. Now the best ETF (which is still OIH here) has a return of 23% (why is it lower than the un-leveraged case? Remember the formula m-s2/2 in my previous article.) , but the Universal Portfolio has a return of 45%. So now the Universal Portfolio wins. But this is a Pyrrhic victory: if you factor in a transaction cost of 10 basis points, the Universal Portfolio scheme actually returns only 4%. This is the second caveat of Universal Portfolios: because of the frequent rebalancing required, transaction costs tend to eat up all the out-performance.

Now there is a final caveat. The reader may ask why I don’t just pick two stocks instead of two ETF’s to illustrate this scheme. Aren’t most stocks more volatile than ETF’s and therefore much better suited for this scheme? Indeed, most academic papers, including Prof. Cover’s original paper, use a pair of stocks for illustration. But if we do that, we run the risk of introducing survivorship bias. Naturally, if you know ahead of time that none of these two stocks will go bankrupt, the Universal Portfolio scheme may look great. But if you run a simulation where one of the stocks suddenly went bankrupt one day (which tend to be a fairly mathematically discontinuous affair), the Universal Portfolio scheme will most likely not beat holding just the non-bankrupt stock in the beginning. Using ETF’s eliminated this problem. But then ETF’s are far less volatile.

So given all these caveats, is Universal Portfolio really practical? Prof. Cover seems to think so. That’s why he has started a hedge fund to prove it.

20 comments:

  1. interesting, I'm getting a 404 error when trying to view the paper :(

    ReplyDelete
  2. It seems that either Prof. Cover has removed his reprint from public access, or the Stanford server is down. You can read other similar papers by googling Universal Portfolios. -Ernie

    ReplyDelete
  3. Hi,

    Why did you use ETFs as opposed to futures on indexes, commodities and interest rates, which would eliminate issues with suvivorship, allow for more volatility with higher leverage and lower transaction costs? Also, did you try different time periods for portfolio rebalancing such as weekly or monthly? Would rebalancing using these time periods provide some benefits?

    Thanks!

    ReplyDelete
  4. Dear Anonymous,
    Certainly using futures will reduce transaction cost, which is the main reason why this strategy didn't work well for stocks or ETF's. I will post the results on futures here when I get around to doing the calculations.

    I have tried rebalancing at different time period -- the results are not significantly different.

    Thanks for your suggestions!
    Ernie

    ReplyDelete
  5. Hi Ernie,
    Dokuchaev and Savkin have published an improved version of Cover's Ultimate Portfolio theory (in the journal: Insurance:Mathematics and Economics 34 (2004) 409 - 419). Are you able to decode from the heavy maths what their modified version entails?I could not follow the maths..

    ReplyDelete
  6. Hi Anonymous,
    Thank you for the heads-up on the new article on Universal Portfolio. I will see if I have time to read such heavily mathematical article. My past experience has been that the more mathematical an article is, the less likely it will actually be a profitable strategy.
    Ernie

    ReplyDelete
  7. Mr. Chan -

    Have you tried this concept with the 3x (Direxion or other) funds? Also, if we start with the following:

    ETF A - $5000
    ETF B - $5000
    Cash - $10,000

    and both ETF A and ETF B lose value over time, say both drop to:

    ETF A - $3000
    ETF B - $2000
    Cash - $10,000

    How should one rebalance?

    ReplyDelete
  8. Anonymous,
    If one is to use UP for reallocation, one has to know the history of returns of the various instruments. The latest returns alone are not sufficient.

    If one, however, assumes that the allocation is fairly constant, then one should make sure that their percentages in the portfolio are constant. In your example, you need to use cash to buy ETF A and B so that they return to 25% of equity.

    Ernie

    ReplyDelete
  9. Prof. Cover's papers on portfolio theory are posted at http://www.stanford.edu/~cover/papers/paper93.pdf

    ReplyDelete
  10. sorry... that was one particular paper. here's the larger list: http://www.stanford.edu/~cover/portfolio-theory.html

    ReplyDelete
  11. Ernie,

    the paper that your are referring to is available at the Stanford's web site. The hyperlink should start with "www" instead of "itg":


    http://www.stanford.edu/~cover/papers/universal_portfolios.pdf

    Thanks,
    Kirill

    ReplyDelete
  12. Hi Kirill,
    Yes, your link works... thanks!
    Ernie

    ReplyDelete
  13. > this scheme is supposed to work even if the stock prices are totally random

    Well, that is definitely to good to be true..

    ReplyDelete
  14. Dear Ernest,

    Having read your blog a little more and discovered that it's actually quite easy to add comments, I thought I would!

    As part of my philosophy of `learning by doing', I coded up a simple example of volatility pumping based on a paper from yats.com. This of course is a prelude to Cover's Universal Portfolios. The MATLAB code is available from the MATHWORKS website under
    example-of-volatility-pumping and contains references.

    I am happy to receive comments / bug suggestions.

    Best,

    -ed

    ReplyDelete
  15. In a 2004 paper "Can We Learn to Beat the Best Stock" it showed that although the optimal re-balance vector hindsight beats the best stock, when running the Universal portfolio algo by cover it really don't get near the best stock. Surely not in the short term of 3-4 years as their dataset contains. The dataset used in cover original paper is also very very biased.

    ReplyDelete
  16. Hi M,
    Thanks for the link.

    Yes, there are various authors who claim they can construct optimal trading strategies that can overcome transaction costs. Unfortunately, I have never been able to verified their results out-of-sample using a representative universe of instruments.
    Ernie

    ReplyDelete
  17. This seems contradictory to me. At first it sounds like a mean reversion strategy:

    "if you own IBM and MSFT, and IBM went up after one day whereas MSFT went down, you should sell some IBM and use the capital to buy some more MSFT"

    eg: if something goes down (relative to the avg stock in the port), buy more. if it goes up relatively, sell it.

    But later, there is this:

    "try all possible initial allocations, and calculate the hypothetical net worth of the resulting portfolio. Use these hypothetical net worth as weights (after normalizing them by the sum of all net worth), and compute a weighted-average percent allocation. Finally, adopt this weighted average allocation as the new desired allocation and rebalance the portfolio accordingly."

    The easiest hypothetical portfolios to calculate are where each respective stock, except for 1, is 0% and then 1 stock is all that is held. eg: 100%

    If this is the criteria for evaluating and re-balancing, the strategy is simple: Buy the stock that has had the largest percent gain since the beginning date of the evaluation as your only holding. Simply hold 100% of your portfolio in the stock that has gone up the most. Re-balancing only happens when another stock has a larger percent gain in the period.

    I am confused by the seeming contradictory nature of Kelly-criterion betting, which maximizes geometric returns, which seems congruent with the above scheme (eg: "trend following" and "averaging up"), and the seeming opposite idea of mean reversion, or averaging down, which seems parallel to your IBM/MSFT example, as well as the "Shannons Demon" example on page 203 of "Fortune's Formula", where Shannon seems to indicate a geometric random walk can be exploited in a mean reversion scheme of money management.

    Hoping you might clarify..

    ReplyDelete
  18. Hi,
    The daily rebalancing indeed has a mean reversion character, as the IBM/MSFT example shows.
    However, the question is "rebalance to what?" It is rebalancing to an allocation scheme that is constantly changing, and giving more weights to the most profitable allocation in the past. So the allocation scheme itself has a momentum/trending character, but that happens on a slower time scale.

    This is actually quite generic in the financial markets: mean reversion can happen in a very short time scale, even milliseconds, but momentum can take weeks. Both can happen simultaneously, but at different time scales.
    Ernie

    ReplyDelete
  19. the question is "rebalance to what?"

    Right, and it seems in some simplified way that if you consider the "universe" of investable securities to be be 2 "stocks", A & B, that the rebalancing question immediately boils down to a very simple answer: Own ALL of the highest performing stock since inception. Owning any positive percentage of the lesser performing stock be definition results in lower total wealth.

    Abstracting this idea "outward" to include all/any securities, and any time period of arbitrary length, it seems this argument holds of owning 100% of the best performing security, all other allocations result in lower total wealth. eg: Cover's Universal Portfolio theory seems to be a sort of ultimate pathological trend following system.

    The only question seems to be "when" to choose as your start date. It seems at different points such as the 90's you would have owned nothing but AOL, and perhaps Priceline or Amazon recently.

    To me, this has a whiff of Kelly "overbetting" which inevitably leads to ruin, especially if you have chosen to leverage something like the 12.5X (full Kelly bet) as stated in another post. (Overbetting is something that I believe that a competitive society compels its institutions to engage in, and leads to virtually all financial catastrophes, such as Tulip-mania, 2008 housing collapse, etc. Personally believe the 2008 bailouts have "institutionalized" overbetting in the US financial system.)

    I guess I am having a hard time rationalizing Cover's Universal Portfolios, which seems to have a pathological 100% single security solution, and Kelly criterion, which NEVER has a 100% allocation when there is ANY positive chance of loss.



    ReplyDelete
  20. Universal portfolio scheme by Cover would *not* recommend that the allocation be set to just trading the "best" stock at 100%. It just gives the highest weight to this allocation scheme, but all other less profitable schemes also get their weights based on how profitable they were.

    Ernie

    ReplyDelete