Tuesday, May 27, 2008

Parameterless trading models

A portfolio manager that I used to work for like to pronounce that his trading models have "no free parameters". As is customary in our secretive industry, he would not elaborate further on his technique.

Lately, I begin to understand what a trading model with no free parameter means. It doesn't mean that it does not contain any lookback period for calculating trends, or thresholds for entry or exit. I think that would be impossible. It just means that all such parameters are dynamically optimized in a moving lookback window. This way, if you ask: "Does the model have a fixed profit cap?", the trader can honestly reply: "No, profit cap is not an input parameter. It is determined by the model itself."

The advantage of a parameterless trading model is that it minimizes the danger of overfitting the model to multiple input parameters. (The so-called "data-snooping bias".) So the backtest performance should be much closer to the actual forward performance.

Now, it is quite computationally challenging to optimize all these parameters just-in-time for your next order, but it is often even more difficult to do that in a backtest, given that a multidimensional optimization need to be performed for each historical bar. As a result, I personally have seldom traded parameterless models, until I get to research my regime-switching model. That model is almost parameterless (I left out a few parameters from optimization because of a lack of time, not because of any technical difficulties).

The reason backtest optimization can now be done within a few minutes is due to my use of Alphacet Discovery's server-based optimization engine. There may be other optimization software out there that performs similar functions efficiently -- I welcome comments from the reader.

6 comments:

Anonymous said...

I think it is indeed impossible to create a "parameterless model". Even if you make them dynamic, still you always need some other parameters to govern the process of calculating the dynamic parameters.
Meanwhile a parameter models can perform really good, even the simple ones. Check some free trading models on my blog:
http://quantslibrary.blogspot.com/

Anonymous said...

I see the point and the simplicity, in my own little world I've come up with a nice set of strategies that only take one param and that is capital allocation when activating a strategy to a real account. the back testing builds metrics in terms of percentages that are universal. The dynamic or adaptive params can be adjusted on a session basis even intra session and allow for strategies to be dynamically traded that are currently performing well in the current market condition. Life goes on.

Anonymous said...

Hi Ernie,

Re: your suggestion to take a mean of top performing parameters in an optimisation.

I certainly see the logic in this approach and have a couple of questions:

What function would you suggest for determining optimal fitness? My own research suggests that trade frequency is a relevant component (as its presumably associated with statistical significance) in addition to linearity and the gradient of an equity curve.

Secondly, do you have any suggestions for selecting which variants to average? For example, taking the top 99th percentile.

Any thoughts appreciated,

David

Ernie Chan said...

Hi David,
I typically optimize the overall Sharpe ratio. I am not sure how you can optimize both gradient and linearity of the equity curve, and why you would want to do that. Is that some sort of smoothness criterion that you apply to the equity curve?
Ernie

skan said...

If the parameter is a function itself it doesn't mean it's not a parameter it just means it's a function. Changing that function will also change the results. You can also have overfitting problems.

Many times I've seen people using fixed numbers instead of parameters and telling they have few parameters, but the truth is that they have hidden parameters (sometimes they didn't optimized them, sometimes they did before)

Ernie Chan said...

Skan,
In some sense, every strategy is composed of a bunch of functions, so the choice of those functions are certainly subject to overfitting. However, the choice of functions is more constrained than the choice of parameters, so it is less of a problem for choosing functions.
Ernie