V.6:7 (262-264): The value of optimization by Louis P. Lukac and B. Wade Brorsen
Product Description
The value of optimization
by Louis P. Lukac and B. Wade Brorsen
Optimization based on historical data is used widely in the futures industry to determine the most profitable parameter sets for a trading system. The underlying premise is that if some parameter set worked well in the past, it should work well in the future.
Simply picking the best parameter set from historical data and stating the accompanying profit level (called in-sample profits) is of no real value because you cannot see into the future to know what the best parameter set will be. Actual or blind simulation (out-of-sample) results are always much worse than in-sample results, which has led many to question the value of optimization.
There seems to be little agreement on how much data should be used in optimizing parameters, how frequently to re-optimize or whether optimization is even of any value at all. Even the large public futures fund managers who rely heavily on technical analysis cannot agree. A survey of public futures fund managers by B. Wade Brorsen and Scott H. Irwin in The Review of Futures Markets in 1987 showed that the data used by advisors to select parameters ranged from all the available historical data to as little as two years worth. Their frequency of re-optimization ranged from twice a year to once every five years. Several advisors did not optimize at all, they simply used the same parameter across all commodities.
Large amounts of time and money are devoted to finding the optimal parameters for a system. With more and more money being managed technically, and more software available for optimization, the need to test the validity or limits of optimization grows greater. An invalid framework could lead to invalid results and a false sense of security, not to mention the potential loss of money.
We tested various re-optimization strategies to determine if optimization is of any value for predicting correct sets of parameters and to determine if one re-optimization strategy is better than another. Specifically, we statistically compared 10 different re-optimizations to each other and to a random selection strategy using two different trading systems.
Optimization based on historical data is used widely in the futures industry to determine the most profitable parameter sets for a trading system. The underlying premise is that if some parameter set worked well in the past, it should work well in the future.
Simply picking the best parameter set from historical data and stating the accompanying profit level (called in-sample profits) is of no real value because you cannot see into the future to know what the best parameter set will be. Actual or blind simulation (out-of-sample) results are always much worse than in-sample results, which has led many to question the value of optimization.
There seems to be little agreement on how much data should be used in optimizing parameters, how frequently to re-optimize or whether optimization is even of any value at all. Even the large public futures fund managers who rely heavily on technical analysis cannot agree. A survey of public futures fund managers by B. Wade Brorsen and Scott H. Irwin in The Review of Futures Markets in 1987 showed that the data used by advisors to select parameters ranged from all the available historical data to as little as two years worth. Their frequency of re-optimization ranged from twice a year to once every five years. Several advisors did not optimize at all, they simply used the same parameter across all commodities.
Large amounts of time and money are devoted to finding the optimal parameters for a system. With more and more money being managed technically, and more software available for optimization, the need to test the validity or limits of optimization grows greater. An invalid framework could lead to invalid results and a false sense of security, not to mention the potential loss of money.
We tested various re-optimization strategies to determine if optimization is of any value for predicting correct sets of parameters and to determine if one re-optimization strategy is better than another. Specifically, we statistically compared 10 different re-optimizations to each other and to a random selection strategy using two different trading systems.
FOR THOSE ORDERING ARTICLES SEPARATELY:
*Note: $2.95-$5.95 Articles are in PDF format only. No hard copy of the article(s) will be delivered. During checkout, click the "Download Now" button to immediately receive your article(s) purchase. STOCKS & COMMODITIES magazine is delivered via mail. After paying for your subscription at store.traders.com users can view the S&C Digital Edition in the subscriber's section on Traders.com.
| Take Control of Your Trading. |
| Professional Traders' Starter Kit |
| All these items shown below only $299.99! |
| Click Here to Order |