Stocks & Commodities V. 35:09 (18–21, 47): Optimization—Getting It Right by Perry J. Kaufman

Optimization—Getting It Right by Perry J. Kaufman

Creating a trading system is hard work and we want our system to work well. Oftentimes, we get carried away by trying to tweak it so it works the way we want it to. But that can lead to unrealistic results. Here’s one way to set realistic expectations when optimizing trading systems.

We all know the worst about optimization—it leads to overfitting, and overfitting means there is little chance a system will work. It’s not that the system rules are wrong. It’s that the parameters are too finely tuned. They are so closely tied to the exact patterns of the historical data that they won’t work on any data that’s different, which is the most recent data. My experience with an overfitted system is that most trades end up as losses.

Does that mean that optimization is wrong? Optimization itself is not the wrong thing to do. Rather, it’s the way that it’s done and the way the results are interpreted that could be incorrect ...