LETTERS TO S&C
PARAMETRIC CONTROL AND SAMPLE SIZE
"Stopping a system" by John Sweeney (Settlement, December 1991) may have suggested a misleading
lesson to your readers. John introduced what appeared to be a very normal response concerning the
system he was analyzing. Profits were too small, so he decided to introduce stops to improve the results.
In fact, his change improved the system's profitability five-fold.
What John did was add a new parameter without deleting an old one or increasing the sample size. When
parameters are added to a process, freedom, in a statistical sense, is taken away. Profits are increased and
losses get smaller due to the added control. No one should expect otherwise. The trick would have been
to improve profits and reduce losses by substituting the stop control for some other buy/sell decision tool.
Analysts have been plagued for years by the discrepancy between actual trading profits and simulated
performance. The matter boils down to parametric control and sample size. At the risk of sounding
self-serving, passing an actual or simulated record through CSI'S Trading System Performance Evaluator
can help to eliminate possibilities that are likely to be unprofitable in practice.
If you take a loss on the first trade of your new system, you still won't know whether it will work on
Commodity Systems, Inc.
Boca Raton, FL
Bob's point is well taken, but my technique does not restrict degrees of freedom, if I understand what I'm
doing correctly! I made no change in the buy/sell decision rule that specifies the trade. Had I added an
additional rule to the entry decision, I would indeed need to increase the sample size. However, by
keeping the underlying rule unchanged and specifying only the worst loss to accept, I believe we evade
the issue he's raised—which is a problem in 99% of the "systems" people develop. MAE (maximum
adverse excursion) is not a trading system but a loss control or money management technique.
By the way, Bob, since we're on the topic, how about an article explaining your T SPE technique?
—John Sweeney, Technical Editor