Letters To S&C by Technical Analysis, Inc.
NETTING AN EXPERT SYSTEM
Congratulations to Mark Fishman and Dean S. Barr for their description of a hybrid system for market
timing (STOCKS & COMMODITIES, August 1991). Their use of neural networks to generate "IF-THEN"
rules to feed an expert system truly combines the best properties of both technologies.
One shortfall of the article, however, was the lack of information provided by the authors with respect to
the predictive power of the hybrid system. It's easy to create a system that generates profitable buy/sell
signals with the benefit of hindsight. And training a neural network (or expert system) on a series of data
and then evaluating the system's trading performance on the same data is a textbook case of 20/20
One way to test the predictive power of a system is by using out-of-sample testing as described by
Thomas P. Drinka in his article "Finding reliable trading strategies (S TOCKS & COMMODITIES, May 1990).
Using this technique, the authors would begin by training the network on half of the data, say through the
second quarter of 1989. Next, trade the resulting system through the following quarter. Then retrain the
system on the new data and continue "walking forward" through to the present. This technique would
provide some meaningful data as to the system's ability to predict market behavior in the following time
Authors Fishman and Barr did mention that they trained their system on five years of data and it
accurately predicted the succeeding quarter. This is impressive in itself. But additional out-of-sample
testing is clearly in order.
CHRISTOPHER K. SMITH - Scarborough, NY