by Arthur A. Merrill, C.M.T.
Just how meaningful are statistics? Arthur Merrill explains how to find out.
If records show that market behavior exhibited more rises than declines at a certain time in the past,
could it have been by chance? Yes. If a medicine produced cures more often than average, could it have
been luck? If so, how meaningful is the record?
To determine how meaningful a particular statistic is, statisticians set up "confidence levels." If the result
in question could have occurred by chance once in 20 repetitions, you can have 95% confidence that the
result isn't just luck. This level is called "probably significant."
If the result in question could be expected by chance once in a hundred repetitions, you can have 99%
confidence. This level is called "significant."
If the result in question could be expected by chance once in a thousand repetitions, you can have 99.9%
confidence that the result wasn't a lucky record. This level is called "highly significant."
If your statistics are a simple two-way (yes/no, rises/declines, heads/tails, right/wrong), you can easily
determine the confidence level with a simple statistical test — chi squared with Yates' correction, one
degree of freedom.
The chi-squared formula can be seen in Figure 1. If the resulting chi squared is above 10.83, the
confidence level is 99.9%. If the chi-squared result is above 6.64, the confidence level is 99%. If it is
above 3.84, the level is 95%.
A sample calculation can be seen in Figure 2. If the expectation of the result seems to be about even, then
the formula is simplified. The simplified formula is in Figure 3.