Empirical Rule

Suppose that you have a set of data that is approximately normally distributed. Suppose, also, that you know the standard deviation of the data set. Is there much you can discern about the data from this information? Well, as a matter of fact, there is quite a bit, thanks to the empirical rule.

The empirical rule can be used to judge the likelihood of certain values in a dataset, as well as to check for outliers in your data set and much more. What is the empirical rule, and how does it relate to normal distributions and standard deviations?

Definition of the Empirical Rule

The Empirical rule goes by several names, Sometimes it is called the \(95 \%\) rule, the three-sigma rule, or the \(68\)-\(95\)-\(99.7\) rule.

It is usually called the empirical rule as it is a rule informed by many observations of data sets, not a logical or definitive mathematical proof.

Where do the other names come from? Well, there's even more that the empirical rule can tell you, and the clues are in the names. It's all about percentages, and standard deviation.

Empirical Rule Percentages

As mentioned previously, one of the names for the empirical rule is the \(68\)-\(95\)-\(99.7\) rule. This name is actually quite telling when we look at the empirical rule in full. It states

For a set of normally distributed data, approximately \(68\%\) of observations fall within one standard deviation of the mean, approximately \(95\%\) of observations fall within two standard deviations of the mean, and approximately \(99.7\%\) of observations fall within three standard deviations of the mean.

\(68\%\), \(95\%\), \(99.7\%\), get it?

If you remember those three percentages, then you can use them to infer all sorts of normally distributed data sets.

But wait a minute, it's also sometimes called the three-sigma rule, why on earth is that?

Well, the symbol for standard deviation is sigma, \(\sigma\). It is sometimes called the three-sigma rule because it states that almost all observations fall within three sigmas of the mean.

It is a standard convention to consider any observations that lie outside of these three sigmas as outliers. This means that they are not typically expected observations, and aren't indicative of the overall trend. In some applications, the bar for what is considered an outlier might be explicitly stated to be something else, but three sigmas is a good rule of thumb.

Let's take a look at what all of this looks like when put into a graph.

Empirical Rule Normal Distribution Graph

Take the following normal distribution with a mean of \(m\) and a standard deviation of \(\sigma\) as an example.

Fig. 1. Normal Distribution Curve.

It's possible to divide it up according to the empirical rule.

Fig. 2. The empirical rule.

This graphical representation really demonstrates the main takeaways we can make of the empirical rule. It's very clear to see that virtually all observations fall within three standard deviations of the mean. There may very occasionally be outliers, but these are exceedingly rare.

The biggest chunk is clearly the middle \(-\sigma\) to \(\sigma\), just as the empirical rule states.

You may be thinking, 'great this rule seems useful, I'm gonna use it all the time!' But beware, and be careful. The empirical rule only holds true for data that is normally distributed.

Empirical Rule Examples

Let's take a look at some examples to see how we can put all of this into practice.