Demonstrating the 68-95-99.7 rule in statistics using Python

Statistics provides us with powerful tools to analyze and understand data. One of the fundamental concepts in statistics is the 68-95-99.7 rule, also known as the rule of thumb or the three sigma rule. This rule allows us to make important inferences about the distribution of data based on its standard deviation. In this blog post, we will explore the 68-95-99.7 rule and demonstrate how to apply it using Python.

68-95-99.7 Rule Overview

68-95-99.7 rule provides a way to estimate the percentage of data in a normal distribution that is within a certain standard deviation from the mean. According to this rule -

• Approximately 68% of the data falls within one standard deviation of the mean.

• Approximately 95% of the data falls within two standard deviations of the mean.

• Approximately 99.7% of the data falls within three standard deviations of the mean.

These percentages are for data sets that follow a normal distribution (also called a bell curve). Understanding this rule allows us to quickly assess the spread of data and identify outliers or unusual observations.

Implementing the 68-95-99.7 rule in Python

To demonstrate the 68-95-99.7 rule in action, we will use Python and its popular data analysis library NumPy. NumPy provides efficient numerical operations and statistical functions to help us calculate necessary values. Let’s first import the required libraries−

import numpy as np
import matplotlib.pyplot as plt

Next, we will use the numpy.random.normal() function to generate a random data set that follows a normal distribution. We will use mean 0 and standard deviation 1 −

np.random.seed(42)  # Set the random seed for reproducibility
data = np.random.normal(0, 1, 10000)

Now, we can calculate the mean and standard deviation of the data set−

mean = np.mean(data)
std = np.std(data)

To visualize the data and the area covered by the 68-95-99.7 rule, we can create a histogram using the matplotlib.pyplot.hist() function −

plt.hist(data, bins=30, density=True, alpha=0.7)

# Plot the mean and standard deviations
plt.axvline(mean, color='r', linestyle='dashed', linewidth=1, label='Mean')
plt.axvline(mean - std, color='g', linestyle='dashed', linewidth=1, label='1 STD')
plt.axvline(mean + std, color='g', linestyle='dashed', linewidth=1)
plt.axvline(mean - 2*std, color='b', linestyle='dashed', linewidth=1, label='2 STD')
plt.axvline(mean + 2*std, color='b', linestyle='dashed', linewidth=1)
plt.axvline(mean - 3*std, color='m', linestyle='dashed', linewidth=1, label='3 STD')
plt.axvline(mean + 3*std, color='m', linestyle='dashed', linewidth=1)

plt.legend()
plt.xlabel('Value')
plt.ylabel('Density')
plt.title('Histogram of the Dataset')
plt.show()

The resulting histogram will show the distribution of the data with the mean and standard deviation marked with dashed lines.

To calculate the percentage coverage of each range, we can use the cumulative distribution function (CDF) of the normal distribution. The NumPy function numpy.random.normal() generates normally distributed data, but NumPy also provides numpy.random.normal() to calculate CDF −

# Calculate the percentage within one standard deviation
pct_within_1_std = np.sum(np.logical_and(data >= mean - std, data 7lt;= mean + std)) / len(data)

# Calculate the percentage within two standard deviations
pct_within_2_std = np.sum(np.logical_and(data >= mean - 2*std, data <= mean + 2*std)) / len(data)

# Calculate the percentage within three standard deviations
pct_within_3_std = np.sum(np.logical_and(data >= mean - 3*std, data <= mean + 3*std)) / len(data)

print("Percentage within one standard deviation: {:.2%}".format(pct_within_1_std))
print("Percentage within two standard deviations: {:.2%}".format(pct_within_2_std))
print("Percentage within three standard deviations: {:.2%}".format(pct_within_3_std))

When you run this code, you will see the percentage of your data that falls within 1, 2, and 3 standard deviations of the mean.

Percentage within one standard deviation: 68.27%
Percentage within two standard deviations: 95.61%
Percentage within three standard deviations: 99.70%

These results are very consistent with the expected percentages for the 68-95-99.7 rule.

68-95-99.7 Explanation of rules

The percentage covered by each range has a specific interpretation. Data that fall within one standard deviation of the mean are relatively common, while data that fall outside three standard deviations of the mean are considered rare. Understanding these explanations helps make meaningful inferences about the data.

68-95-99.7 Rule Limitations

While the 68-95-99.7 rule is a valuable guideline, it may not apply accurately to data sets that deviate significantly from the normal distribution. When working with such data sets, it is crucial to consider other statistical techniques and conduct further analysis.

Outliers and the 68-95-99.7 Rule

Outliers can greatly affect the accuracy of the percentage covered by each range. These extreme values ​​may distort the distribution and affect the effectiveness of the rules. Proper identification and handling of outliers is important to ensure accurate statistical analysis.

Real life examples

68-95-99.7 Rules apply in all areas. For example, it is critical in identifying defective products in quality control processes, in assessing risk and return on investment in financial analysis, in understanding patient characteristics in healthcare research, and in understanding data distributions in many other areas.

As you dig deeper into the statistics, consider exploring other concepts that supplement the 68-95-99.7 rule. Skewness, kurtosis, confidence intervals, hypothesis testing, and regression analysis are just a few examples of statistical tools that can further enhance your understanding and analysis of your data.

in conclusion

68-95-99.7 Rules are a powerful concept in statistics that allow us to understand the distribution of data based on its standard deviation. By applying this rule, we can estimate the proportion of data that falls within a specific range around the mean. In this blog, we use Python and the NumPy library to generate a random dataset, visualize it, and calculate the percentage coverage of each range. Understanding this rule allows us to make meaningful inferences about the data and identify potential outliers or unusual observations.

The above is the detailed content of Demonstrating the 68-95-99.7 rule in statistics using Python. For more information, please follow other related articles on the PHP Chinese website!