Python for Statistics: Confidence Intervals with NumPy and SciPy

Importing Libraries:

• NumPy (denoted by import numpy as np) offers fundamental functions for numerical operations and data structures.
• SciPy (denoted by from scipy import stats) provides advanced statistical functions.

Sample Data:

• You'll need your sample data as a NumPy array. This could represent measurements, survey responses, or any other quantitative data points.

Confidence Level:

• The confidence level determines how certain you want to be that the true population parameter (often the mean) falls within the interval. Common confidence levels are 90% (0.90) or 95% (0.95).

Calculations:

• Mean and Standard Deviation:

  • Calculate the mean of your sample data using np.mean(data).

• Confidence Interval Bounds:

  • The lower bound is calculated by subtracting the margin of error from the sample mean.

Interpretation:

• The confidence interval represents a range of values that are likely to contain the true population mean with a certain level of confidence (based on your chosen confidence level).

Here's an example code snippet demonstrating this process:

import numpy as np
from scipy import stats

# Sample data
data = np.random.normal(loc=50, scale=10, size=100)

# Confidence level
confidence_level = 0.95

# Calculate statistics
mean = np.mean(data)
std_dev = np.std(data, ddof=1)

# Calculate z-score
z = stats.norm.ppf(1 - (1 - confidence_level) / 2)

# Calculate margin of error
margin_of_error = z * std_dev / np.sqrt(len(data))

# Confidence interval bounds
lower_bound = mean - margin_of_error
upper_bound = mean + margin_of_error

# Print the confidence interval
print(f"Confidence Interval (CI) for population mean at {confidence_level*100}% confidence level:")
print(f"Lower bound: {lower_bound:.2f}")
print(f"Upper bound: {upper_bound:.2f}")

This code generates a random sample from a normal distribution (loc=50, scale=10) and computes the 95% confidence interval for the population mean. Remember to replace the sample data with your own data for practical applications.

Example 1: Confidence Interval for Population Mean (Normal Distribution)

This code calculates the confidence interval for the population mean assuming the data follows a normal distribution:

import numpy as np
from scipy import stats

# Sample data (replace with your actual data)
data = [52, 48, 55, 49, 51, 50, 53, 54, 57, 58]

# Confidence level
confidence_level = 0.90

# Calculate statistics
mean = np.mean(data)
std_dev = np.std(data, ddof=1)

# Calculate z-score
z = stats.norm.ppf(1 - (1 - confidence_level) / 2)

# Calculate margin of error
margin_of_error = z * std_dev / np.sqrt(len(data))

# Confidence interval bounds
lower_bound = mean - margin_of_error
upper_bound = mean + margin_of_error

# Print the confidence interval
print(f"Confidence Interval (CI) for population mean at {confidence_level*100}% confidence level:")
print(f"Lower bound: {lower_bound:.2f}")
print(f"Upper bound: {upper_bound:.2f}")

import numpy as np
from scipy import stats

# Sample data (number of successes, total trials)
successes = 20
trials = 100

# Confidence level
confidence_level = 0.95

# Calculate proportion
proportion = successes / trials

# Calculate margin of error (using normal approximation for large enough trials)
margin_of_error = stats.norm.ppf(1 - (1 - confidence_level) / 2) * np.sqrt(proportion * (1 - proportion) / trials)

# Confidence interval bounds
lower_bound = proportion - margin_of_error
upper_bound = proportion + margin_of_error

# Print the confidence interval
print(f"Confidence Interval (CI) for population proportion at {confidence_level*100}% confidence level:")
print(f"Lower bound: {lower_bound:.4f}")
print(f"Upper bound: {upper_bound:.4f}")

Remember to replace the sample data (data in example 1 and successes, trials in example 2) with your own data and choose the appropriate calculation method based on the underlying distribution of your data (normal for means, binomial for proportions).

Bootstrap Confidence Intervals:

The bootstrap method is a non-parametric approach that resamples your data with replacement to create an empirical distribution. Confidence intervals are then calculated based on this resampled distribution. This method is particularly useful when the underlying data distribution is unknown or non-normal.

Here's an example using scipt.stats.bootstrap:

import numpy as np
from scipy import stats

# Sample data (replace with your actual data)
data = [52, 48, 55, 49, 51, 50, 53, 54, 57, 58]

# Confidence level
confidence_level = 0.95

# Define the statistic function (e.g., mean)
def statistic(data):
  return np.mean(data)

# Bootstrap confidence interval
bootstrap_ci = stats.bootstrap(data, statistic=statistic, confidence_level=confidence_level)

# Print the confidence interval
print(f"Bootstrap Confidence Interval for mean at {confidence_level*100}% confidence level:")
print(f"Lower bound: {bootstrap_ci[0]:.2f}")
print(f"Upper bound: {bootstrap_ci[1]:.2f}")

Using Libraries like statsmodels:

The statsmodels library provides functions for various statistical models, including confidence interval calculations. Here's an example using statsmodels.NormalDist for confidence intervals on the mean of a normal distribution:

import numpy as np
from statsmodels import api as sm

# Sample data (replace with your actual data)
data = [52, 48, 55, 49, 51, 50, 53, 54, 57, 58]

# Confidence level
confidence_level = 0.90

# Model and fit
model = sm.NormalDist(data)
fit = model.fit()

# Confidence interval
ci = fit.conf_int(alpha=1-(confidence_level))  # alpha is 1-confidence level

# Print the confidence interval
print(f"Confidence Interval (CI) for population mean at {confidence_level*100}% confidence level:")
print(f"Lower bound: {ci[0]:.2f}")
print(f"Upper bound: {ci[1]:.2f}")

These are just two examples, and the best method depends on your specific data and assumptions. Make sure to research the appropriate method for your situation.