When the population standard deviation is unknown, the one-sample t-test is generally used:

t = (x̄ − μ₀) / (s / √n)

The t-test uses n − 1 degrees of freedom.

5.11 Worked Example: One-Sample z-Test

5.12 Worked Example: One-Sample t-Test

5.13 One-Sample Test for a Population Proportion

A population proportion test is used when the variable of interest represents a binary outcome, such as success/failure, defective/non-defective, employed/unemployed, or yes/no.

The sample proportion is:

p̂ = x / n

where x is the number of observations possessing the characteristic and n is the sample size.

For a one-sample proportion z-test, the test statistic is:

z = (p̂ − p₀) / √[p₀(1 − p₀)/n]

Notice that the hypothesised proportion p₀, rather than the sample proportion, is used in the standard error when constructing the hypothesis test statistic.

5.14 Worked Example: One-Sample Proportion Test

5.15 Critical-Value and p-Value Approaches

There are two common approaches to hypothesis testing.

Critical-Value Approach:

The calculated test statistic is compared with a critical value obtained from the relevant probability distribution.

For example, in a two-tailed z-test at α = 0.05:

Reject H₀ if |z| > 1.96.

p-Value Approach:

The calculated p-value is compared directly with α.

Reject H₀ if p-value ≤ α.

When the assumptions are satisfied, both approaches lead to the same statistical decision.

5.16 Assumptions and Conditions

A statistical test should not be selected merely because a formula is available. The assumptions underlying the method must also be considered.

For a one-sample mean test, relevant considerations include:

• the observations should be obtained using an appropriate sampling method;
• observations should be independent when independence is required;
• the population distribution should be reasonably appropriate for a t-test when the sample is small;
• extreme outliers should be investigated; and
• the standard deviation used in the calculation must correspond to the chosen test.

For a one-sample proportion test, the sample should be appropriate for the binomial setting, and the normal approximation should be reasonable. A common rule is to check that both np₀ and n(1 − p₀) are sufficiently large.

5.17 Common Mistakes in Hypothesis Testing

• Writing the alternative hypothesis incorrectly: The alternative must reflect the research question.
• Using the wrong test statistic: A z-test and t-test do not have identical assumptions.
• Using the sample proportion in the null standard error: For the standard one-sample proportion z-test, p₀ is used.
• Confusing statistical significance with practical importance: A statistically significant difference may be too small to have meaningful practical consequences.
• Interpreting the p-value as the probability that H₀ is true: This is incorrect.
• Ignoring the direction of the alternative hypothesis: One-tailed and two-tailed tests use different rejection regions.
• Claiming that failure to reject proves H₀: It only means that the evidence was insufficient to reject it under the chosen procedure.
• Reporting only a numerical decision: A proper conclusion should be stated in the context of the research problem.

5.18 Key Points

• Hypothesis testing evaluates claims about population parameters using sample evidence.
• The null hypothesis represents the assumed condition being tested.
• The alternative hypothesis represents the competing claim.
• The significance level determines the maximum tolerated probability of a Type I error.
• A p-value measures how unusual the observed evidence would be under H₀.
• For p-value ≤ α, H₀ is rejected.
• For p-value > α, H₀ is not rejected.
• One-sample mean tests may use z or t statistics depending on the assumptions and information available.
• A one-sample proportion test uses the hypothesised population proportion in its standard error.
• Statistical conclusions should always be interpreted in the context of the original problem.

6.1 Introduction

Many statistical investigations involve comparing two groups rather than examining a single population. A researcher may want to know whether two teaching methods produce different average examination scores, whether two manufacturing processes have different mean output levels, or whether the proportion of customers preferring one product differs from the proportion preferring another.

Two-sample hypothesis testing provides statistical procedures for determining whether an observed difference between two samples is sufficiently large to support a conclusion about the corresponding populations.

The central principle is important: two sample statistics will almost always differ to some degree simply because samples vary. Statistical inference determines whether the observed difference is larger than would reasonably be expected from sampling variation alone.

6.2 Comparing Two Population Means

Suppose two populations have means μ₁ and μ₂. A common question is whether the population means are equal.

The null hypothesis may be:

H₀: μ₁ = μ₂

which can equivalently be written as:

H₀: μ₁ − μ₂ = 0

A two-sided alternative is:

H_a: μ₁ ≠ μ₂

For a one-sided test, the alternative may instead be μ₁ > μ₂ or μ₁ < μ₂.

6.3 Difference Between Two Sample Means

The observed difference between two independent sample means is:

x̄₁ − x̄₂

If the null hypothesis states that the population means are equal, then the expected population difference is zero.

The test asks whether the observed difference is sufficiently large relative to its standard error.

For independent samples with population standard deviations known, the standardised statistic is:

z = [(x̄₁ − x̄₂) − (μ₁ − μ₂)] / √(σ₁²/n₁ + σ₂²/n₂)

When the population standard deviations are unknown, t-based methods are generally used.

6.4 Independent Two-Sample t-Test

The independent two-sample t-test is commonly used to compare the means of two independent populations when population standard deviations are unknown.

The general form of the test statistic is:

t = [(x̄₁ − x̄₂) − Δ₀] / SE(x̄₁ − x̄₂)

where Δ₀ is the hypothesised difference between the population means. In the common equal-means test, Δ₀ = 0.

When the population variances are not assumed to be equal, Welch's two-sample t-test is generally preferred because it does not require the equal-variance assumption.

6.5 Pooled and Welch Approaches

In practical statistical computing, Welch's method is often a useful default for independent two-sample mean comparisons because it avoids requiring equal population variances.

6.6 Worked Example: Two-Sample Comparison of Means

6.7 Paired and Independent Samples

Students must distinguish between independent samples and paired samples.

Two samples are independent when observations in one sample do not naturally correspond to particular observations in the other sample.

Paired samples arise when observations are naturally matched. Examples include:

• measurements taken from the same individual before and after treatment;
• the same machine measured before and after maintenance;
• matched individuals in two experimental conditions.

For paired data, the analysis is usually performed on the differences within each pair rather than treating the two sets of observations as independent.

6.8 Comparing Two Population Proportions

Two-sample proportion tests are used when the objective is to compare the proportions of a characteristic in two populations.

Let p₁ and p₂ represent the population proportions.

A common null hypothesis is:

H₀: p₁ = p₂

or equivalently:

H₀: p₁ − p₂ = 0

The alternative hypothesis may be:

H_a: p₁ ≠ p₂

for a two-sided test, or it may specify one direction.

6.9 Sample Proportions

For each sample, the sample proportion is calculated as:

p̂₁ = x₁/n₁

and

p̂₂ = x₂/n₂

The observed difference is:

p̂₁ − p̂₂

Under the null hypothesis that p₁ = p₂, the two samples can be combined to obtain a pooled estimate of the common population proportion.

The pooled proportion is:

p̂ = (x₁ + x₂) / (n₁ + n₂)

The standard error under H₀ is:

SE = √[p̂(1 − p̂)(1/n₁ + 1/n₂)]

The test statistic is:

z = (p̂₁ − p̂₂) / SE

6.10 Worked Example: Two-Sample Proportion Test

6.11 Interpreting Statistical Outcomes

A statistical decision should not be presented without an interpretation. The conclusion must connect the mathematical result to the original research question.

Example of a Poor Conclusion:

“Reject H₀ because z = 2.41.”

This statement gives the decision but does not explain what it means.

Better Conclusion:

“Because the test statistic falls in the rejection region, the null hypothesis is rejected at the 5% significance level. There is sufficient evidence to conclude that the two population means differ.”

A strong statistical conclusion should identify:

• the significance level;
• the statistical decision;
• the direction or nature of the evidence where appropriate; and
• the practical context of the investigation.

6.12 Statistical Significance Versus Practical Significance

A statistically significant difference is not automatically an important real-world difference.

For example, a very large sample may detect a difference of only 0.2 units and produce a very small p-value. The result may be statistically significant while having little practical importance.

Conversely, an important difference may fail to reach statistical significance when the sample size is small or the data are highly variable.

Researchers should therefore consider both statistical evidence and the practical magnitude of the observed difference.

6.13 Confidence Intervals for Two-Sample Comparisons

Hypothesis testing can be complemented by confidence intervals. Instead of only asking whether two population parameters differ, a confidence interval estimates the plausible size of the difference.

For example, suppose a 95% confidence interval for μ₁ − μ₂ is:

(1.5, 6.2)

Because zero is not contained in the interval, the result is consistent with rejecting H₀: μ₁ − μ₂ = 0 at the corresponding 5% two-sided significance level, assuming the interval and test use compatible procedures.

If the confidence interval were:

(−1.2, 5.4)

zero would be included, indicating that equality of the population means remains plausible at the corresponding significance level.

6.14 Statistical Computing for Two-Sample Tests

Modern statistical analysis frequently uses software to calculate test statistics, degrees of freedom, p-values and confidence intervals. Software does not eliminate the need for statistical reasoning. The researcher must still identify the correct test, verify assumptions and interpret the output correctly.

For example, in R, an independent two-sample t-test can be performed using:

t.test(group1, group2)

By default, R's t.test() function uses Welch's two-sample t-test for independent samples.

A two-sample proportion test can be performed using functions such as prop.test(), depending on the specific analysis and assumptions.

The important principle is that statistical software should be treated as a computational tool rather than a substitute for understanding the statistical method.

6.15 Common Mistakes in Two-Sample Testing

• Ignoring whether samples are paired or independent: The sampling structure determines the appropriate method.
• Assuming equal variances without justification: The pooled t-test requires an equal-variance assumption.
• Comparing sample means without considering sampling variability: A difference between sample means does not automatically imply a population difference.
• Using the wrong standard error: Mean comparisons and proportion comparisons use different standard-error formulas.
• Forgetting the direction of the alternative hypothesis: A two-tailed test is different from a one-tailed test.
• Interpreting a p-value as an effect size: The p-value describes evidence against H₀; it does not directly measure the practical magnitude of an effect.
• Reporting only “significant” or “not significant”: A useful conclusion should include the estimated difference and its context where possible.

6.16 Key Points

• Two-sample tests determine whether differences observed between two samples provide evidence of differences between two populations.
• Population means can be compared using appropriate z- or t-based procedures.
• Welch's t-test does not require the two population variances to be equal.
• The pooled t-test assumes equal population variances.
• Paired samples require different treatment from independent samples.
• Two population proportions can be compared using a two-sample proportion test.
• Under the null hypothesis of equal proportions, a pooled proportion is used in the standard two-proportion z-test.
• Statistical significance should not be confused with practical importance.
• Confidence intervals provide information about the plausible magnitude of a population difference.
• Statistical software can perform calculations efficiently, but the analyst remains responsible for choosing and interpreting the method.

6.17 Chapter Review

Two-sample hypothesis testing extends the principles of statistical inference to situations in which two populations or groups must be compared. The central question is whether the observed difference between sample statistics can reasonably be attributed to sampling variation or provides evidence of a genuine population difference.

For population means, the analysis depends on the structure of the samples and assumptions concerning population variability. Independent samples are commonly analysed using two-sample t procedures, with Welch's method providing a robust approach when equal variances cannot be assumed. Paired observations require analysis of within-pair differences.

For population proportions, the two-sample proportion test compares the observed proportions using a standard error based on the null hypothesis. In both mean and proportion comparisons, the statistical decision must ultimately be translated into a clear statement about the research problem.

The most important principle is that statistical testing is not simply a calculation. A sound analysis requires an appropriate research question, correctly specified hypotheses, suitable data, valid assumptions, an appropriate test statistic, correct interpretation of the p-value or critical value, and a conclusion stated in the context of the investigation.

Core Formula Reference:

One-sample mean z-test:

z = (x̄ − μ₀) / (σ / √n)

One-sample mean t-test:

t = (x̄ − μ₀) / (s / √n)

One-sample proportion test:

z = (p̂ − p₀) / √[p₀(1 − p₀)/n]

Two-sample mean comparison:

t = [(x̄₁ − x̄₂) − Δ₀] / SE(x̄₁ − x̄₂)

Two-sample pooled proportion:

p̂ = (x₁ + x₂) / (n₁ + n₂)

Two-sample proportion z-test:

z = (p̂₁ − p̂₂) / √[p̂(1 − p̂)(1/n₁ + 1/n₂)]

Chapter Seven: Regression and Correlation

Understanding relationships between variables, measuring association, building regression models, and interpreting statistical results.

7.1 Understanding Relationships Between Variables

A relationship between variables exists when changes in one variable are associated with systematic changes in another variable. Statistical methods allow researchers to determine whether such relationships exist and, where appropriate, quantify them.

Consider the following examples:

• Hours spent studying and examination scores.
• Temperature and electricity consumption.
• Advertising expenditure and product sales.
• Age and resting heart rate.
• Rainfall and agricultural yield.
• Household income and household expenditure.

In each case, the researcher has at least two variables. The purpose of statistical analysis is not simply to observe that the variables change, but to determine whether their changes are systematically related.

7.2 Correlation

Correlation is a statistical technique used to measure the degree to which two quantitative variables are linearly associated.

Correlation answers questions such as:

• Do the two variables tend to increase together?
• Does one variable tend to increase when the other decreases?
• Is there little or no linear relationship between them?
• How strong is the observed linear relationship?

The most commonly used measure for linear correlation between two quantitative variables is the Pearson product-moment correlation coefficient, usually represented by r.

The Correlation Coefficient:

The Pearson correlation coefficient is defined as:

where:

• x = an observed value of the first variable;
• y = an observed value of the second variable;
• x̄ = mean of the first variable;
• ȳ = mean of the second variable; and
• r = Pearson correlation coefficient.

The value of r always lies between −1 and +1:

7.3 Direction of Correlation

Positive Correlation:

A positive correlation occurs when larger values of one variable tend to be associated with larger values of the other variable.

For example, suppose students who spend more hours studying generally obtain higher examination scores. If the relationship is approximately linear, the two variables will have a positive correlation.

As x increases, y tends to increase.

Negative Correlation:

A negative correlation occurs when larger values of one variable tend to be associated with smaller values of the other variable.

For example, suppose fuel consumption per kilometre decreases as vehicle efficiency increases. If higher efficiency values are associated with lower fuel consumption, the variables have a negative association.

As x increases, y tends to decrease.

Zero or Weak Linear Correlation:

A correlation coefficient close to zero indicates little or no linear association. It is important to emphasize the word linear. A correlation close to zero does not necessarily mean that the variables have no relationship at all. A strong curved relationship, for example, may have a Pearson correlation close to zero.

7.4 Strength of Correlation

The strength of a correlation is determined primarily by the magnitude of r, that is, by how close the absolute value of r is to 1.

For example:

• r = 0.91 indicates a strong positive linear association.
• r = 0.62 indicates a moderate positive linear association.
• r = −0.88 indicates a strong negative linear association.
• r = −0.18 indicates a weak negative linear association.
• r = 0.03 indicates almost no linear association.

There is no single universally applicable set of numerical boundaries for "weak," "moderate," and "strong" correlation. Interpretation should consider the subject area, sample size, measurement quality and purpose of the analysis.

7.5 Worked Example: Calculating Correlation

7.6 Scatter Plots and Correlation

A scatter plot is one of the most useful graphical tools for examining the relationship between two quantitative variables. Each observation is represented by a point whose horizontal coordinate is the value of one variable and whose vertical coordinate is the value of the other.

A scatter plot can reveal:

• the direction of a relationship;
• the approximate strength of a relationship;
• nonlinear patterns;
• outliers;
• clusters of observations; and
• changes in variability across the range of a variable.

A scatter plot should normally be examined before interpreting a correlation coefficient. A numerical value alone can hide important features of the data.

7.7 Correlation Does Not Imply Causation

This is one of the most important principles in statistical analysis.

Suppose a study finds a strong positive correlation between the number of ice creams sold and the number of people visiting a beach. It would be incorrect to conclude that buying ice cream causes people to visit the beach.

A third variable, such as temperature, may influence both quantities. Higher temperatures may cause more people to visit beaches and may also increase ice cream sales.

Thus, an observed association can arise because:

• one variable affects the other;
• the second variable affects the first;
• a third variable affects both;
• the association occurs by chance; or
• the relationship is influenced by the way the data were collected.

7.8 Regression Analysis

Regression analysis is a statistical method used to describe and quantify the relationship between a response variable and one or more explanatory variables.

Regression has two closely related purposes:

1. Explanation: understanding how the response variable changes as an explanatory variable changes.
2. Prediction: estimating the response for given values of the explanatory variable.

In the simplest case, there are two variables:

• Independent or explanatory variable, x: the variable used to explain or predict.
• Dependent or response variable, y: the variable being explained or predicted.

For example, if a researcher studies how advertising expenditure affects sales, advertising expenditure may be treated as x, while sales may be treated as y.

7.9 Simple Linear Regression Model

The simple linear regression model describes the relationship between one response variable and one explanatory variable.

The population model is commonly written as:

where:

• Y = response variable;
• X = explanatory variable;
• β₀ = population intercept;
• β₁ = population slope;
• ε = random error term.

The fitted sample regression equation is written as:

Here, ŷ represents the predicted value of the response variable.

The Intercept:

The intercept b₀ is the predicted value of y when x = 0.

The intercept may or may not have a meaningful practical interpretation. If a value of zero for x is outside the range or context of the data, the intercept should not be given an unrealistic interpretation.

The Slope:

The slope b₁ represents the estimated change in the response variable for a one-unit increase in the explanatory variable.

For example, if a regression equation is:

then the slope is 5. For every one-unit increase in x, the predicted value of y increases by 5 units.

7.10 Estimating the Regression Coefficients

The least-squares method is commonly used to estimate the coefficients of a simple linear regression model.

The slope is calculated using:

The intercept is then calculated as:

The least-squares regression line is the line that minimizes the sum of the squared differences between the observed values and their corresponding predicted values.

7.11 Worked Example: Fitting a Regression Line

7.12 Using a Regression Equation for Prediction

Once a regression equation has been obtained, it can be used to calculate predicted values.

The predicted value is not necessarily the actual score of the student. It is the value supplied by the fitted statistical model.

7.13 Residuals

A regression model rarely predicts every observation perfectly. The difference between the actual observed value and the value predicted by the regression equation is called a residual.

Using the regression equation:

Suppose a student studied for 6 hours and obtained an actual score of 70.

The predicted score is:

Therefore:

Now suppose another student studied for 3 hours and obtained a score of 60.

The predicted score is:

Therefore:

A positive residual means that the observed value is above the predicted value. A negative residual means that the observed value is below the predicted value.

7.14 The Coefficient of Determination

The coefficient of determination, denoted by R², measures the proportion of variation in the response variable that is explained by the fitted regression model.

In simple linear regression, when Pearson correlation is used, the coefficient of determination is related to the correlation coefficient by:

For example, if:

then:

Therefore, the model explains approximately 64% of the variation in the response variable through its linear relationship with the explanatory variable.

The remaining 36% represents variation not explained by this simple linear relationship and may be associated with other variables, measurement error, random variation, or model limitations.

7.15 Regression and Correlation Compared

Although regression and correlation are closely related, they answer different questions and should not be treated as identical techniques.

7.16 Assumptions of Simple Linear Regression

Statistical conclusions from a regression model depend on assumptions about the data and the underlying process. Important assumptions of ordinary simple linear regression include the following.

1. Linearity:

The mean relationship between the explanatory variable and response variable should be adequately represented by a straight line.

2. Independence:

The observations or errors should be appropriately independent of one another. Dependence can occur in time-series data, repeated measurements and clustered observations.

3. Constant Variance:

The variability of the errors should remain reasonably constant across the relevant range of the explanatory variable. Unequal variance is commonly referred to as heteroscedasticity.

4. Normality of Errors:

For many small-sample inferential procedures involving regression coefficients, the error terms are assumed to be approximately normally distributed. This assumption is about the errors, not necessarily about the explanatory variable itself.

5. Appropriate Data Structure:

The variables should be measured appropriately, and important features such as outliers and influential observations should be investigated.

7.17 Interpreting Regression Results

Consider the following fitted regression model:

Suppose x represents hours of training and y represents productivity units per day.

The intercept is 12.5. Mathematically, the model predicts 12.5 productivity units when training hours are zero. Whether that interpretation is practically meaningful depends on whether zero training hours is relevant to the population being studied.

The slope is 2.8. Therefore, each additional hour of training is associated with an estimated increase of 2.8 productivity units per day in the predicted response.

If the model has R² = 0.72, then approximately 72% of the observed variation in productivity is explained by the fitted linear relationship with training hours.

7.18 Statistical Significance in Regression and Correlation

In statistical inference, researchers may wish to determine whether an observed relationship provides sufficient evidence of an association in the underlying population.

For example, a hypothesis concerning a population correlation may be expressed as:

Similarly, in simple regression, a hypothesis may be formulated about the population slope:

If the null hypothesis about the slope is rejected, there is statistical evidence that the explanatory variable is linearly associated with the response in the population represented by the study.

Statistical significance should not be confused with practical importance. A very small effect may become statistically significant in a sufficiently large sample, while an important effect may fail to reach statistical significance in a small or highly variable sample.

7.19 Outliers and Influential Observations

An outlier is an observation that differs substantially from the general pattern of the data. Outliers can have a strong effect on correlation and regression results.

For example, suppose most observations show a positive relationship between income and expenditure, but one observation has an extremely high income and an unusually low expenditure. That observation may substantially change the estimated regression line.

Researchers should therefore inspect scatter plots and residuals rather than relying exclusively on numerical output.

7.20 Limitations of Regression and Correlation

• Association is not causation: A strong correlation does not establish a causal relationship.
• Correlation measures linear association: Pearson correlation can fail to capture important nonlinear relationships.
• Outliers can distort results: A few unusual observations can substantially affect both correlation and regression estimates.
• Extrapolation can be dangerous: Predictions made far outside the range of observed explanatory-variable values may be unreliable.
• Omitted variables may matter: A simple regression model may exclude variables that influence the response.
• Measurement error matters: Poorly measured variables can weaken or distort observed relationships.
• Model assumptions may fail: Nonlinearity, unequal variance or dependence among observations can make standard regression methods inappropriate or misleading.

7.22 Applications of Regression and Correlation

Regression and correlation are used extensively across academic disciplines, industry and government because many real-world decisions involve relationships between variables.

Business and Economics:

Businesses may investigate relationships between advertising expenditure and sales, price and demand, income and consumption, or production volume and operating costs.

Engineering:

Engineers can use regression to model relationships between operating conditions and system performance, estimate equipment behaviour and develop empirical models from experimental observations.

Healthcare:

Researchers may examine relationships between physiological measurements, treatment variables and health outcomes. Regression models can also be used to predict outcomes from several explanatory variables.

Agriculture:

Regression can be used to study relationships between rainfall, fertilizer application, soil characteristics and crop yield.

Education:

Researchers may investigate relationships between attendance, study time, prior achievement and examination performance.

Data Science:

Regression is a fundamental modelling technique in data science. It is used for prediction, estimation, exploratory analysis and as a foundation for more advanced statistical and machine-learning methods.

7.23 Worked Interpretation of a Complete Regression Result

7.24 Key Points

• Correlation measures the strength and direction of linear association between two quantitative variables.
• The Pearson correlation coefficient, r, ranges from −1 to +1.
• A positive correlation means that larger values of one variable tend to be associated with larger values of the other.
• A negative correlation means that larger values of one variable tend to be associated with smaller values of the other.
• The magnitude of the correlation coefficient indicates the strength of the linear association.
• A correlation close to zero does not necessarily mean that there is no relationship; a nonlinear relationship may exist.
• Correlation does not establish causation.
• Regression models the relationship between a response variable and one or more explanatory variables.
• Simple linear regression uses one explanatory variable and one response variable.
• The simple regression equation is commonly written as ŷ = b₀ + b₁x.
• The intercept represents the predicted response when the explanatory variable is zero.
• The slope represents the estimated change in the response associated with a one-unit increase in the explanatory variable.
• Residuals are calculated as observed value minus predicted value.
• The coefficient of determination, R², measures the proportion of variation in the response explained by the regression model.
• Scatter plots are important for examining the form and quality of relationships between variables.
• Outliers and influential observations can substantially affect regression and correlation results.
• Regression models depend on assumptions such as linearity, independence and appropriate error behaviour.
• Regression and correlation are widely applied in business, engineering, healthcare, agriculture, education and data science.

7.25 Chapter Summary

Regression and correlation provide fundamental statistical tools for studying relationships between variables. Correlation focuses on measuring the direction and strength of linear association, while regression develops an explicit mathematical model that can be used to explain relationships and generate predictions.

The Pearson correlation coefficient r ranges from −1 to +1. Its sign indicates the direction of the linear association, while its magnitude indicates the strength. A scatter plot should normally accompany correlation analysis because graphical examination can reveal patterns, curvature and unusual observations that a single numerical coefficient may conceal.

Simple linear regression expresses the relationship between a response variable and one explanatory variable through the model ŷ = b₀ + b₁x. The slope indicates the estimated change in the response associated with a one-unit increase in the explanatory variable, while the intercept represents the predicted response when the explanatory variable is zero.

The least-squares method is commonly used to estimate regression coefficients. Once a regression equation has been obtained, it can be used to calculate predicted values, while residuals provide a measure of the difference between observed and predicted values.

The coefficient of determination, R², provides an important measure of model explanatory power. In simple linear regression, R² is equal to the square of the Pearson correlation coefficient. Nevertheless, a high R² does not establish causation and does not guarantee that the model is appropriate in every context.

Effective regression and correlation analysis therefore requires more than calculating a coefficient. The analyst must understand the variables, inspect the data graphically, consider assumptions, identify unusual observations, interpret the numerical results in context and avoid conclusions that go beyond the evidence.

8.1 Chapter Introduction:

Many statistical observations are collected repeatedly over time. A business may record monthly sales, a government may record annual population figures, a hospital may record the number of patients treated each week, and a financial institution may record daily exchange rates. When observations are arranged according to time, they form a time series.

Time series analysis provides statistical methods for examining how a variable changes over time and for identifying systematic patterns such as long-term growth, seasonal variation and irregular movements. These methods are useful for describing historical behaviour, supporting planning and making forecasts.

Another important statistical tool is the index number. Index numbers provide a convenient way of measuring relative changes in variables such as prices, quantities, production, wages and living costs. Instead of comparing large numerical values directly, an index expresses change relative to a selected reference period.

8.2 Learning Objectives:

At the end of this chapter, the student should be able to:

• define a time series and identify its major components;
• explain the importance of time series analysis;
• distinguish between trend, seasonal, cyclical and irregular movements;
• calculate and interpret simple measures of time series change;
• explain moving averages and their application;
• define index numbers and explain their purpose;
• calculate simple price and quantity indices;
• distinguish between weighted and unweighted index numbers;
• interpret commonly used index numbers;
• identify limitations associated with time series and index-number methods; and
• apply elementary time series and index-number techniques to practical problems.

8.3 Key Terminology:

• Time series: A set of observations on a variable recorded in chronological order.
• Trend: The long-term general direction of movement in a time series.
• Seasonal variation: A systematic pattern that repeats within a fixed period, such as a year.
• Cyclical variation: Long-term fluctuations around a trend that are commonly associated with economic or business cycles.
• Irregular variation: Unpredictable movements caused by unusual or random events.
• Moving average: An average calculated over successive groups of observations to smooth short-term fluctuations.
• Index number: A statistical measure that expresses the relative change in a variable or group of variables compared with a reference period.
• Base period: The period selected as the reference for an index number.
• Price index: An index measuring changes in prices.
• Quantity index: An index measuring changes in quantities.
• Weighted index: An index in which different items receive different levels of importance or weight.

8.4 Meaning of Time Series

A time series is a sequence of observations collected for the same variable at different points or periods of time and arranged in chronological order.

For example, the following table represents the annual sales of a company:

The observations are a time series because sales are measured repeatedly and arranged chronologically.

Time series may be recorded at different frequencies, including:

• hourly;
• daily;
• weekly;
• monthly;
• quarterly; and
• annually.

The appropriate frequency depends on the phenomenon being studied. Electricity demand may be studied hourly, while population growth is commonly studied annually.

8.5 Importance of Time Series Analysis

Time series analysis is important because observations collected over time often contain information that cannot be understood by examining isolated observations.

Understanding historical behaviour: A time series allows researchers to determine whether a variable has generally increased, decreased or remained relatively stable.

Forecasting: Historical patterns can be used to develop forecasts of future observations, although forecasts are always subject to uncertainty.

Planning: Businesses and governments can use historical patterns to support decisions concerning production, staffing, inventory and resource allocation.

Monitoring: Time series can help identify unusual changes. For example, a sudden decline in hospital attendance may require investigation.

Policy evaluation: A government may compare a variable before and after a policy intervention to determine whether substantial changes occurred.

8.6 Components of a Time Series

A time series may contain several types of movement. The four classical components are trend, seasonal variation, cyclical variation and irregular variation.

8.6.1 Trend:

The trend represents the long-term general direction of a series.

A trend may be:

• upward;
• downward; or
• approximately constant.

For example, if a company's annual revenue increases from ₦20 million to ₦35 million over several years despite occasional declines, the series may have an upward trend.

8.6.2 Seasonal Variation:

Seasonal variation consists of movements that recur at approximately the same time within a fixed period, usually within a year.

Examples include:

• increased retail sales during festive periods;
• higher demand for certain agricultural products during particular seasons;
• increased hotel occupancy during holiday periods; and
• changes in electricity demand associated with recurring weather conditions.

8.6.3 Cyclical Variation:

Cyclical variation refers to fluctuations occurring over longer periods and often associated with expansions and contractions in economic activity.

A business may experience increasing sales during economic expansion and declining sales during an economic downturn. Such movements may extend over several years rather than repeating at a fixed annual interval.

8.6.4 Irregular Variation:

Irregular variation consists of unpredictable movements caused by unusual events.

Examples include natural disasters, major strikes, unexpected supply disruptions, political crises or other exceptional events.

8.7 Time Series Models

The components of a time series can be represented using different models.

In an additive model, the observed value is represented conceptually as:

Y = T + S + C + I

where:

• Y = observed value;
• T = trend;
• S = seasonal variation;
• C = cyclical variation; and
• I = irregular variation.

In a multiplicative model, the components are represented as proportions or factors:

Y = T × S × C × I

The additive model is appropriate when the size of fluctuations remains relatively constant, while the multiplicative model is often useful when the size of fluctuations changes in proportion to the level of the series.

8.8 Measuring Change in a Time Series

A simple way of examining a time series is to calculate the absolute and relative changes between observations.

Absolute Change:

The absolute change is:

Absolute Change = Current Value − Previous Value

Percentage Change:

The percentage change is:

Percentage Change = [(Current Value − Previous Value) / Previous Value] × 100

8.9 Moving Averages

A moving average is an average calculated from a fixed number of consecutive observations and then moved forward through the series.

Moving averages are useful for reducing short-term fluctuations and making the underlying trend easier to observe.

For a three-period moving average:

Moving Average = (Y₁ + Y₂ + Y₃) / 3

8.10 Meaning of Index Numbers

An index number is a statistical measure used to express the relative change in the value of a variable or group of variables between two periods or situations.

The period selected for comparison is called the base period. The base-period index is normally assigned a value of 100.

If an index rises from 100 to 120, the variable has increased by 20% relative to the base period.

If an index falls from 100 to 85, the variable has decreased by 15% relative to the base period.

8.11 Types of Index Numbers

Index numbers may be classified according to what they measure.

8.11.1 Price Index:

A price index measures changes in prices between two periods.

Examples include consumer price measures, producer price measures and other specialized price indices.

8.11.2 Quantity Index:

A quantity index measures changes in quantities such as production, sales volume or consumption.

8.11.3 Value Index:

A value index measures changes in total monetary value. Since value is generally price multiplied by quantity, changes in a value index can reflect changes in both prices and quantities.

8.12 Simple Price Index

A simple price index for a single commodity is calculated as:

P₁₀ = (P₁ / P₀) × 100

where P₁ is the current-period price and P₀ is the base-period price.

8.13 Simple Quantity Index

The simple quantity index is:

Q₁₀ = (Q₁ / Q₀) × 100

where Q₁ is the current-period quantity and Q₀ is the base-period quantity.

8.14 Simple Aggregate Price Index

When several commodities are involved, their prices can be combined using a simple aggregate price index:

P₀₁ = (ΣP₁ / ΣP₀) × 100

This method gives each commodity equal importance regardless of the quantity purchased.

8.15 Weighted Index Numbers

A simple index may be inappropriate when different commodities have substantially different levels of importance. A weighted index assigns weights to reflect relative importance.

For example, in measuring changes in household expenditure, food may receive a larger weight than an item that represents only a small proportion of household spending.

Two important weighted price-index approaches are the Laspeyres and Paasche indices.

Laspeyres Price Index:

The Laspeyres price index uses base-period quantities as weights:

L = [Σ(P₁Q₀) / Σ(P₀Q₀)] × 100

Paasche Price Index:

The Paasche price index uses current-period quantities as weights:

P = [Σ(P₁Q₁) / Σ(P₀Q₁)] × 100

8.16 Interpretation of Index Numbers

A common mistake is to interpret an index of 120 as a 120% increase. It actually means that the current level is 120% of the base level, which corresponds to a 20% increase.

8.17 Applications of Time Series Analysis

Time series techniques are used in many areas of statistical practice.

• Business: analysing sales, revenue, demand and inventory.
• Finance: examining prices, returns, interest rates and exchange rates.
• Healthcare: monitoring disease incidence, patient attendance and hospital activity.
• Agriculture: analysing crop production, rainfall and commodity prices.
• Government: monitoring population, employment, inflation and public expenditure.
• Energy: analysing electricity consumption and energy demand.
• Transportation: studying traffic volume, passenger demand and travel patterns.

8.18 Applications of Index Numbers

Index numbers are particularly useful when many related quantities must be summarized using a single measure.

• measuring changes in consumer prices;
• monitoring inflation;
• comparing production levels across periods;
• studying changes in wages and salaries;
• analysing changes in export and import prices;
• comparing economic activity across periods; and
• supporting business and government decisions.

8.19 Limitations

Time series analysis has several limitations. Historical patterns do not guarantee future behaviour. Sudden events may disrupt an established pattern, and forecasts become less reliable as the forecasting horizon increases.

Index numbers also require careful construction. The selection of the base period, commodities, weights, quality adjustments and data sources can influence the resulting index.

Another limitation is that an index is a summary measure. It may conceal important differences between individual items. An overall price index may rise even though the prices of some individual products have fallen.

8.21 Key Points

• A time series consists of observations arranged chronologically.
• The major components of a classical time series are trend, seasonal, cyclical and irregular variation.
• Moving averages can reduce short-term fluctuations and reveal underlying patterns.
• Index numbers measure relative changes between a current period and a reference period.
• The base period is normally assigned an index value of 100.
• Price indices measure price changes, while quantity indices measure quantity changes.
• Weighted indices account for differences in the importance of individual items.
• Laspeyres indices use base-period quantities as weights.
• Paasche indices use current-period quantities as weights.
• Time series and index numbers are widely used in business, government, finance, healthcare and other fields.

8.22 Chapter Summary

Time series analysis provides methods for studying observations recorded over time. A time series may contain long-term trend, seasonal variation, cyclical movements and irregular fluctuations. Elementary methods such as percentage change and moving averages allow analysts to describe and smooth time-dependent data.

Index numbers provide another important statistical tool for measuring relative changes. A single index can summarize changes in prices, quantities or values across periods. Simple indices are appropriate in basic situations, while weighted indices are useful when different items have different levels of importance.

Both time series analysis and index numbers are therefore valuable tools for converting large collections of statistical observations into measures that can support interpretation, comparison, planning and decision-making.

9.1 Chapter Introduction:

Statistics is frequently concerned with populations of people. Governments, researchers, health organizations, businesses and development agencies need reliable measures of population size, composition, births, deaths, migration and other demographic characteristics.

Demography is the systematic study of human populations, particularly their size, structure, distribution and changes resulting from fertility, mortality and migration. Demographic measures provide numerical descriptions of these characteristics.

Statistical studies often cannot collect information from every member of a population. A sample is therefore selected from the population. However, the usefulness of a sample depends partly on its size. A sample that is too small may produce imprecise estimates, while an unnecessarily large sample may waste time, money and other resources.

This chapter introduces important demographic measures and explains the basic principles used to determine an appropriate sample size.

9.2 Learning Objectives:

At the end of this chapter, the student should be able to:

• define demography and demographic measures;
• explain the importance of demographic statistics;
• calculate basic demographic rates and ratios;
• distinguish between crude and specific rates;
• explain fertility, mortality and population-growth measures;
• define sample size;
• identify factors affecting sample-size determination;
• calculate sample size using basic formulas;
• understand the role of confidence level, precision and variability in sample-size estimation; and
• apply sample-size concepts to practical statistical studies.

9.3 Key Terminology:

• Demography: The systematic study of human populations, including their size, composition, distribution and changes.
• Population: The group of individuals or units about which a statistical study is concerned.
• Birth rate: A measure relating the number of live births to the population.
• Death rate: A measure relating deaths to the population.
• Fertility: The actual reproductive performance of a population.
• Mortality: The occurrence of death within a population.
• Migration: Movement of people from one geographical area to another.
• Sample size: The number of observations or units included in a sample.
• Confidence level: The long-run proportion associated with the confidence procedure used to construct confidence intervals.
• Margin of error: The maximum tolerated difference between an estimated value and the corresponding population quantity under a specified confidence procedure.
• Parameter: A numerical characteristic of a population.
• Proportion: A fraction of a population possessing a specified characteristic.

9.4 Meaning and Importance of Demography

Demography examines population characteristics and the processes that cause populations to change. These processes include births, deaths and migration.

Demographic information is important because population characteristics affect the demand for education, healthcare, housing, transportation, employment, food, water and other services.

For example, a rapid increase in the population of children may indicate future demand for schools and teachers. An ageing population may increase demand for healthcare and social support services.

9.5 Sources of Demographic Data

Important sources include:

• Population censuses: Systematic enumeration of a population at a specified time.
• Civil registration: Continuous recording of vital events such as births and deaths.
• Sample surveys: Collection of information from a sample rather than the entire population.
• Administrative records: Information generated through government or institutional activities.

Each source has strengths and limitations. A census can provide extensive population information but may be expensive and difficult to conduct. Administrative records may be continuously available but can contain missing or inconsistent information.

9.6 Population Size and Composition

Population size refers to the number of people in a specified population at a particular time.

Population composition describes the characteristics of the population, such as age, sex, geographical location and other relevant characteristics.

Population composition is important because two populations of the same size may have very different needs. A population containing a large proportion of young children has different requirements from one containing a large proportion of older adults.

9.7 Crude Birth Rate

The crude birth rate (CBR) expresses the number of live births in a population relative to the population size, commonly per 1,000 people.

CBR = (Number of live births during the year / Mid-year population) × 1,000

9.8 Crude Death Rate

The crude death rate (CDR) expresses the number of deaths occurring in a population relative to the population size, commonly per 1,000 people.

CDR = (Number of deaths during the year / Mid-year population) × 1,000

9.9 Natural Increase

The population may increase naturally when the number of births exceeds the number of deaths.

The natural increase is:

Natural Increase = Births − Deaths

The natural increase rate can be obtained by subtracting the crude death rate from the crude birth rate:

Rate of Natural Increase = CBR − CDR

9.10 Fertility Measures

Fertility refers to actual childbearing within a population. Fertility measures often focus on women of reproductive age rather than the entire population because the entire population includes people who are not at risk of giving birth.

The general fertility rate (GFR) is commonly expressed as:

GFR = (Number of live births / Number of women aged 15–49) × 1,000

9.11 Infant Mortality Rate

The infant mortality rate (IMR) measures deaths of infants under one year of age relative to live births. It is commonly expressed per 1,000 live births.

IMR = (Deaths of infants under age 1 / Live births) × 1,000

This measure is widely used as an indicator of population health, although its interpretation requires attention to data quality and demographic context.

9.12 Migration and Population Change

Population change is not determined only by births and deaths. Migration also affects population size.

Immigration refers to movement into a population or geographical area, while emigration refers to movement out.

Net migration can be expressed as:

Net Migration = Immigration − Emigration

A simplified population-change relationship is therefore:

Population Change = Births − Deaths + Immigration − Emigration

This equation demonstrates that population growth can result from natural increase, positive net migration, or both.

9.13 Ratios and Rates

A ratio compares two quantities, while a rate generally relates the occurrence of an event to a population exposed to the possibility of that event occurring over a specified period.

For example, a sex ratio compares the number of males with the number of females, whereas a death rate relates deaths to a population over a specified period.

9.14 Meaning of Sample Size

Sample size is the number of units, observations or participants included in a statistical sample.

Suppose a researcher wants to estimate the proportion of university students who use a particular online learning platform. Studying every student may be impractical. The researcher may instead select a sample and use the sample result to estimate the population proportion.

The sample must be sufficiently large to provide the desired level of precision, but the required size depends on several factors.

9.15 Factors Affecting Sample Size

Important considerations include:

• Population variability: Greater variability generally requires a larger sample to achieve a specified precision.
• Confidence level: A higher confidence level generally requires a larger sample.
• Margin of error: A smaller acceptable error requires a larger sample.
• Population size: For finite populations, population size can influence the final sample requirement.
• Study design: Complex sampling designs may require adjustments to the simple random-sampling calculation.
• Expected response rate: If some selected participants are unlikely to provide usable responses, additional units may need to be selected.
• Available resources: Time, personnel and financial resources may constrain the feasible sample size.

9.16 Sample Size for Estimating a Population Proportion

For a sufficiently large population under a simple random-sampling framework, an elementary formula for estimating a population proportion is:

n = Z²p(1 − p) / E²

where:

• n = required sample size;
• Z = critical value associated with the selected confidence level;
• p = anticipated population proportion;
• 1 − p = complement of the anticipated proportion; and
• E = desired margin of error expressed as a proportion.

Common approximate standard-normal critical values include:

9.17 Worked Sample-Size Calculation

9.18 Effect of the Margin of Error

The formula shows that sample size is inversely proportional to the square of the margin of error:

n ∝ 1/E²

This means that demanding greater precision can substantially increase the required sample size.

9.19 Sample Size for a Population Mean

When the objective is to estimate a population mean and a reasonable estimate of the population standard deviation is available, an elementary sample-size formula is:

n = (Zσ / E)²

where:

• σ = estimated population standard deviation;
• Z = critical value corresponding to the selected confidence level; and
• E = desired margin of error.

9.20 Finite Population Correction

The large-population sample-size formula may produce a sample that is unnecessarily large when the target population itself is relatively small.

For a finite population of size N, a commonly used adjustment is:

n_adjusted = n / [1 + (n − 1)/N]

where n is the sample size calculated without the finite population correction.

9.21 Allowing for Nonresponse

In practical surveys, some selected participants may fail to respond or may provide unusable information. If the expected response rate is less than 100%, the initial sample requirement can be increased.

If n usable responses are required and the expected response rate is r, expressed as a proportion, the number to approach can be estimated as:

n_contact = n / r

9.22 Sample Size and Study Design

A mathematical sample-size formula does not by itself guarantee a good study. The sampling design must also be appropriate.

For example, a sample of 500 individuals selected using a severely biased method may provide less useful information than a smaller probability-based sample that adequately represents the target population.

Important design considerations include:

• definition of the target population;
• sampling frame;
• sampling method;
• expected variability;
• desired precision;
• confidence level;
• possible nonresponse;
• cost and feasibility; and
• ethical and practical constraints.

9.23 Sample Size Does Not Eliminate Bias

A larger sample primarily improves precision when the sampling procedure is appropriate. It does not automatically eliminate systematic bias.

For example, suppose a researcher wants to estimate the average income of all adults in a city but collects responses only from a high-income neighbourhood. Increasing the number of respondents within that neighbourhood does not make the sample representative of the entire city.

9.24 Common Mistakes in Demographic Calculations

• Using the wrong population denominator.
• Confusing a rate with a ratio.
• Forgetting the multiplier, such as 1,000.
• Using the total population when a specific population is required.
• Interpreting a rate per 1,000 as a percentage.
• Ignoring the time period associated with a demographic rate.
• Failing to distinguish births from women of reproductive age when calculating fertility measures.

9.25 Common Mistakes in Sample Size Estimation

• Using a percentage as a whole number instead of converting it to a proportion.
• Using the wrong critical value for the selected confidence level.
• Forgetting to square the Z value.
• Failing to square the margin of error in the proportion formula.
• Rounding the final sample size downward.
• Ignoring expected nonresponse.
• Assuming that a larger sample automatically produces an unbiased study.
• Applying a simple random-sampling formula without considering a more complex study design.

9.26 Key Points

• Demography is the systematic study of human populations.
• Demographic change is strongly influenced by fertility, mortality and migration.
• Demographic data may come from censuses, civil registration, surveys and administrative records.
• The crude birth rate and crude death rate commonly use the mid-year population as the denominator and are expressed per 1,000 population.
• The rate of natural increase can be obtained from the crude birth rate minus the crude death rate.
• Sample size is the number of observations or units included in a statistical sample.
• Confidence level, margin of error, variability, population size and study design influence sample-size requirements.
• For estimating a population proportion, the elementary formula is n = Z²p(1 − p)/E².
• When the population proportion is unknown, p = 0.5 is commonly used for a conservative calculation under this formula.
• For estimating a population mean, an elementary formula is n = (Zσ/E)² when an appropriate estimate of σ is available.
• Finite population corrections can reduce the required sample when the target population is relatively small.
• Nonresponse should be considered when planning the number of units to approach.
• Sample size alone does not eliminate sampling bias.

9.27 Chapter Summary

Demographic measures provide quantitative descriptions of populations and their changes. Births, deaths and migration are major processes responsible for changes in population size and structure. Measures such as crude birth rate, crude death rate, general fertility rate and infant mortality rate allow researchers and policymakers to summarize important demographic phenomena.

Sample size estimation is an essential part of statistical study design. The appropriate sample size depends on the purpose of the study, the required precision, the confidence level, population variability, population size, sampling design and practical considerations such as nonresponse.

For proportion estimates, the relationship between sample size, confidence level, variability and margin of error can be expressed using the formula n = Z²p(1 − p)/E². For mean estimates, an elementary formula is n = (Zσ/E)². These formulas provide useful starting points, but sound statistical practice requires consideration of the entire research design rather than sample size alone.

DTS 204: Statistical Computing, Inference and Modelling

Chapters 10–11

10.1 Chapter Introduction

Statistical computing involves the use of computational methods and software to collect, organize, calculate, analyze and interpret statistical information. Although many statistical calculations can be performed manually, real-world datasets are often too large or complex for practical hand calculation. Programming therefore provides an efficient way to perform statistical operations accurately and repeatedly.

Python is a general-purpose programming language that is widely used for data analysis, scientific computing, statistics and machine learning. Its usefulness in statistical computing comes from both the language itself and its extensive collection of libraries designed for numerical and data-oriented work.

In this chapter, the emphasis is not on learning the entire Python language. Instead, the objective is to introduce the programming concepts necessary to perform basic statistical computations and to show how programming can support statistical reasoning.

10.2 Learning Objectives

At the end of this chapter, the student should be able to:

• explain the meaning of statistical computing;
• describe the role of Python in statistical computing;
• identify basic Python structures used in statistical calculations;
• store numerical observations in Python data structures;
• calculate the arithmetic mean using Python;
• calculate population and sample variance using Python;
• calculate correlation between two variables using Python;
• interpret statistical results obtained through Python; and
• distinguish between performing a statistical calculation and interpreting its statistical meaning.

10.3 Key Terminology

• Statistical computing: The use of computers, algorithms and software to perform statistical calculations, analysis and modelling.
• Python: A general-purpose, high-level programming language widely used in scientific computing, data analysis and statistics.
• Variable: A characteristic or quantity whose value can differ among observations.
• Observation: A recorded value or set of values relating to an individual case, subject or unit of analysis.
• Mean: A measure of central tendency obtained by dividing the sum of observations by the number of observations.
• Variance: A measure of dispersion based on the average squared deviation from the mean.
• Correlation: A statistical measure describing the direction and strength of a linear relationship between two quantitative variables.
• Library: A collection of pre-written programming functions and tools that can be imported and used in a program.
• Array: A structured collection of numerical values that can be processed efficiently using numerical computing software.

10.4 What Is Statistical Computing?

Statistical computing is the application of computational techniques to statistical problems. It includes numerical calculation, data manipulation, simulation, statistical modelling, visualization and computational inference.

Consider a researcher who has examination scores for 50,000 students. Calculating the mean, variance and relationships between variables manually would be impractical. A computer program can perform these calculations rapidly and consistently.

Statistical computing is therefore particularly valuable when:

• datasets contain many observations;
• calculations must be repeated frequently;
• several statistical procedures must be applied to the same data;
• complex mathematical operations are required;
• results must be reproduced or verified; or
• statistical models must be fitted to data.

10.5 Why Python Is Useful for Statistical Computing

Python combines a relatively readable programming syntax with a large ecosystem of scientific and statistical libraries.

Important libraries include:

10.6 Basic Python Concepts for Statistical Computing

Before performing statistical calculations, a student must understand a few basic programming concepts.

Variables:

A variable provides a name through which a value can be stored and referenced.

Here, scores is a variable containing five numerical observations.

Lists:

A Python list can contain multiple values in an ordered collection.

Individual elements can be accessed using their positions. Python indexing begins at zero.

The result is 18.

Functions:

A function is a reusable block of code that performs a particular operation.

The len() function returns the number of elements in the list.

10.7 Computing the Mean

The arithmetic mean is one of the most commonly used measures of central tendency.

Worked Example 10.1: Mean of Examination Scores:

Question: Five students obtained the following scores:

60, 70, 75, 80, 90

Given:

• Scores = 60, 70, 75, 80, 90
• n = 5

Required: Mean score.

Formula:

x̄ = Σx / n

Solution:

First calculate the sum:

Σx = 60 + 70 + 75 + 80 + 90 = 375

Therefore:

x̄ = 375 / 5 = 75

Answer: The mean score is 75.

Computing the Mean in Python:

The result is:

The computer returns 75.0 because Python represents the division result as a floating-point number. Statistically, this represents the same mean of 75.

10.8 Computing Variance

Variance measures the extent to which observations differ from their mean. A small variance indicates that observations tend to be close to the mean, whereas a large variance indicates greater dispersion.

The distinction between population and sample variance is important. Dividing by n − 1 in the sample case provides the commonly used unbiased estimator of population variance under the standard assumptions.

Worked Example 10.2: Population Variance:

Question: Calculate the population variance of:

2, 4, 6, 8, 10

Given:

• Data = 2, 4, 6, 8, 10
• N = 5

Required: Population variance.

Step 1: Calculate the mean.

μ = (2 + 4 + 6 + 8 + 10) / 5 = 6

Step 2: Calculate deviations from the mean.

Step 3: Sum the squared deviations.

Σ(x − μ)² = 16 + 4 + 0 + 4 + 16 = 40

Step 4: Divide by N.

σ² = 40 / 5 = 8

Answer: The population variance is 8.

Computing Variance in Python:

The population variance is 8, while the sample variance is:

40 / 4 = 10

This illustrates why the denominator matters when computing variance.

10.9 Interpreting Variance

Variance is expressed in squared units. For example, if the original variable is measured in kilograms, its variance is expressed in kilograms squared.

This sometimes makes variance less intuitive to interpret directly. The standard deviation, which is the square root of variance, returns the measurement to the original unit.

For example, if variance is 25 kg²:

Standard deviation = √25 = 5 kg

Variance is nevertheless fundamental because it is used extensively in statistical inference, probability theory, regression and statistical modelling.

10.10 Computing Correlation

Correlation measures the strength and direction of a linear association between two quantitative variables.

The commonly used Pearson correlation coefficient is denoted by r. Its value lies between −1 and +1.

A positive correlation means that larger values of one variable tend to be associated with larger values of the other. A negative correlation means that larger values of one variable tend to be associated with smaller values of the other.

Worked Example 10.3: Perfect Positive Correlation:

Suppose the following data are recorded:

The variables follow an exact positive linear pattern. Their Pearson correlation is:

r = 1

This indicates a perfect positive linear relationship in this particular dataset.

Computing Correlation in Python:

The result is approximately 1.0.

10.11 A More Realistic Correlation Example

Consider the following observations of study hours and examination scores:

The relationship is positive, but the points do not form a perfectly straight line. Python can calculate the Pearson correlation coefficient:

The resulting value will be close to +1, indicating a strong positive linear association.

The correct interpretation is that students with more study hours in this dataset tend to have higher scores. It would not be correct to conclude solely from this correlation that increasing study hours necessarily causes a particular increase in examination scores.

10.12 A Basic Statistical Computing Workflow in Python

A simple statistical computing task can follow the sequence below:

1. Define the problem. Determine what statistical question needs to be answered.
2. Obtain the data. Collect or load the observations.
3. Inspect the data. Check values, data types and possible errors.
4. Prepare the data. Handle missing or inappropriate values where necessary.
5. Calculate descriptive statistics. Compute measures such as mean and variance.
6. Analyze relationships. Calculate measures such as correlation where appropriate.
7. Interpret the results. Connect numerical results to the original statistical question.
8. Communicate the findings. Present results clearly using appropriate tables, graphs and explanations.

The important point is that statistical computing is not simply about writing code. A correct program can still produce an inappropriate analysis if the statistical question, data or interpretation is wrong.

10.13 Common Errors in Statistical Python Programs

Confusing Population and Sample Variance:

Using the sample variance when a population variance is required, or vice versa, produces a different result.

Using the Wrong Variable:

A correlation calculation requires two corresponding variables. The first observation of one variable must correspond to the first observation of the other.

Ignoring Missing Values:

Missing observations can affect statistical calculations. They should be identified and handled appropriately rather than silently ignored without understanding the consequences.

Confusing Correlation with Causation:

A statistical association does not automatically demonstrate a cause-and-effect relationship.

Rounding Too Early:

Intermediate calculations should generally retain sufficient precision. Premature rounding can introduce unnecessary errors into later calculations.

10.14 Key Points

• Statistical computing uses computational methods to perform and support statistical analysis.
• Python is widely used for numerical and statistical computing.
• Python libraries extend the language with specialized statistical and numerical capabilities.
• The arithmetic mean is obtained by dividing the sum of observations by their number.
• Variance measures the dispersion of observations around their mean.
• Population variance uses N in the denominator, while the conventional sample variance uses n − 1.
• Correlation measures the direction and strength of linear association between quantitative variables.
• Correlation values range from −1 to +1.
• Correlation does not, by itself, establish causation.
• Programming performs calculations efficiently, but statistical reasoning is required to determine whether the calculations and interpretations are appropriate.

10.15 Chapter Summary

Statistical computing provides a practical bridge between statistical theory and computational analysis. Python is particularly useful because it combines an accessible programming language with powerful libraries for numerical computation, data manipulation, visualization and statistical analysis.

In basic statistical computing, numerical observations can be stored in Python structures and processed using functions and mathematical operations. The arithmetic mean provides a measure of central tendency, variance measures dispersion, and correlation describes the direction and strength of a linear relationship between two quantitative variables.

Successful statistical computing requires more than obtaining numerical output. The analyst must understand the statistical definition of each measure, select an appropriate method, verify the data and interpret the result in relation to the original problem.

11.1 Chapter Introduction

Statistical analysis begins long before a statistical test or model is applied. Raw data collected from surveys, experiments, administrative systems, sensors or other sources are rarely ready for immediate analysis. They may contain values in different formats, incorrect entries, duplicate observations, inconsistent labels or measurements that require transformation.

Data processing refers to the systematic organization, manipulation and preparation of data so that it can be analyzed appropriately. The purpose is not to change data simply to make it look better. Rather, data processing should preserve the information contained in the original observations while making the dataset accurate, consistent and suitable for analysis.

This chapter focuses on sorting and ranking data, basic processing operations, preparation of quantitative data and evaluation of data before statistical analysis.

11.2 Learning Objectives

At the end of this chapter, the student should be able to:

• explain the meaning and purpose of data processing;
• distinguish raw data from prepared data;
• sort observations according to specified variables;
• rank quantitative observations;
• describe the major steps involved in data preparation;
• identify common data-quality problems;
• evaluate quantitative data before statistical analysis;
• distinguish legitimate data transformation from inappropriate alteration of data; and
• prepare a simple dataset for statistical analysis.

11.3 Key Terminology

• Raw data: Data in the form in which they are initially collected or obtained.
• Data processing: The organization, transformation and manipulation of data to make them suitable for analysis or use.
• Sorting: Arranging observations according to the values of one or more variables.
• Ranking: Assigning an ordered position to observations according to the values of a variable.
• Data preparation: The set of activities used to organize, clean, transform and structure data before analysis.
• Data validation: Checking whether data satisfy specified rules or expected conditions.
• Data cleaning: Detecting and correcting or appropriately handling errors, inconsistencies and problematic observations.
• Quantitative data: Numerical data representing quantities that can be measured or counted.
• Outlier: An observation that is unusually distant from the general pattern of the other observations.
• Variable: A characteristic measured or recorded for each observation.

11.4 Meaning of Data Processing

Data processing converts raw observations into a structured form that can be examined statistically.

For example, suppose a researcher records the ages of six participants as:

24, 19, 31, 22, 19, 27

The values are valid observations, but several operations may be useful before analysis. The researcher may sort the observations, determine their ranks, calculate descriptive statistics and inspect whether any value appears unusual.

Data processing may involve:

• checking the structure of the dataset;
• sorting observations;
• ranking observations;
• checking for missing values;
• checking for duplicate records;
• checking variable types and units;
• detecting impossible values;
• standardizing categories and formats;
• transforming variables where statistically justified; and
• creating an analysis-ready dataset.

11.5 Raw Data and Prepared Data

Raw data are not necessarily bad data. They are simply data that have not yet undergone the processing required for the intended analysis.

Consider a dataset containing:

This dataset is already structured, but it may still require validation. The analyst must determine whether ages are plausible, whether scores fall within the permitted range, whether department names are consistently recorded and whether each row represents a unique student.

11.6 Sorting Data

Sorting means arranging observations according to the values of one or more variables.

Data can be sorted in:

• ascending order, from smallest to largest; or
• descending order, from largest to smallest.

Example 11.1: Sorting Examination Scores:

Suppose the scores are:

72, 45, 88, 61, 95, 54

Ascending order gives:

45, 54, 61, 72, 88, 95

Descending order gives:

95, 88, 72, 61, 54, 45

Sorting makes patterns easier to inspect and is often useful when identifying minimum values, maximum values, potential outliers and rankings.

Sorting in Python:

11.7 Sorting Tabular Data

When working with a dataset containing several variables, sorting should normally be performed according to a particular variable while keeping the corresponding observations together.

For example, if a student record contains a name, age and score, sorting the score column independently would destroy the relationship between each student and the student's score. The complete rows must therefore move together.

Using pandas, a table can be sorted by a specified column:

The student records remain associated with their correct scores while the rows are reordered.

11.8 Ranking Data

Ranking assigns an ordered position to observations according to their values.

Suppose five students obtain:

95, 82, 76, 68, 60

The ranking is:

Ranking becomes more interesting when observations have tied values. For example:

90, 85, 85, 70

The two observations with a value of 85 occupy the same position in the ordered data. Different ranking conventions can assign tied observations differently, so the chosen method should be understood before interpreting ranks.

11.9 Data Step Processing

The expression data step processing refers broadly to the sequence of operations through which raw observations are read, examined, modified, combined or organized before statistical analysis.

A typical processing sequence can be represented as:

At each stage, the analyst should preserve the meaning of the original data and maintain a record of important transformations.

11.10 Data Validation

Data validation involves checking whether observations conform to expected rules.

For example, suppose examination scores are recorded on a scale from 0 to 100. A value of 87 is plausible, while a value of 143 requires investigation.

Similarly, if age is measured in years, an entry such as −4 is impossible under ordinary circumstances and should be investigated.

Validation may include:

• range checks;
• type checks;
• format checks;
• consistency checks;
• uniqueness checks; and
• logical relationship checks.

Example 11.2: Range Validation:

Suppose a dataset contains:

45, 72, 91, 103, 67, −5, 88

If the variable represents an examination score with a valid range of 0 to 100, the values 103 and −5 violate the expected range.

The correct action is not automatically to replace them with convenient values. The original source should first be investigated to determine whether they resulted from data-entry errors, coding problems or some legitimate but unusual circumstance.

11.11 Data Cleaning

Data cleaning is the process of identifying and appropriately handling errors, inconsistencies, missing values, duplicates and other data-quality problems.

Common problems include:

11.12 Preparing Data for Analysis

Data preparation should be guided by the statistical question that the analyst intends to answer.

A typical preparation process includes the following stages:

1. Understand the Dataset:

Identify what each row represents, what each variable measures, the units of measurement and the source of the observations.

2. Check Variable Types:

Determine whether variables are numerical, categorical, dates, identifiers or other forms of information. Incorrect variable types can lead to incorrect analysis.

3. Check Missing Values:

Determine whether observations are incomplete and understand why values are missing where possible.

4. Check for Duplicates:

Determine whether duplicate rows represent genuine repeated observations or accidental duplication.

5. Check Validity:

Examine whether values fall within reasonable or permitted ranges.

6. Standardize Data:

Ensure that units, labels, date formats and categorical values are represented consistently.

7. Transform Where Necessary:

Apply mathematically justified transformations required by the analysis, while preserving the original variable whenever appropriate.

8. Verify the Final Dataset:

After processing, inspect the dataset again before statistical analysis begins.

11.13 Quantitative Data

Quantitative data represent numerical quantities. Examples include age, income, height, weight, temperature, examination scores and production output.

Quantitative variables can generally be divided into:

• Discrete variables: Variables that take countable values, such as number of children or number of defective products.
• Continuous variables: Variables that can take values across an interval, such as height, weight, temperature or time.

The distinction matters because the appropriate statistical methods and interpretations may depend on the nature of the variable.

11.14 Evaluating Quantitative Data

Before analyzing numerical data, the analyst should evaluate several characteristics.

Central Tendency:

The mean, median and mode provide different descriptions of the centre of a dataset.

Variation:

Measures such as range, variance and standard deviation describe how widely observations are distributed.

Distribution:

The analyst should consider whether the data are approximately symmetric, skewed, multimodal or otherwise unusual.

Outliers:

Unusually large or small observations should be investigated. An outlier is not automatically an error.

Units:

The unit of measurement must be known. A value of 75 is meaningless without knowing whether it represents kilograms, years, marks, degrees or another quantity.

Precision:

The number of decimal places should be appropriate to the measurement process. Recording a measurement to many decimal places does not necessarily make it more accurate.

11.15 Worked Example: Evaluating a Quantitative Dataset

Question: A researcher records the following daily sales values in thousands of currency units:

12, 15, 14, 16, 13, 15, 17, 14, 100

Evaluate the dataset before performing further analysis.

Step 1: Inspect the values.

Most observations lie between 12 and 17, while one observation is 100.

Step 2: Identify a potential outlier.

The value 100 is dramatically larger than the other observations and should therefore be investigated.

Step 3: Do not automatically delete the value.

The value could be a data-entry error, or it could represent a genuine exceptional sales day.

Step 4: Verify the source.

If the original record shows that the correct value was 10.0 rather than 100, the error can be corrected. If the value of 100 is genuine, it should normally remain in the dataset, although its influence on statistical analysis may need to be considered.

Conclusion: Data evaluation requires statistical reasoning rather than automatic deletion of unusual observations.

11.16 Data Transformation

Data transformation involves changing the representation or scale of a variable for a justified analytical purpose.

Common transformations include:

• converting units;
• creating a new variable from existing variables;
• standardizing values;
• aggregating observations;
• converting categorical representations; and
• applying mathematical transformations such as logarithms when appropriate.

Example 11.3: Unit Conversion:

Suppose temperature is recorded in degrees Celsius but an analysis requires Fahrenheit.

Formula:

F = (9/5)C + 32

For 25°C:

F = (9/5)(25) + 32 = 45 + 32 = 77°F

The transformation changes the representation of the measurement but does not change the underlying physical temperature.

11.17 Creating Derived Variables

A derived variable is created from one or more existing variables.

Suppose a student's continuous assessment score is 30 and examination score is 55. If the total score is defined as their sum:

Total = Continuous Assessment + Examination

then:

Total = 30 + 55 = 85

In Python:

Derived variables should have clearly documented definitions so that another analyst can understand how they were produced.

11.18 Standardization of Quantitative Data

Standardization is a transformation commonly used to express an observation in terms of its distance from the mean measured in standard deviations.

Worked Example 11.4:

Question: A student's score is 80, the class mean is 70 and the standard deviation is 5. Calculate the student's z-score.

Given:

• x = 80
• x̄ = 70
• s = 5

Formula:

z = (x − x̄) / s

Solution:

z = (80 − 70) / 5 = 10 / 5 = 2

Answer: The student's z-score is 2.

This means the student's score is two sample standard deviations above the class mean.

11.19 Data Processing with Python

Python can be used to automate many data-processing operations. For example:

This short program creates a dataset, displays it, sorts the records by score in descending order and calculates the mean and sample variance of the scores.

The important statistical principle is that the code is implementing operations that have already been defined conceptually. The analyst should understand what each operation means before using it.

11.20 Maintaining Data Integrity

Data preparation must not become an opportunity to manipulate results. An analyst should distinguish between cleaning data and changing data to obtain a desired conclusion.

Good data-processing practice includes:

• preserving the original dataset;
• documenting important transformations;
• recording reasons for corrections;
• checking suspicious values against source records;
• avoiding unjustified deletion of observations;
• using consistent processing rules;
• maintaining reproducible procedures; and
• keeping processed data traceable to the original source.

11.21 Common Mistakes in Data Processing

Sorting One Column Independently:

Sorting a single column without moving the corresponding rows can destroy relationships between variables.

Deleting Outliers Automatically:

An unusual value may be a genuine observation. It should be investigated before removal.

Replacing Missing Values Without Justification:

Missing observations should not automatically be replaced by the mean or another value without considering why the data are missing and how the replacement affects the analysis.

Ignoring Units:

Combining values measured in different units can produce meaningless results.

Changing the Original Data:

Editing the only copy of the raw dataset makes it difficult to verify the analysis or identify processing errors.

Confusing Precision with Accuracy:

A measurement reported to many decimal places is not necessarily more accurate than a measurement reported to fewer decimal places.

Failing to Document Transformations:

If a variable is converted, standardized, aggregated or otherwise transformed, the procedure should be documented.

11.22 Key Points

• Data processing prepares raw observations for meaningful statistical analysis.
• Sorting arranges observations according to one or more variables.
• Ranking assigns ordered positions to observations.
• Data validation checks whether observations satisfy expected rules.
• Data cleaning addresses missing values, duplicates, inconsistencies and errors.
• Quantitative data should be evaluated for centre, variation, distribution, outliers, units and precision.
• Data transformation changes the representation of data for a justified analytical purpose.
• Derived variables can be created from existing variables using clearly defined rules.
• Standardization can express observations relative to a mean and standard deviation.
• Unusual observations should be investigated rather than automatically deleted.
• The original dataset should be preserved and important processing steps documented.
• Data preparation must improve analytical validity rather than manipulate results.

11.23 Chapter Summary

Statistical analysis depends heavily on the quality and structure of the data supplied to the analytical procedure. Data processing provides the systematic steps required to transform raw observations into an analysis-ready dataset.

Sorting and ranking help organize observations and reveal patterns. Validation checks whether values satisfy expected conditions, while cleaning addresses errors, missing observations, duplicates and inconsistencies. Quantitative data should be evaluated with attention to central tendency, variation, distribution, outliers, units and measurement precision.

Data transformation can include unit conversion, creation of derived variables and standardization. Such transformations must have a clear statistical or analytical purpose and should be documented. Python and libraries such as pandas can automate many processing tasks, but the analyst remains responsible for ensuring that the operations are statistically appropriate.

Ultimately, good data preparation protects the validity of statistical analysis. A sophisticated statistical model cannot compensate for poorly prepared or misunderstood data. The quality of the conclusions therefore depends not only on the statistical method used, but also on the quality and integrity of the data-processing process.

Chapter Introduction

Regression analysis and designed experiments are two important areas of statistical computing. Regression provides a systematic method for studying relationships between variables and for using observed data to estimate or predict outcomes. Designed experiments, on the other hand, provide a structured approach to collecting data when a researcher deliberately changes one or more factors in order to investigate their effects on a response.

Modern statistical analysis relies heavily on computing because practical datasets may contain hundreds, thousands, or millions of observations. Statistical software makes it possible to calculate regression coefficients, assess model performance, examine experimental effects, produce diagnostic plots, and summarize results efficiently.

This chapter introduces the computational principles behind regression analysis and experimental data analysis. The emphasis is on understanding what the statistical procedures accomplish, how the calculations are performed, and how their results should be interpreted.

12.1 Statistical Computing in Regression Analysis

Statistical computing in regression analysis refers to the use of computational procedures, algorithms and statistical software to fit, evaluate and interpret regression models.

Although regression equations can be calculated manually for small datasets, computational methods become essential when the dataset contains many observations or several explanatory variables.

A typical computational regression procedure involves:

1. collecting and preparing the data;
2. identifying the response and explanatory variables;
3. specifying an appropriate regression model;
4. estimating the model parameters;
5. calculating predicted values and residuals;
6. evaluating the fitted model;
7. checking assumptions and diagnostic information; and
8. interpreting and communicating the results.

12.2 Simple Linear Regression

Simple linear regression examines the relationship between one response variable and one explanatory variable.

The population model is commonly written as:

Y = β₀ + β₁X + ε

where:

• Y = response variable;
• X = explanatory variable;
• β₀ = population intercept;
• β₁ = population slope;
• ε = random error.

After estimating the parameters from sample data, the fitted regression equation is written as:

ŷ = b₀ + b₁x

Here, ŷ represents the predicted value of the response, while b₀ and b₁ are estimates of the intercept and slope.

Interpretation of the Regression Coefficients:

The intercept b₀ represents the predicted value of the response when X equals zero, provided that X = 0 is meaningful within the context of the study.

The slope b₁ represents the estimated change in the response associated with a one-unit increase in X.

12.3 Computing Regression Coefficients

For a simple linear regression model, the least-squares estimates of the slope and intercept can be calculated using:

b₁ = Σ(x_i − x̄)(y_i − ȳ) / Σ(x_i − x̄)²

and

b₀ = ȳ − b₁x̄

where x̄ and ȳ are the sample means of X and Y.

Worked Example 1: Calculating a Regression Equation:

Question: A researcher records the number of hours studied by five students and their corresponding examination scores.

Given: X = 1, 2, 3, 4, 5 and Y = 50, 55, 65, 70, 80.

Required: Find the least-squares regression equation.

Step 1: Calculate the means.

x̄ = (1 + 2 + 3 + 4 + 5) / 5 = 3

ȳ = (50 + 55 + 65 + 70 + 80) / 5 = 64

Step 2: Calculate the deviations and products.

Therefore:

Σ(X − x̄)(Y − ȳ) = 75

Σ(X − x̄)² = 10

Step 3: Calculate the slope.

b₁ = 75 / 10 = 7.5

Step 4: Calculate the intercept.

b₀ = 64 − (7.5 × 3) = 41.5

Answer:

ŷ = 41.5 + 7.5x

The model estimates that each additional hour of study is associated with an increase of approximately 7.5 examination-score points.

12.4 Prediction Using a Regression Model

Once a regression equation has been fitted, it can be used to calculate predicted responses for values of the explanatory variable within an appropriate range of the observed data.

Worked Example 2:

Question: Using the regression model

ŷ = 41.5 + 7.5x

estimate the examination score for a student who studies for 4.5 hours.

Given: x = 4.5 hours.

Solution:

ŷ = 41.5 + 7.5(4.5)

ŷ = 41.5 + 33.75 = 75.25

Answer: The predicted examination score is 75.25.

This is a model-based prediction, not a guarantee that the student's actual score will be exactly 75.25.

12.5 Residuals and Model Error

A regression model rarely predicts every observation perfectly. The difference between an observed value and its predicted value is called the residual.

Residual = Observed value − Predicted value

In symbols:

e_i = y_i − ŷ_i

Worked Example 3:

Suppose a model predicts a student's examination score as 72, but the actual score is 78.

Residual = 78 − 72 = 6

The positive residual indicates that the actual score was six points higher than the model predicted.

If the actual score had been 68, the residual would be:

68 − 72 = −4

A negative residual indicates that the model overpredicted the observed outcome.

12.6 Measuring the Fit of a Regression Model

One important measure used to describe the explanatory power of a regression model is the coefficient of determination, denoted by R².

For a simple linear regression:

R² = 1 − SSE / SST

where:

• SSE is the sum of squared errors or residual sum of squares;
• SST is the total sum of squares.

R² indicates the proportion of variation in the response that is explained by the fitted regression model.

Example:

If a regression model has R² = 0.81, then approximately 81% of the observed variation in the response is explained by the model, while the remaining 19% is not explained by the model.

R² should not be interpreted as proof that the model is correct or that the explanatory variable causes the response.

12.7 Multiple Regression

When more than one explanatory variable is used, the model is called multiple regression.

A general multiple linear regression model can be written as:

ŷ = b₀ + b₁x₁ + b₂x₂ + ⋯ + b_px_p

For example, house price might be modelled using floor area, number of bedrooms and distance from a city centre.

Multiple regression is particularly useful when an outcome is influenced by several measurable factors.

12.8 Statistical Computing for Regression

Statistical software and programming languages such as R and Python provide functions for fitting regression models, calculating model statistics and producing diagnostic visualizations.

A typical computational workflow consists of:

1. import the dataset;
2. inspect variable names and data types;
3. check for missing or invalid observations;
4. identify the response and predictor variables;
5. fit the regression model;
6. examine coefficients and statistical measures;
7. calculate predictions;
8. inspect residuals and diagnostic plots; and
9. communicate the findings.

For example, in R, a simple linear regression can be fitted using the lm() function.

model <- lm(score ~ hours, data = students)

The fitted model can then be examined using:

summary(model)

Predictions can be obtained using:

predict(model, newdata = data.frame(hours = 4.5))

The computational tool performs the numerical calculations, but the researcher remains responsible for choosing an appropriate model and interpreting the results correctly.

12.9 Designed Experiments

A designed experiment is a planned study in which researchers deliberately vary one or more factors and observe the resulting changes in a response variable.

For example, an agricultural researcher may investigate how fertilizer type affects crop yield. The researcher selects fertilizer treatments, applies them under controlled conditions and measures the resulting crop yields.

Designed experiments are different from observational studies because the researcher actively controls or assigns experimental conditions.

Basic Components of an Experiment:

• Experimental unit: The smallest unit to which a treatment is assigned.
• Factor: A variable deliberately manipulated by the researcher.
• Level: A particular setting of a factor.
• Treatment: The experimental condition assigned to an experimental unit.
• Response: The outcome measured after applying a treatment.
• Randomization: The use of random allocation to reduce systematic bias.
• Replication: Repeating treatments across multiple experimental units.
• Control: Maintaining appropriate conditions against which treatment effects can be compared.

12.10 Principles of Experimental Design

Randomization:

Randomization assigns treatments using a random mechanism. Its purpose is to reduce systematic differences between treatment groups and help support valid statistical inference.

Replication:

Replication involves applying the same treatment to multiple experimental units. Replication provides information about variability and improves the precision with which treatment effects can be estimated.

Control:

Control refers to managing experimental conditions so that differences in the response can more reasonably be attributed to the factors under investigation.

Blocking:

Blocking groups similar experimental units together before treatments are assigned. This can reduce the effect of known sources of variation.

12.11 Completely Randomized Design

A completely randomized design is an experimental design in which treatments are assigned randomly to experimental units without first forming blocks.

Suppose three fertilizers, A, B and C, are being compared using 15 experimental plots. Five plots are randomly assigned to each fertilizer.

The response variable might be crop yield measured in kilograms per plot.

12.12 Randomized Block Design

A randomized block design is used when experimental units can be grouped according to a characteristic that is expected to influence the response.

Within each block, treatments are randomly assigned.

For example, if four teaching methods are being compared across students with substantially different prior academic abilities, students might first be grouped into blocks based on prior performance. Each teaching method can then be randomly assigned within the blocks.

The objective is to separate the variation caused by the blocking factor from the variation caused by the treatments.

12.13 Analysis of Variance in Designed Experiments

Analysis of variance (ANOVA) is a statistical method used to determine whether observed differences among group means are sufficiently large relative to within-group variation to provide evidence of differences among the underlying population means.

For a one-factor experiment, total variation can conceptually be divided into:

Total Variation = Between-Group Variation + Within-Group Variation

The corresponding sums of squares are commonly expressed as:

SST = SSB + SSW

where:

• SST = total sum of squares;
• SSB = between-group sum of squares;
• SSW = within-group sum of squares.

The ANOVA F-statistic is calculated as:

F = MSB / MSW

where MSB is the mean square between groups and MSW is the mean square within groups.

A sufficiently large F-statistic provides evidence against the null hypothesis that all treatment means are equal, subject to the assumptions of the ANOVA model.

Worked Example 4: One-Way ANOVA Structure:

Suppose three production methods are tested and the resulting output values are:

The first step is to calculate the mean for each method:

Mean A = (12 + 14 + 13) / 3 = 13

Mean B = (18 + 17 + 19) / 3 = 18

Mean C = (20 + 21 + 19) / 3 = 20

The treatment means differ, but this alone does not establish that the treatments have different population effects. ANOVA compares the variation among these means with the variation within the groups.

In practical statistical computing, software calculates the sums of squares, degrees of freedom, mean squares, F-statistic and corresponding probability value. The researcher then interprets these quantities in the context of the experiment.

12.14 Regression and Experimental Design: The Relationship

Regression and designed experiments are closely related. Both use statistical models to explain variation in a response variable.

Regression is frequently used with observational or experimental data to quantify relationships between a response and one or more predictors. Experimental design determines how observations are collected so that treatment effects can be estimated efficiently and with reduced bias.

In many practical studies, regression and experimental design are combined. For example, regression can be used to model a response as a function of treatment variables and other relevant explanatory variables.

12.15 Statistical Computing with Experimental Data

Computational analysis of experimental data usually follows a structured sequence.

1. Data entry: Record treatment assignments and measured responses accurately.
2. Data checking: Identify missing values, impossible observations, incorrect labels and data-entry errors.
3. Exploratory analysis: Calculate descriptive statistics and inspect plots.
4. Model specification: Select an appropriate statistical model based on the experimental design.
5. Model fitting: Estimate model parameters using statistical software.
6. Hypothesis testing: Evaluate whether observed treatment differences provide statistical evidence of effects.
7. Diagnostic analysis: Examine residuals and model assumptions.
8. Interpretation: Translate numerical results into conclusions relevant to the research question.

The computational procedure should always follow the experimental design. A statistical program cannot compensate for poor experimental planning, biased sampling or incorrect treatment assignment.

12.16 Regression Diagnostics

Before interpreting a regression model, the analyst should examine whether the model provides a reasonable representation of the data.

Important diagnostic considerations include:

• Linearity: The relationship between predictors and response should be appropriately represented by the model.
• Independence: Observations or errors should satisfy the independence assumptions appropriate to the study design.
• Constant variance: The variability of errors should be reasonably stable where this assumption is required.
• Normality of errors: For certain inferential procedures, the error distribution is assumed to be approximately normal.
• Influential observations: Individual observations can sometimes exert an unusually large influence on the fitted model.

Residual plots are particularly useful because they allow the analyst to examine patterns that may not be obvious from numerical summaries alone.

12.17 Interpretation of Statistical Output

Statistical software can produce a large amount of output. Correct interpretation requires more than reading individual numbers.

For regression analysis, important output may include:

• estimated coefficients;
• standard errors;
• test statistics;
• probability values;
• confidence intervals;
• R² and adjusted R²;
• residual statistics; and
• diagnostic information.

For experimental analysis, output may include:

• group or treatment means;
• sums of squares;
• degrees of freedom;
• mean squares;
• F-statistics;
• probability values;
• confidence intervals; and
• post-hoc comparisons where appropriate.

12.18 Worked Applied Example: Regression in Engineering

Question: An engineer wants to examine whether machine operating temperature can be used to predict energy consumption. A fitted model is:

ŷ = 25 + 1.8X

where X is temperature in °C and ŷ is energy consumption in kWh.

Required: Estimate energy consumption at 30°C and interpret the slope.

Solution:

ŷ = 25 + 1.8(30)

ŷ = 25 + 54 = 79 kWh

Answer: The predicted energy consumption at 30°C is 79 kWh.

The slope of 1.8 means that a one-degree Celsius increase in operating temperature is associated with an estimated increase of 1.8 kWh in energy consumption, according to the fitted model.

The interpretation should not be extended beyond the range of temperatures for which the model was developed without appropriate justification.

12.19 Common Mistakes in Regression and Experimental Analysis

1. Confusing correlation with causation: A strong statistical relationship does not by itself establish a causal relationship.

2. Interpreting the intercept automatically: The intercept may have little practical meaning when X = 0 is outside the observed or scientifically meaningful range.

3. Extrapolating excessively: Predictions far outside the range of the observed predictor values can be unreliable.

4. Ignoring residuals: A high R² does not guarantee that a model satisfies its assumptions.

5. Treating statistical significance as practical significance: Statistical evidence and practical importance are related but different concepts.

6. Using the wrong experimental design: The statistical analysis must reflect how the data were collected.

7. Failing to randomize: Systematic treatment assignment can introduce bias.

8. Confusing replication with repeated measurements: Repeated measurements on the same experimental unit are not automatically equivalent to independent experimental replications.

9. Treating software output as the analysis itself: Software calculates statistics, but the researcher must determine whether the model and assumptions are appropriate.

10. Ignoring data quality: Incorrect, missing or inconsistently recorded data can produce misleading statistical conclusions.

12.20 Key Points

• Statistical computing makes regression and experimental analysis practical for large and complex datasets.
• Simple linear regression models the relationship between one response variable and one explanatory variable.
• The fitted regression equation is commonly written as ŷ = b₀ + b₁x.
• The slope represents the estimated change in the response associated with a one-unit increase in the predictor.
• A residual is the difference between an observed value and its model-predicted value.
• R² describes the proportion of response variation explained by a regression model under the model formulation.
• Multiple regression allows several explanatory variables to be included in a model.
• Regression diagnostics are necessary for evaluating whether a fitted model is reasonable.
• A designed experiment deliberately varies one or more factors to investigate their effects on a response.
• Randomization, replication, control and blocking are fundamental principles of experimental design.
• A completely randomized design assigns treatments randomly without forming blocks.
• A randomized block design groups similar experimental units before treatment assignment.
• ANOVA separates total variation into components associated with different sources.
• The F-statistic compares between-group variation with within-group variation in appropriate ANOVA settings.
• Statistical software performs computations efficiently, but correct interpretation depends on sound statistical reasoning and appropriate study design.

12.21 Chapter Summary

Statistical computing provides the computational foundation for analysing regression and experimental data efficiently. Regression analysis is used to quantify relationships between variables, estimate model parameters and generate predictions. In simple linear regression, the least-squares method produces an estimated intercept and slope that define the fitted regression equation.

Regression analysis involves more than calculating coefficients. Predicted values, residuals, R², statistical tests and diagnostic procedures help the analyst determine whether the fitted model is useful and appropriate. Multiple regression extends the approach by allowing several explanatory variables to be included simultaneously.

Designed experiments provide a structured framework for collecting data when researchers deliberately manipulate experimental factors. Important concepts include experimental units, factors, levels, treatments, responses, randomization, replication, control and blocking.

Analysis of variance provides a fundamental computational method for analysing many designed experiments. It compares variation between treatment groups with variation within groups and provides a formal framework for determining whether observed differences among treatments are statistically meaningful.

The effective use of statistical computing therefore requires both computational competence and statistical judgment. Software can perform complex numerical calculations quickly, but meaningful conclusions depend on appropriate data, sound experimental design, suitable statistical models, careful diagnostics and context-aware interpretation.

Chapter Review

Before proceeding, the student should be able to explain how regression models are fitted computationally, interpret regression coefficients and residuals, explain the purpose of R², distinguish regression analysis from experimental design, and describe how ANOVA is used to analyse treatment variation.

Chapter Introduction

Statistical computing provides practical methods for performing calculations that would otherwise be difficult, repetitive, or time-consuming to carry out manually. As statistical problems become more complex, researchers increasingly rely on computational techniques and specialized statistical software.

One important computational technique is the Monte Carlo method. Monte Carlo methods use repeated random sampling and numerical computation to study uncertain systems, approximate quantities that may be difficult to calculate analytically, and investigate the behaviour of statistical procedures.

Statistical packages provide another important component of statistical computing. Programs such as SPSS, SAS, Minitab, GENSTAT, EPI-INFO and SYSTAT provide facilities for data entry, data management, descriptive statistics, statistical inference, modelling, visualization and reporting.

This chapter introduces the principles behind Monte Carlo methods and examines the role, strengths and applications of major statistical packages. The objective is not merely to identify software names, but to understand how statistical computing environments support the complete process of statistical analysis.

13.1 Introduction to Monte Carlo Methods

The Monte Carlo method is a numerical technique in which random sampling is used repeatedly to estimate a quantity, solve a computational problem, or study the behaviour of a system.

The method is particularly useful when a problem contains uncertainty or when a direct mathematical solution is difficult, complicated, or computationally expensive.

The central idea is simple:

Suppose a researcher wants to estimate the probability that a randomly selected value from a population satisfies a particular condition. Instead of deriving the probability analytically, the researcher can generate many random observations, count how many satisfy the condition, and estimate the probability from the observed proportion.

Monte Carlo methods therefore connect probability, random sampling and numerical computation.

13.2 Why Monte Carlo Methods Are Important

Many statistical and scientific problems involve uncertainty. Analytical mathematics can provide exact solutions for some problems, but not all problems have simple closed-form solutions.

Monte Carlo methods provide an alternative computational approach.

Complex probability calculations: Simulation can approximate probabilities when direct calculation is difficult.

Uncertainty analysis: Researchers can examine how uncertain inputs affect uncertain outputs.

Statistical inference: Simulation can be used to investigate the behaviour of estimators and hypothesis tests.

Model evaluation: Researchers can simulate data from a model and determine whether statistical methods recover known characteristics.

Risk analysis: Monte Carlo methods are widely applicable when decisions depend on uncertain future outcomes.

Numerical approximation: Some mathematical quantities can be approximated using random sampling when deterministic numerical methods are inconvenient.

13.3 The Basic Monte Carlo Principle

Consider a random variable X whose expected value is:

If independent observations X₁, X₂, ..., X_n are generated from the same distribution, the sample mean is:

Under appropriate conditions, as the number of simulations increases, the sample mean tends to approach the expected value.

This is one of the fundamental principles that makes Monte Carlo estimation possible.

13.4 A Simple Monte Carlo Example

A classical illustration of Monte Carlo simulation is the estimation of the probability of obtaining a particular outcome from repeated random trials.

Example 13.1: Estimating the Probability of a Head:

Suppose a fair coin is simulated 10,000 times. Let the number of simulated heads be 5,038.

Required: Estimate the probability of obtaining a head.

Formula:

Solution:

Number of heads = 5,038

Total number of trials = 10,000

Therefore:

Answer: The estimated probability of obtaining a head is 0.5038, or 50.38%.

The theoretical probability for a fair coin is 0.5. The difference occurs because the simulation is based on a finite number of random trials. Increasing the number of trials generally produces an estimate closer to the theoretical probability.

13.5 Monte Carlo Estimation of an Area

Monte Carlo methods can also be used to estimate geometric quantities.

Consider a square of side length 2 units centred at the origin. The square extends from −1 to 1 on both the x-axis and y-axis. A circle of radius 1 is placed inside the square.

The area of the square is:

The area of the circle is:

Therefore, the proportion of the square occupied by the circle is:

If random points are generated uniformly inside the square, the proportion of points falling inside the circle can be used to estimate π.

Suppose 100,000 random points are generated and 78,540 fall inside the circle.

Estimated proportion:

78,540 / 100,000 = 0.7854

Since this proportion estimates π/4:

Estimated π = 4 × 0.7854 = 3.1416

Answer: The simulation produces an estimate of π ≈ 3.1416.

This example illustrates the general Monte Carlo principle: random samples can be used to approximate a quantity when the relationship between the random experiment and the quantity of interest is known.

13.6 Steps in a Monte Carlo Experiment

A properly designed Monte Carlo analysis normally follows a logical sequence.

13.7 Random Numbers and Pseudorandom Numbers

Monte Carlo simulation requires values that behave like random observations. Computers, however, are deterministic machines. They normally generate pseudorandom numbers using mathematical algorithms.

A pseudorandom number generator produces a sequence of values that appears random according to specified statistical properties.

A random seed determines the starting point of many pseudorandom number generators.

For example, if a statistical program is instructed to use the same seed before generating a sequence of pseudorandom numbers, it can often reproduce the same sequence. This is valuable when researchers need reproducible computational results.

13.8 Replication and Simulation Accuracy

Monte Carlo estimates are subject to simulation variability. A different sequence of random numbers may produce a slightly different result.

One way to improve the reliability of a simulation is to increase the number of replications.

For example, suppose a simulation estimates a probability using only 100 trials. The result may vary substantially between runs. Increasing the number of trials to 10,000 or 1,000,000 generally reduces the random fluctuation in the estimated proportion.

However, increasing the number of simulations does not correct a badly specified model. If the probability distribution or assumptions used in the simulation are inappropriate, running the incorrect model more times will not make the model correct.

13.9 Applications of Monte Carlo Methods in Statistics

Monte Carlo methods have numerous applications in statistical computing.

Sampling distribution studies: Researchers can simulate repeated samples to investigate the behaviour of an estimator.

Confidence intervals: Simulation can be used to investigate the coverage properties of confidence intervals.

Hypothesis testing: Simulated data can be used to study the distribution of test statistics under specified assumptions.

Regression: Researchers can simulate regression data to investigate estimator bias, variability and prediction performance.

Risk analysis: Uncertain quantities can be simulated to estimate probabilities of adverse outcomes.

Bayesian statistics: Simulation-based methods are important in computational Bayesian inference, particularly for models where direct calculations are difficult.

Experimental design: Different experimental designs can be evaluated computationally before actual data collection.

13.10 Advantages and Limitations of Monte Carlo Methods

13.11 Statistical Packages

A statistical package is software designed to provide computational tools for statistical analysis. Depending on the package, these tools may include data entry, data cleaning, descriptive statistics, probability calculations, hypothesis testing, regression, analysis of variance, multivariate analysis, graphics and reporting.

Statistical packages reduce the amount of manual arithmetic required in statistical analysis. More importantly, they provide tested algorithms and organized workflows that allow researchers to analyze large datasets efficiently.

However, statistical software does not replace statistical reasoning. A computer can calculate a statistic correctly while the researcher uses an inappropriate statistical method or incorrectly interprets the result.

13.12 General Functions of Statistical Packages

Although statistical packages differ in design and specialization, many provide common categories of functionality.

• Data entry: Creating or importing datasets.
• Data management: Sorting, filtering, recoding and transforming variables.
• Descriptive statistics: Calculating measures such as mean, median, variance and standard deviation.
• Data visualization: Producing charts, plots and graphs.
• Probability analysis: Working with probability distributions and probability calculations.
• Inferential statistics: Performing confidence intervals and hypothesis tests.
• Regression and modelling: Estimating relationships between variables and building statistical models.
• Experimental analysis: Supporting methods such as analysis of variance and experimental design.
• Reporting: Producing tables, graphs and statistical output for interpretation and presentation.

13.13 SPSS

SPSS, historically associated with the Statistical Package for the Social Sciences, is a widely used statistical software environment. It is particularly common in social sciences, education, health research, business and survey-based research.

Important Features:

• data entry and data management;
• descriptive statistics;
• frequency analysis;
• cross-tabulation;
• hypothesis testing;
• correlation and regression;
• analysis of variance;
• survey and questionnaire analysis;
• data visualization; and
• graphical user-interface-based statistical analysis.

SPSS is particularly attractive to beginners because many procedures can be performed through menus and dialog boxes without requiring extensive programming.

Example Application:

A researcher studying the relationship between students' study hours and examination scores could enter the observations into SPSS, calculate descriptive statistics, produce a scatter plot, calculate a correlation coefficient and fit a regression model.

The researcher must still determine whether correlation and regression are appropriate and how their assumptions should be evaluated.

13.14 SAS

SAS is a comprehensive analytics and statistical computing environment widely used in business, government, health research, finance and large-scale data analysis.

SAS is particularly recognized for its programming capabilities, data management facilities and extensive statistical procedures.

Important Features:

• data management;
• statistical analysis;
• regression modelling;
• analysis of variance;
• multivariate analysis;
• large-scale data processing;
• report generation;
• business analytics; and
• programmatic and automated analytical workflows.

Unlike software environments that rely primarily on point-and-click operations, SAS allows analysts to construct reusable programs for complex analytical tasks.

13.15 Minitab

Minitab is statistical software designed to support data analysis, statistical quality control, process improvement and experimental analysis.

It is widely associated with quality management and industrial statistics.

Important Applications:

• descriptive statistics;
• hypothesis testing;
• regression;
• analysis of variance;
• design of experiments;
• quality control charts;
• process capability analysis; and
• statistical process improvement.

For example, a manufacturing organization can use Minitab to examine production measurements, identify process variation and determine whether a process is statistically stable.

13.16 GENSTAT

GENSTAT is a statistical software system with strong applications in agricultural research, experimental design, biological sciences and related fields.

Its capabilities include statistical modelling, experimental design, analysis of variance, regression and data analysis.

GENSTAT has been particularly useful where researchers need to analyze structured experiments involving treatments, blocks, repeated observations and other experimental arrangements.

Example Application:

Suppose an agricultural researcher compares four fertilizer treatments using a replicated experimental design. GENSTAT can be used to organize the experimental data and conduct an analysis appropriate to the design.

The software does not determine the scientific validity of the experiment. The researcher must correctly specify the experimental design and identify the appropriate statistical model.

13.17 EPI-INFO

Epi Info is a software environment developed for public health and epidemiological applications. It provides facilities for data collection, database management, analysis and visualization.

It has been particularly useful in public health investigations where researchers need to collect and analyze health-related information rapidly.

Applications:

• epidemiological investigations;
• public health surveys;
• questionnaire development;
• data entry and management;
• descriptive epidemiological analysis;
• outbreak investigation; and
• presentation of public health data.

For example, during an investigation of a suspected disease outbreak, researchers may collect information about individuals, exposure factors and disease status and then use statistical analysis to examine patterns in the collected data.

13.18 SYSTAT

SYSTAT is a statistical software environment designed for data analysis and statistical modelling. It has supported applications involving descriptive statistics, inferential statistics, regression, analysis of variance and graphical analysis.

Its facilities can be used in scientific, academic, business and research settings where statistical analysis and visualization are required.

Like other statistical packages, its usefulness depends on the researcher's ability to select appropriate methods and correctly interpret the output.

13.19 Comparison of Major Statistical Packages

The table should not be interpreted as a strict ranking. Statistical software evolves, features change across versions, and many packages can perform similar statistical procedures. The appropriate choice depends on the research question, data, required methods, institutional environment, reproducibility requirements and user's expertise.

13.20 Choosing a Statistical Package

The choice of statistical software should be based on the analytical problem rather than simply on the popularity of a particular package.

Nature of the data: Consider whether the dataset contains continuous variables, categorical variables, repeated measurements, time series, spatial information or other specialized structures.

Statistical method: The package should support the statistical procedures required for the research.

Data size: Large datasets may require software capable of efficient data management and processing.

Ease of use: A graphical interface may be useful for beginners and researchers performing routine analyses.

Programming capability: Programming-based environments are particularly useful when analyses must be repeated, automated or documented precisely.

Reproducibility: Analytical workflows should ideally be recorded so that the analysis can be repeated and independently checked.

Visualization: The ability to produce clear and informative graphs may be important when communicating results.

Institutional requirements: Universities, research organizations and companies may have established software environments and licensing arrangements.

13.21 Statistical Packages and the Statistical Workflow

Statistical software is most effective when integrated into a sound statistical workflow.

This workflow demonstrates why statistical computing is more than pressing buttons in a software package. Sound analysis requires statistical reasoning before, during and after computation.

13.22 Common Mistakes in the Use of Statistical Software

1. Choosing a method because it is available. The availability of a statistical procedure does not mean that it is appropriate for a particular research problem.

2. Ignoring assumptions. Many statistical procedures depend on assumptions concerning independence, distributional form, variance or other properties of the data.

3. Treating software output as the conclusion. Numerical output must be interpreted in the context of the research question.

4. Failing to inspect the raw data. Errors in data entry, missing observations and unusual values can substantially affect results.

5. Relying on default settings without understanding them. Software defaults may not always correspond to the requirements of a particular analysis.

6. Reporting excessive decimal places. A computer can display many decimal places, but this does not mean that the data justify such precision.

7. Confusing statistical significance with practical importance. A statistically significant result may have little practical importance, while an important practical effect may fail to achieve statistical significance in a small study.

8. Failing to document computational procedures. Researchers should retain sufficient information about data preparation, methods, software and settings to support reproducibility.

13.24 Monte Carlo Methods versus Traditional Analytical Methods

13.25 Integrated Example: Monte Carlo and Statistical Software

13.26 The Role of Reproducibility in Statistical Computing

Reproducibility means that another researcher, using the same data, computational procedures and relevant assumptions, can reproduce the reported analysis and obtain consistent results.

Reproducibility is especially important in Monte Carlo studies because random-number generation is involved.

A reproducible simulation should document:

• the probability distributions used;
• the parameters of those distributions;
• the number of simulation replications;
• the random-number generation method where relevant;
• the random seed when reproducibility requires it;
• the statistical procedures used to summarize results;
• the software and relevant version information; and
• important computational settings.

Similarly, statistical analyses performed using statistical packages should retain sufficient information about data preparation and analytical procedures to allow the work to be checked or repeated.

13.27 Statistical Packages in Modern Statistical Computing

The traditional statistical packages introduced in this chapter form part of a much broader statistical computing environment. Modern statistical analysis may involve programming languages, statistical packages, spreadsheets, databases, cloud platforms and specialized analytical systems.

The underlying principles remain the same: data must be properly collected and prepared, statistical methods must be appropriate to the problem, computations must be performed correctly, assumptions must be examined, and results must be interpreted in context.

For an undergraduate student of statistical computing, the most important skill is therefore not memorizing software menus. It is understanding the relationship between the statistical problem, the data, the method, the computation and the interpretation.

13.28 Key Points

• Monte Carlo methods use repeated random sampling and numerical computation to investigate uncertain systems or estimate quantities.
• Monte Carlo simulation is particularly useful when analytical solutions are difficult or inconvenient.
• A typical Monte Carlo procedure involves defining a problem, specifying a probability model, generating random values, repeating the simulation and summarizing the results.
• Pseudorandom number generators are commonly used by computers to produce sequences suitable for simulation.
• A random seed can be used to make a pseudorandom sequence reproducible.
• Increasing the number of simulations generally improves the precision of a Monte Carlo estimate but does not correct an incorrect model.
• Monte Carlo methods have applications in statistical inference, regression, risk analysis, experimental design and many other areas.
• Statistical packages provide tools for data management, statistical analysis, modelling and visualization.
• SPSS is widely used for accessible statistical analysis, particularly in social sciences, education, health and survey research.
• SAS provides extensive programming, data management and analytical capabilities and is widely used in large-scale statistical and business applications.
• Minitab is strongly associated with statistical quality control, process improvement and experimental analysis.
• GENSTAT has important applications in experimental design, agriculture, biology and scientific research.
• Epi Info supports epidemiological and public health data collection and analysis.
• SYSTAT provides general statistical analysis, modelling and graphical capabilities.
• No statistical package can replace statistical reasoning.
• Correct statistical computing requires appropriate methods, suitable data, valid assumptions, accurate computation and meaningful interpretation.

13.29 Chapter Summary

Monte Carlo methods provide a powerful computational approach to problems involving uncertainty. Instead of relying entirely on analytical derivation, a Monte Carlo method generates random observations according to a specified model and uses repeated computation to estimate quantities of interest. The method can be applied to probability estimation, numerical approximation, statistical inference, risk analysis, regression studies and experimental design.

The accuracy of a Monte Carlo estimate depends on the number of simulations and the quality of the underlying model. Increasing the number of replications generally reduces simulation variability, but it cannot compensate for incorrect assumptions or a poorly specified probability model.

Statistical packages provide the computational infrastructure required for many statistical analyses. SPSS, SAS, Minitab, GENSTAT, Epi Info and SYSTAT are examples of established statistical software environments with different areas of emphasis. SPSS is widely used for accessible general statistical analysis; SAS is particularly strong in programming, data management and large-scale analytics; Minitab is important in quality control and process improvement; GENSTAT is prominent in experimental and agricultural research; Epi Info supports epidemiological and public health analysis; and SYSTAT provides general statistical and modelling capabilities.

Ultimately, statistical computing is not simply the use of software to produce numerical output. Effective statistical analysis requires the researcher to understand the problem, prepare the data, select appropriate methods, perform computations correctly, evaluate assumptions and communicate the meaning of the results.

14.1 Chapter Introduction:

Statistical analysis is ultimately concerned with making informed statements about data and using those statements to understand a population, evaluate alternatives, predict outcomes, or support decisions. A statistical calculation by itself is rarely the final objective. The important question is what the result means and how it should influence action.

In earlier chapters, statistical methods such as estimation, hypothesis testing, regression, correlation, time-series analysis and statistical modelling were introduced. These methods provide mathematical tools for extracting evidence from data. However, every statistical method is based on assumptions, and the conclusions obtained from an analysis depend on whether those assumptions are reasonably satisfied.

This chapter brings together the major ideas of statistical inference and modelling. It explains how statistical assumptions affect the validity of results, how models are constructed and interpreted, how statistical outcomes should be communicated, and how statistical evidence can be translated into practical decisions.

14.2 Learning Objectives:

At the end of this chapter, the student should be able to:

• explain the meaning of statistical inference and statistical modelling;
• identify important assumptions underlying common statistical methods;
• distinguish between statistical assumptions, models, estimates and conclusions;
• interpret statistical results in their proper context;
• distinguish statistical significance from practical significance;
• explain uncertainty associated with statistical conclusions;
• identify common problems that can lead to misleading statistical conclusions;
• communicate statistical findings clearly to technical and non-technical audiences;
• evaluate the real-world implications of statistical results; and
• apply statistical evidence appropriately to decision-making problems.

14.3 Key Terminology:

• Statistical inference: The process of using sample data to draw conclusions about a population or an underlying process.
• Statistical model: A mathematical representation of relationships, variation or uncertainty in observed data.
• Assumption: A condition that is expected to hold for a statistical method or model to produce valid results.
• Parameter: A numerical characteristic of a population, such as a population mean or population proportion.
• Statistic: A numerical quantity calculated from sample data.
• Estimate: A value calculated from sample data and used to approximate an unknown population parameter.
• Confidence interval: An interval calculated from sample data that provides a range of plausible values for an unknown population parameter under a specified confidence procedure.
• Hypothesis test: A statistical procedure used to evaluate evidence concerning a stated hypothesis about a population or process.
• Statistical significance: Evidence from a statistical procedure indicating that an observed result would be relatively unusual under a specified null hypothesis.
• Practical significance: The extent to which a statistical result is large or important enough to matter in a real-world context.
• Uncertainty: The lack of complete knowledge about a population, process, parameter or future outcome.
• Decision-making: The process of selecting an action from available alternatives using relevant evidence, objectives and constraints.

14.4 Statistical Inference

Statistical inference is the process of using information obtained from a sample to make conclusions about a population.

In most practical situations, studying every member of a population is difficult, expensive or impossible. A researcher therefore obtains a sample and uses statistical methods to learn about the larger population.

For example, suppose a manufacturer produces 100,000 electric motors. Testing every motor may be impractical. A sample of 500 motors may be selected and tested. The observed failure rate in the sample can then be used to estimate the failure rate of the entire production population.

Statistical inference commonly involves:

• estimation of population parameters;
• construction of confidence intervals;
• hypothesis testing;
• prediction of future observations;
• comparison of populations or treatments; and
• modelling relationships between variables.

14.5 From Data to Statistical Conclusion

A sound statistical analysis can be viewed as a sequence of connected stages:

The quality of the final decision depends on every stage. Poor-quality data can produce poor estimates. An inappropriate model can produce misleading predictions. Violated assumptions can make statistical tests unreliable. Finally, even a statistically correct result can be misinterpreted when it is removed from its practical context.

14.6 Statistical Assumptions

Statistical assumptions are conditions concerning the data, sampling process or probability structure that support the validity of a statistical method.

Different statistical procedures require different assumptions. There is therefore no single list of assumptions that applies to every statistical analysis.

14.6.1 Independence:

Independence means that the relevant observations do not provide dependent information in a way that violates the statistical model.

For example, measurements obtained from 100 randomly selected individuals may reasonably be treated as independent in some studies. However, measurements collected repeatedly from the same individual are generally not independent because observations from the same person may be related.

Ignoring dependence can result in underestimated uncertainty and misleading conclusions.

14.6.2 Random Sampling:

Many inferential procedures assume that the sample is obtained through an appropriate sampling process. Random sampling helps reduce systematic selection effects and provides a basis for generalizing findings from a sample to a population.

A sample that consists only of individuals who voluntarily respond to an online survey may not represent the target population if the characteristics of respondents differ systematically from those who do not respond.

14.6.3 Normality:

Some statistical procedures assume that a variable, an error term, or a sampling distribution follows a normal distribution, depending on the method being used.

The normality assumption should not be treated as a requirement for every statistical analysis. Many procedures are reasonably robust to departures from normality, particularly with sufficiently large and appropriately collected samples.

14.6.4 Homogeneity of Variance:

Some comparison procedures assume that the variability of observations is similar across groups. This condition is commonly described as homogeneity of variance.

For example, when comparing examination scores from two groups using a method that assumes equal variances, substantially different variances may affect the validity of the conventional procedure.

14.6.5 Linearity:

Linear regression assumes that the relationship represented by the model is appropriately described as linear in the relevant predictors and response structure.

If the true relationship is strongly curved but a straight-line model is imposed without appropriate transformation or modelling, predictions and interpretations may be poor.

14.6.6 Absence of Serious Outliers:

An outlier is an observation that is unusually distant from the general pattern of the data. Outliers may result from measurement errors, data-entry errors, unusual but genuine observations, or other causes.

Outliers should not automatically be deleted. Their origin should be investigated before deciding how to handle them.

14.7 Why Assumptions Matter

Statistical assumptions matter because statistical methods do not operate independently of the conditions under which they are used.

Consider a simple example involving a sample mean. Suppose a researcher obtains a sample mean of 72 and wishes to use it to estimate a population mean. The value 72 is a statistic calculated from the sample. Its usefulness as an estimate depends partly on how the sample was obtained and the characteristics of the underlying population.

If the sample was selected systematically from only the highest-performing members of the population, the sample mean may not provide a reliable estimate of the overall population mean, even though the arithmetic calculation is correct.

14.8 Checking Statistical Assumptions

Assumptions should be evaluated using a combination of knowledge about the study design, descriptive statistics, graphical methods and, where appropriate, formal diagnostic procedures.

For example, in a regression analysis, a researcher may examine:

• scatter plots to investigate relationships between variables;
• residual plots to investigate systematic patterns;
• the distribution of residuals;
• the spread of residuals across fitted values;
• unusual or influential observations; and
• the design and sampling procedure used to collect the data.

The goal is not simply to prove that assumptions are true. In real data analysis, assumptions are often approximations. The objective is to determine whether departures from the assumptions are serious enough to affect the intended conclusions.

14.9 Statistical Models

A statistical model is a mathematical representation of a data-generating process that incorporates both systematic relationships and random variation.

A simple statistical model can be expressed conceptually as:

Observed outcome = systematic component + random variation

For example, a simple linear regression model can be written as:

Y = β₀ + β₁X + ε

Where:

• Y is the response variable;
• X is the explanatory or predictor variable;
• β₀ is the intercept;
• β₁ is the slope coefficient; and
• ε represents random error or unexplained variation.

The model does not claim that every observation lies exactly on a straight line. Instead, it provides a mathematical framework for describing the systematic relationship while allowing for unexplained variation.

14.10 Types of Statistical Models

Statistical models can be classified in several ways depending on the purpose and mathematical structure of the analysis.

14.11 Statistical Methodologies

A statistical methodology is a systematic approach for obtaining, analysing and interpreting statistical information.

Important methodologies encountered in statistical computing include:

• Descriptive statistics: summarizing observed data using measures such as the mean, median, standard deviation, percentages and graphical displays.
• Estimation: using sample data to estimate unknown population parameters.
• Hypothesis testing: evaluating evidence concerning statistical hypotheses.
• Regression analysis: investigating relationships between response and explanatory variables.
• Correlation analysis: measuring the strength and direction of association between variables.
• Time-series analysis: studying observations collected sequentially over time.
• Experimental analysis: evaluating differences between treatments or experimental conditions.
• Simulation: using computational procedures to imitate random or complex processes.

14.12 Interpretation of Statistical Results

Interpretation is the process of explaining what a statistical result means in relation to the research question, population, variables and study design.

A statistical result should never be interpreted as an isolated number. Its meaning depends on the context in which it was obtained.

Example: Mean:

Suppose the average daily electricity consumption of a group of households is reported as 14.8 kWh.

The number 14.8 kWh tells us the average observed consumption. However, interpretation requires additional information. We may need to know the number of households, the period of observation, the sampling method, the variability of consumption and the characteristics of the households.

Example: Correlation:

Suppose the correlation between advertising expenditure and sales revenue is r = 0.82.

This indicates a strong positive linear association in the analysed data. It means that higher advertising expenditure tends to be associated with higher sales revenue in the observed data.

However, correlation alone does not establish that advertising expenditure caused the increase in sales. Other variables may contribute to the observed relationship.

14.13 Statistical Significance and Practical Significance

One of the most important distinctions in statistical interpretation is the difference between statistical significance and practical significance.

A statistically significant result provides evidence against a specified null hypothesis under the assumptions of the statistical procedure. It does not automatically mean that the effect is large, important or useful in practice.

For example, suppose a new production process reduces the average production time by 0.02 minutes and a statistical test produces a very small p-value because the sample contains hundreds of thousands of observations.

The result may be statistically significant, but the reduction may have little practical importance if the cost of implementing the new process is very high.

14.14 Understanding the p-Value

The p-value is a probability calculated under the assumptions of a specified statistical model and null hypothesis. It measures how compatible the observed data, or something more extreme according to the test statistic, are with the null hypothesis.

A small p-value provides evidence against the null hypothesis, while a large p-value indicates that the data do not provide strong evidence against the null hypothesis under the specified test.

However, the p-value should not be interpreted as the probability that the null hypothesis is true.

The p-value must be interpreted together with the effect size, confidence interval, study design, sample size and practical context.

14.15 Confidence Intervals and Uncertainty

Statistical conclusions based on samples are uncertain because a different sample would generally produce somewhat different results.

A confidence interval provides a range of values generated by a specified confidence procedure. It communicates both an estimate and information about sampling uncertainty.

Worked Example: Confidence Interval for a Mean:

Question: A random sample of 64 students has a mean examination score of 68.5. Suppose the population standard deviation is known to be 8. Determine a 95% confidence interval for the population mean.

Given:

• Sample mean, x̄ = 68.5
• Population standard deviation, σ = 8
• Sample size, n = 64
• 95% confidence level
• For a 95% confidence interval, z_0.025 = 1.96

Required: Confidence interval for the population mean.

Formula:

x̄ ± z × σ/√n

Solution:

First calculate the standard error:

σ/√n = 8/√64 = 8/8 = 1

The margin of error is:

1.96 × 1 = 1.96

Therefore:

68.5 ± 1.96

The lower limit is:

68.5 − 1.96 = 66.54

The upper limit is:

68.5 + 1.96 = 70.46

Answer: The 95% confidence interval is approximately (66.54, 70.46).

The interval provides a range of plausible values for the population mean under the specified confidence procedure.

14.16 Interpreting Regression Models

Regression models are particularly useful because they can describe relationships between variables and provide predictions.

Consider the fitted regression equation:

ŷ = 25 + 4.2X

Suppose Y represents monthly sales in thousands of naira and X represents the number of advertising campaigns.

The intercept is 25. According to the model, when X = 0, the predicted value of Y is 25 thousand naira.

The slope is 4.2. For a one-unit increase in X, the model predicts an average increase of 4.2 thousand naira in Y, assuming the model and its interpretation are appropriate.

Prediction Example:

For 5 advertising campaigns:

ŷ = 25 + 4.2(5)

ŷ = 25 + 21 = 46

Answer: The predicted sales are 46 thousand naira, or ₦46,000, according to the model.

This is a model-based prediction, not a guarantee that actual sales will equal ₦46,000.

14.17 Interpreting Model Fit

A model should not be judged solely by whether it produces a numerical answer. The analyst must determine whether the model provides an adequate representation of the data for the intended purpose.

Model evaluation may involve:

• examining residuals;
• comparing predicted and observed values;
• evaluating measures of predictive performance;
• examining confidence intervals;
• checking model assumptions;
• investigating influential observations;
• comparing alternative models; and
• considering whether the model is appropriate for the scientific or practical question.

A model can fit historical data well and still perform poorly on new data. This is one reason why training and testing procedures are important in predictive modelling.

14.18 Overfitting and Underfitting

Overfitting occurs when a model captures not only the underlying pattern in the training data but also random fluctuations or noise. Such a model may perform well on the data used to construct it but poorly on new observations.

Underfitting occurs when a model is too simple to capture important patterns in the data.

14.19 Statistical Results and Real-World Implications

Statistical results become meaningful when they are connected to the problem that generated the data.

Consider a public-health study investigating whether a new intervention reduces average waiting time in hospitals. Suppose the analysis finds a statistically significant reduction of 1.5 minutes.

The statistical result tells us that the observed difference provides evidence against the null hypothesis under the assumptions of the analysis. However, decision-makers must ask additional questions:

• Is a 1.5-minute reduction practically important?
• How much does the intervention cost?
• Does the intervention introduce other risks?
• Does the improvement occur across all hospitals or only some?
• How precise is the estimated reduction?
• Are there other benefits or disadvantages?
• Can the result reasonably be generalized to other hospitals?

Statistical analysis therefore provides evidence for decision-making, but it does not replace professional judgement, domain knowledge, ethical considerations or economic analysis.

14.20 Statistical Evidence Does Not Automatically Imply Causation

One of the most important principles in statistical interpretation is that association does not automatically imply causation.

Suppose data show that students who spend more hours using educational software tend to obtain higher examination scores. The observed association does not by itself prove that the software caused the higher scores.

Other factors could contribute to the relationship, including:

• previous academic ability;
• motivation;
• access to educational resources;
• study time;
• teacher support; or
• other variables not included in the analysis.

Strong causal conclusions generally require an appropriate research design, such as a well-designed randomized experiment, or carefully justified observational methods with suitable assumptions.

14.21 Common Sources of Misleading Conclusions

Even technically sophisticated statistical analyses can produce misleading conclusions when the underlying study or interpretation is flawed.

Biased Sampling:

If some members of the population are systematically more likely to be included than others, the resulting sample may not adequately represent the population.

Small or Inadequate Samples:

A sample that is too small may produce estimates with substantial uncertainty and may have insufficient ability to detect important effects.

Multiple Testing:

When many hypotheses are tested without appropriate consideration of multiplicity, the probability of obtaining apparently significant findings by chance can increase.

Confounding:

A confounding variable is associated with both the explanatory variable and the outcome and can distort the apparent relationship between them.

Measurement Error:

Incorrect or imprecise measurement can reduce the quality of statistical conclusions.

Data Leakage:

In predictive modelling, information that would not actually be available when making a future prediction must not improperly enter the training process. Such leakage can make model performance appear better than it really is.

Misinterpretation of p-Values:

Researchers sometimes treat a p-value as a direct measure of the size or importance of an effect. It is neither.

14.22 Communicating Statistical Results

Statistical analysis is only useful when its findings can be communicated accurately to the intended audience.

A good statistical report should normally identify:

• the problem or research question;
• the data and population of interest;
• the statistical method used;
• important assumptions;
• the principal numerical results;
• the uncertainty associated with the results;
• the interpretation of the findings;
• important limitations; and
• the practical implications.

Technical Audience:

A technical audience may require detailed information about the statistical model, parameter estimates, standard errors, confidence intervals, diagnostic procedures and assumptions.

Non-Technical Audience:

A non-technical audience generally benefits from clear explanations of the main finding, its practical importance, uncertainty and implications rather than an excessive collection of statistical terminology.

14.23 A Framework for Reporting Statistical Findings

A useful structure for communicating a statistical result is:

1. State the question: What problem was investigated?
2. Describe the data: What population and sample were involved?
3. State the method: What statistical procedure or model was used?
4. Report the result: Present the relevant estimate, test statistic, confidence interval, p-value or model measure.
5. Interpret the result: Explain what the number means in context.
6. Discuss uncertainty: State how precise the result is and identify important limitations.
7. Explain practical implications: Describe what the result could mean for the decision or problem.

Example of Good Reporting:

Suppose a study estimates that a new teaching method increases the average examination score by 6.2 marks, with a 95% confidence interval of 3.1 to 9.3 marks.

A suitable interpretation would be:

“The study estimated that the new teaching method is associated with an average increase of 6.2 examination marks. The 95% confidence interval ranges from 3.1 to 9.3 marks, indicating uncertainty around the estimated effect. The estimated improvement may therefore be educationally meaningful, although the final decision should also consider implementation cost, study design and the population to which the findings are intended to apply.”

14.24 Statistical Analysis and Decision-Making

Decision-making involves selecting an action when there may be uncertainty about the consequences. Statistics helps quantify and communicate that uncertainty.

A decision-maker may use statistical evidence to:

• compare alternative products;
• evaluate medical treatments;
• forecast demand;
• estimate financial risk;
• determine whether a manufacturing process has changed;
• evaluate educational interventions;
• allocate limited resources;
• assess environmental measurements; and
• plan future operations.

Statistical evidence should normally be combined with other relevant information such as cost, risk, feasibility, legal requirements, ethical considerations and organizational objectives.

14.25 Worked Decision-Making Example

Question: A company is considering replacing its current machine with a new machine. The current machine produces an average of 100 units per hour. A sample of observations from the new machine gives a mean of 108 units per hour. The company estimates that the new machine will cost ₦5,000,000 to install. How should the statistical result be considered in the decision?

Given:

• Current production rate = 100 units/hour
• Estimated new-machine production rate = 108 units/hour
• Estimated improvement = 8 units/hour
• Installation cost = ₦5,000,000

Step 1: Quantify the estimated improvement.

108 − 100 = 8 units/hour

The estimated increase is therefore 8 units per hour.

Step 2: Determine whether the evidence supports a genuine improvement.

A hypothesis test or confidence interval should be used to determine whether the observed difference is sufficiently supported by the data. The decision should not be based solely on the difference between the two sample means.

Step 3: Evaluate practical importance.

Even if the statistical analysis establishes that the new machine increases production, management must determine whether the additional production justifies the ₦5,000,000 investment.

Step 4: Consider uncertainty.

The estimated increase of 8 units per hour is based on sample data and therefore contains uncertainty. A confidence interval for the difference would provide useful information about the plausible size of the improvement.

Step 5: Consider other factors.

Maintenance cost, energy consumption, downtime, product quality, training requirements, machine life and safety may all influence the final decision.

Conclusion: Statistical analysis can establish evidence concerning the production improvement, but the final investment decision should combine that evidence with economic, operational and technical considerations.

14.26 Decision-Making Under Uncertainty

Statistical decision-making recognizes that uncertainty cannot always be eliminated. Instead, the objective is to make better decisions by quantifying available evidence and understanding the possible consequences.

For example, a retailer deciding how much stock to order may face uncertain future demand. Ordering too little can result in lost sales, while ordering too much can result in excess inventory.

A statistical forecast can estimate expected demand and quantify uncertainty. The decision-maker can then combine the forecast with storage costs, ordering costs and the consequences of shortages.

This illustrates an important principle:

14.27 Statistical Modelling in Different Fields

14.29 Key Points

• Statistical inference uses sample information to draw conclusions about populations.
• Statistical models provide mathematical representations of relationships, variation and uncertainty.
• Statistical methods depend on assumptions concerning the data, sampling process and model structure.
• Important assumptions may include independence, appropriate sampling, normality, homogeneity of variance and linearity, depending on the method.
• Assumptions should be evaluated rather than blindly accepted.
• A statistically correct calculation does not guarantee a scientifically valid conclusion.
• Statistical significance and practical significance are different concepts.
• A p-value should be interpreted in relation to a specified null hypothesis and statistical procedure.
• Confidence intervals communicate estimates together with information about uncertainty.
• Regression models can describe relationships and support prediction, but predictions are subject to model assumptions and uncertainty.
• Overfitting occurs when a model captures noise in training data rather than generalizable structure.
• Correlation does not by itself establish causation.
• Statistical results must be interpreted within the context of the research question and study design.
• Statistical evidence should be combined with practical, technical, economic and ethical considerations when making decisions.
• Effective statistical communication requires clear reporting of methods, results, uncertainty, limitations and implications.
• The purpose of statistical decision-making is not to eliminate uncertainty but to make better-informed decisions under uncertainty.

14.30 Chapter Summary

Statistical inference provides a framework for learning about populations from sample data. Because sample observations contain only partial information about a population, statistical conclusions are inherently uncertain. Estimation, confidence intervals, hypothesis testing, regression and other statistical procedures provide methods for quantifying and interpreting that uncertainty.

Statistical models provide structured representations of relationships and variation in data. Their usefulness depends on appropriate assumptions, suitable data and a model that is relevant to the problem being investigated. Assumptions such as independence, appropriate sampling, normality, homogeneity of variance and linearity may be important depending on the statistical method.

The interpretation of statistical results requires more than reading numerical outputs from statistical software. Estimates, confidence intervals, p-values, regression coefficients and other measures must be interpreted in the context of the research question and study design. Statistical significance should be distinguished from practical significance, and associations should not automatically be interpreted as causal relationships.

Statistical modelling and inference become particularly valuable when their results are connected to real-world decisions. In healthcare, finance, manufacturing, education, agriculture, government and many other fields, statistical evidence can support decisions involving uncertainty. Nevertheless, statistical results should be considered alongside domain knowledge, costs, risks, feasibility, ethical requirements and other relevant information.

The complete process can therefore be summarized as:

Data → Assumptions → Model → Statistical Analysis → Inference → Interpretation → Uncertainty → Practical Implications → Decision

Effective statistical computing is therefore not simply the ability to calculate statistical quantities. It is the ability to select appropriate methods, understand their assumptions, evaluate their results, communicate evidence clearly and use statistical knowledge responsibly in solving real problems.

14.31 Worked Review Examples

Example 1: Interpreting a Confidence Interval:

A study estimates that the average monthly household expenditure on electricity is ₦48,000, with a 95% confidence interval of ₦44,000 to ₦52,000.

Interpretation: The sample provides an estimate of ₦48,000 for the population mean, while the specified 95% confidence procedure produces an interval from ₦44,000 to ₦52,000. The interval communicates uncertainty about the population mean rather than claiming that every household spends within that range.

Example 2: Interpreting a Regression Coefficient:

A fitted model is:

ŷ = 120 + 15X

If X represents hours of employee training and Y represents productivity units per day, the slope of 15 means that, according to the fitted model, a one-hour increase in training is associated with an estimated increase of 15 productivity units per day, within the range and conditions for which the model is appropriate.

Example 3: Statistical Versus Practical Importance:

A study with a very large sample finds that a new procedure reduces average waiting time by 0.4 seconds with a very small p-value.

The result may be statistically significant. However, if the reduction has no meaningful effect on service quality or operating efficiency, its practical significance may be negligible.

14.32 End-of-Chapter Exercises

Part A: Basic Understanding:

1. Define statistical inference.
2. What is a statistical model?
3. Distinguish between a parameter and a statistic.
4. What is meant by a statistical assumption?
5. State four assumptions that may be relevant to statistical analysis.
6. Explain why statistical conclusions based on samples are uncertain.
7. Define statistical significance.
8. Define practical significance.
9. What is an outlier?
10. Explain why correlation does not necessarily imply causation.

Part B: Understanding and Interpretation:

1. Explain why a statistically significant result may have little practical importance.
2. Explain why statistical software cannot replace statistical reasoning.
3. Discuss the importance of checking assumptions before applying a statistical procedure.
4. Explain how confidence intervals help communicate uncertainty.
5. Distinguish between overfitting and underfitting.
6. Explain why sample selection affects the validity of statistical inference.
7. Discuss why a large sample does not automatically eliminate bias.

Part C: Applied Questions:

1. A company reports that a new production method increases output by 3%. Explain what additional statistical information would be useful before deciding whether to adopt the method.
2. A researcher obtains a correlation of r = 0.91 between two variables. Explain what the result means and identify what cannot be concluded from it alone.
3. A regression model predicts that a machine will produce 750 units per day. Explain why the prediction should not automatically be treated as a guaranteed outcome.
4. A study reports a p-value of 0.02. Explain the appropriate interpretation and identify two incorrect interpretations.
5. A confidence interval for a population mean is reported as (42.5, 48.7). Explain what information the interval provides.

Part D: Higher-Order Questions:

1. A hospital introduces a new appointment system. The mean waiting time decreases from 42 minutes to 39 minutes, and the difference is statistically significant. Discuss the statistical and practical considerations that should be examined before recommending the system for nationwide implementation.
2. A predictive model performs extremely well on its training data but poorly on new observations. Explain the likely problem and propose appropriate approaches for evaluating and improving the model.
3. A researcher finds a strong association between social-media use and academic performance. Construct a reasoned argument explaining why the association should not automatically be interpreted as a causal effect.
4. A government agency must choose between two public-health interventions. Explain how statistical evidence, uncertainty, costs, risks and practical considerations can be combined to support the decision.
5. Discuss the statement: “The purpose of statistical analysis is not merely to obtain significant results but to produce reliable evidence that can support sound decisions.”