Estimation

In statistical inference, we use sample data to draw conclusions about the entire population. Estimation is the process of determining the most likely value (or range of values) for an unknown population parameter, such as the true mean compressive strength of a concrete mix ($\mu$) or the true proportion of defective rivets in a bridge ($\pi$).

A point estimate is a single value calculated from sample data. For example, the sample mean $\bar{x}$ is a point estimate of the population mean $\mu$. The sample variance $s^2$ is a point estimate of the population variance $\sigma^2$.

Not all estimators are created equal. A good estimator should be:

• Unbiased: The expected value of the estimator equals the true population parameter (e.g., $E[\bar{X}] = \mu$). If we take many samples, the average of our estimates will center exactly on the true value.
• Minimum Variance (Efficient): Among all unbiased estimators, the one with the smallest variance (tightest spread) is preferred. It consistently provides estimates closer to the true value.

How do statisticians derive these formulas (like $\bar{x}$ or $s^2$) in the first place?

• Method of Moments: Equates sample moments (like the sample mean or variance) to population moments to solve for unknown parameters.
• Maximum Likelihood Estimation (MLE): Finds the parameter value that makes the observed sample data the most "likely" to have occurred. It is the most robust and widely used mathematical method for deriving estimators.

Because a point estimate will almost never exactly equal the true parameter due to sampling error, we construct a Confidence Interval (CI). A CI provides a range of values and a level of confidence (e.g., 95%) that the true parameter lies within that range.

The term $(\text{Critical Value} \times \text{Standard Error})$ is called the Margin of Error ($E$).

If we know the true standard deviation $\sigma$ (rare in practice, but possible with extensive historical data), we use the Standard Normal ($Z$) distribution.

• $\bar{x}$: Sample mean
• $Z_{\alpha/2}$: Critical value from Standard Normal distribution
• $\sigma$: Population standard deviation
• $n$: Sample size

This is the most common scenario. We must estimate $\sigma$ using the sample standard deviation $s$. Because of this added uncertainty, we use the wider Student's t-distribution with $n-1$ degrees of freedom.

• $\bar{x}$: Sample mean
• $t_{\alpha/2, n-1}$: Critical value from Student's t-distribution with $n-1$ degrees of freedom
• $s$: Sample standard deviation
• $n$: Sample size

Used to compare the averages of two distinct populations (e.g., comparing the strength of concrete from two different suppliers).

• $\bar{x}_1, \bar{x}_2$: Sample means of populations 1 and 2
• $Z_{\alpha/2}$: Critical value from Standard Normal distribution
• $\sigma_1^2, \sigma_2^2$: Population variances of populations 1 and 2
• $n_1, n_2$: Sample sizes of populations 1 and 2

Used for categorical data (e.g., the percentage of structural beams failing an inspection). Let $p$ be the sample proportion (number of successes divided by sample size). If the sample size is large enough (both $np \ge 5$ and $n(1-p) \ge 5$), the sampling distribution of $p$ is approximately normal.

• $p$: Sample proportion
• $Z_{\alpha/2}$: Critical value from Standard Normal distribution
• $n$: Sample size

Estimating the variability of a process, crucial for quality control. Because the sampling distribution of the sample variance ($s^2$) is not symmetric, we use the heavily skewed Chi-Square ($\chi^2$) distribution with $n-1$ degrees of freedom. The interval is not symmetric around $s^2$.

While a confidence interval bounds a population parameter (like the mean $\mu$), engineers often need to bound future individual measurements.