When we calculate a statistic from a sample, such as the sample mean, we usually intend to learn something about a larger population. The sample statistic, however, is not expected to be exactly equal to the corresponding population parameter. A different random sample would probably produce a slightly different result.
The standard error measures this sampling uncertainty.
More precisely, the standard error of an estimator is the standard deviation of its sampling distribution: the distribution of the values that the estimator would produce across many hypothetical samples drawn in the same way from the same population.
The standard error of the mean
Suppose we want to estimate a population mean \(\mu\) using the sample mean \(\bar{x}\). If we repeatedly drew samples of the same size, each sample would produce a different value of \(\bar{x}\). Some sample means would be slightly higher than the true population mean and others slightly lower.
The standard error of the sample mean describes how widely these sample means would vary.
When the observations are independent and come from the same population, the estimated standard error of the mean is:
$$ SE(\bar{x}) = \frac{s}{\sqrt{n}} $$where:
- \(s\) is the sample standard deviation;
- \(n\) is the sample size.
The formula shows two important things. The standard error is larger when the observations themselves are more variable, and smaller when the sample contains more observations.
Because the sample size appears under a square root, increasing the sample size has diminishing returns. To reduce the standard error by half, we generally need four times as many observations.
Standard deviation and standard error
The standard deviation and the standard error are related, but they describe different kinds of variation.
The standard deviation describes the variation among individual observations. For example, it may describe how much individual heights differ from their average height.
The standard error of the mean describes the uncertainty in the estimated mean. It tells us how much the sample mean would tend to change if we repeated the sampling procedure.
A dataset may therefore have a large standard deviation but a small standard error. Individual observations may vary greatly, while the mean is estimated relatively precisely because the sample is large.
The distinction can be summarised as follows:
- standard deviation: variation in the data;
- standard error: uncertainty in an estimate derived from the data.
What the standard error is used for
The standard error is one of the main links between descriptive statistics and statistical inference. It allows us to move from a result observed in one sample to statements about the population from which the sample was drawn.
Confidence intervals
A confidence interval combines an estimate with its standard error to express the precision of that estimate.
For a sample mean, an approximate confidence interval has the form:
$$ \bar{x} \pm \text{critical value} \times SE(\bar{x}) $$For example, under appropriate assumptions, a 95% confidence interval may be calculated using a critical value from the Student’s \(t\) distribution:
$$ \bar{x} \pm t_{0.975,n-1} \times SE(\bar{x}) $$A small standard error produces a narrow confidence interval, indicating a more precise estimate. A large standard error produces a wider interval, reflecting greater uncertainty.
Hypothesis tests
Standard errors are also used in statistical hypothesis testing. A test statistic often compares the distance between an observed estimate and a value proposed by a null hypothesis with the estimate’s standard error:
$$ \text{test statistic} = \frac{\text{estimate} - \text{hypothesised value}} {SE(\text{estimate})} $$This ratio asks whether the observed difference is large relative to the amount of sampling variation that would normally be expected.
Comparing estimates
Standard errors can help us compare estimates from different groups, periods, or models. For example, when comparing the mean outcome of two groups, the uncertainty of the difference depends on the standard errors of both group means.
Standard errors are also reported for regression coefficients, proportions, correlations, risk estimates, and many other statistics. The exact formula changes, but the underlying interpretation remains the same: the standard error measures how much an estimate would vary across repeated samples.
A simple example
Suppose a sample of 100 people has an average height of 175 cm and a standard deviation of 10 cm. The estimated standard error of the mean is:
$$ SE(\bar{x}) = \frac{10}{\sqrt{100}} = 1 \text{ cm} $$The standard deviation of 10 cm describes the variation among individual heights. The standard error of 1 cm describes the uncertainty in the estimated population mean.
If the sample instead contained 400 people with approximately the same standard deviation, the standard error would be:
$$ SE(\bar{x}) = \frac{10}{\sqrt{400}} = 0.5 \text{ cm} $$The variation among individuals has not changed, but the larger sample gives a more precise estimate of the mean.
What a small standard error does not guarantee
A small standard error means that an estimator is relatively stable under repeated sampling according to the assumed sampling process. It does not necessarily mean that the estimate is close to the true population value.
A large but systematically biased sample may produce a very small standard error while still giving a misleading estimate. Standard errors do not automatically account for problems such as:
- biased sampling;
- measurement errors;
- missing data;
- dependent observations;
- an inappropriate statistical model.
The usual formulas also depend on assumptions about how the data were collected and how observations relate to one another. When these assumptions do not hold, alternative methods may be needed, such as clustered or heteroskedasticity-robust standard errors, survey-design corrections, or bootstrap estimates.
The central idea
The standard error does not describe uncertainty in an abstract sense. It describes the expected variation of a particular estimator under a particular sampling procedure and statistical model.
It is therefore best understood as a measure of sampling precision:
How much would this estimate tend to change if we repeated the study under the same conditions?
This is why the standard error is fundamental to confidence intervals, hypothesis tests, and most forms of statistical inference.
Αφήστε ένα σχόλιο
Τα σχόλια ελέγχονται πριν δημοσιευθούν και εμφανίζονται με την επόμενη ενημέρωση του ιστότοπου.