StreamSnagSample study guide. Made by StreamSnag from an invented lecture, written for this purpose and not taken from any real course. The guide, quiz and flashcards were written by AI from the lecture's captions and slide text. The times open the sample study page, which has no video.Reading editionStudy guideStudy pageAll samples
Statistics for the life sciences · Study guide

From a sample to a claim: standard error, confidence intervals and p-values

  • 28 min lecture
  • 5 key ideas
  • 2 review sheets
  • 8 questions
  • 9 flashcards

Made by StreamSnag with AI from the transcript of this lecture. Every part points to the minute it comes from: check anything important against the lecture itself.

Big picture

The lecture asks how a single sample can support a claim about a whole population. It first shows that the sample mean varies from sample to sample in a predictable way (the sampling distribution). That variation is measured by the standard error, which gives confidence intervals, with t for small samples. The same logic yields hypothesis tests and p-values, and the lecture ends with their misreadings, error types and multiple testing.

After this lecture you should be able to

  • explain the difference between a parameter and a statistic and describe the sampling distribution of the mean (centre, spread, shape)
  • calculate a standard error and a 95% or 99% confidence interval from a sample mean, s and n, using t when the sample is small
  • distinguish standard deviation from standard error and predict how sample size changes the standard error
  • interpret a 95% confidence interval correctly, as a statement about the method and not about one interval
  • carry out a one-sample hypothesis test with z, decide whether to reject H0, and link the result to the confidence interval
  • compare type I and type II errors and power, and decide how to adjust alpha with a Bonferroni correction when running many tests

Key ideas

1

Why does the sample mean vary between samples, and how is that variation described?Core

01:35 · slide 2also at02:2403:0004:4505:0905:21

The sample mean varies because each survey picks different students, while the population mean stays fixed. The population is everybody the question is about, and its true average, mu, is a parameter: a fixed number you never learn exactly. The sample mean, x-bar, is a statistic and changes with every new sample. Repeat the survey thousands of times and draw a histogram of the means, not of students: that is the sampling distribution of the mean. It is centred on mu, so the sample mean is unbiased. Its spread is sigma over root n. Its shape is approximately normal, even if the population is not, by the central limit theorem.

In the lecturer's words“That group, everybody we care about, is the population, and the true average for that whole population gets a Greek letter, mu.”
Example

A survey of 64 students gave a mean of 6.8 hours, but a different 64 students might give 6.6 or 7.1, even though the true mean has not moved.

Ask yourself

Why can the sampling distribution of the mean be centred on mu and still be different from the distribution of individual students' sleep?

2

How does the standard error differ from the standard deviation, and how does sample size affect it?Core

07:36 · slide 4also at07:1607:4408:0108:25

The standard deviation measures how much individual people differ, while the standard error measures how much the sample mean would vary from survey to survey. The standard error is the standard deviation divided by the square root of n. Collecting more data does not shrink the standard deviation, because students are as varied as ever, but it does shrink the standard error. Since n sits under a square root, the gain is slow: to halve the standard error you need four times the data. Confusing the two is the most common exam mistake.

In the lecturer's words“The standard deviation, one point six hours, tells you how much individual students differ from each other.”
Example

With s = 1.6 hours and n = 64, SE = 1.6/8 = 0.2 hours, or 12 minutes. Going to 256 students gives 1.6/16 = 0.1 hours: four times the work for twice the precision.

Ask yourself

Why does collecting more data shrink the standard error but leave the standard deviation unchanged?

3

How is a 95% confidence interval built, and what does '95%' actually refer to?Core

09:54 · slide 5also at09:2510:3011:1411:5012:10

A 95% confidence interval is the sample mean plus or minus 1.96 standard errors, and the 95% describes the method, not any single interval. The sampling distribution is normal, and in a normal distribution 95% of values fall within 1.96 standard deviations of the centre. So 95% of the time x-bar lands within 1.96 standard errors of mu, and so mu is within that distance of x-bar. Higher confidence widens the interval. Because mu is fixed, it is either in your interval or not; the random thing is the interval. Across repeated surveys, about 95 in 100 intervals would contain mu.

In the lecturer's words“the sample mean, plus or minus one point nine six times the standard error.”
Example

For the sleep survey, 1.96 × 0.2 ≈ 0.39, so the interval is 6.8 ± 0.39, or 6.41 to 7.19 hours. At 99% the multiplier is 2.576, giving 6.28 to 7.32.

Ask yourself

Why is it wrong to say there is a 95% chance the true mean lies between 6.41 and 7.19, and what does the 95% refer to instead?

4

Why do small samples need Student's t, and how does it change the interval?

14:18 · slide 7also at15:2315:3815:5616:13

Small samples need Student's t because the sample standard deviation s is a wobbly estimate of the true sigma, and ignoring that makes intervals too narrow, claiming more precision than the data justify. The t distribution has fatter tails, and its multiplier depends on the degrees of freedom, which for one sample is n minus one. A bigger multiplier widens the interval. As the sample grows, t settles towards the normal value, so for large samples it makes little difference. In practice people use t whenever sigma is unknown, which is always.

In the lecturer's words“With a small sample it is not, it is itself quite wobbly, and if we ignore that, our intervals come out too narrow.”
Example

With n = 16 and s = 1.6, SE = 0.4 and there are 15 degrees of freedom. The multiplier is 2.131 instead of 1.96, so the margin is about 0.85 and the interval runs 5.95 to 7.65 hours. At 30 degrees of freedom it is 2.042, and at 100 it is 1.984.

Ask yourself

How do the smaller sample and the larger t multiplier each contribute to the wider interval at n = 16 compared with n = 64?

5

How does a hypothesis test turn the sample mean into a p-value and a decision?Core

18:12 · slide 8also at17:1717:4818:3319:3520:11

A hypothesis test measures how surprising your sample mean would be if a claim were true, turns that into a probability, and compares it with a threshold. First state the null hypothesis, the claim being tested. Then express the gap between sample mean and claim in standard errors: this is the test statistic. Then find the p-value: the probability, if the null were true, of landing that far or further away in either direction. Finally compare it with alpha, chosen in advance, conventionally 0.05. If p is below alpha, reject the null; otherwise you fail to reject, which is not proof, since absence of evidence against is not evidence for.

In the lecturer's words“If the null were true, how often would a sample land three and a half standard errors away or further, in either direction?”
Example

The leaflet claims mu = 7.5 hours. The sample gives (6.8 − 7.5)/0.2 = −3.5 standard errors, a p-value of about 0.0005 (five in ten thousand), so the null is rejected. For a claim of 7 hours, the distance is one standard error, p ≈ 0.32, so we fail to reject.

Ask yourself

Why do we say we fail to reject a null of 7 hours rather than saying we have proved the mean is 7, and how does this match the confidence interval?

Review sheets

Building and interpreting a confidence interval

A confidence interval turns one sample mean into a range for the true mean of all students, built from the standard error and a multiplier such as 1.96 for 95%. The rest of the lecture needs it because the '95%' wording is the most commonly misinterpreted statement in the course and loses exam marks.

Step by step

  1. First get the standard error. In the sleep survey the standard deviation was 1.6 hours and n was 64, whose square root is 8. So 1.6 divided by 8 is 0.2 hours, which is 12 minutes. 07:16
  2. The standard error of 0.2 hours says how much the sample mean would differ from survey to survey. The standard deviation of 1.6 hours says how much individual students differ. Mixing these two up is the most common exam mistake. 07:44
  3. The sampling distribution is normal, and in a normal distribution 95% of values fall within 1.96 standard deviations of the centre. So 95% of the time the sample mean lands within 1.96 standard errors of the true mean. That is the basis of the interval. 09:25
  4. If the sample mean is within that distance of the true mean, the true mean is within the same distance of the sample mean. So a 95% interval is the sample mean plus or minus 1.96 times the standard error. 09:25
  5. For the survey, 1.96 times 0.2 is 0.392, about 0.39. The interval for the mean sleep of all students is 6.8 ± 0.39, so 6.41 to 7.19 hours. The plus-or-minus part, 0.39, is the margin of error. 09:54
  6. For more confidence, use a 99% interval. The multiplier rises from 1.96 to 2.576, and 2.576 times 0.2 is about 0.52. The interval becomes 6.28 to 7.32 hours: more confidence gives a wider interval. 10:53
  7. Now the meaning of 95%. The true mean is a fixed number, either inside this interval or not; the interval is the random thing, since it comes from a random sample. So 95% describes the procedure: of 100 repeated surveys, about 95 intervals would contain the true mean and about 5 would miss. 12:10

How it connects

The standard error step links to the lecture's idea of how the standard error differs from the standard deviation. The interval's meaning of '95%' is the lecture's idea of what 95% actually refers to.

Common mistakes

  • MistakeTreating the standard error and the standard deviation as the same number.InsteadThe standard deviation (1.6 hours) is about how individual students differ; the standard error (0.2 hours) is about how the sample mean varies between surveys.
  • MistakeSaying there is a 95% chance the true mean lies between 6.41 and 7.19.InsteadThe true mean is fixed, either in the interval or not. The 95% describes the procedure, not this one interval.
  • MistakeThinking the true mean is the random quantity that jumps around.InsteadThe true mean is fixed; the interval is random because it comes from a random sample.
  • MistakeExpecting higher confidence to come with the same interval width.InsteadMore confidence means a wider interval: the multiplier rises from 1.96 to 2.576, so 99% gives 6.28 to 7.32 hours.

Go back to

From test statistic to p-value, and what p does not mean

This topic is the four-step procedure that turns a sample mean into a p-value and a decision about a claim, together with what the p-value does and does not tell you. The rest of the lecture needs it because it is the formal use of the standard error and links testing to confidence intervals.

Step by step

  1. A test begins by stating the null hypothesis, the claim being tested. In the sleep example the leaflet says students get seven and a half hours a night, so the null is that the mean mu equals 7.5. 17:17
  2. Next ask how surprising the data would be if that claim were true, measured in standard errors. The sample mean is 6.8, the claim is 7.5, so the difference is minus 0.7, and the standard error is 0.2. 17:17
  3. Divide the difference by the standard error: minus 0.7 divided by 0.2 is minus 3.5. This is the test statistic, here a z score. The sample mean lies 3.5 standard errors below what the leaflet claims. 17:48
  4. Turn the test statistic into a probability. If the null were true, how often would a sample land 3.5 standard errors away or further, in either direction? The answer is about 0.0005, five times in ten thousand, and this is the p-value. 18:12
  5. Compare p with a threshold alpha chosen in advance, by convention 0.05. Here p is far below it, so the null is rejected: the data are not compatible with 7.5 hours. 18:33
  6. The p-value is the probability of data at least as extreme as yours, if the null is true. It starts by assuming the null, so it is the probability of the data given the hypothesis, not of the hypothesis given the data. p = 0.03 does not mean a 3% chance the leaflet is right. 20:50
  7. A small p does not mean a big effect, since it depends on effect size and sample size. With 10,000 students and a mean only 0.05 hours (3 minutes) below the claim, SE is 1.6/100 = 0.016, z is about 3.1 and p about 0.002. 21:58

How it connects

The test statistic is built from the standard error, which depends on sample size (with 10,000 students, SE = 1.6/100 = 0.016). The test also mirrors the 95% confidence interval: the interval is exactly the set of values a 5% level test would not reject.

Common mistakes

  • MistakeA p-value of 0.03 means there is a 3% chance the null hypothesis is true.InsteadThe p-value is the probability of the data given the null, not the probability of the hypothesis given the data.
  • MistakeA very small p-value means the effect is big or important.Insteadp depends partly on effect size and partly on sample size; 3 minutes of sleep can be highly significant with 10,000 students.
  • MistakeA p-value above 0.05 shows there is no effect.InsteadIt may just mean the sample was too small to detect the effect.
  • MistakeTreating the test and the confidence interval as unrelated results that could disagree.InsteadThe 95% interval is exactly the set of values a 5% level test would not reject: 7.5 lay outside and was rejected, 7 lay inside and was not.

Go back to

Key figures

Slide 3: open the study page to see this figure

Skewed histogram of individual sleep times beside the narrow normal sampling distribution of means

What it shows. A skewed histogram of individual sleep times beside a narrow bell curve of sample means, illustrating the sampling distribution of the mean.

What the lecturer said. Repeating the survey thousands of times with fresh samples and plotting the means gives the sampling distribution of the mean, a distribution of a statistic, not of individuals. It is centred on the true mean μ, its spread is σ over root n, and by the central limit theorem its shape is approximately normal even though sleep times are lopsided.

04:11 · slide 3

Slide 6: open the study page to see this figure

100 confidence intervals around the true mean, misses in red

What it shows. 100 horizontal confidence intervals drawn around a vertical line at μ, with the intervals that miss μ shown in red.

What the lecturer said. The lecturer said μ is fixed and the interval is the random part, so the 95% describes the procedure: of 100 repeated surveys, about 95 intervals contain the true mean and about 5 (the red ones) miss. With one survey you never know whether your interval is one of the red ones.

11:32 · slide 6

Slide 7: open the study page to see this figure

Table of t multipliers by degrees of freedom

What it shows. A table of t multipliers (t*) by degrees of freedom: 15 → 2.131, 30 → 2.042, 100 → 1.984, ∞ → 1.960.

What the lecturer said. With a sample of 16 and 15 degrees of freedom, the multiplier is 2.131 instead of 1.96, giving a margin of error of about 0.85. As the sample grows, t settles towards the normal value: 2.042 at 30 degrees of freedom and 1.984 at 100.

14:00 · slide 7

Easily confused

Standard deviationStandard error
What it describesHow much individual students differ from each otherHow much the sample mean would differ from survey to survey
Value in the survey1.6 hours0.2 hours
With more dataDoes not shrink; students stay as varied as everDoes shrink as the sample grows
07:36 · slide 4
Type I error (alpha)Type II error (beta)
What it isRejecting a null hypothesis that is actually true (false positive)Failing to reject a null hypothesis that is false (a miss)
Its probabilityAlpha, which the researcher choosesBeta; one minus beta is the power of the test
23:37 · slide 10

Formulas and numbers

As given in this lecture. Where a value matters beyond the exam, check it against a current source.

  • The spread of the sampling distribution of the mean is the population standard deviation divided by the square root of the sample size, sigma over root n. 05:21 · slide 3
  • As a rule of thumb, a sample size of about thirty is enough for approximate normality of sample means, a bit more if the population is very skewed. 06:13 · slide 3
  • A 95% confidence interval is the sample mean plus or minus 1.96 times the standard error. 09:54 · slide 5
  • A 99% interval uses the multiplier 2.576 instead of 1.96, giving about 6.8 ± 0.52, or 6.28 to 7.32 hours in the survey. 10:53 · slide 5
  • With 15 degrees of freedom the t multiplier is 2.131 instead of 1.96, giving a margin of error of about 0.85. 15:38 · slide 7
  • As sample size grows the t multiplier approaches the normal value: 2.042 at 30 degrees of freedom and 1.984 at 100. 16:13 · slide 7
  • The test statistic is the difference between sample mean and claimed mean divided by the standard error: (6.8 − 7.5)/0.2 = −3.5, a z score, meaning 3.5 standard errors below the claim. 17:48 · slide 8

Self-test

Answer each one from memory, out loud or on paper, before you open the answer. Then mark yourself honestly.

  1. Explain

    A student says: 'The population mean mu is 6.8 hours in one survey and 7.1 in the next, so mu keeps changing.' Using the lecture's terms parameter and statistic, explain what is wrong with this statement.

    Show answer

    6.8 and 7.1 are sample means (x-bar), which are statistics and change with each new sample. The parameter mu is a fixed but unknown number that does not move. Only the statistic varies from sample to sample.

    A good answer includes
    • 6.8 and 7.1 are sample means, statistics
    • mu is a fixed, unknown parameter
    • statistic changes with each new sample

    Review this part01:35 · slide 2

    How did you do?
  2. Explain

    Sleep times in the population are lopsided, with a long tail of short sleepers. Why can we still use a bell-curve (normal) model for the sample mean, and what rule of thumb says when this is reasonable?

    Show answer

    The sampling distribution of the mean is approximately normal by the central limit theorem, even if the population is not normal. Averages of thirty or more sleep times still pile up in a bell shape. About thirty is enough, a bit more if the population is very skewed.

    A good answer includes
    • central limit theorem
    • sampling distribution of the mean is approximately normal
    • about thirty, more if very skewed

    Review this part05:36 · slide 3

    How did you do?
  3. Apply

    A study of resting heart rate has s = 12 beats per minute and n = 36, giving a mean of 70. The researchers then quadruple the sample to n = 144, assuming s stays at 12. What happens to the standard deviation and the standard error? Give numbers.

    Show answer

    The standard deviation stays about 12, since individuals are as varied as before. The standard error falls from 12/6 = 2 to 12/12 = 1, halved, because n is under a square root and four times the data halves it.

    A good answer includes
    • SD stays about 12
    • SE goes from 2 to 1
    • four times the data halves the SE

    Review this part07:44 · slide 4

    How did you do?
  4. Explain

    A report says: 'There is a 95% chance that the true mean lies between 6.41 and 7.19 hours.' Explain why the lecturer treats this as wrong and give the correct meaning of the 95%.

    Show answer

    The true mean mu is a fixed number, so it is either inside this particular interval or not; the random thing is the interval, which comes from a random sample. The 95% describes the method: if the survey were repeated 100 times, about 95 intervals would contain mu and about 5 would miss.

    A good answer includes
    • mu is fixed, either inside or not
    • the interval is the random thing
    • 95% describes the procedure over repeated surveys

    Review this part11:50 · slide 6

    How did you do?
  5. Go further

    A researcher wants a 99% interval to be as narrow as the 95% interval from the same survey (s = 1.6, n = 64). Using the lecture's multipliers, say what changes with the 99% interval and what the researcher would have to do to restore the narrowness.

    Show answer

    The multiplier rises from 1.96 to 2.576, so the margin of error grows from about 0.39 to 0.52 hours: more confidence gives a wider interval. The only honest way to get an interval that is both confident and narrow is a bigger sample, which shrinks the standard error.

    A good answer includes
    • multiplier rises from 1.96 to 2.576
    • higher confidence widens the interval
    • bigger sample is the honest fix

    Review this part10:53 · slide 5

    How did you do?
  6. Apply

    A sample of n = 100 has mean 52 and standard error 2. A leaflet claims the true mean is 55. Compute the test statistic and say whether a test at alpha = 0.05 would reject, using the confidence interval to check your reasoning (use 1.96).

    Show answer

    z = (52 − 55)/2 = −1.5, so the sample is 1.5 standard errors below the claim. The 95% interval is 52 ± 1.96×2 = 48.1 to 55.9, which contains 55, so the null is not rejected. We would say we fail to reject, not that 55 is proved.

    A good answer includes
    • z = −1.5
    • interval 48.1 to 55.9 contains 55
    • fail to reject, not proved

    Review this part17:48 · slide 8

    How did you do?

Practice quiz

New cases to try the lecture's ideas on. Pick an answer to see why each option is right or wrong.

  1. Apply

    A survey of 10,000 people finds a mean 3 minutes below a claimed value, with p = 0.002. A colleague wants to write 'the effect is large and important because p is so small.' Which response is best?

    Show answer
    • ACorrect. Small p reflects both effect and sample size; statistical and practical significance differ.
    • BConfuses probability of data given the null with probability of the null given data.
    • CIgnores that p also depends on sample size.
    • DReverses the effect of sample size; large n makes small effects significant.

    Review this part21:58 · slide 9

  2. Go further

    A team runs 20 independent tests at alpha = 0.05 and none of the effects are real. They then switch to a Bonferroni correction. What happens, and why?

    Show answer
    • ACorrect. Bonferroni divides alpha by the number of tests, countering about 64% chance of a false alarm.
    • BMultiplies instead of dividing alpha.
    • CApplies alpha to one test, ignoring that risks accumulate across 20 tests.
    • DRight threshold, wrong consequence; a stricter threshold makes misses more likely, not power higher.

    Review this part23:55 · slide 10

Check this

Not from the lecture

The boxes marked “Beyond the lecture (not from your lecturer)” were added by the AI to help. Your lecturer did not say what is in them. They are under:

  • Key idea 1: Why does the sample mean vary between samples, and how is that variation described?
  • Key idea 2: How does the standard error differ from the standard deviation, and how does sample size affect it?
  • Key idea 3: How is a 95% confidence interval built, and what does '95%' actually refer to?
  • Key idea 4: Why do small samples need Student's t, and how does it change the interval?
  • Key idea 5: How does a hypothesis test turn the sample mean into a p-value and a decision?

Glossary

95% confidence interval 09:54 · slide 5
A 95% confidence interval is the sample mean plus or minus 1.96 times the standard error.
Bonferroni correction 25:49 · slide 10
The Bonferroni correction divides alpha by the number of tests; for 20 tests each must beat 0.05/20 = 0.0025. It is strict but honest.
Margin of error 10:35 · slide 5
The plus-or-minus part of the interval, 0.39 hours here, is called the margin of error.
p-value 20:50 · slide 9
A p-value is the probability of obtaining data at least as extreme as the observed data, assuming the null hypothesis is true.
Parameter 01:35 · slide 2
The population is everybody the question is about, and its true average is called mu, which is a parameter.
Power 24:18 · slide 10
Power equals 1 minus beta: the chance of detecting an effect that is really there. Studies usually aim for 80% power.
Sampling distribution of the mean 04:45 · slide 3
The sampling distribution of the mean is the histogram of sample means from repeating the survey thousands of times with fresh samples; it is a distribution of a statistic, not of individuals.
Standard error 07:44 · slide 4
The standard error (0.2 hours in the survey) describes how much the sample mean would vary from survey to survey; confusing it with the standard deviation is the most common exam mistake.
Statistic 02:24 · slide 2
The sample mean is written x-bar and is a statistic; parameters describe the population, statistics describe the sample (P with population, S with sample).
Statistical significance 22:56 · slide 9
Statistical significance means the result is unlikely to be due to chance; practical significance means the effect is big enough to matter. The two must be separated.
Type I error 23:37 · slide 10
A type I error is rejecting a null hypothesis that is actually true (a false positive).

Flashcards

9 cards from this lecture are in your flashcard file and in StreamSnag's review. Each one shows the minute it comes from.

Look through the cards
Population mean: the true average of the whole population is written with the Greek letter mu, a parameter.01:35 · slide 2
Sampling distribution of the mean: how is it built?Histogram of sample means from thousands of repeated surveys04:45 · slide 3
Standard deviation: what does a standard deviation of 1.6 hours describe in the student survey?How much individual students differ from each other07:36 · slide 4
Standard error: what does a standard error of 0.2 hours describe?How much the sample mean varies between surveys07:44 · slide 4
95% confidence interval: sample mean plus or minus 1.96 times the standard error.09:54 · slide 5
Hypothesis test: a sample 3.5 standard errors from the claimed value, in either direction, has p of about 0.0005.18:12 · slide 8
Confidence interval: a 95% interval is the set of which values, in hypothesis-testing terms?Those a 5% level test would not reject20:11 · slide 8
p-value: the probability of data at least as extreme as observed, assuming the null hypothesis is true.20:50 · slide 9
Bonferroni correction: how is the per-test threshold found from alpha?Divide alpha by the number of tests25:49 · slide 10