Hypothesis Testing (t.test, chisq.test)

Reviewed & published by Brayan K

Hypothesis testing is the statistical procedure for deciding whether a pattern in your data is real or could plausibly have arisen by chance, by comparing it against a null hypothesis of "no effect" and measuring the surprise with a p-value.

Part of the free R course at LearnCodingFast — hands-on lessons with worked examples and the output they print, plus practice exercises and a quick quiz.

In this lesson you'll learn the logic of a test and the meaning of a p-value, run one- and two-sample t.test(), switch to the robust wilcox.test(), test categorical counts with chisq.test(), and read every part of R's output — including the confidence interval.

What You'll Learn in This Lesson

• The logic of a null hypothesis and what a p-value means
• How to run one- and two-sample t.test()
• Why R defaults to Welch's t-test (unequal variances)
• When to use the non-parametric wilcox.test()
• How chisq.test() tests association in count data
• How to read the statistic, p-value, and confidence interval

1️⃣ The Logic & the t-test

Every test starts with a null hypothesis (H0) of "no effect," then asks how surprising your data would be if H0 were true. The t.test() compares means. A one-sample test checks a mean against a fixed target.

# A hypothesis test asks: is the pattern in my data REAL, or could it
# easily be chance? We state a NULL hypothesis (H0: "no effect") and
# ask how surprising our data would be IF H0 were true. That surprise
# is the p-value.

# One-sample t-test: is the mean weight different from a target of 50g?
weights <- c(48, 52, 49, 51, 47, 53, 50, 46, 54, 49)

t.test(weights, mu = 50)
# Read the printout, NOT just one number:
#   t          - the test statistic
#   df         - degrees of freedom
#   p-value    - chance of data this extreme if the true mean were 50
#   95% CI     - plausible range for the true mean
#   sample est - the observed mean
#
# If the p-value is below your threshold (commonly 0.05) you REJECT H0
# and conclude the mean differs from 50. Otherwise you don't.

The two-sample version asks whether two groups differ. R's default is Welch's t-test, which doesn't assume the groups share the same variance — the safer choice in practice.

# Two-sample t-test: do two GROUPS have different means? H0 is "the
# two population means are equal".
group_a <- c(20, 22, 19, 23, 21, 18, 24)
group_b <- c(25, 27, 24, 26, 28, 23, 29)

# Formula interface needs a tidy data frame (values + a group factor):
scores <- data.frame(
  value = c(group_a, group_b),
  group = rep(c("A", "B"), each = 7)
)

t.test(value ~ group, data = scores)
# By default R runs WELCH's t-test, which does NOT assume equal
# variances — the safe choice. Add var.equal = TRUE only if you have
# good reason to assume the spreads are the same.

# Same hypothesis, two equivalent calls:
t.test(group_a, group_b)            # supply the two vectors directly

2️⃣ When Normality Fails: wilcox.test

The t-test leans on roughly normal data. With small, skewed, or outlier-ridden samples, switch to the non-parametric wilcox.test(), which compares distributions through ranks and is far more robust.

# t.test assumes roughly normal data. When that's doubtful (small
# samples, skew, outliers, or ranks), use the NON-PARAMETRIC
# wilcox.test, which compares distributions via ranks instead of means.
group_a <- c(20, 22, 19, 23, 21, 18, 24)
group_b <- c(25, 27, 24, 26, 28, 23, 29)

wilcox.test(group_a, group_b)
# H0: the two distributions are the same (no shift). The output gives a
# W statistic and a p-value you interpret exactly like the t-test's.

# One-sample / paired versions exist too:
wilcox.test(group_a, mu = 21)             # one-sample
wilcox.test(group_a, group_b, paired = TRUE)  # paired observations

You read its output the same way — a test statistic (W) and a p-value — but the underlying null is "the two distributions are the same," and no normality is assumed.

3️⃣ Categorical Data: chisq.test

When your data are counts in categories rather than numbers you average, the chi-squared test asks whether two categorical variables are associated. You feed it a contingency table of observed frequencies.

# chisq.test works on COUNTS, not means. It asks whether two
# categorical variables are associated (independent or not).
# Build a contingency table of observed counts:
tbl <- matrix(c(30, 10, 15, 25), nrow = 2,
              dimnames = list(
                treatment = c("Drug", "Placebo"),
                outcome   = c("Recovered", "Not")
              ))
tbl

chisq.test(tbl)
# H0: treatment and outcome are INDEPENDENT (no association).
# The output gives X-squared, df, and a p-value. A small p-value means
# the variables ARE associated — the drug relates to recovery here.

# Goodness-of-fit (one variable vs expected proportions):
rolls <- c(8, 12, 9, 11, 10, 10)        # 60 die rolls, 6 faces
chisq.test(rolls)                       # H0: the die is fair (uniform)

Finally, every test returns an object you can index. Pulling out the $p.value or $conf.int lets you use results in reports and pipelines.

# Pull a single number OUT of a test for use in a report or pipeline.
# Every test returns an object you can index with $.
weights <- c(48, 52, 49, 51, 47, 53, 50, 46, 54, 49)
res <- t.test(weights, mu = 50)

res$p.value        # just the p-value
res$conf.int       # the 95% confidence interval (two numbers)
res$estimate       # the sample mean
res$statistic      # the t value

# tidy() from the 'broom' package turns ANY test into a one-row tibble,
# which is ideal for tables and for testing many groups at once:
# library(broom); tidy(res)

Your turn. Fill in the ___ blank, run it, and read the p-values and intervals.

# 🎯 YOUR TURN — replace each ___ then run it.
# A teacher wants to know if a new method beats the old class average
# of 70, and whether two sections differ.

old_avg  <- 70
new_scores <- c(74, 78, 69, 81, 77, 72, 85, 79)

# 1) One-sample t-test: is the new mean different from 70?
test1 <- t.test(new_scores, mu = ___)   # 👉 replace ___ with old_avg
print(test1)        # look at the p-value and the 95% CI

# 2) Compare two sections with a two-sample (Welch) t-test.
section_x <- c(74, 78, 69, 81, 77)
section_y <- c(66, 71, 68, 73, 70)
test2 <- t.test(section_x, section_y)
print(test2)

# 3) Print JUST the p-value of test2 (hint: use $p.value).
cat("Section comparison p =", round(test2$p.value, 4), "\n")

Mini-Challenge: A/B Test the Right Way

The hardest part of testing is picking the RIGHT test. These are counts by category, so it's a chi-squared problem — not a t-test. Build the table, run it, and write a plain-English conclusion.

# 🎯 MINI-CHALLENGE: choose and run the right test.
#
# You run an A/B test on a website. Of 200 visitors shown layout A,
# 48 signed up; of 200 shown layout B, 72 signed up.
#
# 1. These are COUNTS of a categorical outcome (signed up: yes/no) by
#    group (A/B) -> the right tool is chisq.test, not t.test.
# 2. Build a 2x2 contingency table with matrix() (rows = A/B,
#    columns = signed up / not).
# 3. Run chisq.test() on it. State H0 in a comment.
# 4. Read off the p-value and write a one-line plain-English conclusion
#    about whether layout affects sign-up rate.
# 5. Bonus: extract the p-value with $p.value and round it.
#
# Write it yourself, then run in RStudio or rdrr.io/snippets.

# your code here

Common Errors (and the fix)

• ❌ "p > 0.05 proves there's no effect" — ✅ a non-significant result means you lack evidence, not that the null is true. Absence of evidence isn't evidence of absence.
• ❌ Running t.test on category counts — ✅ counts need chisq.test; t-tests are for numeric means.
• ❌ Formula value ~ group with no data = — ✅ pass the data frame: t.test(value ~ group, data = df).
• ❌ Confusing a tiny p with a big effect — ✅ significance ≠ importance; always check the confidence interval for the effect's size.

Pro Tips

• 💡 Paired data: for before/after on the same subjects use t.test(after, before, paired = TRUE) — it's more powerful.
• 💡 Tidy output: broom::tidy(test) turns any test into a one-row tibble, great for tables.
• 💡 Small tables: with sparse 2x2 counts prefer fisher.test() over chisq.test() for an exact p-value.

📋 Quick Reference — Hypothesis Tests

🎉 Lesson Complete!

• ✅ A test weighs your data against a null hypothesis of no effect
• ✅ The p-value is the chance of data this extreme if H0 were true
• ✅ t.test() compares means; R defaults to robust Welch
• ✅ wilcox.test() is the non-parametric backup
• ✅ chisq.test() tests association in count data
• ✅ Read the p-value AND the confidence interval together

Practice quiz

Continue this course

• Previous: Probability Distributions
• Next: Generalized Linear Models (glm) — Fit logistic regression and other GLMs and interpret the coefficients
• Quick reference: R cheat sheet

Frequently asked questions

What exactly is a p-value, and what does p < 0.05 mean?

A p-value is the probability of seeing data at least as extreme as yours IF the null hypothesis (no effect) were true. A small p-value means your data would be surprising under 'no effect', which is evidence against the null. The 0.05 threshold is a convention, not a law of nature: p < 0.05 means you'd see a result this extreme less than 5% of the time by chance alone, so you 'reject the null'. Crucially, the p-value is NOT the probability that the null is true, and it says nothing about how BIG or important the effect is.

When should I use a t-test versus wilcox.test?

Use t.test when your data are roughly normal (or your samples are large enough that the sample mean is approximately normal by the central limit theorem) and you care about comparing means. Use wilcox.test (the Mann-Whitney / Wilcoxon rank test) when normality is doubtful — small samples, heavy skew, strong outliers, or ordinal data — because it compares distributions through ranks and is robust to those problems. The trade-off is a little less power than a t-test when the data really are normal.

What's the difference between chisq.test and t.test?

They answer different kinds of questions. t.test compares the MEANS of a numeric variable (e.g. average weight in two groups). chisq.test works on COUNTS of categorical variables and asks whether two categories are associated, using a contingency table of frequencies (e.g. is recovery related to treatment?). If your outcome is a number you average, reach for t.test; if your data are counts in categories, reach for chisq.test. Using the wrong one on the wrong data type gives meaningless results.

How do I read the confidence interval in the output?

A 95% confidence interval is a plausible range for the true value (the true mean, or the true difference between two means). If a two-sample interval for the difference does NOT contain 0, that agrees with a significant result (p < 0.05) — zero difference is implausible. If it straddles 0, the data are consistent with no difference. The CI is often more useful than the p-value because it shows both the direction AND the magnitude of the effect, with its uncertainty, rather than collapsing everything into a single yes/no.