Machine Learning with caret

Reviewed & published by Brayan K

caret gives you a single, consistent interface to hundreds of machine-learning algorithms in R — handling data splitting, preprocessing, resampling, tuning, and evaluation through one workflow.

Part of the free R course at LearnCodingFast — hands-on lessons with worked examples and the output they print, plus practice exercises and a quick quiz.

By the end of this lesson you'll split data with createDataPartition(), fit and tune models with train() and trainControl(), and score them with confusionMatrix().

What You'll Learn in This Lesson

• How to make a stratified split with createDataPartition()
• How train() unifies fitting across many algorithms
• How trainControl() configures cross-validation
• How to center and scale predictors with preProcess
• How to tune hyperparameters with a tuneGrid
• How to evaluate a classifier with confusionMatrix()

1️⃣ Splitting Data with createDataPartition()

Before modelling, hold out a test set the model never sees. createDataPartition() makes a stratified split, keeping each class's proportion balanced across train and test.

library(caret)

# createDataPartition makes a STRATIFIED train/test split so the
# outcome's class balance is preserved in both halves.
set.seed(42)
data(iris)

idx   <- createDataPartition(iris$Species, p = 0.8, list = FALSE)
train <- iris[idx, ]
test  <- iris[-idx, ]

nrow(train)   # -> [1] 120
nrow(test)    # -> [1] 30

2️⃣ Fitting with train() and trainControl()

trainControl() says how to resample (here, 5-fold cross-validation); train() does the fitting and reports resampled accuracy. preProcess standardizes predictors first.

library(caret)

# trainControl() defines HOW we resample; train() does the fitting + tuning.
ctrl <- trainControl(method = "cv", number = 5)   # 5-fold cross-validation

set.seed(42)
model <- train(
  Species ~ .,          # formula: predict Species from all other columns
  data       = train,
  method     = "rpart", # a decision tree
  trControl  = ctrl,
  preProcess = c("center", "scale")  # standardize predictors
)

model   # prints accuracy across the tried complexity values (cp)

3️⃣ Tuning Hyperparameters

A tuneGrid lists candidate hyperparameter values. train() fits every combination under the resampling scheme and keeps the best in $bestTune.

library(caret)

# A tuning grid lists the hyperparameter values to try.
# For a k-nearest-neighbours model we tune k.
grid <- expand.grid(k = c(3, 5, 7, 9))

ctrl <- trainControl(method = "cv", number = 5)
set.seed(42)
knn_fit <- train(
  Species ~ .,
  data      = train,
  method    = "knn",
  trControl = ctrl,
  tuneGrid  = grid
)

knn_fit$bestTune    # the k caret selected -> e.g. k = 7

4️⃣ Evaluating with confusionMatrix()

predict() applies the trained model to the held-out test set, and confusionMatrix() reports accuracy, sensitivity, specificity, and kappa.

library(caret)

# predict() applies the model, confusionMatrix() scores it on held-out test data.
preds <- predict(model, newdata = test)
confusionMatrix(preds, test$Species)
#   Accuracy : 0.9667
#   Sensitivity / Specificity reported per class

Your turn. Fill in the ___ blank, run it, and compare with the expected output.

library(caret)

# 🎯 YOUR TURN — replace each ___ then run it.

set.seed(1)
data(iris)
idx   <- createDataPartition(iris$Species, p = 0.7, list = FALSE)
train <- iris[idx, ]
test  <- iris[-idx, ]

# 1) Use 10-fold cross-validation
ctrl <- trainControl(method = "cv", number = ___)   # 👉 replace ___ with 10

# 2) Train a tree (method = "rpart") to predict Species
model <- train(Species ~ ., data = train, method = "rpart", trControl = ctrl)

# 3) Score it on the test set
preds <- predict(model, newdata = test)
confusionMatrix(preds, test$Species)$overall["Accuracy"]

# ✅ Expected: an accuracy near 0.93 (your exact number may vary slightly)

Mini-Challenge: Compare Two Models

Train a kNN and a tree on the same split, then compare their test accuracies. Swapping methods while keeping everything else fixed is exactly what caret makes painless.

# 🎯 MINI-CHALLENGE: compare two models.
#
# Using the iris train/test split:
# 1. trainControl() with method = "cv", number = 5.
# 2. Train a kNN model (method = "knn") and an rpart tree.
# 3. predict() each on the test set.
# 4. Print both accuracies with confusionMatrix(...)$overall["Accuracy"].
#
# ✅ Example output:
#    kNN accuracy:   0.9778
#    Tree accuracy:  0.9556

# your code here

Common Errors (and the fix)

• "could not find function 'train'" — load caret with library(caret) first.
• Non-reproducible results — call set.seed() before splitting and before train().
• Leaking the test set — fit preProcess inside train() so scaling is learned only from training folds, not the test data.
• Wrong tuneGrid columns — the grid's column names must match the method's parameters (e.g. k for knn, cp for rpart).

Pro Tips

• 💡 Use repeated CV: method = "repeatedcv" with repeats gives steadier estimates.
• 💡 Let caret tune: pass tuneLength = 10 to auto-generate a grid when you do not have one in mind.
• 💡 Learn the successor: the same ideas power tidymodels and mlr3.

📋 Quick Reference — caret

🎉 Lesson Complete!

• ✅ createDataPartition() makes a stratified train/test split
• ✅ train() + trainControl() fit and resample any method
• ✅ preProcess centers and scales predictors
• ✅ A tuneGrid searches hyperparameters
• ✅ confusionMatrix() reports accuracy, sensitivity, specificity

Practice quiz

Continue this course

• Previous: Capstone: A Data Analysis Report
• Next: Bayesian Statistics in R — Priors, posteriors and credible intervals with brms and rstanarm
• Quick reference: R cheat sheet

Frequently asked questions

What is caret and what problem does it solve?

caret (Classification And REgression Training) is a wrapper package that gives you one consistent interface to hundreds of modelling algorithms in R. Instead of learning a different function signature for every algorithm, you call train() with a method name. caret also standardizes the surrounding workflow: data splitting, preprocessing, resampling, hyperparameter tuning, and evaluation. That uniformity is what makes it easy to swap a decision tree for a random forest and compare them fairly.

What is the difference between trainControl() and tuneGrid?

trainControl() configures HOW the model is validated — the resampling method (cross-validation, bootstrap, repeated CV), the number of folds, and options like saving predictions. tuneGrid configures WHAT hyperparameters to try — it is a data frame whose columns are the tunable parameters of the chosen method and whose rows are the candidate combinations. train() fits the model for every grid row under the resampling scheme from trainControl(), then keeps the combination with the best resampled performance.

Why split data and use cross-validation instead of fitting on everything?

A model evaluated on the same data it was trained on will look better than it really is, because it can memorize noise. createDataPartition() holds out a test set the model never sees, giving an honest final score. Cross-validation goes further during training: it repeatedly trains on most of the data and validates on the rest, so the performance estimate and the hyperparameter choice are based on out-of-sample behaviour. Together they guard against overfitting.

Is caret still the right choice, or should I use tidymodels?

caret is mature, stable, and widely used, so it remains a solid choice and is well documented. However, its author has shifted active development to tidymodels, a tidyverse-aligned collection (recipes, parsnip, rsample, yardstick, tune) that is more modular and composes with the pipe. mlr3 is another modern, object-oriented framework. For new projects many people now learn tidymodels, but the concepts you learn here — splitting, resampling, tuning, evaluating — transfer directly to any of them.