Model Deployment

A trained model lives in memory. The moment Python exits, it's gone. Serialising means writing the model to a file so a completely separate program — your API server — can load it back later.

Critically, you save more than the weights. You also save the preprocessing recipe (feature order, scaling means and stds, encoders) so serving can repeat it exactly. Run the worked example below, then read the comments about which format to use when.

Other programs can't import your Python model directly. A REST API is a shop counter: a client sends a small JSON order over HTTP, your service runs the model, and hands a JSON prediction back.

FastAPI is the modern Python choice (Flask is the older, simpler alternative). With FastAPI you declare the expected input as a class and it validates every request for you — a missing field or wrong type is rejected automatically before your code runs. Note the two rules below.

Study the real serving shape (this is read-only — you'll write runnable plain Python just below):

# serve.py — the real production shape with FastAPI
# (read this; you run plain Python in the exercises below)
import joblib
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field

app = FastAPI()

# Load the model ONCE at startup, not on every request.
model = joblib.load("model.joblib")

# Pydantic validates the request body for you: wrong type or
# missing field -> automatic 422 error, your code never runs.
class HouseRequest(BaseModel):
    sqft: float = Field(gt=0)   # must be greater than 0
    beds: int   = Field(ge=0)

@app.post("/predict")
def predict(req: HouseRequest):
    # Preprocess EXACTLY as in training (same feature order, same scaling).
    x = [req.sqft, req.beds]
    price = model.predict([x])[0]
    return {"price": round(price, 2), "model_version": "1.3.0"}

@app.get("/health")        # let your load balancer check the service is alive
def health():
    return {"status": "ok"}

# Run it locally:
#   uvicorn serve:app --reload
# Then POST {"sqft": 1800, "beds": 3} to http://localhost:8000/predict

# Expected output (from the POST request):
# {"price": 304000.0, "model_version": "1.3.0"}

Two failures sink more deployments than bad models. First, no input validation — a missing or negative field crashes your service. Second, train/serve skew — the model was trained on scaled features, but serving forgot to scale them the same way, so every prediction is quietly wrong.

The fix for both: validate first, then run the exact same preprocessing you used in training (same feature order, same scaling, same encoders). In the exercise below you'll write a predict() that validates the request and standardises each feature with the saved means and stds before scoring.

"It works on my machine" is the classic deployment failure — a library version differs in the cloud and the service breaks. A Docker container is a sealed shipping crate that freezes the OS, the Python version, and every dependency, so the service runs identically everywhere.

You pin your dependencies in a requirements.txt, copy them and your code into an image, and run it anywhere. Copying dependencies before the code lets Docker cache that slow step.

# Dockerfile — package the service + its exact dependencies
# A container freezes the OS, Python version, and every library so it
# runs identically on your laptop and in the cloud ("missing deps" gone).

FROM python:3.11-slim

WORKDIR /app

# Install deps FIRST (this layer is cached unless requirements change).
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Then copy the code and the serialised model.
COPY serve.py model.joblib ./

EXPOSE 8000
CMD ["uvicorn", "serve:app", "--host", "0.0.0.0", "--port", "8000"]

# Build and run:
#   docker build -t house-api .
#   docker run -p 8000:8000 house-api
#
# Expected output (terminal):
# Uvicorn running on http://0.0.0.0:8000

Shipping once is easy; shipping safely over and over is the job. CI/CD (Continuous Integration / Continuous Delivery) is an automated pipeline — a tool like GitHub Actions runs your tests, builds the Docker image, and deploys it whenever you push a new model. No manual steps means fewer human mistakes.

Once live, monitoring takes over. A model trained on yesterday's data slowly goes stale as the world changes — this is data drift. You can't see it from the code, only from the metrics. Track these and alert when they cross a threshold, then retrain.

Common tools: GitHub Actions / MLflow / DVC for CI/CD, and Prometheus + Grafana / Evidently for monitoring. Start simple — wire these in only once you have real traffic.

These five mistakes break far more deployments than poorly-tuned models. Learn to spot each one.

# Training scaled inputs, serving forgot to:
x = [req.sqft, req.beds]          # raw, unscaled  ❌
price = model.predict([x])

x = [(req.sqft - mean_sqft) / std_sqft,
     (req.beds - mean_beds) / std_beds]   # same as training ✓

price = request["sqft"] * w   # KeyError: 'sqft'  ❌

if "sqft" not in request:
    return {"error": "missing field: sqft", "status": 400}

model = {"version": "1.3.0", "weights": [...]}
return {"price": price, "model_version": model["version"]}

@app.post("/predict")
def predict(req):
    model = joblib.load("model.joblib")   # reloads every call  ❌

model = joblib.load("model.joblib")   # at module top, once ✓

ModuleNotFoundError: No module named 'scikit-learn'  ❌

# requirements.txt
scikit-learn==1.5.0
fastapi==0.115.0
# then: RUN pip install -r requirements.txt  (in the Dockerfile)