Courses/C#/Background Jobs

Advanced Track

Background Jobs

By the end of this lesson you'll be able to move slow work off the web request and onto a background worker — enqueuing jobs, processing them in a loop, running things on a recurring schedule, and reaching for BackgroundService, Hangfire, or Quartz.NET when you need persistence, retries, and idempotency.

What You'll Learn

Why you offload slow work off the request thread instead of blocking the user
The three job shapes: fire-and-forget, delayed, and recurring
Build a queue and a worker loop — the core of every job system
Run long-lived workers with IHostedService / BackgroundService
Use Hangfire and Quartz.NET for persistent, scheduled, retrying jobs
Make jobs idempotent and resilient with retries, poison handling, and persistence

Before you start: make sure you've finished Microservices. You should be comfortable with dependency injection, async/await, and how an ASP.NET request flows through a controller — background jobs build directly on those ideas.

💡 Real-World Analogy

Picture a busy restaurant. When you order, the waiter doesn't stand at your table cooking your meal — that would block them from serving anyone else. Instead they clip your ticket onto a rail in the kitchen and walk away. The chefs work that rail of tickets in order, one after another, at their own pace. A web request is the waiter: it should take the order (enqueue a job) and return to the customer instantly. The background worker is the chef: it pulls tickets off the queue and does the slow cooking. Some tickets are "make this now" (fire-and-forget), some say "fire the dessert in 10 minutes" (delayed), and the daily prep list runs every morning whether anyone ordered it or not (recurring). The whole point is that the dining room never waits on the stove.

Why offload work at all?

A web request should finish in milliseconds. The moment you do something slow inside the request — send an email, resize an image, call a flaky third-party API, build a PDF — the user sits there watching a spinner, your thread is tied up, and a timeout or a crash loses the work entirely.

The fix is to offload: the request records "this needs doing" and returns immediately; a separate worker does the slow part later. That gives you three big wins — fast responses, work that survives a slow downstream service, and the ability to retry failures without the user ever knowing.

Background work comes in three shapes you'll use again and again:

🔥 Fire-and-forget — do this once, soon, in the background (send a welcome email).
⏳ Delayed — do this once, after a wait (a follow-up email in 3 days).
🔁 Recurring — do this on a schedule, forever (a nightly report at 02:00).

📊 The three job shapes

Shape	Runs	Example	Hangfire call
Fire-and-forget	Once, ASAP	Send welcome email	BackgroundJob.Enqueue(...)
Delayed	Once, after a delay	Follow-up in 3 days	BackgroundJob.Schedule(...)
Recurring	On a schedule	Nightly report	RecurringJob.AddOrUpdate(...)

📊 Choosing a tool: HostedService vs Hangfire vs Quartz

Tool	Persistent?	Built-in retries	Dashboard	Best for
BackgroundService	No (in-memory)	You write them	No	Simple in-process workers, timers, queue draining
Hangfire	Yes (database)	Yes, automatic	Yes (/hangfire)	Most web apps — easy, persistent, great UI
Quartz.NET	Yes (optional)	Yes (refire)	No (3rd-party)	Complex CRON, clustering, calendar rules

Rule of thumb: start with BackgroundService for in-process work, reach for Hangfire the moment jobs must survive a restart or retry on failure, and pick Quartz.NET only when you need its advanced scheduling or clustering.

1. Queues and Workers — the core idea

Strip away the frameworks and every job system is the same two pieces: a queue of pending work and a worker that drains it. A Queue<string> is a first-in, first-out line — Enqueue adds to the back, Dequeue removes from the front. The request side enqueues and returns instantly; the worker side loops, pulling one job at a time. Read this worked example, run it, then you'll build the same loop yourself.

Worked example: enqueue jobs, then drain them in a loop

Read every comment, run it, and watch jobs come out in the order they went in.

Try it Yourself »

using System;
using System.Collections.Generic;

class Program
{
    static void Main()
    {
        // A background job system is, at heart, a QUEUE and a WORKER.
        // The web request just ENQUEUES work and returns immediately;
        // a separate worker drains the queue and does the slow part.

        // Queue<string> = a first-in, first-out (FIFO) line of tickets.
        Queue<string> jobs = new Queue<string>();

        // --- The "request" side: enqueue work and return fast ---
 
...

Your turn. The program below queues three jobs and processes them — but three pieces are missing. Fill in the ___ blanks using the hints, then run it.

🎯 Your turn: a queue and a worker loop

Fill in the ___ blanks, then check jobs run in FIFO order.

Try it Yourself »

using System;
using System.Collections.Generic;

class Program
{
    static void Main()
    {
        // 🎯 YOUR TURN — fill in the blanks marked with ___ then run it.

        Queue<string> jobs = new Queue<string>();

        // 1) Add three jobs to the queue (the request side).
        jobs.Enqueue("backup-database");
        jobs.___("send-report");        // 👉 same method as the line above: Enqueue
        jobs.Enqueue("clear-cache");

        // 2) Drain the queue: keep going while there 
...

2. Recurring Schedules — running only when due

A recurring job shouldn't run every time the worker wakes up — only when its interval has elapsed. The pattern is a small piece of bookkeeping: count how long it's been since the last run, and when that count reaches the interval, run the job and reset the counter. Real schedulers do this with CRON expressions and timestamps, but the logic is identical. Fill in the two ___ blanks below to make a job that fires every third tick.

🎯 Your turn: run a job only when its interval is due

Make the job fire on ticks 3, 6 and 9 by completing the due-check.

Try it Yourself »

using System;

class Program
{
    static void Main()
    {
        // 🎯 YOUR TURN — a recurring job runs only when its INTERVAL is due.
        // We simulate "ticks" of a clock and run the job every 3rd tick.

        int interval = 3;     // run the job every 3 ticks
        int ticksSinceRun = 0;

        for (int tick = 1; tick <= 9; tick++)
        {
            ticksSinceRun++;

            // 1) The job is DUE only when enough ticks have passed.
            if (ticksSinceRun ___ interva
...

3. IHostedService & BackgroundService

.NET has the worker loop built in. IHostedService is the interface for "something that starts when the app starts and stops when it stops"; BackgroundService is the convenient base class — you just override ExecuteAsync and write your loop. Pair it with a Channel<T> (a thread-safe async queue) and a controller can enqueue work that the worker drains in the background. Note the cancellation token and the try/catch: a real worker must shut down gracefully and must never die because one job threw.

Worked example: a BackgroundService draining a queue

A long-running worker that processes an in-memory job channel.

Try it Yourself »

// ══════════════════════════════════════════════
// IHostedService / BackgroundService — built into .NET
// ══════════════════════════════════════════════
// A BackgroundService is a long-running task the .NET host starts at
// boot and stops on shutdown. No external dependencies needed — perfect
// for an in-process worker loop draining a queue.

using System;
using System.Threading;
using System.Threading.Tasks;
using System.Threading.Channels;
using Microsoft.Extensions.Hosting;

public reco
...

4. Hangfire — persistent jobs with retries

A plain BackgroundService keeps its queue in memory, so a restart loses every pending job. Hangfire fixes that by storing jobs in a database: they survive restarts, retry automatically when they throw, and show up in a built-in dashboard at /hangfire. All three job shapes have a one-liner — BackgroundJob.Enqueue for fire-and-forget, BackgroundJob.Schedule for delayed, and RecurringJob.AddOrUpdate (with a stable id) for recurring.

Worked example: Hangfire fire-and-forget, delayed & recurring

The three job shapes with Hangfire's fluent API and a CRON schedule.

Try it Yourself »

// ══════════════════════════════════════════════
// Hangfire — persistent jobs with automatic retries + dashboard
// ══════════════════════════════════════════════
// Hangfire stores every job in a database (SQL Server, PostgreSQL,
// Redis...). Jobs SURVIVE a restart, retry automatically on failure,
// and you get a /hangfire dashboard showing history.

using System;
using Hangfire;

// Program.cs — setup
// builder.Services.AddHangfire(cfg => cfg
//     .UseSqlServerStorage(connectionString))
...

5. Quartz.NET — enterprise scheduling

Quartz.NET is the heavyweight option. It splits the job (a class implementing IJob) from the trigger (when it fires), so one job can have many triggers, each with a full CRON expression and misfire handling for runs missed during downtime. It also supports clustering, so a job runs on exactly one node in a farm. Reach for it when Hangfire's scheduling isn't expressive enough — otherwise Hangfire is simpler.

Worked example: Quartz.NET job + CRON trigger

Separate the job from its trigger and schedule it with CRON.

Try it Yourself »

// ══════════════════════════════════════════════
// Quartz.NET — enterprise scheduling (rich CRON, clustering)
// ══════════════════════════════════════════════
// Quartz separates the JOB (what to do) from the TRIGGER (when to do it).
// One job can have several triggers; triggers support full CRON and
// "misfire" handling for missed runs.

using System.Threading.Tasks;
using Quartz;

// A job: implement IJob and put the work in Execute.
public class InvoiceJob : IJob
{
    public Task Execut
...

🔎 Deep Dive: idempotency, retries & persistence

Background jobs will run more than once. A worker can crash after doing the work but before marking it done, so the job runs again on retry. That means every job must be idempotent — running it twice has the same effect as running it once. "Charge the card" is dangerous; "charge the card if this order isn't already paid" is safe.

Retries are your safety net for transient failures (a network blip, a locked row). Hangfire and Quartz retry automatically; in a hand-rolled worker you add it yourself, usually with exponential backoff (wait 1s, then 2s, then 4s…). After a few failures a job is "poison" — move it to a dead-letter store and alert, rather than retrying forever.

// Idempotent: a guard makes a second run a no-op.
if (order.IsPaid) return;          // already done — nothing to do
order.Charge();
order.IsPaid = true;

// Poison handling: stop after N attempts, don't loop forever.
if (attempt >= maxAttempts) {
    MoveToDeadLetter(job);          // park it, alert a human
    return;
}

Persistence is what makes all this survive a restart. A queue in memory vanishes when the process dies; a queue in a database (Hangfire) or on a broker (RabbitMQ, Azure Service Bus) is still there when the app comes back. If losing a job would matter, the queue must be persistent.

Pro Tips

💡 Never do slow work in the request: enqueue it and return. The user gets a fast response and the work retries on its own if it fails.
💡 Write every job idempotent: assume it can run twice. Guard with a check (if (order.IsPaid) return;) so a retry is harmless.
💡 Pass IDs, not objects: Hangfire and Quartz serialise arguments. Pass an orderId and reload the order inside the job, not a fat object graph.
💡 Give recurring jobs a stable id: RecurringJob.AddOrUpdate("nightly-report", ...) — the id stops re-deploys creating duplicate schedules.
💡 Honour the cancellation token: pass stoppingToken to your awaits so the worker stops promptly and cleanly on shutdown.
💡 Keep jobs small and DI-friendly: create a scope inside the job for scoped services like a DbContext; don't capture them across loop iterations.

Common Errors (and the fix)

Doing long work on the request thread: sending an email or building a PDF inside a controller blocks the response and risks a timeout. Enqueue a job and return immediately — let the worker do the slow part.
Non-idempotent jobs: a job that charges a card or sends an email unconditionally will double-charge or double-send when it retries. Guard it (if (order.IsPaid) return;) so a second run is a no-op.
No retry or poison handling: letting a job die on the first transient error loses work; retrying forever on a permanent error spins the worker and floods logs. Retry with backoff, then dead-letter after N attempts.
Losing jobs on restart: an in-memory queue (a plain Queue<T> or Channel<T>) is gone the moment the process stops. If the work matters, use a persistent store — Hangfire's database or a message broker.
"Cannot resolve scoped service from root provider": you injected a scoped service (like DbContext) into a singleton BackgroundService. Inject IServiceProvider and call provider.CreateScope() inside the loop instead.

📋 Quick Reference

Task	Code	Notes
Add to a queue	jobs.Enqueue(x)	Adds to the back (FIFO)
Take next job	jobs.Dequeue()	Removes from the front
Long-running worker	class W : BackgroundService	Override ExecuteAsync
Fire-and-forget	BackgroundJob.Enqueue(...)	Hangfire, runs once now
Delayed	BackgroundJob.Schedule(...)	Hangfire, after a delay
Recurring	RecurringJob.AddOrUpdate(...)	Hangfire, CRON + stable id
Quartz job	class J : IJob	Implement Execute

Frequently Asked Questions

Q: Can't I just use Task.Run to do work in the background?

For a quick, fire-and-forget bit of work it sometimes seems fine, but it's risky: the work isn't persisted, won't retry, and can be killed mid-flight when the app recycles. Use a real job system (BackgroundService + a queue, or Hangfire) so work survives and retries.

Q: What does "idempotent" actually mean here?

A job is idempotent if running it twice has the same effect as running it once. Because retries can re-run a job, design each one so a repeat is harmless — usually by checking whether the work is already done before doing it.

Q: BackgroundService or Hangfire — how do I choose?

Use BackgroundService for simple, in-process work where losing a job on restart is acceptable. The moment jobs must survive restarts, retry automatically, or be visible in a dashboard, switch to Hangfire.

Q: Why pass an ID into a job instead of the whole object?

Hangfire and Quartz serialise job arguments to the database. A big object can fail to serialise or go stale by the time the job runs. Pass a small ID and reload the current data inside the job.

Q: What's a "poison" message?

A job that fails every time it runs — bad data, a permanent error. Retrying it forever wastes resources, so after a few attempts you move it to a dead-letter store and alert a human, instead of looping.

Mini-Challenge: a tiny scheduler

No blanks this time — just a brief and an outline. Build a tiny scheduler that holds a few jobs, each with its own interval, and on every tick runs the ones that are due while skipping the rest. This is exactly the bookkeeping a real recurring scheduler does. Run it and check your output against the expected lines in the comments.

🎯 Mini-Challenge: run due jobs, skip not-yet-due ones

Schedule a heartbeat (every 2 ticks) and a report (every 5 ticks) over 5 ticks.

Try it Yourself »

using System;
using System.Collections.Generic;

// 🎯 MINI-CHALLENGE: a tiny scheduler
// Each job has a name, an interval (how often it should run) and a
// "ticksSinceRun" counter. On every tick:
//   - add 1 to every job's ticksSinceRun
//   - if ticksSinceRun >= interval: run it (print "Running NAME") and reset to 0
//   - otherwise skip it
//
// Set up two jobs:  "heartbeat" every 2 ticks,  "report" every 5 ticks.
// Loop ticks 1..5 and let the scheduler decide what runs each tick.
//
// ✅
...

🎉 Lesson Complete

✅ Offload slow work off the request thread — enqueue and return immediately
✅ Three shapes: fire-and-forget, delayed, and recurring
✅ Every job system is a queue plus a worker that drains it (FIFO)
✅ IHostedService/BackgroundService give you a built-in long-running worker
✅ Hangfire adds persistence, automatic retries, and a dashboard; Quartz.NET adds rich scheduling
✅ Make jobs idempotent, add retries with poison handling, and persist anything you can't lose
✅ Next lesson: Performance Profiling — measure and benchmark your .NET apps

Background Jobs

What You'll Learn

💡 Real-World Analogy

Why offload work at all?

📊 The three job shapes

📊 Choosing a tool: HostedService vs Hangfire vs Quartz

1. Queues and Workers — the core idea

Worked example: enqueue jobs, then drain them in a loop

🎯 Your turn: a queue and a worker loop

2. Recurring Schedules — running only when due

🎯 Your turn: run a job only when its interval is due

3. IHostedService & BackgroundService

Worked example: a BackgroundService draining a queue

4. Hangfire — persistent jobs with retries

Worked example: Hangfire fire-and-forget, delayed & recurring

5. Quartz.NET — enterprise scheduling

Worked example: Quartz.NET job + CRON trigger

🔎 Deep Dive: idempotency, retries & persistence

Pro Tips

Common Errors (and the fix)

📋 Quick Reference

Frequently Asked Questions

Mini-Challenge: a tiny scheduler

🎯 Mini-Challenge: run due jobs, skip not-yet-due ones

🎉 Lesson Complete

Cookie & Privacy Settings