Hangfire for Background Processing

1. The Problem Before Background Jobs

An HTTP request thread is the wrong place to execute slow, retry-prone, or mission-critical operations.

2. What is Hangfire?

Hangfire is an open-source .NET library that allows you to create, persist, and process background jobs reliably — without requiring separate standalone services or complex external broker management for standard workflows.

• Persistent Storage: Jobs are persisted to Redis — guaranteeing they survive application restarts or server crashes.
• In-Process Execution: Runs seamlessly inside any .NET application (ASP.NET Core API, Console Apps, Windows Services).
• Built-in Dashboard: Ships with a real-time web dashboard for queue health inspection, metrics, and manual 1-click retries.
• Horizontal Scalability: Scale out worker instances by pointing multiple servers at the same shared storage.

3. Four Kinds of Jobs in Hangfire

4. Architecture & Queue Isolation

In our microservices, we configure dedicated Hangfire servers and queues to guarantee that high-frequency background traffic never starves critical payment operations:

5. Production Implementation Examples

Program.cs Configuration with Redis

var redisConn = ConnectionMultiplexer.Connect(redisConnectionString);

builder.Services.AddHangfire(cfg => cfg
    .UseRedisStorage(redisConn, new RedisStorageOptions {
        Prefix = "OPO-MpesaService:"
    }));

builder.Services.AddHangfireServer(o => {
    o.Queues = new[] { "mpesa-queue" };
    o.WorkerCount = 1;
});

Recurring Worker with Safe Row Claiming (FOR UPDATE SKIP LOCKED)

// Registered in ApplicationStarted to ensure Redis connectivity
app.Lifetime.ApplicationStarted.Register(() =>
{
    RecurringJob.AddOrUpdate<MpesaScraper>(
        "MpesaScraper",
        x => x.ScrapeUnsent(),
        "*/3 * * * * *" // Polling every 3 seconds
    );
});

// SQL query for non-blocking concurrent worker polling:
SELECT * FROM "disbursement"
WHERE "issent" = false AND "systemretries" < 3
ORDER BY "id" LIMIT 20
FOR UPDATE SKIP LOCKED;

6. Best Practices & Key Takeaways