Queues, Batches, Cron Jobs & Job Scheduling

1. Why background jobs exist

A web request should finish in well under a second. Sending an email (1–3 s), generating a PDF invoice, calling a slow partner API or resizing images would make the user wait and tie up a PHP worker. Instead the request puts a small message ("send receipt for order 187") on a queue and returns immediately; a separate worker process picks it up and does the work.

Queues also absorb spikes. During a flash sale 50,000 orders arrive in ten minutes; the order is saved instantly and the confirmation emails drain over the next minutes without crashing the mail server.

Checkout with a queue

1. User clicks Pay
2. Save order + payment (fast DB transaction)
3. Dispatch SendOrderConfirmation, ReserveStock, NotifyWarehouse
4. Return 201 to the user
5. Workers process jobs in the background

2. Laravel queues: jobs, workers, retries

A job that is safe to retry

class SendPaymentReceipt implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 5;
    public array $backoff = [10, 60, 300];       // seconds between attempts
    public int $timeout = 30;

    public function __construct(public Payment $payment) {}   // only the id is serialized; reloaded in handle()

    public function handle(Mailer $mail): void
    {
        if ($this->payment->receipt_sent_at) return;           // idempotent: a retry must not email twice
        $mail->to($this->payment->user)->send(new PaymentReceiptMail($this->payment));
        $this->payment->update(['receipt_sent_at' => now()]);
    }

    public function failed(Throwable $e): void
    {
        Log::error('Receipt failed', ['payment' => $this->payment->id, 'error' => $e->getMessage()]);
    }
}

// Dispatch only after the DB transaction commits, otherwise the worker may not find the payment yet
SendPaymentReceipt::dispatch($payment)->onQueue('emails')->afterCommit();

Running workers

php artisan queue:work redis --queue=payments,emails,default --tries=3 --max-time=3600
php artisan queue:failed            # list failed jobs
php artisan queue:retry all         # retry them after fixing the bug
php artisan queue:restart           # workers reload code after a deployment

Drivers: database (simple, works on shared hosting, which is what this portal uses), Redis (fast, most common), SQS, or Kafka/RabbitMQ via packages. In production a process manager (Supervisor, systemd, a Kubernetes Deployment) keeps workers running. On cPanel without Supervisor, a cron entry runs queue:work --stop-when-empty every minute.

Queue priority comes from the order in --queue=payments,emails,default: the worker always empties "payments" first.

3. Batches: many jobs tracked as one

A batch groups many jobs and gives you progress, a single "all done" callback and failure handling. Typical uses: generate monthly statements for 2 million customers, import a 500,000-row product CSV in 1,000-row chunks, send a festive-offer email to every customer segment.

Monthly bank statements as a batch

// php artisan queue:batches-table && php artisan migrate   (creates job_batches)
$jobs = Account::query()->where('status', 'active')->select('id')
    ->lazyById(1000)->chunk(500)
    ->map(fn ($ids) => new GenerateStatements($ids->pluck('id')->all(), '2026-09'));

$batch = Bus::batch($jobs->all())
    ->name('Statements 2026-09')
    ->allowFailures()                                       // one bad account must not stop 2M others
    ->then(fn (Batch $b) => Notification::route('mail', 'ops@bank.test')->notify(new StatementsReady($b->id)))
    ->catch(fn (Batch $b, Throwable $e) => Log::warning('statement chunk failed', ['batch' => $b->id]))
    ->finally(fn (Batch $b) => Cache::forget('statements:running'))
    ->onQueue('statements')
    ->dispatch();

// Progress for an admin screen
$batch = Bus::findBatch($id);
echo "{$batch->processedJobs()}/{$batch->totalJobs} ({$batch->progress()}%), failed: {$batch->failedJobs}";

Chains are the sequential cousin: Bus::chain([new CaptureFunds($order), new GenerateInvoice($order), new NotifyWarehouse($order)])->dispatch() runs each job only after the previous one succeeded. Use a chain when order matters, a batch when the items are independent.

4. Cron and the Laravel scheduler

Cron is the Unix service that runs commands at set times. Each line has five time fields (minute, hour, day of month, month, day of week) and a command. Managing dozens of cron lines on every server is error-prone, so Laravel needs only ONE cron entry that runs schedule:run every minute, and the schedule itself lives in code, under version control.

On GoDaddy cPanel: Cron Jobs > Common settings "Once per minute" > command: /opt/alt/php83/usr/bin/php /home/USER/career-portal/naukri-com/artisan schedule:run >> /dev/null 2>&1

Cron syntax

# ┌ minute (0-59)  ┌ hour (0-23)  ┌ day of month  ┌ month  ┌ day of week (0=Sun)
  *              *              *              *       *      php /var/www/artisan schedule:run   # every minute
  30             2              *              *       *      backup.sh                          # 02:30 daily
  0              */6            *              *       *      import-feeds.sh                    # every 6 hours
  0              9              1              *       *      statements.sh                      # 09:00 on the 1st

app/Console/Kernel.php schedule for a bank and a shop

protected function schedule(Schedule $schedule): void
{
    // Banking
    $schedule->command('interest:accrue')->dailyAt('00:05')->timezone('Asia/Kolkata')
        ->withoutOverlapping()->onOneServer();                  // never run twice, even with 5 servers
    $schedule->command('statements:generate --month=previous')->monthlyOn(1, '01:00');
    $schedule->command('loans:mark-overdue')->dailyAt('06:00')->environments(['production']);

    // Payments
    $schedule->command('payments:reconcile')->hourly()->runInBackground()
        ->onFailure(fn () => Notification::route('slack', config('alerts.slack'))->notify(new ReconcileFailed()));
    $schedule->command('payments:expire-pending')->everyFiveMinutes();   // cancel unpaid orders after 15 min

    // E-commerce
    $schedule->command('carts:send-abandoned-reminders')->hourlyAt(15);
    $schedule->job(new RebuildSearchIndex())->weeklyOn(0, '03:00');     // queued job on schedule
    $schedule->command('queue:prune-batches --hours=48')->daily();
}

withoutOverlapping() puts a lock in the cache so a slow run is not started again a minute later; onOneServer() makes sure only one of several servers runs it (requires a shared cache such as Redis). Long tasks should dispatch jobs instead of doing all the work inside the scheduled command.

php artisan schedule:list shows every task and its next run; php artisan schedule:test runs one now.

5. Spring Boot: @Async, @Scheduled and ShedLock

Scheduled interest posting with a distributed lock

@SpringBootApplication
@EnableScheduling
@EnableAsync
public class BankApplication { }

@Component
class InterestJobs {
    @Scheduled(cron = "0 5 0 * * *", zone = "Asia/Kolkata")         // sec min hour day month weekday
    @SchedulerLock(name = "accrueInterest", lockAtMostFor = "PT2H")  // ShedLock: one instance only
    public void accrueInterest() {
        accounts.streamActiveSavings().forEach(interestService::accrueForToday);
    }

    @Scheduled(fixedDelay = 300_000)                                 // 5 min after the previous run ends
    public void expirePendingPayments() { payments.expireOlderThan(Duration.ofMinutes(15)); }
}

@Service
class ReceiptService {
    @Async                                                            // runs on a thread pool, caller does not wait
    public CompletableFuture<Void> send(Payment p) { mailer.send(p); return CompletableFuture.completedFuture(null); }
}

@Async keeps work in the same JVM; if the instance restarts, in-flight work is lost. For anything important (money, emails customers rely on) use a durable queue or Kafka topic instead. Spring Batch is the framework for large chunked jobs (read-process-write with restartability), the counterpart of Laravel batches for bulk statement or settlement files.

6. Kafka as a queue, and showing progress in React

A Kafka topic can replace a job queue when many independent consumers need the same event (see the Kafka events lesson). Each consumer group gets every message; inside one group partitions spread the work across instances, like queue workers.

React: poll batch progress for an admin screen

function BatchProgress({ id }) {
  const [batch, setBatch] = useState(null);
  useEffect(() => {
    const timer = setInterval(async () => {
      const b = await (await fetch(`/api/admin/batches/${id}`)).json();
      setBatch(b);
      if (b.finished_at) clearInterval(timer);
    }, 3000);
    return () => clearInterval(timer);
  }, [id]);
  if (!batch) return <Spinner />;
  return <progress max="100" value={batch.progress}>{batch.progress}%</progress>;
}

7. Interview questions