Process Large Datasets in Chunks

User::all() is fine when you have two hundred users. It is a production incident when you have two million. It loads every row into memory at once, hydrates a model for each one, and only then lets you start working. Chunking processes large datasets in pieces, so memory stays flat no matter how big the table grows.

Loading everything at once

$users = User::all();

foreach ($users as $user) {
    $user->recalculateScore();
}

On a large table this either exhausts PHP's memory limit or runs so long that the worker gets killed halfway through. Either way, nothing tells you which users were processed and which were not.

Process it in chunks

User::chunkById(500, function (Collection $users): void {
    foreach ($users as $user) {
        $user->recalculateScore();
    }
});

Laravel fetches 500 users, passes them to the closure, releases them and fetches the next 500. At any moment only one chunk lives in memory. The total runtime is similar, but the memory curve is a flat line instead of a cliff.

chunk() or chunkById()

chunk() pages through results with LIMIT and OFFSET. If your closure updates the column you are filtering on, rows shift between pages and some are skipped. chunkById() pages by primary key instead, so it stays correct even when the rows change underneath it. When in doubt, use chunkById().

If you prefer a plain loop over a closure, lazyById() streams the same chunks as a lazy collection:

User::where('active', true)->lazyById(500)->each(
    fn (User $user) => $user->recalculateScore()
);

Why this pays off

Flat memory usage. The job needs the same memory for a thousand rows as for ten million.

No surprise timeouts. Long imports and recalculations finish instead of dying at the memory limit.

Easy to move into queues. A chunk is a natural unit of work. Each one can become its own queued job when processing gets heavier.

Where chunking is not the answer

For small result sets, chunking only adds complexity. And if you just need to update every matching row with the same value, do it in the database instead of in PHP:

User::where('last_login_at', '<', now()->subYear())->update(['active' => false]);

One query beats a million model hydrations every time.

A useful rule of thumb:

• Small result set → get() is fine.
• Large result set, per-row PHP logic → chunkById() or lazyById().
• Same change for every row → a single update() query.

The practical test

Search your Artisan commands and queued jobs for ::all() and ->get() on tables that grow over time. Any of them iterating over the result is a memory problem waiting for enough data. Swap it for chunkById() before the table gets there.