The Last Mango Problem: Race Conditions in a Real Marketplace

There's one Sindhri mango left at the Latifabad stall. Two people tap "Place order" at the same moment. Who gets it?

If your answer is "whoever was first", you've already made the mistake this post is about. Computers don't do "first" the way people do. Two requests can land in the same millisecond, both read the stock, both see 1, both subtract one, and both leave happy. The farmer now owes somebody a mango that doesn't exist.

I ran into this properly while building GleanGrid, a pre-order marketplace for Hyderabad's farmers markets that I built on my own. This is how it handles the moments where people collide: the last item, the single-use coupon, the nearly full pickup window, the double-tapped button, the payment that arrives late. It also covers a MySQL detail that fooled code which looked perfectly correct to me.

#Why "check, then write" is broken

Here's the naive version. I'd bet most first checkouts look like it; mine did.

$product = Product::find($id);

if ($product->stock_quantity >= $qty) {
    $product->stock_quantity -= $qty;
    $product->save();
}

Run that from two requests at once and both pass the if before either one saves. The gap between the read and the write is tiny, but a busy Saturday morning is exactly when a lot of people land in it together.

#Rule one: lock the thing you're about to change

The fix is to make the read and the write one indivisible step. In MySQL that means a transaction plus a locking read, SELECT … FOR UPDATE. The first request to get the lock holds it until it commits. Everyone else waits in line and then sees the updated number.

$products = Product::whereIn('id', $ids)
    ->where('farmer_profile_id', $farmer->id)
    ->orderBy('id')
    ->lockForUpdate()
    ->get();

Two details matter here.

Lock in a fixed order. A basket can hold several products. If one checkout locks the mangoes and then the honey while another locks the honey and then the mangoes, each ends up waiting for the other forever. That's a deadlock. Sorting by id means everyone queues in the same order, so the cycle can't form.

Merge duplicate lines first. If a basket somehow holds the same product twice, checking each line against the stock separately lets 2 + 2 sneak past a stock of 3. GleanGrid sums the lines per product before it checks anything.

#Rule two: the same goes for coupons

A coupon with a usage limit of one is just another counter that fifty people want to decrement. Same medicine: lock the coupon row, check it, record the redemption and bump the counter, all inside the checkout transaction.

$coupon = Coupon::where('code', $code)->lockForUpdate()->first();
$this->assertUsable($coupon, $order->farmer_profile_id, $subtotal, $user);

CouponRedemption::create([...]);
$coupon->increment('used_count');

Because I don't fully trust myself, the database gets the last word too:

ALTER TABLE coupons ADD CONSTRAINT chk_coupon_usage
  CHECK (usage_limit IS NULL OR used_count <= usage_limit);

If a future change ever forgets the lock, MySQL refuses the write instead of quietly giving away a discount.

#Rule three: the pickup window, and the bug I didn't see

Each farmer offers pickup windows with a capacity, say three orders between 7 and 8 am, so the stall doesn't get swamped. This code looked right to me:

$slot = $farmer->pickupSlots()->lockForUpdate()->find($slotId);

$booked = Order::where('pickup_slot_id', $slot->id)
    ->whereDate('pickup_date', $date)
    ->whereIn('status', Order::OPEN)
    ->count();

if ($booked >= $slot->capacity) {
    throw ValidationException::withMessages(['pickup' => 'That pickup window is fully booked.']);
}

Lock the slot, count the bookings, reject if it's full. The unit test passed. Then I wrote a stress script that fires real, separate PHP processes at checkout at the same instant, and it said this:

Twenty orders in a window built for three. The lock was there. So what went wrong?

#Snapshots under REPEATABLE READ

MySQL's InnoDB engine runs at the REPEATABLE READ isolation level by default. Inside a transaction, a normal SELECT doesn't read the latest data. It reads a snapshot, and that snapshot is fixed at the transaction's first ordinary read.

My checkout's first read was loading the farmer's profile, before it asked for the slot lock. So each request:

1. read the farmer, which fixed its snapshot at that moment;
2. waited in line for the slot lock while other checkouts committed their orders;
3. got the lock and ran count() against its old snapshot, which contained none of those new orders.

The lock serialised the requests perfectly. It just didn't change what they could see. Stock and coupons were never affected, because I read those rows with FOR UPDATE, and locking reads always see the latest committed data, not the snapshot.

I'll admit I stared at this one for a while. What finally cracked it was writing down, step by step, what each transaction could see at each moment. I talked it through with Claude as well, and it asked the question that mattered: when does this transaction take its snapshot? The answer was in my own code, three lines above the lock.

#The one-line fix

Make the count a locking read too:

$booked = Order::where('pickup_slot_id', $slot->id)
    ->whereDate('pickup_date', $date)
    ->whereIn('status', Order::OPEN)
    ->lockForUpdate()
    ->count();

Then I ran it again, this time with fifty buyers:

The lesson I'm keeping: inside a transaction, a lock decides who goes next. It doesn't decide what they see. Anything you use to make a decision under a lock should itself be read with a lock.

#Rule four: the double-tap

The most common race isn't fifty strangers. It's one person on a slow connection pressing "Place order" twice, or checking out in two tabs. GleanGrid wraps the whole checkout in a short, per-customer atomic lock:

$lock = Cache::lock("checkout:{$customer->id}", 15);

if (! $lock->get()) {
    throw ValidationException::withMessages([
        'checkout' => 'Your previous checkout is still being processed. Please wait a moment.',
    ]);
}

In the stress test, ten simultaneous submissions from one account produce exactly one order and nine polite messages. While writing this I also noticed that the "too many open orders" check ran before that lock was taken, which left a small window for going one over the limit. It now runs inside.

#Rule five: the races that don't look like races

Once you start looking, they're everywhere. These three never showed up in a stress test, because they need a person and a clock rather than fifty processes.

The farmer's stale form. A farmer opens the stock form at 7:00, when there are 20 bunches of spinach. Customers buy five while the form sits open. At 7:10 the farmer saves the form, still showing 20, and quietly erases five sales. That's a lost update. Now the form sends back the number it showed along with the new one, and the server works out what happened in between:

private function reconcileStock(Product $locked, int $submitted, ?int $seen): int
{
    if ($seen === null) {
        return $submitted;
    }
    if ($submitted === $seen) {
        return $locked->stock_quantity; // field untouched: keep what's really there
    }

    // subtract whatever sold since the form was opened
    return max(0, $submitted - max(0, $seen - $locked->stock_quantity));
}

The Monday restock. "Apply weekly template" used to read every product, then write its usual quantity back. A reservation made between the read and the write was overwritten. It's now one atomic statement per run, with no read-modify-write loop to fall into:

UPDATE products SET stock_quantity = weekly_quantity WHERE ...;

Two restocks, two e-mails. When a sold-out item comes back, everyone who asked gets one alert. If two restocks fire at once, both could see the same pending alerts. Each alert is now claimed with a conditional update, and only the process that wins the row sends the message:

StockAlert::whereKey($alert->id)
    ->whereNull('notified_at')
    ->update(['notified_at' => now()]) === 1; // only one process ever gets a 1

The same trick, "update where it's still in the state I expect, then check how many rows changed", turns up all over GleanGrid. It's the cheapest correct lock there is.

#Rule six: money is where races get expensive

Online payments came later, and I treated them as the riskiest code in the project. A double charge isn't an awkward message at a stall; it's someone's money.

• One attempt at a time. The payment row is locked and moves from pending to processing exactly once. A second tap while the first is in flight gets "payment in progress", not a second charge.
• Replays are harmless. Every attempt carries a one-time idempotency key, stored under a unique index. If a flaky connection makes the browser resend the same request, the server recognises the key and returns the existing result.

$locked = Payment::whereKey($payment->id)->lockForUpdate()->firstOrFail();

if (PaymentEvent::where('payment_id', $locked->id)->where('idempotency_key', $key)->exists()) {
    return false; // already handled: answer with what happened, charge nothing
}
if ($locked->status === 'processing') {
    throw ValidationException::withMessages(['payment' => 'payment.in_progress']);
}

• Late confirmations are refunded, not lost. If the gateway confirms after the 20-minute window has closed, the stock may already be back on sale. GleanGrid notices the payment is no longer processing, records a late_capture event and refunds automatically.
• Refunds can't run twice either: they go through a conditional UPDATE … WHERE payment_status = 'paid'.

#Rule seven: let the loser see a form error, not a crash

Some races end at a unique index rather than a lock. Two people pick the same username at the same moment, and both pass the "is this taken?" check because neither has saved yet. The database lets one through and rejects the other with a duplicate-key error, which by default becomes a server error page.

GleanGrid maps that one exception, everywhere, to an ordinary validation message:

$exceptions->map(UniqueConstraintViolationException::class, function ($e) {
    $field = collect(['email', 'username', 'code', 'slug'])
        ->first(fn ($f) => str_contains($e->getMessage(), $f)) ?? 'form';

    return ValidationException::withMessages([
        $field => __('validation.unique', ['attribute' => $field]),
    ]);
});

The second person sees "The username has already been taken" under the field, exactly as if they'd been a second slower.

#What about thousands of people at once?

Correctness under concurrency and survival under load are different problems, so they get different tools.

Updates that don't compete are cheap. A thousand people editing their own profiles touch a thousand different rows. InnoDB locks rows, not tables, so those writes never wait on each other.

Hot rows are the real bottleneck. A thousand people buying the same product queue on that one row. Each transaction is short (lock, check, write, commit, a few milliseconds), and the rare deadlock is retried automatically:

DB::transaction(fn () => /* place the order */, attempts: 3);

Floods get turned away early. Every page sits behind a site-wide rate limiter, 300 requests a minute per IP for guests and 600 per signed-in account, with much tighter limits on sign-in, sign-up, password resets, search, checkout and payments. Writes are capped at sixty a minute per account, while page views stay unrestricted so browsing never feels throttled.

RateLimiter::for('global', fn (Request $request) => $request->user()
    ? Limit::perMinute(600)->by('u:'.$request->user()->id)
    : Limit::perMinute(300)->by('ip:'.$request->ip()));

Bots don't get to race at all. Every public form carries a signed ticket issued with the page, so a script can't hammer the sign-up form without first loading it, waiting, and passing a Turnstile or reCAPTCHA check.

Real denial-of-service protection belongs in front of the app. No PHP code can absorb a flood that saturates the network. In production GleanGrid is meant to sit behind Cloudflare, which soaks up volumetric attacks before they reach the server. Because the app only trusts X-Forwarded-For from proxies listed in TRUSTED_PROXIES, rate limits apply to the real visitor rather than the proxy, and nobody can dodge them by faking the header.

#Test it the way it breaks

The most useful thing I built for all this wasn't a lock. It was the script that fires fifty real processes at the same instant against the real database, checks the numbers afterwards and cleans up after itself. Unit tests run in one process, one request at a time, so they'll happily pass code that falls over the moment two requests overlap.

If your application sells anything that can run out, write the test that makes things overlap on purpose. Then ask the question that caught my bug: at this line, what can this transaction actually see?

The whole project, stress script included, is on GitHub. If you've been bitten by a race condition of your own, I'd love to hear the story.