Backend Architecture

The backend question isn't "which framework." It's "what promise am I making to the user, and what has to be true on a server for that promise to hold?" Build it when a promise requires it, not when the architecture diagram looks incomplete.

The concept

A backend exists to do things a client can't be trusted or relied upon to do:

1. Persist across devices and reinstalls, "your data is safe"
2. Enforce rules the client can't, permissions, entitlement, quotas
3. Hold secrets, API keys that must never reach a client
4. Coordinate between users, sharing, collaboration, marketplaces
5. Run work independent of the client, scheduled jobs, webhooks, notifications
6. Aggregate, analytics, leaderboards, recommendations

If none of those apply yet, you don't need one yet. Local storage plus a static app is a complete architecture for a genuinely useful class of product, and it eliminates an entire category of failure.

Three shapes, and what each costs

Default to the highest abstraction that fits. Running infrastructure has no product upside until the bill genuinely hurts.

Serverless vs long-running

Serverless for bursty, request-shaped work. A server for persistent connections, long jobs, or steady high load. Most early products are the former.

Offline-first: the architecture that decides how your app feels

For anything mobile, the single most consequential backend decision is who owns the truth.

 ❌ SERVER-AUTHORITATIVE              ✅ OFFLINE-FIRST
 action → network → wait → render     action → local write → render (instant)
                                              → queue → sync in background

 every screen needs loading + error   no loading states on your own data
 breaks on bad networks               works on a plane, in a tunnel
 spinner is the default experience    server outage is invisible

Offline-first properties you get for free: instant interactions, no loading states on local data, resilience to outages, and a queue that turns a failed write into a delayed one rather than a lost one.

What it costs: conflict handling if the same data changes in two places, and a genuine second code path (see below).

The write queue

The mechanism that makes offline-first work is about thirty lines:

 1. Action happens → write to local store (this is the truth)
 2. Append an operation to a PERSISTENT queue
 3. Try to flush: send each op to the server
 4. On success → remove from queue
    On failure → leave it; retry on next boot / reconnect

Three properties that must hold:

• The queue is persisted, not in memory. An in-memory queue loses writes on exactly the failure it exists to handle.
• Reading the queue is exception-guarded. Corrupt JSON must return an empty queue, not crash at boot.
• Every operation is idempotent. Replaying it must be a no-op.

Idempotency belongs in the database

Client-side dedup can be bypassed by a bug, an old app version, or a second device. A database constraint cannot:

unique (user_id, item_id)          -- a replayed insert is a no-op, always

Put the guarantee where it can't be circumvented, then use upsert … on conflict do nothing.

⭐ Sync is two features, and you will only build one

This is the most commonly shipped-broken piece of backend work, and the reason is structural:

So upload gets exercised thousands of times and download gets exercised zero times. You ship, a user reinstalls, signs in successfully, and sees an empty app.

For every write path, ask: what reads this back, and when? Then test that path on a clean install.

The hydrate path also has non-obvious rules:

• Drop the pending queue first if this is a "returning user" path, those queued writes belong to the throwaway local session and would pollute the real account.
• Re-derive everything derivable rather than restoring it.
• Never restore state a third party owns, entitlement belongs to your payment provider, not your cache (Chapter 27).

Server-side work

Two things almost every product eventually needs a server for:

Webhooks, external services telling you something happened (payment succeeded, subscription cancelled). Rules: verify a shared secret, fail closed if the secret is unset, make handling idempotent (they retry), and return 200 for events you'll never be able to process, otherwise you get infinite retries.

Background jobs, email, exports, cleanup, scheduled work. Serverless cron or a queue (BullMQ, Inngest, Trigger.dev). Keep them idempotent too.

📐 Best practice

Add a backend when a user-facing promise requires it, not before.

Local-first for anything mobile. The device owns the truth; the server is a durable mirror.

Persist the write queue, and guard its read.

Put idempotency in the schema.

Build and test hydrate() explicitly, on a clean install.

Fail closed. Missing config must reject, never allow.

Never trust the client for anything that grants access, money, or permissions.

Keep secrets server-side. Anything in a client binary is public, assume it will be extracted.

Run your platform's security advisor. Managed platforms ship one; it knows things you don't.

Choose your region deliberately based on where your users are.

💀 Common mistakes

Building a backend before a promise requires it. Weeks spent on infrastructure for an unvalidated product.

Server-authoritative mobile apps. Spinners everywhere; broken on bad networks.

In-memory write queues. Lose data on crash, the exact scenario they exist for.

Client-side idempotency only. Bypassed by bugs and old versions.

Building upload without download. Sync that stores data where users can never see it again.

Flushing the wrong session's queue on a returning-user path.

Restoring third-party state from cache, stale entitlement gives away your product.

Webhooks that don't fail closed. Missing secret treated as "no auth required" means anyone can grant themselves anything.

Webhooks that 4xx on unprocessable events. Infinite retry storms.

Secrets in the client. Extracted within hours of release.

Self-hosting too early. You've hired yourself as a sysadmin.

The professional workflow

 1. LIST THE PROMISES that require a server. If none → don't build one.

 2. CHOOSE THE SHAPE — BaaS / managed / self-hosted, by ops load you'll carry

 3. DECIDE WHO OWNS TRUTH
    mobile → local-first, server as mirror
    web app behind login → server-authoritative is fine

 4. DESIGN THE WRITE PATH
    local write → persistent queue → idempotent upsert →
    DB constraint → retry on boot/reconnect

 5. ⭐ DESIGN THE READ-BACK PATH
    who reads this, when? → build hydrate() → TEST ON CLEAN INSTALL

 6. SECURE IT
    row-level rules · secrets server-side · fail closed ·
    access-granting tables not client-writable

 7. WEBHOOKS + JOBS — verified, idempotent, correct status codes

 8. VERIFY AGAINST THE LIVE SYSTEM — name the specific row you expect

Tools, websites & costs

A complete backend is $0/month at launch scale.

Alternatives & trade-offs

BaaS vs your own API. BaaS removes weeks of work and couples you to a vendor's model and pricing. Escaping is real but tractable if you keep business logic in your code rather than in vendor-specific features.

SQL vs NoSQL. Relational data (users, orders, relationships) wants SQL. Document stores suit genuinely document-shaped data. Most products are relational and choose NoSQL for the wrong reasons (Chapter 20).

Local-first vs server-authoritative. Local-first is better UX and more code paths. Server-authoritative is simpler and worse offline. Mobile: local-first. Collaborative web: server-authoritative with optimistic updates.

Sync engines (ElectricSQL, Replicache, PowerSync, Yjs) do the hard parts, conflict resolution, partial replication, properly. Worth it for genuinely collaborative or multi-device-concurrent products; overkill for single-device append-mostly data.

Monolith vs services. Monolith. Always, at this stage.

Checklist

• The backend exists because a user-facing promise requires it
• I chose the highest abstraction that fits my needs
• Mobile: local owns the truth, server is a mirror
• Write queue is persisted and its read is exception-guarded
• Every sync op is an idempotent upsert
• Idempotency is enforced by a database constraint
• hydrate() exists and is tested on a clean install
• Returning-user path drops the stale queue before hydrating
• Third-party-owned state is never restored from cache
• Webhooks verify a secret and fail closed
• Webhooks are idempotent and return 200 for unprocessable events
• No secrets in the client
• I ran the platform's security advisor
• Verified against the live system, naming a specific expected row

📓 Case Study: sync that worked perfectly and was useless

Project: SOLIS. The app ran with no backend for 18 days, entirely on local device storage.

That was the right call, and the trigger for changing it was a specific promise, not an architectural itch:

"what will be the backend of the app, we need to store the current level of the user right?"

The real driver: if a user reinstalls, they lose 90 days of progress. That's a promise the product implicitly made and couldn't keep. The decision was recorded with its cost attached, folding the backend into v1 pushed the launch date out, which is what makes it a decision rather than a drift.

The architecture, stated plainly in the plan:

Still offline-first: local storage stays the cache so the app works offline and the forgiving design is untouched; the backend syncs in the background.

The properties this bought: no loading states on progress screens, works with no network, a failed write becomes a delayed write, and a server outage is invisible to users.

The write queue was about thirty lines and got the important parts right, persisted to local storage (so it survives app kill), guarded with try/catch on read (corrupt JSON returns an empty queue rather than crashing at boot), and every operation an idempotent upsert backed by a database constraint:

unique (user_id, day, lesson_index)   -- replay is a no-op at the DB level

One genuinely clever detail: anonymous authentication was gated behind a meaningful action rather than firing on app launch. Naive anonymous auth creates a user row on every install, inflating the metric the vendor bills on and providing an abuse surface. Deferring it until the user actually started their path solved it without adding friction. The alternative considered and rejected was a captcha, which would have complicated the flow for every real user to defend against a minority.

💀 The bug: sync was upload-only.

For a week, sync worked. Progress uploaded reliably. Sign-in worked. Everything was green. And the entire feature was worthless:

Sync.hydrate() is the mirror of backfill(). Sync was upload-only, so a returning user could sign in successfully and still stare at Day 1.

The promise was "your progress survives a reinstall." What shipped was "your progress is safely stored where you can never see it again."

The cause is exactly the structural one this chapter describes: upload is event-driven and continuous, so it gets exercised constantly by ordinary use. Download runs once, on a rare path, new device, reinstall, sign-in, that never occurs during development, because your dev device already has the data.

The fix surfaced four non-obvious rules, each of which is a bug someone else will ship:

1. Drop the pending queue first on the reinstall path. When linking an identity fails with "already linked" and the app falls back to signing in, the queued writes belong to the throwaway anonymous session on this device. Flushing them pours junk progress into the real account.
2. Restore a position as a derived value, not a raw count. The stored model required care to reconstruct correctly.
3. Handle local-date round-trips. A naive UTC conversion of a start date shifts it by a day depending on timezone, silently changing the user's progress.
4. Deliberately do not restore entitlement. "RevenueCat owns entitlement and a stale flag would hand out a lapsed sub."

That fourth is the sharpest. When restoring state, ask of every field: who owns this? Progress is owned by your database. Entitlement is owned by your payment provider. Restore what you own; re-derive what you don't.

One thing done to a high standard: verification was specific.

Drove the app → the database received: 1 anonymous user, 1 profile, 1 quiz result, and 20 lesson completions (19 via backfill + 1 live via the queue, completing a specific lesson produced row 4:4:pur).

Naming the exact row, and testing both paths (one-time backfill and live queue), is what separates a verification claim from a hope (Chapter 28).

⚠️ A related deviation: the data model changed later, progress moved from a single position pointer to a set of completed items (Chapter 20), which required rewriting both backfill and hydrate. Migrating your data model means migrating your sync. Worth anticipating.

Lessons

1. Add a backend when a promise requires it. Eighteen days without one was correct.
2. Name the cost when you change scope. It makes it a decision rather than a drift.
3. Offline-first for mobile. Local owns truth; the server mirrors it.
4. Persist the queue. An in-memory queue fails at exactly the moment it exists for.
5. Put idempotency in the schema, where client bugs can't bypass it.
6. ⭐ Sync is two features. Download runs once, on a path you never hit in development. Test it on a clean install.
7. Ask "who owns this field?" before restoring it. Entitlement belongs to your payment provider.
8. Drop the stale queue on returning-user paths, or you pollute real accounts.
9. Gate anonymous auth behind a meaningful action, better than adding friction to defend against abuse.
10. Verify against the live system with a named row. "It works" is not a claim.

Next: Chapter 20: Databases and Data Modeling →