Development

Beyond CRUD: Designing APIs That Predictably Scale

Beyond CRUD: Designing APIs That Predictably Scale

CRUD is where most APIs begin, and where too many designs stop. Create, read, update, and delete are useful verbs, but they are not an architecture. Once an application gains real traffic, background jobs, integrations, retries, permissions, and operational requirements, a resource-shaped interface can become a collection of accidental contracts.

Predictable scale is not primarily about handling an impressive number of requests. It is about making behavior understandable as load, teams, and business rules grow. Clients should know what happens when they retry. Operators should know which request caused a change. Developers should be able to add a feature without turning a simple endpoint into a transaction that spans half the system.

Design around capabilities, not database tables

A database table is an implementation detail. Exposing it directly through endpoints such as POST /orders and PATCH /orders/{id} can work early on, but it often leaks the wrong model. An order is not merely a row that can be edited at any time. It has a lifecycle: it may be submitted, paid, fulfilled, cancelled, or refunded. Each transition has rules, side effects, and audit implications.

Instead of treating every meaningful action as a generic update, give important capabilities explicit names:

  • POST /orders/{id}/submit
  • POST /orders/{id}/cancel
  • POST /orders/{id}/refunds

This does not mean every field needs its own endpoint. Basic profile edits or preference updates can remain ordinary resource updates. The distinction is whether the change represents a business operation with validation, state rules, or consequences outside the current row.

Explicit operations make authorization clearer as well. Permission to edit delivery notes is not necessarily permission to cancel a paid order. They also give future maintainers a place to put rules without accumulating a sprawling collection of conditional branches inside one PATCH handler.

Make retries safe before clients force the issue

Networks fail in inconvenient ways. A client can send a request, lose the response, and reasonably retry even though the server completed the original work. If retrying POST /payments creates another charge or retrying an order submission emits duplicate messages, the API has made a normal failure mode dangerous.

For externally visible write operations, accept an idempotency key. Store the key alongside a request fingerprint and the completed result. A repeat request with the same key and equivalent payload returns the original outcome; a request that reuses the key with a different payload should fail clearly.

POST /orders/ord_482/submit
Idempotency-Key: 8f1f7e8c-2a31-4f92-a6ba-18b7e7a63d00
Content-Type: application/json

{"payment_method":"card_29"}

The storage decision matters. Recording the key only after downstream work finishes leaves a race window. A robust implementation reserves the key in the same database transaction that establishes the operation’s authoritative state. If the work involves an external provider, model the workflow deliberately: persist the intent, perform or schedule the external action, and reconcile the final state rather than pretending a distributed transaction exists.

Use transactions for integrity, queues for propagation

A common scaling failure is trying to do everything during the request: write the order, charge a payment provider, send email, update search, notify analytics, and call a partner API. This makes latency depend on every dependency and makes a partial outage look like an application outage.

Keep the synchronous path focused on the result the caller needs immediately. Commit the core state and an outbox event together in one database transaction. A worker can then publish or process that event after commit. The outbox avoids the classic gap where a database write succeeds but event publication fails, or vice versa.

BEGIN;

UPDATE orders
SET status = 'submitted', submitted_at = CURRENT_TIMESTAMP
WHERE id = :order_id
  AND status = 'draft';

INSERT INTO outbox_events (type, aggregate_id, payload)
VALUES ('order.submitted', :order_id, :payload);

COMMIT;

The conditional update is significant: it enforces the permitted transition at the database boundary. Application-level validation is still valuable for useful error messages, but concurrent requests can bypass assumptions made before the write. Check the affected row count and return a conflict when the order is no longer in the expected state.

Consumers must still be idempotent. Queues typically provide at-least-once delivery, so an event can arrive more than once. Store a processed event identifier or make the consumer’s database operation naturally safe to repeat. “Exactly once” is rarely a property provided end to end; it is a behavior constructed through durable state and careful handling of duplicates.

Choose pagination and consistency contracts deliberately

Offset pagination is easy to introduce and often adequate for small, stable admin views. Under changing data, though, OFFSET can become expensive and causes gaps or duplicates as rows are inserted or removed between requests. For large or frequently changing collections, cursor pagination is usually a better contract.

A cursor should be based on a stable ordering, commonly a timestamp plus a unique identifier. The response should document the ordering and include a next cursor only when more results are available. Do not expose an opaque cursor as though it were a permanent public promise unless you are prepared to support its format; encode it as an opaque token and validate it server-side.

Consistency also deserves a plain answer. A list endpoint may return a slightly stale projection while a detail endpoint returns the authoritative record. That is acceptable when stated and when clients do not need immediate read-after-write behavior. If they do, provide a reliable path to the authoritative state rather than relying on timing luck.

Protect the database from convenient queries

Most performance problems are not solved by adding a cache first. They begin with unconstrained reads, missing indexes, expensive joins, and endpoints that accidentally load an object graph one query at a time. Establish limits at the API boundary: cap page sizes, reject unbounded date ranges where appropriate, and define supported sort fields.

Then inspect the query shape. In PHP applications, an ORM can improve productivity, but it does not remove the need to understand generated SQL. Watch for N+1 queries in serializers and relationship accessors. Fetch known relationships intentionally, select only required columns on hot paths, and add indexes that match the filtering and ordering actually used.

Caching is most useful after this foundation is clear. Cache derived, read-heavy data with an explicit invalidation or expiry strategy. Avoid using cache as the only record of a critical write. A cache miss should make the system slower, not incorrect.

Build an operational contract, not just an HTTP interface

Every request that changes state should carry a correlation identifier, either accepted from a trusted caller or generated at the edge. Include it in structured logs, error responses where appropriate, queued events, and downstream calls. When production behavior becomes surprising, the ability to follow one operation across the system is more valuable than a larger pile of unconnected logs.

Run the service in a container with configuration supplied through environment variables or a managed configuration system, not baked into the image. Keep database migrations separate from ordinary web request startup. A deployment should be able to run a backward-compatible migration, roll out code, and then remove obsolete schema only after old application instances are gone.

That discipline supports safe rollout patterns. Add a nullable column before requiring it. Deploy code that can read both old and new shapes before switching writes. Use feature flags for behavior changes that need staged exposure. Scaling safely is often less about faster servers than about making change reversible.

The API is a promise under pressure

A well-designed API does more than move JSON between clients and tables. It encodes business operations clearly, survives retries, keeps authoritative changes atomic, and moves slow side effects into reliable asynchronous work. It sets limits before load imposes them, and it gives operators enough context to diagnose failures.

CRUD remains a useful tool. The mistake is treating it as the destination. Design for the moments when requests duplicate, data changes mid-pagination, dependencies fail, and another developer needs to extend the system six months later. Those are the moments when an API reveals whether it was merely convenient to build or genuinely prepared to scale.

Blog author portrait

Mihajlo

I’m Mihajlo — a developer driven by curiosity, discipline, and the constant urge to create something meaningful. I share insights, tutorials, and free services to help others simplify their work and grow in the ever-evolving world of software and AI.