Development

System Architecture: Build APIs That Predictably Scale and Endure

System Architecture: Build APIs That Predictably Scale and Endure

Most API failures do not begin with a dramatic outage. They begin as small compromises: a controller that knows too much, a database query added “just for now,” a Docker image that works only on one developer’s machine, or a retry that quietly turns a slow dependency into a traffic storm.

System architecture is the discipline of making those decisions visible before they become expensive. The goal is not to build an elaborate platform for an imagined future. It is to give today’s application clear boundaries, predictable behavior under load, and enough room to change safely.

Start with responsibilities, not frameworks

A maintainable backend separates what the system does from how requests reach it. In a PHP API, controllers should translate HTTP concerns into application calls: validate input shape, identify the authenticated user, select an appropriate response code, and little else.

Business rules belong in application services or use-case classes. Persistence belongs behind repositories or dedicated query objects. External integrations deserve their own adapters. This does not mean creating an interface for every class; it means isolating decisions that are likely to change independently.

final class CreateOrder
{
    public function __construct(
        private OrderRepository $orders,
        private PaymentGateway $payments
    ) {}

    public function handle(CreateOrderCommand $command): Order
    {
        $order = Order::fromCommand($command);

        $this->payments->authorize($order->total(), $command->paymentToken());

        $this->orders->save($order);

        return $order;
    }
}

This structure makes the important questions easier to answer. What happens if payment authorization fails? Where is an order saved? Which rules determine its total? When those answers are scattered across controllers, models, listeners, and middleware, changing one behavior becomes a risky search exercise.

Design APIs as durable contracts

An API is a promise to clients, including clients maintained by another team or a future version of your own frontend. Predictable APIs are more valuable than clever APIs because they reduce integration friction and operational ambiguity.

Use resource-oriented URLs where they fit, consistent field names, explicit status codes, and structured error responses. A validation error should not look like an unexpected server failure. A missing resource should not be represented as an empty successful response.

{
  "error": {
    "code": "validation_failed",
    "message": "The request contains invalid fields.",
    "details": {
      "email": ["Must be a valid email address."]
    }
  }
}

Versioning deserves restraint. Adding an optional response field is often compatible; changing a field’s meaning or removing it is not. A new API version is appropriate when the contract must break, not as a routine response to every feature request. Document pagination, filtering, ordering, authorization behavior, idempotency expectations, and error formats as part of the contract rather than treating them as implementation details.

Make writes safe to retry

Networks fail in inconvenient places. A client may time out after the server has accepted a payment request but before it receives the response. If retrying a POST can create duplicate orders, the API has converted a common failure mode into a data integrity problem.

For operations with meaningful side effects, accept an idempotency key and store the result associated with that key. Repeated requests with the same key should return the original outcome, not repeat the action. The key must be handled atomically with the operation; checking for it in one query and recording it later leaves a race condition.

Treat the database as a core architectural boundary

Application code can hide an inefficient query for a surprisingly long time. Production data eventually exposes it. Query patterns should influence schema design: index columns used for selective filtering, joins, and ordering, while recognizing that every additional index adds write cost and storage overhead.

Pagination is a common example. Offset pagination is simple and useful for small administrative lists, but large offsets can become expensive and can produce shifting results as records are inserted. For high-volume, ordered feeds, cursor-based pagination based on a stable sort key is often a better fit.

Transactions should protect a coherent state transition, not wrap every request by default. Keep them short, avoid network calls while holding locks, and define the invariants they preserve. If an order and its line items must either both exist or neither exist, that is a transaction boundary. Sending an email after the transaction commits is usually a separate concern.

For work that must happen after a successful write, an outbox pattern can be valuable: write an event record in the same transaction, then let a worker publish or process it. This avoids the dangerous gap where a database commit succeeds but the process fails before notifying another system.

Use asynchronous work deliberately

Queues are useful for slow, retryable, or non-interactive work: email delivery, document generation, webhook delivery, image processing, and event handling. They are not a universal performance button. Moving essential work to a queue changes the product behavior from immediate to eventual consistency, which clients and support teams need to understand.

A job should have a clear retry policy, a maximum attempt count, and a failure destination that operators can inspect. Retrying every exception forever is not resilience. It can repeatedly charge a provider, overload a failing dependency, or block more valuable work.

  • Retry transient failures such as connection timeouts with bounded backoff.
  • Do not retry invalid input or authorization failures.
  • Make consumers idempotent because queues may deliver work more than once.
  • Record enough context to diagnose failed jobs without storing unnecessary sensitive data.

Make Docker boring and repeatable

Containerization is most helpful when it removes environment drift. A production image should contain the application and its runtime dependencies, not development caches, local credentials, or tools needed only during a build.

FROM php:8.3-cli-alpine

WORKDIR /app
COPY --from=composer:2 /usr/bin/composer /usr/bin/composer
COPY composer.json composer.lock ./
RUN composer install --no-dev --prefer-dist --no-interaction
COPY . .

CMD ["php", "bin/console", "app:worker"]

The exact base image and command will vary by application, but the principle stays constant: build deterministically, configure through environment variables or managed configuration, run as a minimally privileged user where practical, and expose health checks that reflect whether the service can do its intended work.

Deployment should be reversible. Use migrations designed for compatibility with the currently running application, deploy code before relying on new schema behavior when necessary, and avoid destructive schema changes in the same release that stops reading the old structure. A two-step migration is often less exciting and far safer.

Measure before optimizing

Performance work begins with a question: which user-visible path is slow, costly, or unreliable? Instrument request duration, error rates, queue depth, database timings, and dependency failures. Logs should carry a request or correlation identifier so a single failure can be traced across layers.

Caching is effective when paired with explicit invalidation and ownership. Cache stable, expensive reads; avoid caching merely because a cache is available. A stale price, permission, or inventory count may be worse than a slower response. Likewise, connection pooling, eager loading, and batching should solve observed bottlenecks, not become default complexity.

Architecture is a habit of clarity

Systems that endure are rarely defined by one fashionable pattern. They are built from repeatable choices: narrow responsibilities, explicit contracts, transactions that protect real invariants, retries with limits, observable behavior, and deployments that respect failure.

The most practical architecture is not the one with the most layers. It is the one that lets a developer change a requirement, predict the consequences, test the result, and operate the system calmly when reality disagrees with the plan. That is how an API scales not only in traffic, but in trust.

Blog author portrait

Mihajlo

I’m Mihajlo — a developer driven by curiosity, discipline, and the constant urge to create something meaningful. I share insights, tutorials, and free services to help others simplify their work and grow in the ever-evolving world of software and AI.