Pragmatic Database Design: Architecting for AI's Inevitable Integration
AI features rarely arrive as a clean greenfield project. They show up as a request to summarize support tickets, search product documentation semantically, classify incoming records, or help an operator draft a response. The model may be the visible part, but the database determines whether that feature becomes dependable software or an expensive demo.
Pragmatic database design for AI is not about rebuilding every table around vectors or predicting every future model. It is about preserving trustworthy operational data while creating clear places for derived, probabilistic, and revisable AI output to live.
Keep facts separate from interpretations
A database should distinguish between what happened and what a system inferred about it. An order total, a customer-selected address, and an audit event are facts. A suggested category, extracted sentiment, generated summary, or confidence score is an interpretation.
Mixing those concepts into a single row creates trouble quickly. An AI-produced value may need regeneration after a prompt change, model upgrade, data correction, or business-rule revision. A factual value usually has a different lifecycle and a different standard of correctness.
Prefer a source table plus a derived-results table. For example, a document can remain the authoritative record while classifications are stored separately:
CREATE TABLE documents (
id BIGINT PRIMARY KEY,
body TEXT NOT NULL,
updated_at TIMESTAMP NOT NULL
);
CREATE TABLE document_classifications (
id BIGINT PRIMARY KEY,
document_id BIGINT NOT NULL,
label VARCHAR(100) NOT NULL,
confidence DECIMAL(5,4),
model_identifier VARCHAR(255) NOT NULL,
prompt_version VARCHAR(100) NOT NULL,
source_updated_at TIMESTAMP NOT NULL,
created_at TIMESTAMP NOT NULL,
FOREIGN KEY (document_id) REFERENCES documents(id)
);
This structure makes an important question easy to answer: “Which version of the document did this result describe?” It also allows multiple classifications to coexist while a team evaluates a new approach.
Design for provenance, not just output
When a user disputes an automated result, the useful answer is not merely the result itself. It is the context around it: which source data was used, when processing occurred, which model or workflow produced it, and whether a human changed the outcome.
Store enough provenance to investigate behavior without persisting unnecessary sensitive content. The appropriate details depend on the system, but commonly include:
- the source record identifier and source revision or timestamp;
- the model identifier and configuration relevant to the result;
- a prompt or workflow version identifier;
- processing status, timestamps, and error information;
- human review state and any final override.
A version identifier is often more practical than copying a large prompt into every row. Keep the prompt definition in version-controlled application configuration or a dedicated workflow table, then record the immutable version used for each run. The result remains explainable without making ordinary queries unnecessarily heavy.
Treat AI work as asynchronous work
Most AI processing does not belong inside a synchronous web request. A PHP controller that saves a document and then waits for extraction, embedding generation, or a remote model response will eventually create slow requests, awkward retry behavior, and duplicate processing.
Instead, commit the business transaction first and enqueue a job that references the committed record. The worker fetches the current source state, performs the work, and writes the derived result. A status field such as pending, processing, completed, or failed gives both the application and operators a visible lifecycle.
Idempotency matters here. Jobs can be delivered more than once; workers can stop after a remote call but before saving a result. Make writes safe to repeat by using a stable uniqueness rule, such as one result per document, source revision, task type, and workflow version. In PHP, this usually means relying on a database constraint and handling the conflict deliberately, rather than assuming a queue will never retry.
Use an outbox when consistency matters
If saving a record and publishing its processing job must stay aligned, an outbox pattern is worth considering. Write an outbox event in the same database transaction as the source change. A separate publisher sends unsent events to the queue and records successful publication. This avoids the classic gap where the database commit succeeds but the process fails before the queue message is sent.
The pattern adds moving parts, so it is not mandatory for every feature. It is valuable when missing a downstream action would be costly or difficult to detect.
Choose retrieval architecture from the question
AI search is often introduced as “add embeddings,” but retrieval design starts with a simpler question: what should users be able to find?
Exact identifiers, account ownership, date ranges, permissions, and status filters remain conventional database work. Semantic similarity is useful when users express an idea in varied language and need conceptually related content. Many production searches need both.
Keep structured filters explicit and enforce authorization before handing content to any model-driven workflow. Then apply semantic retrieval within the permitted candidate set where possible. This is easier to reason about than treating a vector similarity score as a replacement for application rules.
For documents, choose chunks based on meaning and retrieval needs, not an arbitrary fixed size alone. Store each chunk’s parent document ID, ordering, source revision, and metadata required for filtering. If the document changes, mark prior chunks as stale and regenerate them through the same asynchronous pipeline.
Make schema changes reversible and observable
AI integration evolves quickly because product requirements evolve quickly. A migration that permanently replaces a human-curated field with generated text creates unnecessary risk. Additive changes are usually safer: new tables, nullable columns during transition, and dual-read or dual-write periods when needed.
Operational visibility is equally important. Track queue depth, job age, failure counts, processing duration, and the number of stale or missing derived records. For outputs that affect users, measure review and override behavior too. Those signals reveal whether the system is useful, not merely whether it is running.
Be cautious with raw inputs and outputs in logs. AI workflows can touch customer content, internal documents, or credentials accidentally included in text. Define retention rules, limit access, and log identifiers and diagnostic metadata by default rather than entire payloads.
Build a stable core around an adaptable edge
The durable part of an application is usually its domain model: users, permissions, transactions, inventory, documents, and the rules that govern them. AI belongs at the adaptable edge of that system, producing suggestions, enrichments, and interfaces over well-managed data.
That framing leads to calmer architecture decisions. Preserve canonical records. Version derived behavior. Process work asynchronously. Record provenance. Make retries safe. Use traditional queries where they are strongest, and semantic retrieval where it genuinely improves discovery.
The goal is not to make the database “AI-native.” The goal is to make it resilient when AI changes—as it will—while the business still needs its data to be correct, explainable, and available.