Calling a language model API is straightforward. Building a reliable feature around that call requires more thought.
The biggest mistakes usually happen at the boundaries: permissions, untrusted input, tool execution, response validation and operational behaviour.
Treat the model as an untrusted dependency
An LLM response is not validated business data. It can be wrong, inconsistent or influenced by malicious instructions embedded in retrieved documents or user input.
Validate structured responses against a schema and apply authorization checks in your application. A model’s request to call a tool should never bypass the permissions of the user or service that initiated the request.
Keep credentials and policy on the server
Call model providers from a server-side component rather than exposing API keys in JavaScript shipped to the browser. Store credentials in a secrets manager or platform environment variables, restrict access and rotate keys when needed.
Consider data retention, regional requirements and the kind of information permitted to leave the system before choosing a provider or deployment approach.
Design the failure paths
Plan for provider rate limits, timeouts, intermittent failures and cost spikes. Bound the number of requests and the amount of input or output per operation. Not every failure should be retried: an invalid request or policy rejection will not usually improve through repeated attempts.
Evaluate quality with examples from your domain
Before shipping, define what a successful result looks like. Create a small evaluation set of representative inputs, including edge cases and adversarial content. Track response usefulness, latency and cost as the implementation changes.
Keep business logic outside the prompt
Prompts can guide model behaviour, but they are not an enforcement boundary. Validation, authorization and irreversible actions belong in deterministic application code.
The principle: design an AI feature as an integration with a probabilistic service — not as a replacement for your existing application’s controls.