Designing LLM-powered applications in JavaScript

Large language models feel magical at first. You send text in, something coherent comes out, and it’s tempting to believe the hard part is over. In practice, the real work begins after the first successful response.

Over time, I’ve learned that LLMs don’t behave like typical libraries. They’re unpredictable, expensive, and deeply sensitive to context. Designing systems around those traits matters far more than the model choice itself.

Treat the model as a dependency, not a feature

An LLM is closer to a database or an external service than a helper function. It can fail, slow down, change behavior, or return unexpected output. Good systems assume this upfront.

// Bad: tightly coupled
const reply = await openai(prompt);

// Better: isolated dependency
const reply = await llmClient.complete(messages);

Once the model is abstracted, everything else becomes easier to evolve. Providers can change. Prompts can be refined. Failures can be handled intentionally.

Prompts are part of your architecture

Prompts aren’t strings you sprinkle through your codebase. They define behavior, constraints, and tone. In production systems, they deserve the same care as business logic.

function buildPrompt({ rules, context, question }) {
  return [
    { role: "system", content: rules },
    { role: "assistant", content: context },
    { role: "user", content: question },
  ];
}

Versioning prompts and keeping them centralized prevents accidental regressions and makes iteration safer.

Retrieval beats memory every time

LLMs don’t remember your data. They generate responses based on what you give them. For anything beyond trivial use cases, retrieval-augmented generation becomes essential.

const context = results
  .map(r => r.text)
  .join("\n\n");

buildPrompt({ context, question });

Separating knowledge storage from reasoning keeps systems reliable and updatable without retraining models.

Validate everything that comes back

Language models produce language, not guarantees. If the output feeds another system, it must be validated. Invalid output should fail fast, not quietly.

function safeParse(json) {
  try {
    return JSON.parse(json);
  } catch {
    throw new Error("Invalid model output");
  }
}

Structure, schemas, and guardrails are what turn probabilistic output into dependable software.

User trust is part of the system

A confident but wrong answer erodes trust quickly. Good interfaces show sources, stream responses, and handle uncertainty openly.

When users understand how an answer was produced, they’re far more willing to rely on it — and to forgive its limitations.

The model matters less than the design

Most successful LLM applications aren’t impressive because of the model they use. They work because the surrounding system is thoughtful, observable, and adaptable.

The real craft isn’t prompt writing or API calls. It’s designing systems that make probabilistic tools behave predictably enough to be useful.