Model operations (LLMOps) for models already in production.

LLMOps keeps cost, quality and change in check.

LLMOps is the operating discipline for systems built on large language models. Hosting, backups and monitoring still apply. It adds the parts only language-model systems need.

At SolaryxDev it is the operating side of the systems we build, or take over once we have read them. You license no platform, and we publish no uptime percentage. The system already records a short list of signals, and a person acts on them.

Every run carries a budget it cannot exceed.

Inference is billed per token, so an agent that loops or a prompt that keeps growing becomes a bill nobody approved. The OWASP Top 10 for LLM Applications (2025) lists this as LLM10, Unbounded Consumption, including the 'denial of wallet' case where volume alone runs up the cost.

Token and call budgets are set per run, with alerts. A looping run stops.

Routing is the second lever.

• Long reasoning over documents goes to a large hosted model such as Anthropic Claude or OpenAI GPT, because a silent mistake there is expensive.
• High-volume classification goes to a small self-hosted open-weight model, where the price per call decides the budget.
• Anything that cannot leave the building goes to open weights on your own hardware, whatever the cost comparison says.

The monthly running cost is estimated from your real volumes before you commit.

What happens when a provider retires your model?

You switch, on their schedule. Anthropic gives at least 60 days' notice before retiring a publicly released model, after which requests to it fail, and OpenAI gives at least six months for generally available models but may retire preview models at much shorter notice, such as two weeks.

Before anything changes, the model-upgrade test runs the system's evaluation set against the replacement, and its scores decide whether the switch goes ahead. Model choice is kept reversible from the first build, which turns a forced switch into a routine change.

Open-weight models on your own hardware are never retired under you, so you move to a newer one only when the numbers say it is better.

Slipping quality raises an alarm.

Uptime and response time say nothing about whether the answers are still right, so sampled live traffic is scored against the evaluation set's criteria and an alarm fires when the distribution moves. Thresholds start conservative. They are reviewed as the system earns them.

If the scoring itself stops, that raises its own alarm.

Every change passes the release gate first, from a new prompt to a new model, and one that makes the system refuse less often where it should refuse counts as a regression.

Indexes keep pace with your documents.

A knowledge assistant is only as current as its index. Connectors to the shared drive, the mailbox and the ERP re-index incrementally on a schedule, so a changed file appears in the next day's answers. A dashboard shows which sources are stale, which gives an out-of-date answer a visible cause.

The guardrails survive every upgrade.

Spending money, deleting records and first contact with a customer still wait for a person after every upgrade, because the gate lives in code outside the model, where no new prompt, model or clever input can move it.

Every action is written to an audit trail, from the input that caused it to the state it left behind, so any run can be traced afterwards.

Releases still go out blue/green, with the previous version kept running beside the new one.

Six signals, each wired to an action.

The table shows where each signal comes from and what it sets off.

Systems built elsewhere are read before we run them.

We read the prompts, the tools it can call, what it may write to, what a run costs today, and whether anything tests it at all. Then you get a written assessment, which sometimes says that replacing one part is cheaper than keeping it alive.

The first practical step matches an evaluation engagement: baseline what runs today. The budgets, the gate and the alarms then go in one at a time, each proven before the next.

The setup is quoted as one fixed price.

The setup (the evaluation set, the budgets, the alarms and the upgrade test) is priced against a signed scope, like every engagement. The same proposal quotes the monthly running cost, estimated from your real volumes. The setup is never billed by the hour.