What Actually Breaks When You Put 111 Generative Models Behind One API
Every generative media product ships the same interface now: a prompt box, an upload slot, and a dropdown. The dropdown is the lie. It presents Kling, Wan, Seedance, Vidu, Hailuo, FLUX, Seedream, Hunyuan, PixVerse, Z-Image and LTX as if they were interchangeable settings on one machine — a font picker for video. Behind it, in our production catalogue as of this writing, sit 111 model entries across 17 families and three upstream providers, and almost nothing about them agrees.
This is a field report on the specific ways that disagreement shows up, written from the inside of one such catalogue. None of it is exotic. All of it is the kind of thing you only find after you ship.
The parameter vocabulary does not converge
The naive mental model is that every image or video model takes roughly the same inputs — a prompt, a size, a length — and differs only in weights. The catalogue says otherwise. Counting how often each parameter name appears across our 111 entries:
| Parameter | Models that accept it |
|---|---|
duration | 77 |
resolution | 44 |
aspect_ratio | 15 |
size | 8 |
mode | 2 |
movement_amplitude | 2 |
style | 2 |
The long tail is the easy part; movement_amplitude appearing on exactly two models is a nuisance, not a problem. The hard part is the top of the table, because resolution, aspect_ratio and size are not three names for one concept. They are three incompatible theories of how a user expresses “how big.”
A model taking resolution: 720p has decided the short edge is the unit and the shape comes from somewhere else. A model taking aspect_ratio: 9:16 has decided shape is the unit and pixel count is the vendor’s business. A model taking size wants an explicit pair and will honour whatever you send, including combinations the model was never trained on. Forty-four models hold the first theory, fifteen the second, eight the third — and a single product surface has to present one ratio chip to the user and satisfy all three.
The failure this produces is not a crash. It is worse: a request that succeeds, bills, and returns a correctly-sized image of the wrong composition, because the ratio the user picked was silently resolved against a default the model brought with it. We ended up with a single shape resolver that all three theories are funnelled through, precisely because the alternative — every call site doing its own conversion — had already produced a class of bug where the UI and the outbound provider request could disagree about what the user had asked for.
If you are building this, the rule that saved us is narrow and worth stating plainly: the ratio the user sees and the ratio that goes on the wire must be computed in exactly one place. Two readers of the same fact will eventually disagree.
Half the endpoints answer in a different shape — from the same vendor
Of our 111 entries, 92 are asynchronous: you post a job, get a task ID, and poll. Nineteen are synchronous: you post a job and the finished asset comes back in the response body.
Those nineteen are not a different vendor. They live under the same base URL, behind the same key, documented in the same place. One image model in the set returns {"images": ["<url>"]} directly — no task ID, no polling, nothing to reconcile against a job table. We know this because someone probed it live and left a dated note in the catalogue saying so; the published documentation had implied the async shape.
This is the part that generalises. When you integrate one model you write a client. When you integrate a hundred you have written, without deciding to, a protocol adapter, and the number of protocol variants is not bounded by the number of vendors. It is bounded by the number of times a vendor shipped a new endpoint without checking the old contract. Any design that assumes “one provider, one client” acquires a second, undocumented client the first time this happens, usually as an if statement in the hot path.
Models die, and the dropdown does not notice
This is the failure mode we most underestimated, and the one with the sharpest operational lesson.
On 2026-07-29 we probed every registered endpoint in the catalogue with an empty request body. The technique is crude and effective: a live route rejects the empty body with a validation error, a retired route answers route not found. The result was that fifteen registered models were dead while still being selectable in the product. Choosing any of them was a guaranteed failure — not a degraded result, not a slow result, a hard failure with a user’s credits on the line.
The chronology matters more than the count:
- One video family had been retired earlier in the month. Its replacement was wired in on 2026-07-15. By 07-29 the replacement was dead too. The successor had a shelf life of two weeks.
- Another model was deliberately re-enabled on 06-27 after a liveness probe showed it answering. The vendor closed it again afterwards. Our config said “alive” because it had been alive, on the day someone checked.
The conclusion we drew is that liveness is a runtime property that we had been storing as a configuration property. A model ID in a config file is not a fact about the world; it is a cached observation with no TTL. Treating it as a constant means the cache never expires and the product confidently sells a thing that no longer exists.
What we did about it is deliberately unglamorous. The retirement list is now a test, and the test’s failure message tells the next engineer what to do: re-probe the endpoint and update the dated note before re-enabling anything. The point is not that a test catches vendor retirements — it cannot, it only knows what we last observed. The point is to make un-hiding a model a conscious act backed by a fresh probe, rather than a quiet line in a diff. The 06-27 re-enable that the vendor promptly undid is exactly the event that convinced us the human ritual mattered more than the automation.
The broader version of this, for anyone building on hosted models: your vendor’s deprecation policy is your uptime policy, whether or not you have read it. Aggregators improve your coverage and your integration cost. They do not give you a say in which weights stay online, and they typically will not tell you when a route goes away — you will find out from a support ticket, or from a probe you decided to run.
One dropdown, a 375x swing in unit cost
Our internal cost table carries 334 priced configurations — 305 video, 29 image. Across the video set, the cheapest configuration and the most expensive differ by a factor of 375. On the image side the spread is narrower but still 31x. And 291 of the 334 are priced per second of output, not per call.
Sit with what that means for a product surface. A user picks a model from a dropdown and a duration from a chip row. Neither control is labelled as a cost decision. Both are. The same click, with the same prompt, can be a rounding error or the most expensive thing that happens on your platform that hour, and the difference is which of two adjacent menu items the user’s thumb landed on.
The consequence for pricing design is that any flat “one generation = N credits” scheme is doing an implicit, enormous cross-subsidy — and the users who find the expensive end of the menu are, reliably, your heaviest users. The consequence for engineering is subtler: because most entries are priced per second, cost is a function of a parameter, not of a route. You cannot price a request by looking at which endpoint it hits. You have to price it after resolving duration, resolution and mode — which is to say, in the same normalization layer that already had to reconcile three theories of “how big.”
That is the quiet argument for putting all of this in one place. The shape resolver, the protocol adapter, the liveness state and the cost model look like four separate concerns and are in fact four views of one question: what, precisely, did the user just ask this specific model to do? Every architecture we tried that answered that question in more than one place eventually let the answers drift.
What we would tell someone starting today
Three things, in the order they will bite you.
Normalize once, at the boundary. One resolver owns the translation from what the user picked to what each provider expects, and every consumer — the UI, the outbound request, the cost calculation, the stored record of what happened — reads its output. This feels like over-engineering when the catalogue has six models. It is load-bearing at sixty.
Treat model IDs as leases, not constants. Probe liveness on a schedule, keep the retirement list in code with the date it was verified, and make re-enabling anything require a fresh probe. Assume the model you integrated this quarter will be gone next quarter, because the replacement for the model that was gone last quarter already is.
Make cost a first-class output of the request path, not a report you run later. When the spread across your menu is two and a half orders of magnitude, “we’ll reconcile the bill monthly” is not a plan. It is a way to discover in arrears that a single popular model consumed most of your inference budget.
None of this is a complaint about hosted models. The alternative — self-hosting a hundred families of weights and their inference stacks — is not a business we want to be in, and the aggregator layer earns its cut on integration cost alone. It is an argument that the dropdown is a much bigger abstraction than it looks, that it hides four independent axes of variation rather than one, and that the cost of pretending otherwise is paid in exactly the currency you cannot afford: requests that succeed, bill the user, and return the wrong thing.