TokelyTHE SHORT ANSWER
A multi-model SaaS architecture separates product workflows from model-specific API calls. Keep task definitions, validation, customer budgets and conversation state in your application. Place model selection and provider adapters behind a shared interface so you can change a route without rewriting the feature that uses it.
What should the application interface describe?
Describe the task in product terms. An extraction request might include a document, a schema version, a tenant ID and a deadline. The product should not need to know every provider’s request fields.
The adapter translates this task into a supported API request, then translates the response into a typed result. Preserve useful metadata such as usage, finish reason and request ID. Do not flatten an incomplete output into an ordinary success.
Where should conversation state live?
Keep authoritative state in your application. Store messages, tool outputs and references to retrieved material according to your retention policy. Convert that state into the format the chosen endpoint supports.
This matters during provider migration. A hosted conversation identifier from one service is not necessarily meaningful to another. Tokely’s migration documentation specifies stateless Responses use with store=false and application-supplied history rather than relying on previous_response_id.
How do you handle model capability differences?
Maintain a capability registry: supported endpoints, modalities, tool calling, structured output, context limits and streaming requirements. Validate eligibility before making a paid call.
Treat model aliases carefully. If an alias begins pointing to a new version, an unchanged application can behave differently. Record the model identity returned by the service where available and rerun the relevant evaluations when the underlying version changes.
What makes rollout reversible?
Store route configuration separately from product code. Release a new candidate to a controlled slice of eligible work, compare accepted-result cost and latency, then expand. Keep the prior configuration available.
For actions, separate model planning from execution. A generated instruction should pass authorization and validation before your application sends an email, modifies a record or starts another paid job.
How does a common API help?
A common API reduces the amount of adapter code, but does not make models equivalent. You still own product semantics, validation and customer isolation.
Use Tokely’s catalog to build your initial capability list and the API documentation to implement the supported request formats. Start with one reliable path per task before adding a more complicated router.