All guides
Decision guides
API model or self-hosted model: how to choose
Most teams start with a hosted model API, and some later move part of the work to a model they run themselves. Neither is the default answer. This guide lists the criteria that decide it, when each side wins and what to check first. It does not compare specific models, vendors or prices.
Last reviewed:
Criteria
| Criterion | Hosted API | Self-hosted model |
|---|---|---|
| Data and residency | Prompts and outputs go to the provider under its terms, region options and data agreements. Fine where the data may leave your environment. | Data stays inside infrastructure you choose. Needed where rules or contracts say it may not leave. |
| Speed to start | Usually an account and a key. The fastest way to find out whether the use case works at all. | Needs hosting, serving software and an evaluation first. Slower to start, with more to set up. |
| Model choice and quality | You can use the provider's most capable models. They change on the provider's schedule. | You choose among open-weight models and their licences. Capability differs per model, so test on your own task. |
| Control and change | Versions and retirement follow the provider. Fine-tuning is limited to what the provider offers. | You pin the version, decide when to update, and can fine-tune or inspect the model. |
| Cost shape | You pay for use. Easy to start, and it grows with volume. | You pay for capacity whether it is busy or not. It starts with a fixed effort and can be steadier under sustained load. |
| Latency and placement | Depends on the provider's network path and load. | You decide where it runs, which helps for real-time or on-site use. |
| Operations | The provider runs scaling, updates and uptime. | You, or a partner, run GPUs, updates, monitoring and on-call. |
| Compliance evidence | You rely on the provider's documentation and certifications, and check them against your needs. | You can show exactly where the system runs, but you also own the evidence. |
| Lock-in and exit | Prompts and code can become tied to one provider's features. Keep an abstraction layer. | Open weights ease switching, but your serving stack becomes a dependency. |
When to choose which
A hosted API is usually the better start when
- The data may leave your environment under a signed agreement.
- You are still finding out whether the use case works.
- Volume is occasional or hard to predict.
- Nobody on the team has run model serving before.
- A general-purpose model is good enough for the task.
A self-hosted model is usually the better fit when
- Rules, contracts or secrecy say the data must stay inside your environment.
- You need to pin a model version, inspect it or fine-tune it yourself.
- Load is large and steady, so reserved capacity can be used well.
- Real-time or on-site use means you must decide where it runs.
- You have a team or a partner who can run it and be on call.
Often the answer is both
Use a hosted API for general work and a self-hosted model for the part that handles sensitive data. Route by data class, keep the interface the same, and measure both on the same evaluation set.
What to check first
- Classify the data the use case needs: which parts may leave your environment, and under which agreement.
- Write down residency and contract requirements before you look at any model.
- Build a small evaluation set from real examples, then try a hosted model and an open-weight model on it.
- Estimate load from your own measurements, volume and peaks, before you compare costs.
- Name who would run it, and who is on call when it fails.
- Keep prompts and tools portable so that the choice stays reversible.
Next step
Facing this decision?
Tell us about your situation. We will say which side we would lean towards and what we would test first.
Currently accepting projects for Q4 2026