Once a business decides to add AI features, a strategic question follows quickly: should we call a commercial model through an API, or run an open-source model ourselves, perhaps fine-tuned on our own data? The debate is often framed as ideology, but it is really an engineering and economics question with a fairly predictable answer for most organisations: start with an API, prove value, measure, and adopt heavier options only when specific requirements demand them. This article gives you the framework.

The three options
1. Commercial model APIs. You send prompts to a provider's hosted model and pay per token. You get the most capable models, no infrastructure and rapid iteration. In exchange you depend on the provider's pricing, availability, policies and model updates, and your data travels to their systems under contractual terms.
2. Self-hosted open-source or open-weight models. You download a model and run it on your own servers or rented GPUs. You control data, versions, latency and costs at scale. You also inherit the operational work: hardware, serving software, scaling, monitoring, security and upgrades.
3. Fine-tuning. Additional training on your examples, applicable to either option (some providers offer fine-tuning of their models, and open models can be tuned by you). Fine-tuning changes how a model behaves, not what it knows in any reliable, updatable way.

What fine-tuning is good for, and not
Fine-tuning shines when you need consistent style, tone or structured output formats; when a narrow task, like classifying tickets into your categories or extracting fields from a specific document type, needs high accuracy from a small, cheap model; or when prompts have grown huge with examples and you want to bake them in to reduce cost and latency. It is a poor tool for teaching facts. Knowledge changes, fine-tuned facts are hard to update or verify, and the model can still hallucinate. For facts, use retrieval, as in our vector database article, which is cheaper, updatable and citeable. Also, fine-tuning requires good data: hundreds to thousands of high-quality labelled examples, and an evaluation set to prove it helped.
Quality
The leading commercial models generally offer the best reasoning, instruction following and breadth, particularly for complex or open-ended tasks. Open models have narrowed the gap considerably and, for well-defined tasks, a smaller open model tuned to the job can match or beat a general commercial model. Do not decide by benchmarks. Build an evaluation set from your own use case, run candidates through it and compare quality, latency and cost. The methods in our prompt engineering article apply.
Cost: do the arithmetic
API costs scale linearly with usage and involve no fixed overhead, which makes them ideal at low and moderate volume and for experimentation. Self-hosting has fixed costs: GPU servers or instances running continuously, engineers to operate them, and the risk of paying for idle capacity. The crossover depends on volume, model size and utilisation, and often arrives later than teams expect once engineering time and reliability requirements are included. Estimate honestly: monthly tokens, average prompt and output length, peak load, GPU cost, redundancy, staff time. Reduce API spend first with prompt trimming, caching, cheaper models for simple tasks and batching. Our advice on cloud spending in our cloud cost guide applies to GPUs too.
Privacy, security and compliance
This is the most legitimate reason for self-hosting. Review each provider's data-handling terms: whether inputs are retained, used for training, or processed in particular regions, and which enterprise agreements and certifications exist. Many businesses find commercial providers' business terms sufficient, especially with data minimisation and redaction. Others, such as health, legal, government or defence organisations, or those with contractual data residency rules, may require that data never leaves their environment. Self-hosting avoids sending data to third parties, but you become responsible for securing the servers, the prompts and the logs; see our security audit guide. Consult legal counsel on your obligations; see our privacy law overview. This is not legal advice.
Latency, availability and control
APIs add network latency and are subject to rate limits and outages, so design fallbacks and retries. Self-hosting can deliver very low latency close to your users and predictable capacity, but you must engineer high availability, as in our high-availability guide. Provider model updates can change behaviour or retire versions, so pin versions and regression test; with self-hosted models, you decide when to upgrade. Consider vendor lock-in: keep an abstraction layer in your code and prompts portable so you can switch providers.

Operational reality of self-hosting
- Hardware: GPUs with enough memory for the model size and concurrency; quantisation reduces requirements at some quality cost.
- Serving software: inference servers that batch requests, stream tokens and manage memory efficiently.
- Scaling and reliability: autoscaling on GPU capacity, health checks and failover.
- Security: network isolation, access control, and logging without leaking sensitive prompts.
- Monitoring and evaluation: quality, latency, throughput and cost dashboards; see our monitoring guide.
- Licensing: read model licences carefully, since "open" models differ in commercial-use terms.
- Skills: machine-learning operations experience is scarce and expensive.
A decision framework
- Choose a commercial API when you are exploring, volumes are moderate, tasks are complex, you lack ML operations staff, and provider data terms are acceptable.
- Choose self-hosting when data cannot leave your environment, volume is high and steady, latency or offline operation is critical, or you need deep customisation.
- Add retrieval whenever answers depend on your changing data.
- Add fine-tuning when evaluation shows consistent gaps in style, format or narrow-task accuracy that prompting cannot close, and you have quality data.
- Use multiple models: route easy tasks to small, cheap models and hard ones to strong models.

Avoiding expensive mistakes
- Do not fine-tune first. Prompting and retrieval solve most problems faster and cheaper.
- Do not buy GPUs before you have measured usage. Rent, prove demand and only then commit.
- Do not skip evaluation. Without it you cannot tell whether a change helped.
- Do not build a bespoke platform for a feature that an API call satisfies.
- Design for portability: a thin internal interface to models, prompts in version control and evaluation that runs against any provider.
Example scenarios
A small retailer wants product descriptions and support answers: a commercial API with retrieval over its catalogue and policies is fast and inexpensive. A law firm summarising confidential documents may need a self-hosted model in its own environment or a provider with strict contractual terms. A logistics company classifying millions of short messages a day might fine-tune a small open model to run at a fraction of the per-call cost of an API. A SaaS company building an assistant usually begins with an API, then routes simple requests to cheaper models as usage grows. See also our generative AI integration guide.
The right strategy evolves with your product, and the worst outcome is committing to infrastructure before you know what you need. Our software team builds model-agnostic AI features and can help you evaluate, tune and, where justified, host models, with hosting and maintenance to keep them running.
Put this into practice with CodeLuma
CodeLuma helps you choose and build the right model strategy, starting with a commercial API and prompts, then adding retrieval, fine-tuning or self-hosting only where the numbers justify it.
- Custom software development - tailored systems, integrations and internal tools.
- Managed hosting - monitored, backed-up, Canadian-friendly hosting.
- Maintenance and support plans - updates, monitoring and ongoing improvement.
Start a conversation. Tell us about your project and we will reply with practical next steps, or browse all CodeLuma services. CodeLuma Development Inc. is a Canadian company that works with teams in Windsor, Essex, London, Toronto, Mississauga and across Ontario, throughout Canada and remotely.


