Open-weight model

What it is

An open-weight model ships the numerical parameters produced by training, usually alongside an architecture definition and a licence, so the model can be loaded and run outside the vendor's own API. Well known families released this way include Llama, Mistral, Gemma, Qwen and DeepSeek. The term distinguishes these releases from closed models that are only reachable through a hosted API, and from fully open source projects that also publish data and training recipes.

Why it matters

Open-weight models let teams run AI retrieval, summarisation and classification on their own infrastructure, which matters when data cannot leave a jurisdiction or a private network. They also lower the cost of experimenting with AI-driven search features, since inference can run on rented or owned hardware rather than per-token API calls. For marketers studying AI search, they are a way to test how a model reads and summarises your content without depending on a single vendor's black box.

How it works

Teams download weights from a hub such as Hugging Face, then serve them with an inference runtime and, if needed, quantise them to fit available GPUs or run smaller versions on a laptop. Many adapt the model with fine-tuning or lightweight methods such as LoRA, and pair it with retrieval so answers draw on their own documents. Because the weights are local, teams can inspect outputs, log prompts and responses, and pin a specific version so behaviour does not change underneath them.

When it applies

It applies when you need control over data handling, cost or model versioning, or when you want to test and audit AI behaviour rather than accept a hosted default. It is less relevant when a hosted frontier model is good enough and you have no hosting capacity or compliance driver.

Examples

• A healthcare provider self-hosts an open-weight model so patient enquiries are summarised inside its own network.
• An ecommerce team fine-tunes a small open-weight model on its product catalogue to generate structured attribute tags at scale.
• An agency runs an open-weight model locally to test how different versions of a service page get summarised, before publishing changes.

How it is measured

• Cost per thousand tokens or per request when self-hosted, compared with the equivalent hosted API cost
• Inference latency and throughput on your chosen hardware, measured at realistic concurrency
• Task quality against a fixed evaluation set, scored before and after fine-tuning or a version change
• Share of AI workloads running on self-hosted weights versus external APIs, tracked for cost and data governance reporting