Open vs Closed Models: What 'Open Source AI' Really Means

When you ask a chatbot for a recipe, you're talking to a model. But what does it actually mean for that model to be "open source"? The term now means three completely different things depending on who's talking: the weights you can download, the data that trained it, and the license that governs what you can do with either. Most models marketed as open are missing at least one of these pieces.

Here's the part that surprises engineers: downloading the weights of a model like Llama does not make it open source. Not even close. The Open Source Initiative now has a formal definition, and most "open" models fail it on multiple counts. Understanding which parts are actually open determines whether you can audit a model for bias, reproduce its training, or embed it in a product without a lawyer.

What does "open source AI" actually require?

The Open Source Initiative defines Open Source AI as a system where you can use, study, modify, and share every component without asking permission. This means three concrete artifacts must be available under an OSI-approved license: detailed information about the training data, the complete code used for training and inference, and the model parameters including weights and checkpoints. source

For a machine learning system, the "preferred form for making modifications" is not just source code. It includes data provenance, collection methods, labeling procedures, and processing steps. A skilled person should be able to build a substantially equivalent system from what's provided. source

This is a much higher bar than most people realize. When a company releases model weights on Hugging Face and calls it open source, they are almost certainly leaving out the training data documentation and the actual training scripts. What they've released is open weights, not Open Source AI.

Why do people confuse weights with open source?

The confusion starts with how software engineers think about openness. In traditional open source, the source code is the thing. Download the repo, read the code, build it, run it. The artifact and the blueprint are the same thing.

Model weights are not source code. They are the output of a training process, a compressed representation of patterns found in the training data. Having the weights lets you run inference and fine-tune. It does not let you understand how the model was built, what data shaped it, or whether that data contained copyrighted material or biased content. source

Think of it like getting a compiled binary versus getting the source tree plus the build system plus the compiler flags. The binary runs. You can even patch it. But you cannot reproduce the build, audit the dependencies, or verify that nothing sketchy happened during compilation. Open weights are the binary. Open Source AI is the whole build pipeline.

{
  "type": "comparison",
  "title": "Weights vs. Full Open Source",
  "caption": "Open weights let you run and fine-tune, but open source AI gives you the full picture.",
  "before": {
    "label": "Open Weights",
    "points": [
      "Use the model for inference",
      "Fine-tune on custom data",
      "Deploy behind an API"
    ]
  },
  "after": {
    "label": "Open Source AI",
    "points": [
      "Retrain the model from scratch",
      "Inspect and modify training code",
      "Reproduce the exact training process",
      "Audit data lineage and labeling"
    ]
  }
}

What's actually in a model release?

When a company releases a model, you typically get a set of files. The weights file contains millions or billions of floating-point numbers representing what the model learned during training. A config file describes the architecture: how many layers, the hidden size, the attention mechanism. A tokenizer file maps between text and the token IDs the model actually processes.

What you almost never get is the training data manifest, the data filtering and deduplication scripts, the exact training hyperparameters, or the evaluation harness used to measure the model. These are the parts that would let someone reproduce or audit the training process. Without them, you're accepting the model as a black box that you can run but cannot fully understand.

Some releases go further. EleutherAI, for example, releases training datasets and codebases alongside weights, intentionally aligning with OSI principles. This is the exception, not the norm.

How do licenses reveal what's really open?

The license is the fastest way to cut through marketing language. An OSI-approved license like Apache 2.0 or MIT allows any use, any modification, any redistribution, with no field-of-use restrictions and no revenue thresholds. If a model license says you need a commercial license above a certain revenue level, it is not open source. source

Mistral AI illustrates this spectrum clearly. Some of their models use Apache 2.0, which qualifies as open source. Others use a modified MIT license requiring companies with monthly revenue above 20 million USD to obtain a separate commercial license. That revenue threshold is a field-of-use restriction. It violates OSI's requirement that open source licenses cannot discriminate based on user size or commercial activity.

Meta's Llama license goes further. It imposes usage restrictions and does not provide full transparency into training data. The OSI has explicitly stated this falls short of Open Source AI requirements. source xAI's Grok weights were released with a custom license containing anti-competitive terms. source Hugging Face supports gated models where access requires approval, which also violates the "no permission needed" requirement. source

How do regulators think about this?

Regulators are building their own categories, and they don't always align with OSI definitions. The EU AI Act uses the term "general-purpose AI models" (GPAI) for models trained with large amounts of data using self-supervision at scale. Recital 98 suggests models with at least a billion parameters trained this way should be considered GPAI. source

The Act distinguishes between GPAI models and GPAI models with systemic risk, attaching different obligations to each. Openness of weights matters mainly for documentation exemptions and risk management requirements. A model with widely available weights gets some regulatory relief, but the Act does not use "open source" as a formal category. source

In the US, the NTIA and related executive orders use "dual-use foundation models with widely available model weights" or "open foundation models." Again, the focus is on weight availability, not on OSI-style full openness. source This creates a gap: a model can be "open" for regulatory purposes while failing OSI's definition of Open Source AI.

Why would anyone choose closed?

Closed models accessed via API (application programming interface) centralize control. The provider manages inference infrastructure, safety filters, telemetry, and updates. For the user, this means no GPU (graphics processing unit) provisioning, no model serving code to maintain, and no weight files to secure. It also means the provider sees every prompt and response, which may be a feature or a dealbreaker depending on your data sensitivity. source

Closed providers can also iterate faster on safety. When a new jailbreak technique emerges, they can patch the API layer centrally. Open-weight model deployers must monitor and patch their own deployments. This is real operational overhead that teams often underestimate. source

The tradeoff is dependency. A closed model's capabilities, pricing, and availability are controlled by the provider. If the provider deprecates a model version or changes its acceptable use policy, you adapt or migrate. With open weights, you control the artifact and can run it indefinitely on your own infrastructure. source

Quick Reference

Property	Open Source AI	Open Weights	Closed Model
Weights available	Yes	Yes	No
Training code available	Yes	Usually no	No
Data documentation	Detailed	Minimal or none	None
License type	OSI-approved	Custom, often restricted	Proprietary
Can reproduce training	Yes	No	No
Can audit for bias	Yes	Partial	Limited
Can fine-tune freely	Yes	Usually yes	No
Can deploy commercially	Yes, unrestricted	Often has revenue limits	Via API only
Regulatory category (EU)	GPAI	GPAI	GPAI
Example	EleutherAI models	Llama, Mistral (some)	GPT-4, Claude

graph TD
A["Closed Model"] --> B["Open Weights"]
B --> C["Open Source AI"]
A -->|provides| A1["No public model files"]
B -->|provides| B1["Weights file only"]
C -->|provides| C1["Weights Code Data"]

Frequently Asked Questions

Q: Can I use Llama in a commercial product without paying Meta?

You can use the weights under the Llama license, which allows commercial use but imposes restrictions. It is not an OSI-approved license. Read the specific terms carefully. If your use case triggers the restrictions, you need a separate agreement with Meta. source

Q: What's the practical difference between Apache 2.0 weights and a custom license?

Apache 2.0 gives you perpetual, irrevocable rights to use, modify, and distribute the weights for any purpose. A custom license can add conditions like revenue thresholds, use-case restrictions, or attribution requirements. Custom licenses can also be changed unilaterally by the licensor for future releases. Read the grant clause. source

Q: Does "open weights" mean I can see what data the model trained on?

No. Open weights means you have the final trained parameters. Training data documentation is a separate artifact. Most open-weight releases provide only high-level descriptions of training data sources, not detailed manifests. Without that, you cannot fully audit for copyrighted material or bias.

Q: Are there any truly open source large language models?

Yes, but they are rare. EleutherAI's models are released with weights, training code, and dataset documentation under open source licenses. They intentionally meet OSI's definition. Most other models marketed as open source are actually open weights with custom licenses.

Q: How does the EU AI Act affect my choice between open and closed models?

If you deploy a GPAI model in the EU, you have transparency and documentation obligations regardless of whether it's open or closed. Models with widely available weights get some exemptions from detailed technical documentation requirements. The Act does not exempt open models entirely. Check the specific obligations for your deployment context. source

Test yourself

You're evaluating two models for a customer support chatbot that will handle sensitive financial queries. Model A is closed, accessed via API, with SOC 2 compliance and contractual data processing terms. Model B is "open source" according to its marketing, with downloadable weights under a custom license that prohibits use in financial services without a separate agreement. The training data provenance for Model B is described as "a mix of public web data and licensed corpora." Which model gives you better compliance posture, and why?

Answer: Model A gives you better compliance posture for this specific use case, despite being closed. The API provider offers contractual commitments about data handling and security certifications that you can present to auditors and regulators. Model B's custom license explicitly restricts financial services use, so you'd need a separate agreement anyway, negating the "free" aspect of open weights. More critically, the vague training data description means you cannot verify whether the model was trained on confidential financial data or copyrighted material, which creates IP indemnification risk. For regulated industries, contractual clarity and audit trails matter more than weight availability. If you wanted an open-weight option, you'd need a model with an OSI-approved license that doesn't restrict financial services and with detailed enough data documentation to satisfy your compliance team's due diligence requirements.

What's next

In the next episode, we'll look at fine-tuning: what actually happens to those weights when you train a model on your own data, and why it's both more powerful and more fragile than most people assume.

If you want this kind of breakdown every week (how real AI systems actually work under the hood, not marketing summaries), subscribe to Internals Decoded at internalsdecoded.com.