Weights and parameters — what 3B, 7B and open weights mean
An is a huge pile of learned numbers called , most of them . Learn how they work, what and 7B count, and what give you.
Weight and parameter: two words you will see everywhere
An does not store sentences or facts. It stores numbers, and every answer is calculated from them. Those numbers are called parameters, and the main kind is called weights.
-
weight
Weight
One is a weight, many are weights. In formulas and code it is usually written w.
A number that decides how strongly one input affects the result. It is multiplied by that input.
Like one volume slider on a mixing board.
-
parameter
Parameter
One is a parameter, many are parameters. It is often shortened to params.
Any number a model adjusts by itself while it learns. Weights are the largest group, but not the only one.
Like all the sliders on the board, counted together.
Every weight is a parameter, but not every parameter is a weight.
People often say weights when they mean all the parameters, as in “download the weights”. That file also holds biases and the other learned numbers.
What counts as a parameter
-
Weights weight
Multiplied by an input. Sets how strongly it counts.
-
Biases bias
Added at the end. Shifts the result up or down.
-
Embeddings embedding
The list of numbers that stands for each .
-
Normalization scales normalization coefficient
Learned numbers that keep values in a steady range.
How one weight works
One small unit of a multiplies the input by the weight, then adds the bias. Change the weight and the same input gives a different result.
The same input with three different weights
-
3 × 2 + 1 = 7Large weight: the input counts a lot -
0 × 2 + 1 = 1Zero: the input is ignored -
-1 × 2 + 1 = -1Negative: the input pushes the other way
A real model repeats this with billions of weights, layer after layer. Each calculation stays this simple. There are just a great many of them.
Who sets these numbers? Training does
Nobody types them in. starts from random numbers and corrects them over and over. When it ends, the numbers are saved in a file and stay fixed while the model answers.
-
Start with random numbers
-
Make a prediction
-
Measure the error
-
Nudge every number slightly
Repeat many times
Chatting does not change the weights. A normal chat only uses them. They change only when the model is trained again.
Parameter or hyperparameter?
The names are close, but the jobs differ: people choose hyperparameters before training, and training then finds the parameters.
| Parameter | Hyperparameter | |
|---|---|---|
| Who sets it | Training finds it from the data | A person chooses it |
| When | Updated throughout training | Set before training starts |
| Examples | Weights, biases | Learning rate, batch size, number of layers, epochs |
What 3B and 7B count
The letter after the number counts parameters. It is not a file size. 7B means about 7 billion numbers.
- M
- million 1,000,000
- B
- billion 1,000,000,000
- T
- trillion 1,000,000,000,000
Reading a model name
Some models use only part of their parameters for each token. Their names give both numbers.
Mistral-Large-4 1T A52B
- Mistral-Large-4
- The name of the model
- 1T
- Parameters in the whole model
- A52B
- A means active: parameters used for each token
Active does not mean lighter to load. The whole model still has to sit in memory.
How big is a 7B model?
The size depends on how many each number uses. keeps the count of numbers and stores each one in fewer bits.
weight size ≈ parameter count × bits per parameter ÷ 8
| Model | 32-bit | 16-bit | 8-bit | 4-bit |
|---|---|---|---|---|
| 3B | 12 GB | 6 GB | 3 GB | 1.5 GB |
| 7B | 28 GB | 14 GB | 7 GB | 3.5 GB |
| 70B | 280 GB | 140 GB | 70 GB | 35 GB |
Real files are a little larger because they also hold . Running a model needs extra for the and buffers.
Open weights, open model, open source: not the same
These words describe how much of a model you are given. The further down the list, the more you receive.
-
Closed model
You reach the model only through the company’s app or . The weights stay on its .
Examples Claude, Gemini, the GPT models behind ChatGPT
-
Open weights (open model)
You can download the weights and run them on your own computer. How they were trained may stay private.
Examples Llama, Qwen, DeepSeek, gpt-oss, Mistral Large 4
-
Open source AI
You also get the training and detailed information about the , under terms that let anyone use, change and share them.
Examples OLMo, Pythia, T5. They passed the validation phase of the Open Source Initiative, which is not a certification.
| Weights to download | Training code | Details of the training data | Free to use, change and share | |
|---|---|---|---|---|
| Closed model | No | No | No | No |
| Open weights (open model) | Yes | Varies | Often partial | Depends on the licence |
| Open source AI | Yes | Yes | Yes | Yes |
Open model is an everyday phrase, not a defined standard, and it usually means open weights. Some companies say open source for models that are only open weights, so read the licence.
Open weights still come with a licence
- Apache 2.0 · MIT
- Few limits. You keep the notice and may use the model commercially. Mistral 7B · gpt-oss · DeepSeek V4.1 Flash
- Own licence
- The company’s own terms. Read them before you build a business on the model. Llama · Mistral Large 4
A recent example: Mistral Large 4
Mistral AI published the weights of Mistral Large 4 on in October 2026. The model has about 1T parameters and uses about 52B of them for each token. Its licence is the Mistral Ecosystem Open-Weight License, not : running it as a model service requires a separate commercial licence from Mistral.
Model card on Hugging Face (opens in a new tab)Try a small one yourself
llama3.2:3b is a 3B model. lists the download at 2.0 , small enough for a laptop.
ollama run llama3.2:3bCommon mix-ups
-
“B means byte”
B means billion here. The size of a 7B model depends on the bits per number.
-
“More parameters always means smarter”
More numbers mean more capacity and a heavier model. Training data and design also decide how good it is.
-
“Open weights means open source”
You may get only the finished numbers, not the recipe, and the licence may set limits.
-
“Quantization removes parameters”
It keeps their count and stores each one with fewer bits, so 7B stays 7B.
-
“A function parameter is the same thing”
In programming and APIs, a parameter is a value you pass in. A model parameter is a number learned in training.
-
“Attention weights are model weights”
Attention weights are calculated fresh for each sentence and add up to 1. They are not saved in the model file.
Sources
Figures as shown on these pages on 2026-10-31. Model names and licences change, so check the current page.
- Open Source Initiative — The Open Source AI Definition 1.0(opens in a new tab)
- Open Source Initiative — Open Source AI FAQ(opens in a new tab)
- Google — Machine Learning Glossary(opens in a new tab)
- Ollama — llama3.2(opens in a new tab)
- Hugging Face — mistralai/Mistral-7B-v0.1(opens in a new tab)
- Hugging Face — openai/gpt-oss-20b(opens in a new tab)
- Hugging Face — Qwen/Qwen3-Coder-30B-A3B-Instruct(opens in a new tab)
- Language Models are Few-Shot Learners (GPT-3, arXiv)(opens in a new tab)
- Hugging Face — deepseek-ai/DeepSeek-V4.1-Flash-1017(opens in a new tab)
- Hugging Face — mistralai/Mistral-Large-4-1T-A52B(opens in a new tab)