Definition
Parameters are the internal variables that a neural network learns during training. They include weights and biases that determine how input data is transformed into output.
Scale in Modern Models: - GPT-2: 1.5 billion parameters - GPT-3: 175 billion parameters - GPT-4: ~1.7 trillion parameters (estimated) - Llama 3: 8B to 405B parameters
More Parameters Generally Means: - Greater capacity to learn complex patterns - Better performance on diverse tasks - Higher computational requirements - More training data needed
Examples
When someone says "a 70B model," they mean a model with 70 billion parameters.
Related Terms
Want more AI knowledge?
Get bite-sized AI concepts delivered to your inbox.
Free daily digest. No spam, unsubscribe anytime.