Learn AI /Neural Networks /Softmax
Softmax
- Deeper
- Neural Networks
- Technical readers
Back to the milestone you came from
In five seconds
What it is
A numerical function that turns a list of model scores into positive values that sum to 1, highlighting which outputs are larger relative to the others.
In plain words
Softmax takes a model’s raw output scores and turns them into a set of shares that add up to 1, so you can see which option stands out most relative to the others.
An everyday example
A team lead opens five incoming messages and gives each one an urgency score based on quick signals such as sender, subject line and timing. Those raw scores are hard to compare because they do not form a neat scale. So the team’s triage tool converts them into attention shares that must add up to 100 percent, giving the most urgent-looking message the biggest share. If one message suddenly looks much more urgent, the shares for the others shrink even if their own raw scores did not change.
A human triage tool may use rules, context and judgement, while softmax is only a fixed mathematical transformation. Also, message urgency can involve several messages being genuinely urgent at once, but softmax is mainly suited to cases where outputs compete for one total share. For tasks where multiple labels can all be true independently, a different output function is usually more appropriate.
Remember thisSoftmax converts competing scores into positive shares that sum to 1. The largest score gets the largest share, but that share is not automatically a reliable measure of confidence.
Understand it
Understand it
Why it matters
Softmax affects how a model presents confidence at the point where people often make decisions. If you read classifier outputs, compare model options or set thresholds, you need to know that softmax shows relative competition among classes, not certainty. This helps you avoid over-trusting a high score, choose better review rules and spot when a task may need a different output design, especially when several labels could all be true at once.
How softmax turns scores into a distribution
After reading the diagram, the reader should be able to explain that softmax takes raw class scores, amplifies relative differences, and normalises them so the outputs sum to 1.
01
1. Raw scores in
Start with three class scores, such as topic A = 2.0, topic B = 1.0, topic C = 0.1. These are model outputs before normalisation.
Show three horizontal bars labelled A, B and C with values 2.0, 1.0 and 0.1. Bars can extend left or right from a zero line if needed, but this example stays positive.
02
2. Exponentiate each score
Apply the exponential function to each score. Larger scores grow more quickly, which increases separation between classes.
Show the transformed values beside each class: exp(2.0), exp(1.0), exp(0.1). Under each, show approximate numbers 7.39, 2.72 and 1.11, with noticeably longer bars than before.
03
3. Sum the transformed values
Add the exponentiated values to get the total used for normalisation.
Display a sum box below the three transformed bars: 7.39 + 2.72 + 1.11 = 11.22. Draw connector lines from each transformed bar to the sum box.
04
4. Divide by the total
Each class gets its exponentiated value divided by the total. The outputs are positive and add up to 1.
Show final values for A, B and C as 0.66, 0.24 and 0.10 with percentage-style bars. Include a small note: 0.66 + 0.24 + 0.10 = 1.00.
A left-to-right flow diagram with one example vector moving through three stages: raw scores, exponentiation, and normalisation. Each stage shows the same three classes in aligned rows. Read from left to right to see how relative size becomes a distribution over classes.
Hovering or tapping a class highlights that class across all stages, so the reader can track one value from raw score to final share.
Describe this without the diagram
The diagram explains softmax as a four-step process using three example classes. Step 1 shows raw scores: A has 2.0, B has 1.0, and C has 0.1. Step 2 applies the exponential function, producing about 7.39, 2.72 and 1.11. Step 3 adds these values to get 11.22. Step 4 divides each transformed value by 11.22, giving final outputs of about 0.66, 0.24 and 0.10. The key point is that larger raw scores become disproportionately larger after exponentiation, then all values are scaled to sum to 1.
With reduced motion: A static four-column diagram shows the same example without animation: raw scores, exponentiated values, the total sum, and final normalised outputs. Matching colours connect each class across the columns.
Softmax usually appears at the output end of a neural network for a task with several competing classes, such as languages, document types or object labels. The input to softmax is a list of scores, often called logits. These scores are not yet probabilities, and they can be any real numbers, including negatives.
Softmax transforms those scores in two main steps. First, it applies the exponential function to each score, which makes larger scores grow much faster than smaller ones. Then it divides each exponentiated value by the sum of all of them. The result is a set of positive numbers that add up to 1.
This means softmax does not judge each class independently. Every output depends on all the scores in the list. If one class score rises, the shares available to the others fall. That is why softmax is commonly used for mutually exclusive classes, where the task requires one main answer.
People often confuse softmax with probability itself. Softmax outputs are often treated like probabilities, but that interpretation can be misleading if the model is poorly calibrated. In practice, softmax gives a convenient relative distribution over classes, not a guarantee that the top class is correct at that exact rate.
How “Sharing attention across a list of messages” maps onto the real thing
| In the story | In the AI system |
|---|---|
| raw urgency scores for each message | logits or unnormalised output scores |
| attention shares that add up to 100 percent | softmax outputs that sum to 1 |
| one message getting a bigger share reduces the others | outputs are coupled across classes, not independent |
A worked example
For one paragraph in the quarterly report, the classifier must select a single label: results, expenditure or risks.
Starting from: Model logits: results = 1.8, expenditure = 0.7, risks = -0.2
- Exponentiate the logits: exp(1.8) is about 6.05, exp(0.7) is about 2.01 and exp(-0.2) is about 0.82.
- Add the exponentiated values: 6.05 + 2.01 + 0.82 = 8.88.
- Divide each value by 8.88: results is about 0.68, expenditure 0.23 and risks 0.09. The outputs are positive and sum to 1.
- For a single-label task, the classifier selects results because it has the largest softmax value. The reviser can still override that suggestion after reading the paragraph.
Result: Softmax produces a distribution across the three competing labels. Results receives about 68 per cent of the total, but that figure reflects the model’s relative scores, not a verified 68 per cent probability that the label is correct.
In a United Nations workplace
At a shared services centre, a reviser in the translation service is checking a quarterly report to a donor before it goes for clearance at close of business. An internal classifier assigns each paragraph one subject label so that the relevant terminology list can be applied.
Softmax converts the classifier’s raw scores for labels such as results, expenditure and risks into shares that sum to 1. The reviser can see both the leading label and whether the alternatives are close enough to warrant a manual check.
Before you try thisA high softmax score is not proof that the label is correct or that the model is well calibrated. Drafts containing personal data, partner-sensitive details or uncleared financial information must not be entered into any tool unless it is approved for that material, because the text could be retained, disclosed or processed outside authorised controls.
Where you may hear it
- During a vendor demonstration to an ICT governance board
- In a model evaluation meeting before an internal classifier is released
- While reviewing misclassified examples with an information management officer
- In a technical discussion about why model scores should not be read as certainty
Avoid confusion
Avoid confusion
Often confused with
Activation function
A neural network usually uses an activation function to transform a unit’s weighted input into an output, often adding the non-linear behaviour needed to learn complex patterns.
Probability (in model outputs)
A number attached to a model’s possible outputs that shows how much of its total estimated likelihood the model assigns to each option.
Common misconceptions
Common misconception: Softmax outputs are always true probabilities.
Correction: They have the mathematical form of a probability distribution because they are positive and sum to 1, but that does not guarantee calibration. A model can assign 0.99 to a wrong class. Whether the numbers match real-world frequencies depends on the model, the data and any calibration method used afterwards.
Common misconception: Softmax decides the class by itself.
Correction: Softmax only transforms scores into a normalised distribution. The highest class after softmax is the same as the highest class before softmax, because the transformation preserves the ranking of the logits. The choice rule, such as taking the maximum or applying a threshold, is a separate step.
Common misconception: Each softmax output is computed independently.
Correction: In softmax, every output depends on all class scores because each one is divided by the same total sum. If one score increases, the others usually decrease in relative share even when their own logits stay unchanged. This is different from functions used in multi-label tasks, where outputs can be treated more independently.
Quick check
Which statement best describes what softmax does in a multi-class classifier?
Softmax normalises a set of class scores into a distribution over the listed classes. Because the outputs must share a total of 1, they are linked to one another. A high value can still be wrong, so softmax does not guarantee correctness or remove the need for judgement.
Go deeper
Go deeper
The simple version
Softmax takes a model’s raw output scores and turns them into a set of shares that add up to 1, so you can see which option stands out most relative to the others.
The technical version
Given logits z_i for classes i, softmax computes exp(z_i) divided by the sum of exp(z_j) over all classes j. The exponential emphasises differences in larger logits, and the normalisation couples all outputs so they form a categorical distribution over classes. Adding the same constant to every logit does not change the result, which is why implementations often subtract the maximum logit first for numerical stability. Softmax is typically used for single-label multi-class prediction and is often paired with cross-entropy loss during training. It should be distinguished from sigmoid outputs used for multi-label settings, where classes do not have to compete for one total mass. Although softmax outputs are often read as probabilities, they may be poorly calibrated and should not automatically be interpreted as reliable confidence estimates.
Reviewed external resources
EncyclopaediaSoftmax function (opens in a new tab)Wikipedia
This link gives a broad reference-style overview of the softmax function, including definition and mathematical form, and suits readers who want a general starting point.
Intermediatelink checked 2026-07-31
InteractiveTensorFlow Playground (opens in a new tab)TensorFlow
This interactive tool lets readers experiment with neural-network behaviour in the browser and suits hands-on learners who want to see output changes directly.
Starterlink checked 2026-07-31
VideoNeural networks, explained visually (opens in a new tab)3Blue1Brown
This video gives a visual explanation of neural networks and helps readers who learn best from intuition and animated structure rather than formal notation.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Softmax (opens in a new tab)YouTube search
This search results page helps readers find different video explanations of softmax and suits people who want to compare teaching styles rather than rely on one source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Softmax (opens in a new tab)Reddit search
This search results page helps readers find community discussions and common points of confusion about softmax, and suits readers who want informal explanations and debate.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
Softmax converts competing scores into positive shares that sum to 1. The largest score gets the largest share, but that share is not automatically a reliable measure of confidence.