presentsUNAIVERSE
The UN AI timeline

    Learn AI /Large Language Models /Next-token prediction

    Next-token prediction

    Also called: next word prediction, autocomplete.

    Detail

    In five seconds

    What it is

    A language model produces text by repeatedly estimating probabilities for small pieces of text that could come next in the current sequence.

    In plain words

    The system looks at the text so far and keeps selecting a small piece of text to add next, one step at a time.

    An everyday example

    While you type a message, your phone may offer several likely continuations based on what is already on the screen. You choose one suggestion, it is added to the message, and the phone immediately offers new choices based on the longer text. A language model follows a much larger and faster version of this loop. It scores possible tokens, which may be whole words, parts of words or punctuation, selects one, adds it and starts again. The message may become smooth and plausible even if the continuation is not true.

    Where the story stops working

    A student may consciously use rules, background knowledge, and test strategy. A language model does not work like a human exam taker. It computes patterns over tokens and outputs one token at a time. Also, real models often predict subword pieces rather than full words, so the exam analogy is simpler than the actual mechanism.

    Remember thisA language model extends its context one token at a time. Fluent wording can result from this repeated prediction process, but fluency does not establish accuracy.

    Understand it

    Understand it

    Why it matters

    If you know that a system writes by next-token prediction, you are less likely to mistake smooth wording for verified truth. This helps with practical decisions: when to trust style but check facts, why small prompt changes can shift the answer, why randomness settings affect wording, and why a model may continue a pattern even when the pattern is misleading. For communications work, it explains both the usefulness and the limits of AI drafting tools: they can help with phrasing, summaries, and reformulation, but they still need human review for accuracy, tone, risk, and context.

    How a reply is built one token at a time

    After viewing this, a reader should be able to say that a language model generates text by repeatedly choosing the next token from the current context.

    1. 01

      1. Start with context

      The model receives the text so far, such as the user's prompt or a partly written reply.

      Draw a text box labelled 'Current text' containing: 'The meeting starts at'. Use a muted background.

    2. 02

      2. Score candidates

      The model assigns a score to many possible next tokens from its vocabulary.

      To the right, show a small ranked list: '9', 'noon', 'the', '.' with horizontal bars of different lengths. Label it 'Possible next tokens'.

    3. 03

      3. Choose one token

      A rule selects the next token, either the top option or one sampled from likely options.

      Highlight '9' in a strong colour. Add a small note: 'selection rule chooses one'.

    4. 04

      4. Append and repeat

      The chosen token is added to the text, creating a new context for the next step.

      Show the text box updated to 'The meeting starts at 9'. Add a loop arrow to another candidate list for the next token, such as 'a.m.', 'today', '.'.

    Show an ordered sequence of panels. Each panel adds one new token to the growing text. Above the text, show a short list of candidate next tokens with different scores, then highlight the chosen one. Read left to right to see that the output is built through repetition of the same step.

    Clicking each step reveals the updated text and a new candidate list; a replay control restarts the sequence from the original prompt.

    Describe this without the diagram

    This diagram explains that text generation happens in repeated small steps. Step 1 shows the current text: 'The meeting starts at'. Step 2 shows several possible next tokens with different scores, such as '9', 'noon', 'the', and '.'. Step 3 highlights one chosen token, here '9'. Step 4 adds that token to the text, making 'The meeting starts at 9', then repeats the same process for the following token. The key idea is that the model does not produce the whole sentence at once. It extends the text one token at a time based on the text already present.

    With reduced motion: A static four-panel sequence shows the same process without animation: current text, candidate token scores, selected token, and updated text with a note that the loop repeats.

    A chat assistant does not write a whole answer in one step. It reads the text it has been given so far and predicts the next token, meaning a small piece of text such as a word, part of a word, punctuation mark, or number fragment.

    What goes in is the current text sequence, often called the context. That includes your prompt and any earlier words already generated in the reply. The model then calculates scores for many possible next tokens in its vocabulary, which is the set of tokens it can output.

    What happens next depends on the generation settings. The system may choose the highest-scoring token, or it may sample from several likely options to make the output less repetitive. After one token is chosen, it is added to the text, and the same process repeats again and again.

    What comes out is a completion built one token at a time. This often produces fluent language, but fluency is not proof of factual accuracy, reasoning quality, or reliable intent. Next-token prediction explains how text is produced, not whether the content is correct or wise.

    How “The predictive text strip” maps onto the real thing

    In the storyIn the AI system
    the words already on the exam linethe current context the model reads
    the student's guess for the next wordthe predicted next token
    repeating one word at a timeautoregressive text generation step by step
    old papers the student practised ontraining data used to learn patterns

    A worked example

    Completing a line in a lessons-learned note

    Starting from: Prompt: “The pilot showed that early consultation can”

    1. The model treats the prompt as the current sequence and scores every possible next token in its vocabulary.
    2. Possible high-scoring tokens might include “ improve”, “ help” and “ reduce”. Suppose the decoding method selects “ improve”.
    3. The selected token is appended, producing “The pilot showed that early consultation can improve”. The longer sequence becomes the input for the next prediction.
    4. The model repeats the loop, perhaps selecting “ the”, then “ quality”, then “ of”, then “ drafting”, followed by punctuation.

    Result: The completion becomes: “The pilot showed that early consultation can improve the quality of drafting.” It was generated through successive token selections, not retrieved or planned as a finished sentence.

    In a United Nations workplace

    After an approved drafting pilot, a reporting officer in a Headquarters department prepares the lessons-learned note for clearance by close of business. She uses one draft sentence to show the front office why fluent output sometimes included unsupported claims.

    The tool did not compose the sentence as a complete thought. At each step, it scored possible next tokens using the prompt and the tokens already present, selected one through its decoding method, appended it and repeated the process. Familiar drafting patterns can therefore emerge without factual understanding.

    Before you try thisThe officer must verify every claim and reference against the file. Confidential drafts, personal data and other sensitive material must not be entered into a tool that has not been approved for them, including merely to demonstrate how the pilot worked.

    Where you may hear it

    • A vendor demonstration during a procurement meeting
    • A lessons-learned session after an approved drafting pilot
    • An information session for an internal artificial intelligence tool
    • A meeting of an inter-agency working group on data governance

    Avoid confusion

    Avoid confusion

    Often confused with

    Common misconceptions

    • Next-token prediction means the system only predicts the next word, so it cannot produce long or structured answers.

      The mechanism predicts one token at a time, but it repeats that step many times. Because each new token becomes part of the context, the model can generate paragraphs, tables, code, or other long outputs. The limit is not that it only does one word, but that it builds the sequence incrementally.

    • If the model predicts likely next tokens well, the content must be true.

      Likelihood and truth are different things. A token sequence can be statistically plausible because it resembles patterns in training data, yet still be outdated, invented, biased, or wrong for the specific situation. Fluency is evidence of pattern matching, not proof of factual reliability.

    • Next-token prediction is just the same as autocomplete on a phone keyboard.

      They are related, because both suggest continuations from earlier text. But large language models usually use much larger training sets, larger vocabularies, longer context, and more complex selection methods. Phone autocomplete is a useful comparison, not the full mechanism.

    Quick check

    What does next-token prediction describe most directly?

    Go deeper

    Go deeper

    The simple version

    The system looks at the text so far and keeps selecting a small piece of text to add next, one step at a time.

    The technical version

    In an autoregressive language model, generation is factorised over a sequence: the model estimates a probability distribution for the next token conditioned on the prior tokens in the context window. At each step it scores tokens in its vocabulary, then a decoding method selects one token, for example greedy choice or stochastic sampling. That chosen token is appended to the sequence and fed back as part of the input for the next step. Repeating this loop produces a completion. The mechanism explains local fluency and sensitivity to context, but it does not by itself guarantee factual accuracy, groundedness, or robust multi-step reasoning.

    Reviewed external resources

    Where this came from

    Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

    The one thing to remember

    A language model extends its context one token at a time. Fluent wording can result from this repeated prediction process, but fluency does not establish accuracy.