presentsUNAIVERSE
The UN AI timeline

    Learn AI /Generative Media /Audio and speech generation

    Audio and speech generation

    Also called: text-to-speech, TTS, voice cloning, audio generation.

    Detail

    In five seconds

    What it is

    Tools in this area create new speech, music or other sounds from text, audio examples or other controls, including voices that imitate a particular person.

    In plain words

    These tools create spoken sound from text or from examples of someone’s voice. Some just read text aloud in a computer voice. Others can make new audio that resembles a specific person or generate other kinds of sound.

    An everyday example

    Imagine a radio station that has a script desk and a digital announcer console. A producer types a sentence, then chooses either a standard station voice or a saved voice profile that matches one presenter’s accent and pace. The console reads the new script aloud in seconds, even though that presenter never entered the studio. If the producer instead asks for crowd noise or a door closing sound, the system uses a different audio tool, because not all audio is speech.

    Where the story stops working

    A real model does not keep neat, human-readable 'voice profiles' the way a studio might label presets. It works through learned statistical patterns in data. Also, the analogy makes the process seem fully controlled and reliable, but generated audio can mispronounce words, miss emotional cues, or produce misleadingly realistic imitations.

    Remember thisA believable voice clip may be synthetic, so treat generated audio as useful but not self-authenticating.

    Understand it

    Understand it

    Why it matters

    Non-specialists may hear a convincing voice and assume a real person recorded it. That is no longer a safe assumption. These tools can improve accessibility, multilingual communication and production speed, but they also make impersonation, fraud and false evidence easier. Knowing the difference between ordinary text-to-speech, voice cloning and broader audio generation helps you ask better questions: What was the source? Was consent given? Was it reviewed by a human? Is there provenance information, such as content credentials? Should this audio be trusted for a decision, or verified another way first?

    How text-to-speech, voice cloning and audio generation relate

    Show the difference between reading text aloud, imitating a specific voice and generating other kinds of sound.

    1. 01

      Text-to-speech

      Input: written text. The system converts the words into spoken audio, usually using a built-in synthetic voice.

      Draw column 1 with a text document icon feeding into a box labelled 'speech generator', then to a speaker icon. Add a small label 'generic or selected voice'.

    2. 02

      Voice cloning

      Input: text plus sample recordings of a person’s voice. The system generates new speech that sounds similar to that person.

      Draw column 2 with a text document icon and a microphone/audio sample icon both feeding into a box labelled 'voice-conditioned speech generator', then to a speaker icon. Add label 'specific voice likeness'.

    3. 03

      Broader audio generation

      Input: text prompts, reference audio or other controls. The output can be speech, music, sound effects or mixed audio.

      Draw column 3 with prompt icons feeding into a box labelled 'audio generator', then to outputs showing speech waveform, music note and sound effect icon.

    4. 04

      What it is not

      Speech-to-text turns audio into words, which is the opposite direction. Deepfake refers to deceptive synthetic media and may include cloned voices, but not every generated voice is deceptive.

      Place a footer strip under all columns. On the left, show microphone to text labelled 'speech-to-text: reverse'. On the right, show warning badge next to speaker and face icons labelled 'deepfake: possible misuse, not the whole category'.

    A three-column comparison read left to right. Each column shows typical input, what the system uses, and the output. The first covers TTS, the second voice cloning, the third broader audio generation. A note at the bottom says speech-to-text is the reverse direction and deepfake is a use case, not the whole category.

    Hover or tap each column to highlight its input and output path; footer items remain visible for contrast.

    Describe this without the diagram

    The diagram compares three related tools. First, text-to-speech starts with written text and outputs spoken audio, usually in a built-in synthetic voice. Second, voice cloning starts with written text plus sample recordings of one person and outputs new speech that sounds similar to that person. Third, broader audio generation can start with prompts or reference audio and can output speech, music or sound effects. A note underneath explains that speech-to-text is the reverse process because it turns audio into text, and deepfake is a deceptive use of synthetic media rather than a synonym for all generated audio.

    With reduced motion: A static three-column table shows inputs, system type and outputs for text-to-speech, voice cloning and broader audio generation, with a bottom note contrasting speech-to-text and deepfake.

    Audio and speech generation means creating sound with a model rather than recording a person speaking each line. The input might be written text, music instructions, a short voice sample, or all three. The output is an audio file or live spoken sound.

    These terms are related but not identical. Text-to-speech, often called TTS, starts with written text and reads it aloud. Audio generation is broader: it can include speech, sound effects, music, or mixed audio. Voice cloning means generating speech that sounds like a specific person, usually from sample recordings of that person.

    A model is a pattern-finding system trained on many audio examples. It learns relationships between text, speech sounds, timing, pitch and style. Given an input, it produces a waveform: the signal that a speaker or headphones play as sound.

    This is often confused with speech-to-text, which does the reverse and turns spoken audio into written words. It is also confused with deepfakes. A cloned voice can be a deepfake when it convincingly imitates a real person, but generated speech need not imitate anyone. For example, an accessibility tool that reads text aloud is generated speech without pretending to be a real recording.

    How “An announcer studio with reusable voice settings” maps onto the real thing

    In the storyIn the AI system
    typed script at the desktext prompt for text-to-speech
    standard station voicegeneric synthetic voice
    saved presenter voice profilevoice cloning setup based on sample recordings
    separate tool for crowd noise or sound effectsbroader audio generation beyond speech

    A worked example

    A museum accessibility editor has a cleared exhibit script and must publish short audio guides in English, Arabic and Swahili before a new gallery opens.

    Starting from: Cleared English line: “Look closely at the surface to see where the artist shaped the clay by hand.” The museum has an authorised synthetic guide voice.

    1. The editor enters the English line into a text-to-speech tool and selects the authorised voice.
    2. The system maps the words to pronunciation, timing, stress and intonation, then generates the audio waveform.
    3. Approved translators provide the Arabic and Swahili text, which the editor processes using suitable language and pronunciation settings.
    4. The editor and language reviewers listen to each file, correct pronunciation and emphasis, then export the final audio clips.

    Result: The museum obtains three consistent spoken versions without recording every line in a studio. Human reviewers still verify the translation, pronunciation, tone and match with the cleared script.

    In a United Nations workplace

    At a regional bureau, a reporting officer has cleared the written quarterly progress report for a donor. At 16:00, the front office asks for a two-minute audio summary to accompany it before close of business.

    An approved text-to-speech tool can turn the cleared summary into a draft voice track, applying pronunciation, timing, stress and intonation. The officer can then check names, figures and emphasis against the report before the file is circulated.

    Before you try thisPersonal, protection-sensitive, security-sensitive or procurement-sensitive content must not be entered into a tool unless it is explicitly approved for that material. A colleague’s voice must not be cloned without clear authorisation. Synthetic audio should be identified as such so that it cannot be mistaken for an authentic recording.

    Where you may hear it

    • During an accessibility check before a webcast
    • At a vendor demo in a procurement meeting
    • In a communications clearance meeting for a narrated video
    • When a duty officer is asked to verify a suspicious voice note
    • While testing an automated telephone line before launch

    Avoid confusion

    Avoid confusion

    Often confused with

    Common misconceptions

    • If audio sounds natural, it must be a real recording of a real person.

      Generated speech can now sound very realistic, especially in short clips. Natural sound is not proof of authenticity. If the source matters, check who created it, whether the speaker consented, and whether there is independent verification or provenance information.

    • Text-to-speech, voice cloning and deepfake all mean the same thing.

      They overlap but are not identical. Text-to-speech reads written text aloud, often in a generic synthetic voice. Voice cloning tries to match a specific person’s voice. Deepfake usually refers to deceptive synthetic media, which can include cloned audio but can also involve video or images.

    • A cloned voice reproduces exactly what the person would have said or felt.

      It only imitates patterns in recordings such as accent, pitch and rhythm. It does not provide reliable evidence of intent, emotion, truthfulness or approval by the person whose voice is being imitated.

    Quick check

    Which statement best distinguishes these related terms?

    Go deeper

    Go deeper

    The simple version

    These tools create spoken sound from text or from examples of someone’s voice. Some just read text aloud in a computer voice. Others can make new audio that resembles a specific person or generate other kinds of sound.

    The technical version

    Modern systems for speech and audio generation learn statistical relationships between inputs such as text, phonemes, speaker examples and acoustic features, then produce audio signals. In text-to-speech, a system maps written words to pronunciation and prosody, meaning timing, stress and intonation, before generating a waveform. In voice cloning or speaker-conditioned synthesis, the model also uses features extracted from recordings of a target speaker so the output resembles that voice. Broader audio generation extends beyond speech to music, sound effects or mixed soundscapes. Different architectures can be used, including diffusion-based systems and other neural waveform or acoustic models. This should be distinguished from speech-to-text, which recognises words from incoming audio rather than generating new sound.

    Reviewed external resources

    Where this came from

    Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

    The one thing to remember

    A believable voice clip may be synthetic, so treat generated audio as useful but not self-authenticating.