NARRATION.md: voice, music and sound

Narration used to be entirely the caller's problem. It is not any more: generate_narration reads beats.json and writes one clip per beat to the path stitch_composition already looks in. This doc covers what that tool does, what it deliberately refuses to do, and why generated voiceover usually sounds synthetic.

The short version

generate_narration  ->  public/audio/vo/<beatId>.mp3
stitch_composition  ->  picks them up automatically

Why generated voiceover sounds like a robot

Rarely the engine. Almost always the stretching.

Every engine reads at its own pace, so a clip rarely lands exactly on its beat. The obvious fix is to time-stretch it to fit. A pitch-preserving stretch past roughly ten percent is audible, and past twenty it is the drawly, processed sound people mean by "AI voice". Building the openvidstudio demo, one beat was stretched to 0.645, a 35 percent slowdown, and that single clip was most of why the whole video sounded synthetic.

So generate_narration clamps stretching to 0.9 to 1.12. When a line still does not fit, it says so and names the beat rather than mangling it, because the real fix is one of:

• Too short for the beat. Add a sentence, or shorten the beat.
• Too long for the beat. Cut words, or lengthen the beat.

Both are edits to beats.json, which is yours to change. Stretching audio to hide a mismatch trades a timing problem for an audible one.

Pace

validate_beats enforces SCRIPT.md's 2.3 to 2.9 words per second as a hard floor and ceiling, per beat. It is a floor as well as a ceiling, so a beat can be rejected for being too sparse, which is not the failure most people expect.

plan_beats gives you the word budget per beat up front, so narration is written to a number instead of guessed and revised.

Engines

edge-tts needs pip install edge-tts.

If the text must not leave the machine, use say on macOS, or wire up a local neural model. A local model on CPU is slow: one measured at a real time factor of 41.9x, meaning six minutes of compute for nine seconds of audio, or about three and a half hours for a five minute video. Fine overnight, not fine in a loop.

If you want it to sound genuinely good, record it yourself. A real voice beats every engine here, and for a launch video it is worth the hour.

Silence is silent

stitch_composition skips a missing public/audio/vo/<beatId>.mp3 with no error and no warning. A half generated narration renders successfully and simply plays silent in those beats.

Two things catch it:

• generate_narration reports which beats have audio and which do not
• validate_scenes flags every beat with no narration file

Neither is optional if you care whether the video has a voice.

Music and sound effects

Everything ships synthesized, generated by scripts/gen-sfx.sh with ffmpeg. Nothing is sampled or licensed, so a rendered video carries no third-party audio rights. No attribution, no content ID, no licensing questions.

Regenerate any of them by editing scripts/gen-sfx.sh and running it. The sfx.tsx API does not change, so you can swap in a recorded pack later without touching a scene.

Bringing in a sound the built-in pack doesn't cover

Drop the file into public/imported_audios/, kept separate from public/sfx/ so it's always obvious which sounds carry licence terms and which don't. Reference it from a scene with staticFile("imported_audios/<filename>").

plan_sound_effects automates the search-and-checklist part of this: tell it what the video needs, and it checks the built-in pack first, then returns a Pixabay search link and the exact filename to save for anything genuinely missing (or fetches it automatically via Freesound, given your own API token). Either way, the file lands in public/imported_audios/, and anything carrying a licence gets written to ATTRIBUTION.md there, which belongs in the video description.

Using the music bed

import { MusicBed } from "@openvidstudio/core";

<MusicBed durationInFrames={9000} volume={0.16} />

Keep it quiet. It sits under narration, and 0.16 is already at the edge of noticeable when a voice is speaking over it.

If you want real music instead

Pixabay Music and the YouTube Audio Library are both no-attribution. Drop the file at public/audio/music-bed.mp3 and stitch_composition picks it up, same as narration.