Skip to main content
A film’s soundtrack is mixed once, in assemble, as one filter graph over a silent anchor as long as the picture. The narration and the audio of every clip are always in it. The music, the ambience bed and the effects are the score, and each of them is in the mix only because the project named a file for it. A sound placed on a cue lands on the second that cue resolved to, which Cues explains. To put one sound on one cue from nothing, follow Put a sound effect on a cue.

​
The parts of the mix

The anchor is what fixes the length. The picture carries no audio of its own, so without a silence as long as the film the soundtrack would run as long as whichever layer happened to be longest. decktalk assemble --skip score leaves the music, the ambience and the effects out and mixes the narration and the clip audio alone. Nothing else changes.

​
What [mix] holds

[mix] lists every key that names a file and its level, and [[mix.effect]] every key of an effect. A cued sound takes a caption, and Captions carry the whole soundtrack says what a viewer reads in its place.

​
The score stage

score is the fourth stage of the six. It generates the music, the ambience bed and the effects this project describes, from the prompts in [score], through the sound provider [score] provider names, which is ElevenLabs’s sound and music endpoints with the key and base_url of [elevenlabs]. It spends, which is why it runs where it does: it is the last stage that can buy anything, so decktalk build --to record stops before it and an unpaid draft loop never reaches it. It is a stage like any other, so decktalk build reaches it with --from score, --to score and --skip score, decktalk assemble --skip score mixes without it, and decktalk score runs it alone.
A run with --no-spend reports what it would ask for and buys nothing, which is the plan an author reads before approving a spend. With neither flag, a run that would buy something prices it and asks on a terminal, and a run with nothing left to buy asks nothing. A build asks once about its takes and its sounds added together, and The ceiling is how --max-cost refuses one. --replace-score is the one way to buy again a sound the ledger holds. It acts only with --spend, is confirmed once on a terminal, and never buys a take, while --force never buys a sound. A sound is billed by the second of audio, at the rate in the sound provider’s own table. Each kind states its rate in US dollars per minute of audio as dollars_per_minute in its own table, [score.ambience], [score.effects] or [score.music], and a request costs the seconds it asks for at that rate. The result’s cost reports the rate in the same unit, under the same name, so the rate you typed is the rate you read back. A rate nobody stated is zero from the default layer, and --max-cost refuses a run that buys at one, because the cap would guard a made-up price. The speech rate, [elevenlabs] dollars_per_1000_characters, never prices a sound. Every request the provider is paid for is a sound.charged line on the stream the moment it answers, one per bed or effect and one per part of the music, and a run that bought something reports its spend as charged.

​
What each table declares

Each table holds its prompt and its file beside the settings that size its request, spelled the same way in every table. [score] lists every key of each table, and Configuration every setting with its range. Music longer than [score.music] chunk_max_seconds is asked for in equal parts, because the service will not write a long piece in one answer. Each part is told which one it is so it keeps the key and the tempo, and the parts are joined with a crossfade of [score.music] crossfade_seconds.
decktalk.toml

​
How each generated file reaches the mix

This is the one join to get right. score writes files and the mix reads [mix], so the two have to name the same path. So naming the file in [mix] is enough for the music and the ambience: score writes to the path the mix will read. An effect has no such shortcut, because one generated sound can be cued in several places, so a [[mix.effect]] file points at score/<name>.mp3 by hand.

​
The score directory

Every sound the score buys lands in the score directory, score/ unless [score] dir names another folder inside the project, and you commit it the way you commit takes/. Each file is named by its item, so score/ reads ambience.mp3, tick.mp3, music-part1.mp3 and the ledger beside them. Deleting the build directory then costs nothing: the next run finds what it bought, and joins the music again from its parts under build/score/, which is free. A run without --spend reads the score directory and never writes it, so a clone that commits it, such as the one the GitHub Action checks out, mixes the film with no key and leaves no tracked file changed. --replace-score with --spend buys each item again and writes over the same file, after it asks on a terminal. score/ledger.json records what this project has already bought, keyed by the request that bought it: the endpoint as the sound provider publishes it and the body it sends, and never base_url, so a project that moves to another host of the same service buys nothing again. An item whose request still matches its row and whose audio is still on disk is kept rather than bought again, and the ledger is written after every paid call, so a run stopped half way through a long piece keeps what it has and asks only for the rest. A planned item nobody has bought is a SOUND_MISSING warning against the path a mix would look for, and assemble reports the same code when it meets that path empty. The layer plays as silence and the film is still made, so decktalk build --no-spend on a project with a score exits 0, and --fail-on warning makes it fail. A music file, an ambience file or an effect file the project names and that no score item writes is the author’s own, so its absence is a FILE_MISSING error. That layer plays as silence too, and the build stops after assemble. An effect whose cue did not resolve is a sentence on the stream, and the sound is left out of the mix and out of the captions.

​
Ducking

The music ducks by [mix] music_duck_db under every span of the film that carries speech. At the default of -6 dB against a music_db of -24, the music plays at -30 dB there.
  • A spoken section carries speech from where it starts until the end of its last word, or until it ends when it says nothing at all.
  • A clip section carries speech for the whole of its own length, because its dialogue is its own.
  • Each edge of a span ramps over [mix] duck_ramp_seconds, which is 0.5 seconds by default.
The music therefore begins to rise in the last half second of speech and is at full level through the silence after it, and it stays down under a whole clip.
The music level from 3:14 to 3:25 of the video, across the end of section 4, the clip in section 5, and the start of section 6. The level sits at -24 dB, drops to -30 dB under each spoken span and under the whole clip, and comes back up in the short silence after the last word of section 4.The music level from 3:14 to 3:25 of the video, across the end of section 4, the clip in section 5, and the start of section 6. The level sits at -24 dB, drops to -30 dB under each spoken span and under the whole clip, and comes back up in the short silence after the last word of section 4.

The music rises as the speech of section 4 ends, then stays down under the clip.

The ambience bed is shaped the same way and for a different reason. It plays under the page sections that ask for one, it ramps over [mix] ambience_ramp_seconds, and it reaches [mix] ambience_pad_seconds past each edge of its section, so the room does not appear and vanish exactly on a cut.

​
The mix mechanics

The settings of [mix] set the shape of the mix, beside the files and the levels that are the table’s project content: the ramp of the music duck at each edge of a spoken span or a clip, the ramp and the reach of the ambience bed, and the ramps into and out of a marker’s mute and swell. Configuration lists each with its default and its range.

​
Markers

A markers file raises or mutes the music at spoken phrases, which gives the bed some structure without an edit to its audio. [mix] music_markers names the file.
media/markers.json
markers.json lists every key. A marker resolves exactly as a cue does, against the words of its own section, by the rules in Cues. Markers apply only where there is a music file. A markers file that is not there leaves the music with no structure and says so, a marker whose phrase is not spoken shapes nothing and says so, and a file that is not valid JSON fails the load with the file and the line named.

​
Loudness

​
A clip’s sound is matched to the voice

A clip is made somewhere else, at whatever level its maker left it, so each clip’s sound is matched to the narration before it joins the mix. assemble measures the integrated loudness of the joined narration and of each clip, and gains each clip by the difference, so a film playing in a clip section is as loud as the voice beside it and the loudness pass below lifts both by one gain. A narration that is a placeholder has no level of its own, so a clip is then matched to [audio] target_lufs, which is where the pass would bring a real narration. A clip with no sound is laid at unity.

​
The programme target

After the layers are summed, the loudness pass brings the whole mix to one programme target.
  1. It measures the integrated loudness, the true peak and the loudness range of the mix.
  2. It applies one plain gain, which is the distance from the measured loudness to [audio] target_lufs. A plain gain keeps the dynamics of the mix intact.
  3. It runs a true-peak limiter at 192 kHz, so an inter-sample peak becomes a sample the limiter can see. The limiter sits 0.3 dB under [audio] true_peak_max_dbtp, attacks in 5 milliseconds and releases in 50, which is long enough not to pump on speech.
  4. It measures the result again.
The defaults are -16 LUFS and a -1.5 dBTP ceiling, so the limiter acts on peaks above -1.8 dBTP. [audio] range_max_lu is the loudness range the measurement is made against, and its default is 11 LU. MIX_LOUDNESS reports what the pass could not fix. It is raised when the measured true peak is above the ceiling, and when the measured integrated loudness sits more than 1 LU from the target. It is a warning, because it is a measurement of a film rather than a fault in it, and a reader decides. A run with a placeholder take skips the pass. The placeholder is clicks and silence, and normalising them would move the clicks the offset check listens for.

​
Your own audio files

Point [mix] music, [mix] ambience and each [[mix.effect]] file at any audio file ffmpeg can read. A project that generates nothing needs no [score] table at all. Keep a file of your own at a path no [score] table writes to. An item whose ledger row does not match its request is bought again and written over whatever is at its output path, so a hand-made bed under a generated bed’s path is replaced the next time the prompt changes. decktalk init writes no [mix] sound keys and no [score] tables, so a new project makes no sound beyond its narration until you add them.

​
Next