Build Dynamic NPC Dialogue With AI Agents and Local Voice

The hard part of dynamic NPC dialogue is not getting an agent to produce a witty tavern keeper. The hard part is making the fiftieth rewrite behave like the first: the same character gets the same approved voice, changed lines are easy to find, and an audio asset can be traced back to the source record that made it.

Vois fits the local build lane, not the game runtime. Its desktop project workflow and CLI give a production agent a controlled way to inspect the approved cast, generate reviewed lines, and export stable assets without making a player's game depend on a live writing agent or cloud voice service.

That calls for a small production contract. Let an agent write structured dialogue. Let validation reject bad records. Let a human approve casting and content. Then let Vois turn the accepted records into files. A game should receive audio and a manifest, not an open-ended conversation with an agent.

Keep the agent, voice app, and game runtime in separate lanes

An AI agent is useful for drafting branches, barks, ambient lines, and alternate reactions. It is not a reliable asset registry. Give it a schema and a scene brief, then inspect the data it returns before anything is voiced.

Vois belongs in the build lane. The bundled CLI controls the running desktop app locally. It is a remote control for the installed app, not an independent voice service and not something the player needs to run.

Prompt your agent

I am preparing a local NPC voice build for [GAME], scene [SCENE ID], and locale [LOCALE]. Vois must be open on this machine.

First, verify that the Vois desktop app is available. Then retrieve the complete voice list as structured data and inspect only the approved cast manifest I provide. Do not draft, generate, export, change a cast assignment, or use a fallback renderer.

Return a preflight report with: app availability; scene ID and locale; each stable character ID; the exact matching Vois voice ID; the selected engine; any missing or unsupported cast binding; and a clear "ready for dialogue review" or "blocked" decision.

Review and approval flow

1. Give the agent the reviewed cast manifest, not a free-form request to choose voices.
2. Confirm that each returned exact voice ID matches the approved character ID and that the selected engine suits the target language.
3. Resolve every missing voice, app-status error, or plan limitation before asking for dialogue or generation.
4. Keep the resulting preflight report with the scene's source records.

For an agent that needs command details, point it at the website-hosted Vois CLI skill. The skill documents structured output, projects, scripts, voice lookup, generation, and export without freezing a hand-maintained command list inside your game workflow.

Make dialogue a data contract, not prose with loose labels

Ask the writing agent for structured scene data, then validate it before converting it to a Vois script. A stable character_id must survive renamed characters, localized display names, and prompt rewrites. A line_id must survive a re-run so a changed record has a clear target.

At minimum, the scene record should identify the scene, revision, locale, characters, and lines. Each character needs a stable character ID, human-facing display name, and approved exact Vois voice ID. Each line needs a stable line ID, character ID, intent, and spoken text. Keep the durable ID separate from the display name: if Mara becomes Keeper Mara, innkeeper_mara and its approved voice binding do not change.

Prompt your agent

Draft dialogue for scene [SCENE ID], revision [REVISION], locale [LOCALE]. Return structured records, not prose.

For every character, preserve the stable character ID from the approved cast manifest and return the matching display name only as presentation. For every line, return a unique stable line ID, character ID, intent, text, locale, scene revision, and review status set to "draft". Do not invent a character ID, select or change a voice, emit executable code, or mark any line approved.

After the records, return a validation summary: duplicate line IDs, unknown character IDs, empty text, missing locale or revision, and the number of draft lines by character. If any required field is missing, stop and list the bad records instead of repairing them silently.

Review and approval flow

1. Compare the returned records with the cast manifest. Reject any record that changes a stable ID or introduces an unapproved character.
2. Read the lines by character and scene, checking personality, repetitions, spoilers, player safety requirements, and localization context.
3. Change the review state to approved only for lines the narrative and audio owners accept. Keep rejected and superseded records rather than overwriting them.
4. Record a new review state for every post-approval text change, then send only approved records to the Vois build.

Require at least these checks before a record reaches generation:

• every line_id is unique within the release set
• every character_id exists in the cast manifest
• text is present, nonempty, and within the project's editorial limits
• the locale, scene revision, and content status are explicit
• a line that changes after review gets a new review state, not a silent regeneration

Structured data is useful because machines can parse it predictably, but it is not validation by itself. Use schema validation or equivalent checks in your build workflow. The JSON standard defines the interchange format; your project defines what a valid NPC line means.

Version the cast separately from the dialogue

Do not ask the agent to choose a new voice every time it writes a line. Instead, maintain a reviewed cast manifest that maps stable character IDs to the exact IDs returned by Vois. Library IDs follow the category_slug/voice_slug pattern; cloned and designed voices use the exact saved IDs returned by the app.

Prompt your agent

Using the approved cast manifest for [GAME], inspect the complete Vois voice list as structured data. Match only the existing exact voice IDs in the manifest to their stable character IDs.

Return a cast-review report with: character ID; display name; exact voice ID; voice source; supported engines; selected engine; cast owner; approval date; and a status of matched, missing, or needs review. Do not replace an ID with a similar-sounding voice and do not alter the cast manifest.

Review and approval flow

1. Audition a representative line for each new or changed character before approving a cast revision.
2. Approve dialogue changes through the text workflow and cast changes through this report. They are different decisions.
3. Save the approved cast revision with the scene release. The agent may read the character brief to write in character, but it must not overwrite the manifest.

This boundary means a dialogue revision changes text, while a casting revision changes voice_id. Reviewers can approve those separately, and a designer can hear an entire character set before the agent creates hundreds of variants.

Batch review before you batch-generate

A line that is syntactically valid can still be wrong for the game. Build a review sheet or dashboard from the JSON with line ID, scene, character, intent, text, voice ID, revision, and status. Group it by character and scene so reviewers can spot a broken personality, a repeated instruction, or an accidental spoiler.

Then make approval mechanical. For example, only records with status: "approved" enter the generation queue. Retain the rejected and superseded records in source control or an archive rather than overwriting them. When an agent runs again, compare line IDs and text hashes to identify new, changed, and unchanged records.

The CLI can return machine-readable generation results as well. Request JSON from the generation command and inspect the returned segment count, duration, speaker, voice ID, and segment indexes as part of the batch report. That gives a reviewer something concrete to investigate when a scene has fewer lines than expected or a character was rendered with the wrong cast binding.

Turn approved records into a Vois project and script

Vois scripts use readable speaker tags such as [Mara]: Dialogue text. The build should convert each approved record into that format using the character manifest. It must never treat a display name in an agent response as the authority for a voice binding.

Prompt your agent

The attached scene records and cast manifest have passed review. Build a Vois project for [SCENE ID], but do not export or replace any game asset yet.

Create a script from approved records only. Render each record as a readable speaker-tagged line using the approved display name, while binding each speaker to the exact voice ID from the cast manifest. Select the engine deliberately for [LOCALE], rather than relying on the app's current selection.

Return a build report with: Vois project ID; script ID; source revision; approved line count; lines excluded because their status was not approved; stable character IDs and exact voice IDs used; selected engine; generated segment count; duration; speaker and segment indexes; and any error. Do not compile or publish an audio file until I approve the report.

Review and approval flow

1. Compare the excluded lines and generated segment count with the approved source record count. Investigate any mismatch.
2. Sample the audio by character and scene. Check that every speaker uses the approved voice, the chosen engine is correct for the locale, and the delivery fits the scene.
3. Approve the compiled scene only after narrative and audio owners accept the build report and sample review.
4. Retain the project ID, script ID, source revision, and approved build report with the asset release.

This order keeps the generation configuration in the build record and the agent's writing task separate from the runtime that produces the audio.

Give every shipped line a deterministic name

Games rarely consume one large scene file forever. Dialogue systems usually need a file per line or a manifest that maps dialogue events to files. Generate release assets from approved records with a filename derived from stable IDs, not display names or generated text.

Use a pattern such as locale/scene_id__character_id__sequence.wav. If a character's display name changes, the game lookup does not. If the line text changes, the path can stay stable while the manifest records its new source hash and revision. If your team needs immutable artifacts instead, append the revision or a short content hash and update the manifest deliberately.

Prompt your agent

The scene build for [SCENE ID] is approved for export. Produce release assets from approved records only.

Name every file with the pattern [LOCALE]/[SCENE ID]__[CHARACTER ID]__[SEQUENCE].wav. Create a release manifest that maps each line ID to its source revision, character ID, exact voice ID, locale, target path, source hash, and review status.

Return an export report with the total approved line count, generated file count, manifest path, any skipped or failed line IDs, and a list of output paths. Do not infer filenames from spoken text or overwrite a different scene's release without an explicit instruction.

Review and approval flow

1. Confirm that the agent's expected output count matches the approved line count before export.
2. Verify the manifest maps every dialogue event to the intended stable path and uses only approved voice IDs.
3. Spot-check the exported assets in the game, then approve the release manifest and archive the superseded version.

The game should resolve a dialogue event through that manifest. It should not guess filenames from natural-language text.

Treat cloned voices as an approval workflow

Voice cloning is not a shortcut around rights and consent. Before a cloned voice enters the cast manifest, obtain permission to use it for the intended game work, retain the scope and owner of that permission, and use a clear 10 to 15 second reference sample.

Prompt your agent

Prepare a Vois voice-cloning request for [CHARACTER ID]. I have supplied a clear reference sample and the documented permission scope.

Do not create the clone yet. Return a consent preflight record with: character ID; sample path; sample duration and clarity concerns; permission owner; approved use scope; proposed clone name; and the exact cast-manifest entry that would change. State every missing consent or production prerequisite.

Wait for my explicit approval before creating the clone. After approval, return the exact saved Vois voice ID and update only the reviewed cast-manifest entry. Do not use a clone outside its documented scope.

Review and approval flow

1. Confirm that the source performer or owner has approved the specific character and game use. A technical confirmation cannot establish rights.
2. Approve the preflight record, then audition the saved clone against a representative scene before it enters the cast manifest.
3. If permission changes, remove the clone from the manifest and rebuild affected assets with an approved alternative.

Keep the runtime boring

The finished game needs stable audio files and a lookup table. The content build needs a running desktop app, the CLI, source JSON, cast approvals, and human review. Mixing those two environments creates brittle release behavior and makes debugging much harder.

Keep the agent on the drafting side, keep Vois on the local generation side, and keep the game runtime on the playback side. That division gives each tool a job it can do consistently. More dialogue is then a production problem you can review and rebuild, not a runtime gamble.

For a broader walkthrough of agents driving a voice workspace, read how AI agents control the Vois voice studio. Then use the game dialogue workflow to set up a small cast and one reviewable scene. Get started when you are ready to run the local build, or review current plans.

Sources

• Vois CLI skill
• Vois CLI command reference
• RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format
• JSON Schema: Object reference

The Vois Team