Speech Speed Cannot Be Changed After Learning — Parameters Are Accepted but Ignored

📝 Originally published (in Japanese) at forge.workstyle.tech.

I'm creating voice models for different roles: narrator, call center, sales, and host. I want each to have a distinct speaking style.

Initially, I thought, "I'll train the model once, and adjust speech rate and intonation during synthesis." The synthesis API had a parameter that seemed to control speech rate.

But when I tested it, it didn't work.

Parameters Being Ignored

I measured the same model and text, varying only the speech rate parameter.

for kw in [{}, {"length": 1.15}, {"length": 1.35}, {"length": 0.85}]:
    wav = synth(model_id=MID, text=T, style="Neutral", **kw)
    sec, lvl, f0, rng = acoustics(wav)
    print(f"{kw}  Speech {sec:.1f}s  Rate {mora(T)/sec:.1f}")

{}                Speech 3.9s  Rate 5.3
{'length': 1.15}  Speech 4.0s  Rate 5.2
{'length': 1.35}  Speech 4.1s  Rate 5.1
{'length': 0.85}  Speech 4.0s  Rate 5.2

Whether set to 1.35 (slower) or 0.85 (faster), the speech rate remained between 5.1 and 5.3. This variation occurred with each synthesis, not due to the parameter's effect.

The API accepted the parameter, returned no errors, and produced audio normally. It received the parameter but didn't use it. The server-side implementation wasn't passing the parameter to the synthesizer.

"Can be passed" and "works" are different. Even if a parameter is documented, you must test it.

What Gets "Baked In"

If parameters can't change it, it's determined during training. Many elements are set by the training corpus.

Speech rate. If the corpus is spoken slowly, the model speaks slowly.

Intonation habits. If the corpus includes clips with drawn-out endings, the model will stretch endings even if the script doesn't specify it. One model turned "こんにちは" (hello) into "こんにちわぁ."

Emotional styles. If the corpus includes screaming or laughter, those styles are created. If not, they don't exist.

Sentence patterns. If the corpus uses polite forms like "〜です" and "〜ます," the model leans formal. If it uses casual forms like "〜だよ" and "〜じゃん," it becomes more casual.

Usage must be determined at creation. I redesigned the system with this in mind.

Defining Speech Profiles

I defined "speech profiles" (conv_style) for each use case, switching script sets and quality gate strictness. There are 14 profiles.

_CONV_STYLE_PATHS = {
    "news":         _SAMPLES / "conversational_news_texts.txt",
    "narration":    _SAMPLES / "conversational_narration_texts.txt",
    "support":      _SAMPLES / "conversational_support_texts.txt",
    "presentation": _SAMPLES / "conversational_presentation_texts.txt",
    "sales":        _SAMPLES / "conversational_sales_texts.txt",
    "counseling":   _SAMPLES / "conversational_counseling_texts.txt",
    "guidance":     _SAMPLES / "conversational_guidance_texts.txt",   # IVR, in-house
    "compliance":   _SAMPLES / "conversational_compliance_texts.txt", # Important notices
    "guide":        _SAMPLES / "conversational_guide_texts.txt",      # Tourism, exhibitions
    "mc":           _SAMPLES / "conversational_mc_texts.txt",
    "secretary":    _SAMPLES / "conversational_secretary_texts.txt",
}
# Add polite (reception), casual (VTuber, streaming), and mixed (general, default) for 14 profiles

Scripts reflect actual usage. For call centers: "Thank you for calling. This is the support center." For narrators: "Since our founding, we've been committed to quality."

Omitting Extreme Material for Business Use

Another decision was whether to include extreme acting material (screaming/laughter).

# Business profiles: exclude extreme acting material
BUSINESS_CONV_STYLES = {"polite", "news", "narration", "support",
                        "presentation", "sales", "counseling",
                        "guidance", "compliance", "guide", "secretary"}
# Note: mc (event host/commentator) includes extreme material = not in business set

def build_corpus_plan(conv_style=None):
    per_cat = int(os.getenv("EXTREME_PER_CAT", "8"))
    if (conv_style or "").lower() in BUSINESS_CONV_STYLES:
        per_cat = 0                       # Exclude all extreme material
    ...

This affects the number of styles a model has.

Business          = 12 styles
  Neutral, Joy, Excitement, Pride, Relief, Surprise,
  Fear, Sadness, Shame, Anger, Contempt, Disgust

Casual/Mixed/MC   = 17 styles
  Above 12 + Scream, Laugh, Cry, JoyBurst, Shock

This wasn't about audio quality but accident prevention. The runtime has a "react fully when excited" feature triggered by "Scream" in styles. Only voices with the Scream style react fully.

If a call center voice suddenly laughed, "Ahahahaha!" it would be problematic. By not including the style during setup, I prevent accidents regardless of runtime logic.

⚠️ Business-Like Profiles with Extreme Material

In the code above, mc is excluded from BUSINESS_CONV_STYLES. Event hosts need to be engaging, so extreme material is useful.

However, MC voices appear business-like. "Event host" is a customer-facing role, so it might be mistakenly assigned to a support desk. With 17 styles, including Scream, full reactions are triggered.

I confirmed this through testing.

I documented this exception and added a constraint to voice assignment: "Do not assign MC voices to customer support." Business-like appearance and extreme material don't align, so it can't be inferred from the name.

Adding Variations to the Same Voice

If profiles can't be changed, I create variations for each use. This was lighter than expected.

A voice's identity is determined by (caption, seed), independent of its profile (Voice Gacha Design). Thus, I can create another version with a different profile for the same voice.

INSERT INTO voice_design_jobs (name, caption, seed, progress) VALUES
  ('Shiori (Narration)', 'Calm, intelligent adult female voice...', 1042,
   '{"params":{"conv_style":"narration"}}'),
  ('Shiori (Call Center)', 'Calm, intelligent adult female voice...', 1042,
   '{"params":{"conv_style":"support"}}');

The caption and seed are identical; only the profile differs. Each takes 1-2 hours to create, so I can add more as needed.

One character has two variants: "Streaming (Casual, 17 styles)" and "Business (Polite, 12 styles)."

Accepting "can't change" shifts to "can add." While believing runtime adjustments were possible, I couldn't reach this design.

Impact: Voice Selection Also Needs Profiles

Since profiles are baked into the corpus, voice selection must consider them.

For "a call center voice," the catalog returns conv_style = 'support'. Older models without recorded profiles aren't considered.

Auditing the catalog, only 19 of 76 entries had recorded profiles (4 registration paths, 0 management screens). The rest were pre-profile models or reference audio registered elsewhere, with conv_style as NULL.

The idea of retroactively adding profiles arose, but I decided against it. conv_style records the training corpus, not the model's behavior. Labeling an old model as narration doesn't mean it was trained on narration material. This would fix a label-reality mismatch in the DB.

I marked them as "Profile Unknown," excluding them from automated selection. The UI shows "Usage Unrecorded." I treat this as a fact, not missing data.

Summary

• Receiving a parameter doesn't mean it works. Test critical parameters like speech rate.
• Identify what's baked into the corpus. Speech rate, intonation habits, styles, sentence patterns.
• Prevent accidents through design, not runtime logic. Without the Scream style, full reactions won't trigger.
• Exceptions can't be inferred from names. MCs seem business-like but have extreme material. Document and constrain them.
• Accepting "can't change" shifts to "can add." Creating profile variations for the same voice works well.
• Don't add labels that can't be verified later. Mismatched records harm automation.

Series: Mass-Producing Practical Voices from Diffusion TTS

This series documents designing voices from captions, creating training corpora, and mass-producing role-specific voices. This article is Part 2: Manufacturing.

← Previous: Stricter Quality Gates Keep Flat Takes Alive

→ Next: TTS Changes "Recording Room" Every Time

All 18 Articles

1. TTS Chosen for Quality Was Too Slow for Conversation
2. Voice Gacha Design
3. Machine Screening 24 Narrator Voice Candidates
4. Stricter Quality Gates Keep Flat Takes Alive 5. Speech Rate Can't Be Changed After Training ← You are here
5. TTS Changes "Recording Room" Every Time
6. One Rough Clip Ruins the Whole Style
7. Where Did the AI's Drawn-Out Ending Come From?
8. Allowed Characters List Was Trimming Japanese
9. Hallucination Code Only Failed During Hallucinations
10. Quality Gates Allowed a "3-Character" Quirk
11. Dropping Candidates for Fixable Flaws
12. Flaws Undetectable by Transcription
13. 70 Minutes of Training Material Lost in a Network Blink
14. From "ja" to "JP": Creating a Jabber Model
15. 4 Registration Paths, 0 Management Screens
16. Deployments Overwriting Each Other's Work
17. Thresholds for Unmeasured Metrics Always Fail

The insights are compiled in Mass-Producing Practical Voices from Diffusion TTS Manufacturing Pipeline.