No More Human Needed to Press the Stream Button — How to Create Unmanned Streaming on YouTube/Twitch

📝 Originally published (in Japanese) at forge.workstyle.tech.

Streaming software makes it easy to go live on YouTube or Twitch, but when you try to build the system yourself, you suddenly find yourself lost in a fog of terminology: RTMP, HLS, WebRTC, SRT, ffmpeg. Which one is used where?

This article summarizes the challenges, causes, and solutions encountered while building a system where an AI avatar starts an unmanned live stream at a set time, responds to comments with voice, and automatically ends with a closing message when the time is up. All humans have to do is register the program in advance; no one opens the streaming screen on the day of the stream.

Here's what happens in sequence:

When the program's start time arrives
  → Create a broadcast using the API
  → Bind to the RTMP stream
  → Start the GPU Pod and send out the video
  → Transition to live
  → Closing message when the time is up
  → End the stream, destroy the Pod
  → Archive (VOD) remains

This process runs unmanned on both YouTube and Twitch. First, we'll outline the overall layout, then dive into specifics like simultaneous streaming, API automation, pitfalls in automatic start triggers, termination handling, and latency.

Note that platform protocols and API specifications change, so check each company's latest documentation before implementing. Here, we'll focus on the structural role division and the actual pitfalls encountered.

Fundamental Principle: Live Streaming is Divided into Three Segments

[1. Production]        →      [2. Ingest]      →      [3. Delivery]
 Create video and audio          Deliver to platform      Deliver to viewers
 OBS / Browser / Camera      RTMP, etc.              HLS, etc.
 ~Local              You → Provider            Provider → Viewers

These three segments use completely different technologies with distinct requirements. Much confusion arises from blurring the distinctions between these segments while discussing terms.

You can only choose segments 1 and 2. Segment 3 is the platform's domain and cannot be controlled externally. This is why latency cannot be reduced beyond a certain point, as discussed later.

Why RTMP is Still Used for Ingest

RTMP is a technology from the 2000s. It remains the standard for ingest because the recipients support it:

• Major platforms provide RTMP endpoints (rtmp:// URLs and stream keys)
• Streaming software and libraries have mature implementations
• It's simple to implement since it only involves "connecting via TCP and sending packets"

Newer protocols like SRT and RIST offer better loss resilience, and WebRTC-based ingest (WHIP) is emerging. However, if the recipient doesn't support it, it's useless. In practice, the most mature and widely supported option is chosen, which is RTMP.

One critical point to note is that RTMP doesn't report failures. Even with an invalid stream key, the connection is accepted, and data is silently discarded. The sender cannot distinguish success from failure. Therefore, you must externally verify the receiver's status. This was a significant pitfall (related to the external status verification discussed later).

Where Does WebRTC Fit In?

WebRTC is designed for sub-second bidirectional communication, ideal for video conferencing.

It's not the primary path for large-scale live streaming. Its peer-to-peer structure for each viewer doesn't scale well, and it doesn't integrate with CDN distribution mechanisms (while specialized services offer WebRTC-based low-latency streaming, it's not the main path for general platforms like YouTube or Twitch).

Does that mean it's unused? We used it internally:

[Server] AI voice generation
    ↓ WebRTC (low-latency, bidirectional)
[Browser] Avatar lip-syncs and plays audio
    ↓ Capture screen and audio
[ffmpeg] Encode
    ↓ RTMP
[Platform] → HLS → Viewers

WebRTC is used within production, while RTMP handles ingest. WebRTC's low latency is leveraged between the server and browser, and the output is switched to a scalable delivery mechanism.

Instead of choosing between WebRTC and RTMP, we used the right tool for each segment.

What Does ffmpeg Do?

ffmpeg is often described as a "video conversion tool," but in streaming pipelines, it serves three simultaneous roles:

1. Encoding

Raw video and audio are compressed into deliverable formats like H.264/AAC. This is the most CPU-intensive task and was the bottleneck for simultaneous streams in our setup. Hardware encoders (NVENC, etc.) are used when possible.

2. Multiplexing (mux)

Video and audio are combined into a single stream. A/V sync is determined here. When video and audio arrive via separate paths, you must decide which timestamp to use as the reference.

3. Distribution (tee)

Encoded packets are duplicated to multiple outputs. Simultaneous streaming to YouTube and Twitch is achieved with just this feature. Since encoding happens only once, adding more destinations barely increases CPU load. This distribution is key to the next section on simultaneous streaming.

Simultaneous Streaming to YouTube and Twitch — One ffmpeg Instance is Enough

To stream an AI avatar to both YouTube and Twitch, a naive approach would be to run two renderers, each sending to a different platform. This doubles GPU usage and encoding load.

This is unnecessary. ffmpeg's tee multiplexer duplicates encoded packets to multiple outputs:

Page (video + audio)
  → ffmpeg (encoding happens once)
      → tee ─┬→ rtmp://a.rtmp.youtube.com/live2/<key>
              └→ rtmp://<ingest>.twitch.tv/app/<key>

The command looks like this:

ffmpeg <input specifications> \
  -c:v libx264 -c:a aac <encoding settings> \
  -f tee -map 0:v -map 1:a \
  "[f=flv:onfail=ignore]rtmp://a.rtmp.youtube.com/live2/KEY1|[f=flv:onfail=ignore]rtmp://INGEST.twitch.tv/app/KEY2"

Two key points:

1. Encoding happens once

tee duplicates the encoded stream, so the CPU-intensive H.264 encoding occurs only once. In our setup, the bottleneck was the CPU, not the GPU, so this optimization was effective. Adding more platforms barely increases CPU load.

Conversely, both platforms receive the same quality and bitrate. If you need different resolutions per platform, this method won't work. We used 720p30 uniformly, so it was fine.

2. onfail=ignore allows partial failure

This is crucial. By default, if one output fails, ffmpeg stops entirely. So, if the Twitch connection drops, the YouTube stream also stops.

With onfail=ignore, failed outputs are disconnected, and the rest continue. If one platform has issues, viewers on the other are unaffected. In streaming, "partial failure" is always preferable to "complete failure," so this option is essential.

Platform Differences Appear in "Peripherals," Not Video

Even with identical video, peripheral handling differs:

While video ingest can be unified, peripheral control requires platform-specific code. We normalized events into a common internal format, routing them to a single response logic layer. This isolates platform differences in the ingest and event collection layers, making the response logic platform-agnostic.

Twitch lacks a broadcast concept, so the avatar must be connected for the stream to go live. Unlike YouTube, where a "created but no video" state exists, Twitch failures manifest as "nothing happens." Subsequent broadcast-related discussions focus on YouTube's asymmetric design.

Creating YouTube Broadcasts Without Human Intervention

YouTube requires controlling "broadcast creation and live transition." First, clarify credential boundaries.

Start by Clarifying "API Key vs. OAuth"

Misunderstanding this wastes time. YouTube Data API v3 operations require different credentials:

Initially, we only implemented viewer comment reading, so an API key was enough. Adding automatic broadcast creation required OAuth, a significant hurdle. Practically, any write operation to a channel requires OAuth.

OAuth involves obtaining a refresh token once and using it to update access tokens. This requires one-time human interaction, the only exception to "no human intervention."

Creating Broadcasts

Use liveBroadcasts.insert to create a broadcast. Settings here define the stream's nature:

• Visibility: Set to private during testing. Switch to public only with human approval
• Auto-end: Enabled to close the broadcast if the stream disconnects, reducing cleanup
• Auto-start: Disabled (due to trigger pitfalls, detailed next)
• Low-latency mode (latencyPreference): For interactive streams, viewer latency directly impacts experience, so minimize it

RTMP streams are reused, with IDs resolved via API. This avoids embedding stream keys in settings, simplifying key rotation.

AI-Generated Content Disclosure Flag

This is more about responsibility than implementation.

YouTube requires flagging synthetic media (AI-generated content that could be mistaken for real). This flag is set during broadcast creation. Since our stream uses AI for voice and responses, we enable this flag.

In automated systems, such disclosures might seem optional. However, automated systems should adhere to the same checks as manual ones. Platforms don't waive responsibilities for automation. As more flags emerge, grouping "disclosure items" near broadcast creation code simplifies additions.

Spec-Compliant Code, But the Stream Doesn't Start

Creating a broadcast, sending video via RTMP, and enabling enableAutoStart should automatically start the stream when video arrives—or so we thought.

It didn't. Video arrived (active stream status), the broadcast existed, but the status remained ready.

The cause:

enableAutoStart doesn't trigger for broadcasts bound to already active streams.

Why This Order?

Our setup reused streams (RTMP endpoints and keys), creating broadcasts per program:

1. Create broadcast with `liveBroadcasts.insert`
2. Bind to existing stream with `liveBroadcasts.bind`
3. Start renderer Pod and begin RTMP ingest

If the stream was already active (from previous programs or tests), binding in step 2 means YouTube sees an "already streaming" stream with a new broadcast attached.

Auto-start likely triggers on the transition from inactive to active. Binding to an already active stream doesn't trigger this transition, so it fails.

Viewer URLs kept changing, requiring constant resharing. Initially, we tweaked insert parameters, but the issue was event ordering.

Solution: Explicitly Trigger Transitions Instead of Auto-Start

We abandoned auto-start:

enableAutoStart = False
monitorStream   = Disabled

Instead, the scheduler explicitly calls liveBroadcasts.transition:

On stream `active` confirmation
  → Call `transition(broadcastStatus='live')`
  → Retry on failure

Retries are needed because there's a lag between stream activation and YouTube readiness. The first attempt often fails, but a retry after a short wait succeeds.

Explicit state-based actions are more reliable than event-driven ones. The former relies on observable states, while the latter depends on internal platform states. "Automagic" features often lack documented triggers, making explicit control vs. automation a reliability design choice.

Additional Pitfalls

403 errors on bind. liveBroadcasts.bind sometimes returned 403 errors, not permanent permission issues but temporary timing issues. We solved this with staged retries:

Retry after 20s → 40s → 60s (max 4 attempts)

In this domain, 403 doesn't always mean "permission denied." Treating temporary failures as permanent ones leads to unnecessary abandonment.

Silent streams are treated as audio-less. During failure recovery, restored state snapshots sometimes skipped greetings, resulting in silent starts. YouTube treats these as audio-less streams, undesirable for live broadcasts. Platform health must be considered, not just application logic, when deciding whether to output audio.

Focus on Termination Handling

Automated streaming often overlooks termination. Starting is motivated, but ending is not—and unterminated streams incur ongoing charges.

We implemented:

• Timed normal termination: End with a closing message when time's up. Allow message completion before closing
• Live transition timeout: If the stream doesn't go live within a set time, treat it as failed, close the broadcast, and destroy the Pod. Without this, GPU charges continue for non-functional streams
• Post-termination verification: Confirm GPU Pods are truly destroyed after stream end

1. Post-termination verification is a routine check. Don't trust "should be terminated." We've avoided several instances of unnoticed lingering Pods during testing. Automated streaming is harder to ensure termination than to start. Unstarted streams are noticed immediately; unterminated ones are noticed in bills.

Honestly, archive (VOD) status checks post-stream still return 403 with API keys, leaving OAuth implementation or warning-based operation as pending decisions. Even read operations require OAuth if accessing non-public channel data.

Where Does 15–30 Second Viewer Latency Come From?

Interactive streams are sensitive to viewer latency. Here's the breakdown:

The dominant factor is segment 3, the platform's domain. HLS delivery involves creating segments (multi-second chunks) for CDN distribution, inherently introducing multi-segment latency. This latency is traded for scalability.

Low-latency modes (latencyPreference) reduce this but don't eliminate it. Viewer latency is beyond our control, so optimize all reducible settings and design UX assuming 15–30 second latency.

We shifted focus from "reducing latency" to minimizing unresponsive periods. Instantly respond to comments with pre-synthesized acknowledgments. Total time remains unchanged, but the experience is vastly different. Design UX assuming unreducible viewer latency.

End-to-End Testing Confirmations

Implementation alone isn't enough; we tested with actual programs:

• Both platforms went live simultaneously
• Comment responses appeared on both
• Closing messages played, and both streams ended
• GPU Pods were automatically destroyed

One issue was external status verification. Twitch's public live status endpoint is cached, returning offline immediately after stream start. This led to false failure detections.

Poll status until it stabilizes for start and end. Single checks are unreliable. Since RTMP doesn't report failures, repeatedly verify receiver status externally.

Key Takeaways for Automation

Design choices effective for unmanned streaming:

• Segment-based thinking. Live streaming comprises production/ingest/delivery. Only the first two are controllable
• Encode once, distribute with tee. Adding platforms barely increases load (onfail=ignore is essential). All outputs share the same quality
• Channel writes require OAuth. Reads use API keys. Clarify this early
• Flag AI-generated content. Automation doesn't exempt platform responsibilities
• Explicit state-based actions over event triggers. enableAutoStart fails for already active streams. Explicitly call transition
• Treat temporary failures as transient. Retry bind 403s with staged delays (20→40→60s, max 4 attempts)
• Independently manage stream lifecycle (start/continue/end). Unterminated streams incur charges. Include live transition timeouts and post-termination verification
• Avoid client-side (browser) state. Treat pages as disposable
• Repeatedly verify receiver status externally. Sender logs don't confirm success. Poll for status, considering caching
• Accept 15–30 second viewer latency as unreducible. Design UX accordingly

Confusion arises from mixing segment-specific terms. Separating segments clarifies choices. The hardest parts of unmanned streaming are ensuring termination and avoiding dependence on platform internal states. Unstarted streams are noticed immediately; unterminated ones and silently dropped ingest are noticed in bills and misdiagnoses.