What Actually Happens When Android's Hardware Encoder Hits SVC

A livestream can look perfectly healthy and still get flagged as broken. That's the situation I kept running into on Android, and it took a while to figure out why.

I work on the video team at Stream, where I spend most of my time on the Android SDK. Video calls and live streams on Stream use a system called scalable video coding (SVC), which lets a phone encode video once and lets the server deliver different resolutions to viewers based on their bandwidth. It's an efficient design, especially for mobile, where every extra encode costs battery and CPU.

The setup I want to walk through here started simple, with one publisher, one viewer, and everything worked as expected. Then a second viewer joined, and the quality score on that stream dropped and never recovered. Nothing about the stream had visibly changed.

That single symptom sent me down a debugging path with two convincing false leads before I found the actual cause, which turned out to be a limitation baked into Android's hardware video encoders themselves.

This piece is about what that debugging process looked like.

What SVC actually is

To back up for a second on how SVC actually works: a single encoded stream carries multiple layers inside it. Spatial layers are different resolutions, something like 180p, 360p, and 720p. Each spatial layer also carries temporal layers, which are different frame rates, something like 7.5, 15, and 30 fps.

The publisher encodes all of this once. The server, in Stream's case the SFU, looks at each subscriber's bandwidth and forwards only the layers that subscriber can handle.

Simulcast solves the same problem differently.

The publisher encodes three separate streams at three separate resolutions, and the server picks which stream to forward. It works, but it costs three encoders' worth of CPU and battery on the sending device. On mobile, where compute and battery are both limited, SVC is the better fit, assuming the encoder can actually produce the layers it's asked for.

That assumption turned out to be the entire problem.

The setup that worked

Here's where the debugging starts.

One publisher on Android, one subscriber on the other end. The SFU tells the publisher to send a single spatial layer with three temporal layers inside it, a mode we call L1T3. The hardware VP9 encoder handles this without issue. Network conditions were good, the stream looked good, and the SFU scored it well. This was the baseline.

The moment it broke

Then a second subscriber joined the call. With two subscribers now needing different resolutions, the SFU asked the publisher to switch modes, this time requesting three spatial layers and three temporal layers, a mode called L3T3_KEY.

As soon as that switch happened, the SFU's quality score for the stream dropped and never recovered.

The obvious thing to check first was the SFU's scoring system, which grades every published stream. This stream scored badly the moment the encoder switched configurations, and stayed bad for the rest of the call.

Something about producing three layers instead of one was breaking the stream, but at that point I had no idea what.

False lead #1: the encoder thrashing loop

The first thing I noticed in the logs was that the hardware encoder was getting torn down and recreated over and over, roughly every 200 milliseconds. That kind of churn is never good, so I dug into why it was happening.

Android's hardware VP9 encoder can't change its resolution or its buffer type while running. Any time WebRTC's quality scaler asks it to adjust resolution, the encoder has to shut down completely and start over, rather than adjusting in place. Software encoders don't have this restriction. Hardware encoders do.

Here's the loop that creates.

The encoder starts at 720p. It gets torn down and rebuilt for buffer type reasons. When it comes back up, it has a pile of frames waiting, which throws off the bandwidth estimator's read on the connection. The quality scaler decides 720p isn't sustainable and drops the encoder to 540p. Now the quantization parameter is calibrated for 720p bitrate but running at 540p, so the scaler tries to climb back to 720p. That triggers another full teardown and rebuild. The whole cycle repeats every five to ten seconds, and what you see as a viewer is the publisher's resolution oscillating between 720p and 540p.

I confirmed this by turning off the quality scaler entirely, using WebRTC's degradation preference setting. That stopped the resolution oscillation completely. But the SFU still scored the stream as poor. The thrashing loop was real, and we fixed it, but it wasn't the reason the stream kept getting marked as low quality.

False lead #2: the phantom bitrate

The next clue came from the stats. WebRTC reported the stream's target bitrate at around 71 kbps, while the encoder was clearly outputting close to 1.4 Mbps at 720p. That gap looked like it explained the quality problem on its own.

It turned out to be a reporting quirk in libWebRTC, not a real problem.

WebRTC calculates target bitrate using an index that assumes simulcast, where three separate streams each have their own SSRC (a unique stream identifier), and the target bitrate for a given layer comes from summing values across those indexes. In SVC, there's only one stream and one SSRC, so the index is always zero. That means WebRTC's target bitrate calculation always lands on the base layer's slice of the total bitrate, around 70 kbps, regardless of what the encoder is actually sending.

The wrong number comes from SendStatisticsProxy in libWebRTC. It maps each SSRC to a simulcast index, then reports the bitrate of that spatial layer as the target bitrate.

// send_statistics_proxy.cc
// Store target bitrates per substream stats.
for (auto& [ssrc, substream] : stats_.substreams) {
  std::optional<size_t> simulcast_index;
  for (size_t i = 0; i < rtp_config_.ssrcs.size(); ++i) {
    if (rtp_config_.ssrcs[i] == ssrc) {
      simulcast_index = i;
      break;
    }
  }
  if (!simulcast_index.has_value()) {
    substream.target_bitrate = std::nullopt;
    continue;
  }
  substream.target_bitrate =
      DataRate::BitsPerSec(allocation.GetSpatialLayerSum(*simulcast_index));
  if (substream.target_bitrate == DataRate::Zero()) {
    substream.target_bitrate = std::nullopt;
  }
}

The function it calls, GetSpatialLayerSum, adds up all the temporal layers on one spatial layer.

// video_bitrate_allocation.cc
// Get the sum of all the temporal layer for a specific spatial layer.
uint32_t VideoBitrateAllocation::GetSpatialLayerSum(
    size_t spatial_index) const {
  RTC_CHECK_LT(spatial_index, kMaxSpatialLayers);
  return GetTemporalLayerSum(spatial_index, kMaxTemporalStreams - 1);
}

This is correct for simulcast. SSRC 0, 1, and 2 map to index 0, 1, and 2, so S0, S1, and S2 each get their own target bitrate.

It is wrong for VP9 SVC. There is only one SSRC, so simulcast_index is always 0. The stat becomes GetSpatialLayerSum(0), which covers the S0 layer only. S1 and S2 never count toward the target bitrate. On L3T3, that S0 slice is the ~71 kbps I saw.

Once I understood the formula, the mismatch made sense.

The encoder was sending the full bitrate as intended. The stats weren't built to report it correctly for a single-SSRC stream. Another dead end, but a useful one to know if you're ever debugging SVC stats and the numbers don't add up.

The real root cause

With both of those explanations ruled out, I checked whether the encoder was actually producing three layers in the first place.

It wasn't.

The hardware VP9 encoder was only ever producing one layer, the highest one requested, and giving that single layer the entire bitrate. It never produced the two lower spatial layers. The SFU expected three layers because it had asked for L3T3, got one flat 720p stream instead, and marked the stream as poor, which was the accurate read of what was happening.

To rule out a device-specific issue, we tested across Samsung and Pixel devices, covering both Qualcomm and Tensor chipsets. Every device showed the same behavior. Hardware VP9 encoding on Android has this limitation across the platform, confirmed on every device and chipset we tested, regardless of brand.

Software VP9, running through libvpx, doesn't have this problem. It produces all three spatial layers correctly and doesn't need to tear itself down to change resolution. The tradeoff is CPU and battery, which is a real cost, and the reason nobody wants to run software encoding by default in the first place.

What this looks like at scale

To see how much this actually mattered outside of a controlled test, we looked at real production data from one of Stream's livestreaming customers using VP9 across platforms.

In that data, Android streams using the hardware VP9 encoder were consistently scored as poor. Streams using the VP9 software did noticeably better, and the remaining problems stemmed from CPU and battery limits rather than missing layers.

Neither path fully solves the problem.

Hardware can't produce the layers SVC needs. Software can, but not without a real device cost. And because quality scores show up in the UI as network indicators, most people watching would assume the publisher had a bad connection.

In most of these cases, the network was fine. The hardware just wasn't capable of what was being asked of it.

Can you detect this ahead of time?

Given all of this, the natural next question is whether you can detect this ahead of time and avoid asking a device to do something its encoder can't handle.

Android's MediaCodec APIs let you query a fair amount about a given hardware encoder: supported resolutions, bitrates, profile levels, and temporal layering schemas. Android 17 (API 37) added a new method, getSupportedLayeringSchemas(), which sounded promising. In practice, it only returns temporal schemas like webrtc.svc.l1tN. There's no API that reports whether a hardware encoder can produce spatial layers.

That means there's no way to check in advance whether a given device's hardware VP9 encoder actually supports the SVC mode you're about to ask it for. The only way to find out is to ask it and inspect the resulting bitstream.

Where this leaves us: broadcast mode

Given that neither hardware nor software VP9 fully solves this on Android, we've been building an alternative approach on the SFU side, which we're calling broadcast mode. It's still in development and not yet in production.

The idea is to stop asking the publisher's device to produce multiple layers at all. Instead, the publisher sends one clean stream at whatever hardware codec and resolution the device handles best. The server then transcodes that single stream into the quality ladder viewers need, generating the lower resolutions server-side instead of on the device.

This removes the CPU drain, the missing spatial layers, and the encoder recreation problem, since the phone only has to do one thing it's already good at. The tradeoff moves to the server, where transcoding adds real compute cost. That's not something we'd turn on for every use case, but it fits broadcast-style setups with one publisher and many viewers well, which is exactly the case where this problem shows up most.

We're planning to make this available as an opt-in SDK option for livestream and broadcast use cases once it's ready.

Takeaways

A few things I'd want any developer building live video on Android to take away from this.

Hardware VP9 on Android only supports temporal SVC. Spatial layers are silently dropped, not degraded, dropped.
Software VP9 gives you real SVC, but the CPU and battery cost is real too.
There's currently no API on Android that lets you check spatial SVC support ahead of time. You find out from the bitstream itself.
If you're seeing SVC quality problems on Android that seem to show up only once more than one viewer joins, check this first.