π Originally published (in Japanese) at forge.workstyle.tech.
TL;DR The Electron streaming window reached 492% CPU while ffmpeg used 28%. The recording size was unset, so capture used the Retina display resolution, and MediaRecorder plus ffmpeg encoded the video twice. Passing size and frame rate limits and copying the recorded format reduced recording-side CPU usage to about one-thirtieth.
The Cause: Recording Size and Double Encoding
There were two main causes.
The first was that I hadn't specified the size for the screen recording. The code was getDisplayMedia({ video: true }). Although the settings screen displayed 1280Γ720 at 30fps, those values were not actually being reflected in the recording.
Without a specification, it was recording at the native Retina resolution. The screen size was something like 3456Γ2234. For someone like me who was estimating streaming load based on the settings values, this was a major blind spot. Even though I intended to record a small window, I was actually capturing a massive display.
The second cause was that the video was being encoded twice. MediaRecorder encoded the video to VP8 using the CPU, and then ffmpeg converted that video into H.264. The device's VideoToolbox encoder was only being used on the ffmpeg side. The VP8 encoding for the recording remained stuck on the CPU.
Even the recording phase, which happens before the streaming, involves heavy encoding work. By measuring the processes separately, I was able to identify exactly where the load was occurring.
Deciding the Recording Format First
In the revision, we first passed the size and frame rate to getDisplayMedia with max. This informs the recording side of the upper limit.
Next, when using MediaRecorder, we try H.264 first. If the device's encoder can record in a usable format, we don't need to re-create the video with ffmpeg. The actual recorded format was video/mp4;codecs=avc1,mp4a.40.2.
In that case, ffmpeg only needs to re-wrap the video with -c:v copy. Here, specifying -b:v or -r would result in re-creating the video after all. If we're just copying, there's no need for extra specifications.
However, we can't know what format MediaRecorder can record in until we check the window side. So, we've changed it to launch ffmpeg after receiving the format notification from the window.
When changing the order, we first investigate the reason why it can be stopped. If it's due to ffmpeg being unavailable or a key being missing, we can inform the user before opening the window. If we open the window first and then display "cannot send", it will appear to the user as if the stream is being transmitted.
The recording side is about 1/30
After modification, I measured it 4 times at 20-second intervals on the same Mac. The CPU usage rate on the recording side was 14.8%, 16.3%, 19.6%, and 19.2%. ffmpeg was at 0.1%. The load average was 3.28, 3.48, 3.27, and 4.16.
The load on the recording side has become about 1/30. The load average of 3 to 4 includes the load of the application itself, such as LLM and embedding. The load of the distribution itself is almost negligible.
Let me note one thing. The modified episode was after the distribution slot of the receiver ended, so I haven't reconfirmed the reach to YouTube. The reach was confirmed in the episode before the modification. At that time, I distributed for 9 minutes and 30 seconds, the image quality was "very good" (ιεΈΈγ«θ―γ), and the bitrate was 3064 kbps. The fact that the load decreased after modification and the fact that the reach after modification was confirmed are separate.
When Load is Too High, the App Automatically Stops Streaming
Even if the load is reduced, operational concerns remain. The biggest issue this time was that users couldn't stop the stream. I was able to stop it through another method, but users didn't have the same option.
To address this, I implemented a mechanism where the app itself stops streaming when the load is too high. During streaming, the app checks the average load over the past minute every 10 seconds. If the load exceeds the number of CPUs Γ 2.5 for three consecutive checks (approximately 30 seconds), the app automatically stops the stream.
It won't stop if the load spikes momentarily. If the load drops even once, the counter resets. When stopping, the app also displays the reason and how to fix it. For example, it might suggest reducing the recording resolution.
By the time the app becomes too slow for users to operate, asking them to stop it manually is already too late. Detecting anomalies and stopping the stream should be part of the streaming feature's design.
Key takeaways
- Set size and frame rate limits in
getDisplayMedia. - Check the format actually produced by MediaRecorder before starting ffmpeg.
- When the format can be copied, use
-c:v copyand avoid settings that re-encode the video. - Monitor load during streaming and stop automatically if high load persists.
