This is a sanitized engineering note about server-side audio noise reduction for WebRTC calls.
Source article:
https://www.lodan.me/posts/server-side-webrtc-noise-reduction-pion-ffmpeg-rnn/
What the prototype tests
The goal is not to replace WebRTC's built-in audio processing. The narrower test is:
- receive a WebRTC Opus track with Pion
- read RTP packets in
OnTrack - decode Opus payloads to PCM
- pipe raw PCM into FFmpeg
- apply the
arnndnRNN noise reduction filter - validate the output as a file before considering real-time forwarding
Why this boundary matters
RTP, Opus, PCM, and FFmpeg raw audio input are different boundaries. If the PCM format is wrong, FFmpeg may still produce a file, but the result should not be trusted.
For example, if the Go side writes int16 PCM, the FFmpeg input format should be reviewed as s16le, not casually treated as s32le.
Production concerns
The prototype is useful because it isolates the audio path, but production use needs more work:
- buffering and latency
- CPU and memory isolation
- FFmpeg process lifecycle
- model choice
- packet loss and jitter
- RTP timestamps
- audio/video sync
- whether the processed audio is returned to WebRTC or only recorded
The full article has diagrams and the longer explanation:
https://www.lodan.me/posts/server-side-webrtc-noise-reduction-pion-ffmpeg-rnn/
