Building Real-Time Visitor Calls with WebRTC

One of the most challenging parts of this project was enabling real-time communication between an unknown visitor at the door and the homeowner. I wanted the experience to feel like a normal phone call: when an unknown person or spoofing attempt was detected, the homeowner should receive an incoming call notification, accept it, view the live camera feed, talk to the visitor, and remotely unlock the door.

Why WebRTC?

My initial thought was to stream video through the backend using HTTP. While simple, it introduced unnecessary latency and bandwidth overhead.

Camera
  │
  ▼
Backend
  │
  ▼
Mobile App

For a security system, every second matters. I needed low-latency audio and video communication, which made WebRTC the obvious choice.

Architecture

The final architecture consists of three major components:

┌─────────────────┐
│ Flutter App     │
└────────┬────────┘
         │
         │ WebSocket
         ▼
┌─────────────────┐
│ Signaling Hub   │
│ (Go Backend)    │
└────────┬────────┘
         │
         │ WebSocket
         ▼
┌─────────────────┐
│ Door Peer       │
│ (Pion WebRTC)   │
└─────────────────┘

The signaling server is responsible for exchanging SDP offers, answers, and ICE candidates between the mobile application and the door device.

Signaling Layer

WebRTC cannot establish a connection on its own. Both peers need a way to exchange connection metadata before media can flow.

I implemented a signaling hub using WebSockets inside the Go backend.

The signaling flow looks like this:

Door Peer
    │
    │ Offer
    ▼
Signaling Hub
    │
    ▼
Flutter App
    │
    │ Answer
    ▼
Signaling Hub
    │
    ▼
Door Peer

Additional messages are exchanged throughout the session:

offer
answer
ice-candidate
call_request
call_accepted
hangup

The signaling server acts purely as a message router. Once the connection is established, media flows directly through WebRTC.

Building the Door Peer

The door side of the connection is implemented using Pion WebRTC.

The peer is responsible for:

Creating the WebRTC PeerConnection
Managing SDP offers and answers
Handling ICE candidates
Capturing video and audio
Streaming media to the mobile application

Door Peer
   │
   ├── PeerConnection
   ├── Video Track
   ├── Audio Track
   └── Signaling Client

Capturing Video

Interfacing directly with camera drivers would have added significant complexity. Instead, I leveraged FFmpeg as a media ingestion layer.

The video pipeline looks like this:

Laptop Camera
      │
      ▼
   FFmpeg
      │
      ▼
 VP8 Encoder
      │
      ▼
WebRTC Video Track
      │
      ▼
  Flutter App

FFmpeg captures frames from the webcam, encodes them as VP8, and feeds them into a WebRTC track managed by Pion.

Capturing Audio

Audio turned out to be significantly more difficult than video.

The final pipeline uses FFmpeg to capture microphone input and encode it using Opus, which is the standard audio codec used by WebRTC.

Laptop Microphone
        │
        ▼
     FFmpeg
        │
        ▼
   Opus Encoder
        │
        ▼
WebRTC Audio Track
        │
        ▼
   Flutter App

Receiving Audio from the Mobile App

The communication is fully bidirectional.

When the homeowner speaks through the mobile application:

  Flutter Microphone
         │
         ▼
       WebRTC
         │
         ▼
     Door Peer
         │
         ▼
      Speaker

Incoming audio tracks are received through Pion and played back on the door device.

NAT Traversal

A major challenge with WebRTC is that devices are typically hidden behind NAT routers.

To solve this, I configured a STUN server:

stun:stun.l.google.com:19302

The connection process becomes:

  Door Peer
     │
     ▼
 STUN Server
     │
     ▼
Public Address 
  Discovery
     │
     ▼
 Flutter App

This allows both devices to discover routable addresses and establish a direct peer-to-peer connection.

Call Flow

The complete call flow in the system is:

       Unknown Visitor
              │
              ▼
      Face Recognition
              │
              ▼
     Incoming Call Event
              │
              ▼
   WebSocket Notification
              │
              ▼
Flutter Incoming Call Screen
              │
              ▼
        User Accepts
              │
              ▼
 SDP Offer / Answer Exchange
              │
              ▼
   ICE Candidate Exchange
              │
              ▼
      WebRTC Connected
              │
              ▼
     Live Audio + Video

Lessons Learned

The most difficult part of implementing WebRTC was not creating the peer connection itself—it was handling the surrounding infrastructure.

Challenges included:

SDP negotiation
ICE candidate management
Audio codec compatibility
FFmpeg media pipelines
Session lifecycle management
Connection cleanup and recovery

Once these pieces were in place, the system was able to provide real-time communication with low latency, making remote visitor verification practical and responsive.

WebRTC Architecture in My Smart Door Project