Real-Time Multimodal AI Meeting & Sales Agent Suite
Autonomous Full-Duplex Google Meet & Zoom Voice Agent + Copilot
Full-duplex multimodal AI meeting agent and copilot that autonomously joins Google Meet/Zoom calls via MeetStream.ai, conducting live voice sales consultations with sub-400ms latency and real-time SSE lead scoring.
The Engineering Challenge
Enterprise sales consultations and discovery calls demand deep technical qualification, instantaneous objection handling, and synchronized CRM note-taking. High-value prospect calls frequently suffer from delayed follow-ups and unstandardized BANT scoring.
Core ProblemCascaded STT→LLM→TTS architectures experience 3+ second delays, catastrophic audio buffering during turn interruptions (barge-in), and complete failure to handle 48kHz native video meeting audio across Google Meet, Zoom, and MS Teams.
- Sub-400ms glass-to-glass turn latency across live WebRTC audio streams
- Zero-click autonomous Google Meet and Zoom bot injection via MeetStream.ai & LiveKit
- Sample rate bridging between 48kHz meeting audio and Gemini 16kHz input / 24kHz output
- Continuous sliding window context compression without losing qualification facts on 15+ min calls
System Architecture
End-to-end data pipelines and component isolation boundaries.
Ultra-low-latency raw binary PCM stream processed over WebSockets with vectorized NumPy DSP resamplers bridging 48kHz audio to Gemini 3.1 Flash Live (16kHz in / 24kHz out). Features a 7-phase sales consultation engine, dynamic client profiling, and a concurrent Silent Observer streaming sentiment and BANT scores via Server-Sent Events (SSE).
Key Engineering Decisions
Vectorized NumPy DSP Audio Resampling
Engineered ultra-low-overhead vectorized buffer resamplers bridging 48kHz WebRTC audio streams directly to Gemini Multimodal Live API (16kHz input / 24kHz output) over raw binary WebSockets.
Automated 7-Phase Sales Consultation Engine
Built a structured conversational state machine with dynamic client profiling (technical vs non-technical), automated language switching, and strict pricing/anti-hallucination guardrails.
Concurrent Silent Observer & Real-Time Copilot via SSE
Decoupled analysis agent delivering live speech diarization, sentiment tracking (-1.0 to +1.0), and BANT lead scoring (0–100) streamed via Server-Sent Events to an interactive monitoring dashboard without interrupting audio.
Production Verification & Results
- Achieved human-like conversational turn-taking with natural barge-in (<400ms response time)
- Engineered 25k → 10k token sliding window compression enabling 15+ minute uninterrupted discovery calls
- Delivered live speech diarization, sentiment tracking, and BANT lead scoring via real-time SSE stream
- Successfully integrated autonomous meeting join pipeline across Google Meet, Zoom, and Teams via MeetStream.ai
Production Stack & Tooling
Next Iterations & Roadmap
- Real-time visual screen-sharing comprehension during live product demos
- Automatic bidirectional CRM sync into HubSpot & Salesforce post-call
Want to build something similar?
I specialize in taking complex agentic workflows, duplex voice architectures, and high-throughput computer vision models into production.