VoiceOps: reviewing sales calls
Scores sales calls against a rubric and remembers context across calls. Caching cut a slow database read from 720 ms to 2 ms; routing work to the right model cut chat replies from 46 s to 11 s.
- Role
- Lead AI Systems Engineer
- Built with
- TypeScript, Node.js / Express, PostgreSQL + pgvector, Deepgram STT, DeepSeek, Mistral Embeddings, React
Results
- 720ms → 2ms SWR cached reads
- 46s → 11s interactive chat latency
- Deterministic memory merge
In more detail
Multi-tenant sales call intelligence platform featuring a timed Generator → Critic → Deterministic Memory Merge pipeline, stale-while-revalidate (SWR) read-through caching (cutting remote DB latency from 720ms to 2ms), and workload-split model routing (cutting interactive chat turns from 46s to 11s).
Key decisions
Generator → Critic → Deterministic Memory Merge Pipeline
Processes each call transcript (or live Deepgram audio upload) through a timed Generator extraction pass and Critic verification pass, followed by a zero-LLM deterministic TypeScript memory merge scoped strictly by accountId, customerId, and repId.
Read-Through SWR Cache & Workload-Split Model Routing
Eliminated N+1 remote Postgres round-trips with joined queries and an in-flight deduplicated SWR cache (720ms → 2ms), while splitting background reasoning models from fast interactive models to cut chat response time from 46s to 11s.