Irham

Project: AURA Protocol (MoodSync Jukebox) I built AURA Protocol, a real-time, room-based social music platform that uses biometric facial detection to autonomously curate a shared soundtrack. Using a "Room Code" system (similar to Kahoot), users join a synchronized session where their webcams act as sensors. The system continuously analyzes facial telemetry to detect the collective mood of the room and instantly plays music that matches that specific energy. What problem does it solve? Shared listening experiences often suffer from the "paradox of choice" or a disconnect between the music and the actual vibe of the people in the room. Manually skipping tracks or searching for songs kills the social momentum. AURA solves this by removing the UI friction entirely—the music evolves naturally based on subconscious emotional feedback (facial expressions), ensuring the "vibe" is always in sync without anyone needing to touch a button. How does it use ElevenLabs? ElevenLabs serves as the "AI DJ" and the narrative soul of the application. Once a mood is detected and a song is selected, we use the ElevenLabs API to generate a high-fidelity, context-aware voiceover. This AI DJ doesn't just announce the song; it comments on the room’s detected state (e.g., "I see the energy is picking up! Let's keep this momentum going with..."). This creates a highly immersive and "living" atmosphere that feels responsive and human. How does it use turbopuffer? turbopuffer is the high-performance semantic engine that makes the "Sync" possible. We store a vast library of song metadata and emotional embeddings within turbopuffer. When facial detection outputs a mood vector, we perform an ultra-low latency vector search in turbopuffer to find the most semantically relevant track. Its incredible speed allows the Jukebox to transition and react to mood changes in milliseconds, providing a seamless experience that would be impossible with traditional keyword-based databases.

Submitted 16 Apr 2026