ElevenLabs Speech Engine: add voice to an existing second brain

Start from a working text knowledge base

Thanks to ElevenLabs for sponsoring this video.

Luke builds a voice agent for his YouTube archive using an Obsidian/Codex second brain and ElevenAPI Speech Engine. The archive already works as a text knowledge base; the voice layer makes that existing agent conversational. Starting with checked text answers gives you a baseline for later voice tests.

1. Follow the Obsidian/Codex wiki section at 1:06 and choose a small set of channel sources you are comfortable using. Keep original files linked to derived pages.
2. Ask the archive a factual question in text, inspect its source and record the answer. Include a question whose answer is absent so you know how missing knowledge is handled.

Use the project and connect the voice layer

The exact code resource is https://github.com/lukejbyrne/second-brain-voice. The product destination is https://try.elevenlabs.io/luke-speech-engine. The recording moves to ElevenAPI at 4:34 and running the application with API keys at 6:44.

1. Open the repository and follow its documented setup for the demonstrated app. Add the required keys locally through the project’s configuration, keeping them out of shared files.
2. Run the app and confirm a spoken question reaches the same archive-backed agent. Compare its response with the text baseline before changing the voice or interface.

Test the conversation, not only the voice sound

The published description says Speech Engine handles speech, transcription, turn-taking, voice activity detection and interruptions. These are separate parts of the interaction: clear audio alone does not show that the agent heard the question correctly or stayed grounded in the archive.

1. Repeat a short known question and inspect the recognised text and answer where the app exposes them. Try a longer question with a natural pause.
2. Interrupt an answer and observe whether the conversation resumes appropriately. Compare voice options using the same wording so content differences do not obscure the audio comparison.

Review grounding and keep a repeatable test set

The video tests voices, grounding and live answers from 11:36. Use those checks to assess whether the voice layer preserves the usefulness of the original knowledge base.

1. Repeat the known and missing-answer questions by voice. Follow available citations back to the archive and correct source problems before adding more content.
2. Record remaining transcription, interruption or retrieval failures with the exact question. Keep the working project and test questions together for the next update.

The takeaway

The voice layer should preserve the evidence behind the text agent’s answers. Compare the complete conversation with a checked text baseline and inspect each failure at its own layer.