Under the Hood
The full technology stack powering automated D&D session archival.
Voice Capture — Native Discord Recording
GygaxBot joins the Discord voice channel and records each speaker to separate WAV files. Per-speaker audio isolation ensures clean transcription even with overlapping voices, crosstalk, and background noise. Recording starts and stops with simple bot commands, capturing the entire session as raw audio.
Transcription — faster-whisper
Per-speaker audio files are processed through faster-whisper, a CTranslate2-optimized Whisper implementation that runs locally. Speaker tags are preserved throughout, producing a full session transcript with timestamps and speaker attribution. The result is a structured document ready for scene extraction.
Scene Generation — Nemotron / Ollama
The transcribed session is sent to a locally-hosted LLM via Ollama (currently Nemotron). A carefully crafted prompt instructs the model to identify 4-6 key dramatic moments, prioritizing combat, dramatic events, exploration, and atmospheric scenes. The model outputs structured JSON with scene titles, visual descriptions for image generation, narration scripts for TTS, character lists, and locations.
Scene Illustration — ComfyUI + SDXL Lightning
Each scene's visual description is enhanced with campaign-specific style tags (western frontier aesthetic, sepia tones, warm sunset lighting) and sent to ComfyUI running locally on GPU with SDXL Lightning. The system automatically finds character reference images on disk and includes them for visual consistency — Chad always looks like a centaur, Threx always looks like a red dragonborn. Alternative backends including Gemini image generation are available as fallback.
Voice Narration — Kokoro TTS
Kokoro TTS provides high-quality dramatic narration with 10 available voice profiles. Each campaign can be assigned a specific narrator voice for consistency across sessions. The TTS service runs locally on CPU, outputs 24kHz WAV files, and integrates directly with the pipeline for fully automated narration generation.
RAG Archival — ChromaDB + SentenceTransformers
Every session is chunked and indexed into a ChromaDB vector store using sentence transformer embeddings. This enables semantic search across all campaign history — ask "when did the party find the cannon?" and get relevant transcript sections, scene descriptions, and narration from any session. The RAG system powers campaign context recall and cross-session narrative continuity.
Orchestration — FastAPI + Async Pipeline
The entire pipeline is orchestrated by a FastAPI server. When GygaxBot triggers archival, the server processes everything asynchronously: transcription, scene extraction, TTS narration (parallel), and image generation (GPU, sequential). Results are posted back to Discord and published to the web archive. Campaign management, party rosters, dice rolling, and session history are all handled through the bot.
