m-bain/whisperX

…perX [0] is a library that combines generic whisper speech recognition models with some additional tools like discrete-time-warping to create a transcription with more accurate time stamp alignment - something that was very helpful when generating subtitles. But because most of this logic just lives in the python library, using this in Elixir requires writing a lot more tooling around the existing bumblebee whisper implementation [1]). but, 2) It's way easier to ship models I built and trained entirely with Elixir's ML ecosystem - EXLA, NX, Bumblebee. I trained a few models doing basic visual recognition tasks (detecting scene transitions, credits, title cards, etc), using the existing CLI…

Whisper-X does speaker-annotated transcripts nicely. I’ve used it for running multi-hour TTRPG sessions with friends and it worked hassle-free after setup. https://github.com/m-bain/whisperX

…yet it completes in a fraction of the time using batching [0]. Yes, if you check their community integrations section on faster-whisper [1], you can see a lot of different CLIs, GUIs, and libraries. I recommend WhisperX [2], it's the most complete CLI so far and has features like diarization which whisper.cpp does not have in a production-ready capacity. [0] https://github.com/SYSTRAN/faster-whisper#benchmark [1] https://github.com/SYSTRAN/faster-whisper#community-integrations [2] https://github.com/m-bain/whisperX

For this use case, why not use Whisper to transcribe the audio, and then an LLM to do a second step (summarization or answering questions or whatever)? If you need diarization, you can use something like https://github.com/m-bain/whisperX

whisperx does this all quite well and can be run with `uvx whisperx` https://github.com/m-bain/whisperX

Yes for parakeet, but only comparing benchmark results for canary. Whisper also has severe hallucinations on silence and noise and WhisperX helps a lot, it adds voice activity detection i.e. a model to detect when someone speaks, to filter the input before running whisper. https://github.com/m-bain/whisperX