Joseph Keshet

Blog

For humans, speech feels natural and effortless. Yet it is one of the most complex signals we know. Every utterance carries far more than words: it encodes intent, identity, emotion, and even hints at a person’s health. This richness is what drives my research. I’m passionate about understanding and quantifying speech in all its dimensions. My work spans the intersection of machine learning and computational speech and language processing, focusing on areas like automatic speech recognition (ASR or “speech-to-text”), speech synthesis (“text-to-speech”), speaker recognition, laboratory phonology automation, and the analysis of pathological speech.

In this blog, I’ll be presenting highlights from my research. Some of my colleagues wrote some of the posts.

Whisper-Medusa: Using multiple Decoding Heads to Achieve 1.5X Speedup
Medusa model builds on top of Whisper by predicting multiple tokens per iteration, which significantly improves speed with small degradation in Word…
Whisper-Medusa: Using multiple Decoding Heads to Achieve 1.5X Speedup
Self-supervised Speaker Diarization
We propose an entirely unsupervised deep-learning model for speaker diarization. Specifically, the study focuses on generating high-quality neural speaker representations without any annotated data, as well as on estimating secondary…
Speeding up and slowing-down speech with exceptional quality
ScalerGAN is a novel deep learning-based method for adjusting speech speed while preserving the speaker’s timbre and pitch, offering exceptional audio quality. It leverages GAN architecture to enhance the listening experience across…
DeepFry: Deep Neural Network Algorithms for Identifying Vocal Fry
Deep learning algorithms to detect vocal fry (creaky voice) in fluent speech.
DeepFry: Deep Neural Network Algorithms for Identifying Vocal Fry
AI for Speech Therapy and Language Acquisition
Learning a new language involves constantly comparing speech productions with reference productions from the environment. Early in speech…
AI for Speech Therapy and Language Acquisition
The Ubiquity of Machine Learning and its Challenges to Intellectual Property
A method for embedding watermarks in neural network models to protect them from unauthorized use or distribution.
The Ubiquity of Machine Learning and its Challenges to Intellectual Property