OpenAI Whisper Now Available in Azure OpenAI and AI Speech

Microsoft recently announced that the OpenAI Whisper model is now available for users of the Azure OpenAI and Azure Speech AI services.

The OpenAI Whisper model is a neural network and speech recognition system that can transcribe audio in 57 languages and translate those languages into English. It’s trained on 680,000 hours of multilingual and multitask data collected from different corners of the web.

In July, Microsoft’s Azure AI Services team announced OpenAI Whisper’s preview. While OpenAI already offers the Whisper API separately, developers can now build applications using OpenAI’s Whisper model in Azure OpenAI Service. The model comes with the same features and functionalities as the OpenAI Whisper API. The Whisper model’s REST APIs for transcription and translation are also available within the Azure OpenAI platform.

For Azure AI Speech users, using the new Whisper model with the existing Azure AI Speech batch transcription API opens up new capabilities. One is transcribing large audio files or a large number of files stored on the Azure platform. Users can transcribe files of up to 1GB in size and batch up to 1000 files in a single round.

The new Azure recognition results will include word-level timestamps, which identify where in the audio each word appears. Another beneficial feature of the OpenAI Whisper model in Azure AI Speech is speaker diarization, which identifies different speakers and their speech segments within an audio file. These new capabilities are available on top of all the existing Azure AI Speech features.

Microsoft also revealed that users will soon get customization options in Azure AI Speech. The Custom Speech feature will allow them to train Whisper on their own data within Azure Speech, which will result in enhanced recognition accuracy and consistency.

Microsoft has supported OpenAI with over $10 billion in investments, which make the tech giant its biggest investor. As such, Microsoft is heavily implementing OpenAI’s advanced AI technologies across its products. In June, Microsoft introduced Azure AI Content Safety, a content moderation system available within the Azure AI platform.