WhatsApp Voice Note Support

overview your customers are busy many would rather speak than type now your whatsapp chatbot understands voice messages just as well as text customers can chat hands free, in their own language, and your bot gets them the help they need faster what is the feature? voice note support allows users to interact with the chatbot using whatsapp voice messages instead of typing the voice note is converted into text using a speech to text (stt) provider and processed by the chatbot like any other text message the feature supports any form of language and multilingual conversations (if enabled on the bot) and can leverage ai to identify user intents from natural spoken queries benefits improves accessibility and user convenience enables hands free interactions similar to our web bot supports multilingual voice queries handles natural and conversational user requests improves intent detection for longer spoken queries using ai (ai intent assist) pre requisites an active chatbot configured and published on whatsapp update the stt provider configuration enable multilingual languages (as per requirements) the ai intent assist feature should be enabled channel supported web whatsapp how to enable enable the voice note feature for the bot from the bot admin console in case you don't have access, raise a request to your account manager / project manager they will enable the voice note toggle from the admin console (limited to internal users) select the preferred stt provider configure the required stt settings important go to bot settings > solution settings > enable ai intent assist save and publish the bot configurations stt provider select the speech to text provider used for audio transcription we support sarvam sarvam (best for latency and regional languages, but average for international languages) whisper (best for international languages, average for regional languages, and latency) gemini (better for all languages, and cost efficient, but high latency) failure message configure the message displayed when a voice note cannot be processed due to transcription failure, unsupported audio, or other processing issues maximum audio duration voice notes longer than 1 minute are not supported and will trigger the configured failure message how does it work? user sends a voice note on whatsapp the whatsapp audio is received by our chatbot system and sent to the configured stt provider the stt provider detects the language and transcribes the audio into text the chatbot processes the transcript as a regular text message nlp attempts to identify the intent if nlp cannot identify the intent, ai intent assist attempts to map the query to the closest matching intent using ai the bot continues the conversation using the identified intent and configured flow example user voice note "mujhe meri policy renew karni hai" transcribed text "mujhe meri policy renew karni hai" detected intent (detected via ai intent assist) renew policy bot response the bot initiates the policy renewal journey known limitations 1\ audio quality & speech recognition the accuracy of voice note transcription depends on the quality of the audio received by the speech to text (stt) provider transcription accuracy may reduce in cases such as background noise or multiple speakers low microphone quality or low audio volume strong accents or unclear pronunciation domain specific terms, product names, abbreviations, or uncommon words for best results, users should record voice notes in a quiet environment and speak clearly 2\ language & multilingual support voice note transcription and language detection are performed by the configured stt provider while the feature supports multilingual conversations and mixed language inputs (e g , hinglish, benglish, tanglish), accuracy may vary depending on language support provided by the stt model regional accents and dialects mixed language sentence structures in some cases, similar languages or heavily accented speech may be detected incorrectly 3\ long queries & multiple requests voice notes are best suited for short, natural language queries as the length of a voice note increases transcription accuracy may decrease intent detection becomes more challenging multiple requests in a single voice note may not be handled consistently example "i want to renew my policy, download my policy schedule, and check my claim status " the bot may identify only the primary intent or ask the user to continue with a single request maximum supported voice note duration 1 minute 4\ structured data collection voice notes are not recommended when the bot is collecting structured information during a flow examples include pan number aadhaar number mobile number otps policy numbers customer ids email addresses vehicle registration numbers stt providers may convert spoken numbers or characters into words rather than the expected format example spoken "9930123456" transcript "nine nine three zero one two three four five six" this can result in validation failures or incorrect data capture for such inputs, users should enter the information as text 5\ ai intent assist scope ai intent assist helps identify user intents from conversational voice notes when traditional nlp matching is unable to find a suitable intent this is most effective for product enquiries policy information claim assistance callback requests faq type questions however, ai intent assist is primarily used for intent discovery and does not guarantee improved accuracy during structured data collection steps within an ongoing flow 6\ dependency on stt providers voice note performance depends on the configured third party stt provider factors such as language coverage transcription quality language detection accuracy processing latency may vary across providers and may improve over time as providers update their models