WhatsApp Voice Note Support
Overview
Your customers are busy. Many would rather speak than type. Now your WhatsApp chatbot understands voice messages just as well as text. Customers can chat hands-free, in their own language, and your bot gets them the help they need faster.
What is the Feature?
Voice Note Support allows users to interact with the chatbot using WhatsApp voice messages instead of typing. The voice note is converted into text using a Speech-to-Text (STT) provider and processed by the chatbot like any other text message.
The feature supports any form of language and multilingual conversations (if enabled on the bot) and can leverage AI to identify user intents from natural spoken queries.
Benefits
- Improves accessibility and user convenience
- Enables hands-free interactions similar to our web bot
- Supports multilingual voice queries
- Handles natural and conversational user requests
- Improves intent detection for longer spoken queries using AI (AI Intent Assist)
Pre-requisites
- An active chatbot configured and published on WhatsApp.
- Update the STT provider configuration.
- Enable multilingual languages (as per requirements).
- The AI Intent Assist feature should be enabled.
Channel Supported
- Web
How to Enable
- Enable the Voice Note feature for the bot from the Bot Admin Console
- In case you don't have access, raise a request to your Account Manager / Project Manager
- They will enable the Voice Note toggle from the Admin console. (limited to internal users)
- Select the preferred STT provider.
- Configure the required STT settings.
- Important: Go to Bot Settings > Solution Settings > Enable AI Intent Assist
- Save and publish the bot.
Configurations
- STT Provider: Select the Speech-to-Text provider used for audio transcription. We support Sarvam
- Sarvam (best for latency and regional languages, but average for international languages)
- Whisper (best for international languages, average for regional languages, and latency)
- Gemini (better for all languages, and cost-efficient, but high latency)
- Failure Message: Configure the message displayed when a voice note cannot be processed due to transcription failure, unsupported audio, or other processing issues.
- Maximum Audio Duration: Voice notes longer than 1 minute are not supported and will trigger the configured failure message.
How Does it Work?
- User sends a voice note on WhatsApp.
- The WhatsApp audio is received by our Chatbot system and sent to the configured STT provider.
- The STT provider detects the language and transcribes the audio into text.
- The chatbot processes the transcript as a regular text message.
- NLP attempts to identify the intent.
- If NLP cannot identify the intent, AI Intent Assist attempts to map the query to the closest matching intent using AI.
- The bot continues the conversation using the identified intent and configured flow.
Example
User Voice Note: "Mujhe meri policy renew karni hai"
Transcribed Text: "Mujhe meri policy renew karni hai"
Detected Intent: (detected via AI Intent Assist) Renew Policy
Bot Response: The bot initiates the policy renewal journey.
Known Limitations
1. Audio Quality & Speech Recognition
The accuracy of voice note transcription depends on the quality of the audio received by the Speech-to-Text (STT) provider.
Transcription accuracy may reduce in cases such as:
- Background noise or multiple speakers
- Low microphone quality or low audio volume
- Strong accents or unclear pronunciation
- Domain-specific terms, product names, abbreviations, or uncommon words
For best results, users should record voice notes in a quiet environment and speak clearly.
2. Language & Multilingual Support
Voice note transcription and language detection are performed by the configured STT provider.
While the feature supports multilingual conversations and mixed-language inputs (e.g., Hinglish, Benglish, Tanglish), accuracy may vary depending on:
- Language support provided by the STT model
- Regional accents and dialects
- Mixed-language sentence structures
In some cases, similar languages or heavily accented speech may be detected incorrectly.
3. Long Queries & Multiple Requests
Voice notes are best suited for short, natural language queries.
As the length of a voice note increases:
- Transcription accuracy may decrease
- Intent detection becomes more challenging
- Multiple requests in a single voice note may not be handled consistently
Example:
"I want to renew my policy, download my policy schedule, and check my claim status."
The bot may identify only the primary intent or ask the user to continue with a single request.
Maximum supported voice note duration: 1 minute
4. Structured Data Collection
Voice notes are not recommended when the bot is collecting structured information during a flow.
Examples include:
- PAN Number
- Aadhaar Number
- Mobile Number
- OTPs
- Policy Numbers
- Customer IDs
- Email Addresses
- Vehicle Registration Numbers
STT providers may convert spoken numbers or characters into words rather than the expected format.
Example:
- Spoken: "9930123456"
- Transcript: "Nine nine three zero one two three four five six"
This can result in validation failures or incorrect data capture.
For such inputs, users should enter the information as text.
5. AI Intent Assist Scope
AI Intent Assist helps identify user intents from conversational voice notes when traditional NLP matching is unable to find a suitable intent.
This is most effective for:
- Product enquiries
- Policy information
- Claim assistance
- Callback requests
- FAQ-type questions
However, AI Intent Assist is primarily used for intent discovery and does not guarantee improved accuracy during structured data collection steps within an ongoing flow.
6. Dependency on STT Providers
Voice note performance depends on the configured third-party STT provider.
Factors such as:
- Language coverage
- Transcription quality
- Language detection accuracy
- Processing latency
may vary across providers and may improve over time as providers update their models.