This guide focuses on the receptionist role: what it should do, how to design the call flow, how it connects to FreePBX, and how transfers to your extensions work. If you want a broader setup walkthrough first, see our post on VEXYL as a FreePBX AI agent.

What a FreePBX AI receptionist does

Think about what your best receptionist handles in a day. Most of it fits into a short list:

• Greets and identifies the need: “Good morning, Sharma and Associates. How can I help?”
• Routes by name, department or reason: “Can I speak to Priya in accounts?” or “I have a question about my invoice.”
• Answers common questions: office hours, address, directions, which documents to bring.
• Takes messages when the person is busy or it is after hours, and sends them to the right inbox or system.
• Screens calls: asks for the caller’s name and reason before putting them through.
• Speaks the caller’s language, switching between English and Hindi, Tamil or Malayalam as needed.

A FreePBX AI assistant can do all of these. What it should not do is make decisions that need authority, such as approving discounts or handling complaints. Those go to a person, quickly.

AI receptionist vs a FreePBX IVR menu

For a deeper comparison of menus and conversational systems, read our guide to AI IVR systems.

Designing the receptionist call flow

Before touching the dialplan, write the flow on paper. It will save hours of rework.

1. Build the routing table

List every destination the receptionist can transfer to: extensions, ring groups and queues, with the names and reasons that should lead there. For example, “accounts, billing, invoice, payment” all go to the accounts ring group. Include alternative names people use, nicknames and common misspellings of staff names.

2. Write the greeting

Short and open. Name the organisation, say it is an automated assistant, and ask how it can help. Do not list departments; that is what you are moving away from.

3. Confirm before transferring

“I’ll connect you to the accounts team. Is that right?” A one-second confirmation prevents most wrong transfers.

4. Decide what happens when nobody answers

If the transfer target is busy or does not pick up, the call should come back to a sensible place: a queue, voicemail or the AI taking a message. Plan this in FreePBX the same way you would for any inbound route.

5. Write the after-hours behaviour

Outside office hours, the receptionist can answer FAQs, take a message with name, number and reason, and tell the caller when they will hear back. FreePBX time conditions can send calls to a different bot or the same bot with after-hours instructions.

6. Add the information it should know

Hours, holidays, address, directions, parking, services and prices you are happy to share. Keep it short and accurate, and assign someone to update it.

How the AI receptionist connects to FreePBX

FreePBX runs on Asterisk, so the cleanest connection is Asterisk’s AudioSocket application, which streams call audio to a voice gateway over TCP. The steps, at a high level:

1. Install the voice gateway on the FreePBX server or a nearby one.
2. Add a custom dialplan context in extensions_custom.conf that answers the call, creates a session ID, posts the caller’s details (number, name, channel, which bot and language) to the gateway, and hands the audio over with AudioSocket().
3. Create a Custom Destination in FreePBX that points at that context.
4. Point an inbound route, a time condition or an IVR option at the Custom Destination.
5. Configure transfers so the AI can send callers back into FreePBX extensions, ring groups or queues.

The core of the dialplan looks like this:

[voice-assistant]

exten => _X.,1,Answer()

same => n,Set(SESSION_UUID=${TRIM(${SHELL(uuidgen)})})

same => n,AudioSocket(${SESSION_UUID},127.0.0.1:8080)

same => n,Hangup()

In production you add a step before AudioSocket() that posts the caller ID, channel name, bot and language to the gateway. The Asterisk AudioSocket documentation has the full dialplan, firewall settings and per-number bot selection.

If you prefer SIP, FreePBX can also reach the gateway over a pjsip trunk, and the SIP trunk guide includes the FreePBX GUI settings.

How transfers to extensions work

Transfers are what make an AI agent a receptionist rather than an answering machine. With AudioSocket, when the AI decides to transfer, the gateway sends an HTTP request with the call’s Asterisk channel name to a transfer endpoint you provide. That endpoint moves the channel, for example with an AMI Redirect into a queue or extension context, and replies with success. This is why the channel name must be posted with the caller details at the start of the call: without it, the transfer cannot find the call.

For multiple destinations, the usual pattern is to have the AI choose a destination from your routing table and pass it along, so the transfer script can redirect to the right extension, ring group or queue.

Testing checklist before going live

• Call from a mobile in a noisy place and ask for each department by different names.
• Ask for staff by first name, full name and nickname.
• Interrupt the greeting to check barge-in works.
• Test transfers to busy and unanswered extensions.
• Test after-hours routing and message taking.
• Ask something the receptionist does not know and confirm it offers a transfer instead of guessing.
• Test in every language your callers use.

How VEXYL fits

The VEXYL Voice Gateway is built for this setup. It connects to FreePBX and Asterisk over AudioSocket (or SIP), serves different bots, languages and voices per number from one install, and supports transfers through your transfer endpoint on AudioSocket calls or SIP REFER on SIP calls. It handles barge-in, echo and line noise, and plays a short filler sound while the AI thinks so callers do not hear dead air.

You pick the speech providers for your callers’ languages, including Sarvam, Google, Deepgram and the self-hosted VEXYL-STT, and the LLM behind the bot. Because it runs alongside FreePBX on your own server, call audio and transcripts can stay inside your network, as described in our guide to self-hosted AI voice agents. The free Developer edition supports 4 concurrent calls, which is enough to run a receptionist pilot on a real number.

FAQ

Does FreePBX have a built-in AI receptionist?

FreePBX provides IVRs, ring groups, queues and time conditions, but an AI receptionist typically comes from an external voice gateway connected through Asterisk AudioSocket or a SIP trunk.

Can the AI receptionist transfer calls to FreePBX extensions?

Yes. On AudioSocket calls, the gateway calls a transfer endpoint that moves the Asterisk channel to an extension, ring group or queue. On SIP calls, it can use SIP REFER.

Can I keep my existing FreePBX IVR?

Yes. You can add the AI receptionist as one IVR option, use it only after hours, or put it on a single inbound route while the rest stays unchanged.

What does a FreePBX AI voice agent need to run?

A FreePBX or Asterisk system with AudioSocket available, the voice gateway on the same or a nearby server, and API keys for the speech and AI providers you choose, unless you self-host them.

How many calls can it handle at once?

That depends on your licence, server and provider limits. The free Developer edition allows 4 concurrent calls; Professional allows 25.

Ready to give your FreePBX an AI receptionist? Download the free Developer edition, or talk to the VEXYL team for help with setup.

Originally published at vexyl.ai.

What Does Voice AI Actually Cost in 2026?

Before we get to our numbers, it’s worth understanding what you’re really paying for when you deploy voice AI. Every call involves at least five cost layers: the platform fee, speech-to-text (STT), the language model (LLM), text-to-speech (TTS), and telephony. Cloud platforms bundle these — and add margin at every layer.

Vapi, the most widely used voice AI platform for developers, advertises pricing starting at $0.05 per minute. But that’s just the platform fee. Once you add STT ($0.01/min), LLM processing ($0.02–$0.20/min depending on model), TTS (~$0.04/min), and telephony, real-world costs land between $0.13 and $0.31 per minute. At current exchange rates, that’s ₹11 to ₹26 per minute. And if you’re in healthcare? HIPAA compliance costs an additional $1,000 per month — a flat fee regardless of your call volume.

Retell AI, often cited as a more transparent alternative, prices AI voice agents at $0.07+ per minute as a base rate. For Indian deployments requiring Indian language support, these costs go even higher because you’re forced to use whatever providers the platform supports — you don’t get to optimise the stack for your use case.

The difference isn’t marginal. It’s structural.

What Do Real Production Numbers Look Like?

The VEXYL cost analytics dashboard for a live healthcare deployment (Feb 1–19, 2026) shows the following:

• Total calls processed: 156
• Total minutes: 248.1 min
• Average call duration: 1.6 minutes (95 seconds)
• Total AI cost: ₹205.88
• Cost per minute (blended): ₹0.83
• Savings vs Vapi: 88.5% (₹6.37/min cheaper)

The cost split is equally revealing: Sarvam AI for STT accounts for 60% of spend (₹124.03), while Google TTS and LLM combined make up the remaining 40% (₹81.85). This breakdown tells you something important — the biggest cost driver is speech recognition, not the language model. Choosing the right STT provider for your language and use case is the single highest-leverage optimisation decision you can make.

What I find most significant in this data is the cost-per-minute trend line. It’s flat. Consistent at ₹0.83 across every active day in the reporting period. No usage spikes. No surprise overage charges. No end-of-month billing shock. That predictability is something cloud platforms structurally cannot offer — because their incentive is consumption, not efficiency.

How Does VEXYL Achieve ₹0.83 Per Minute?

There’s no magic here. The architecture makes the economics possible.

Bring Your Own Keys (BYOK). VEXYL doesn’t proxy your API calls through our infrastructure and mark up the cost. You connect directly to Sarvam AI, Google Cloud, or whichever providers you choose using your own API keys. You pay provider rates — full stop. No platform margin, no intermediary fee.

Provider selection for Indian languages. For Malayalam and other Indian languages, Sarvam AI delivers superior transcription quality compared to generic English-trained models. Using the right provider for the right language isn’t just about accuracy — it directly affects your STT costs per minute.

TTS caching. A significant portion of what a voice agent says is predictable — greetings, confirmations, common responses. VEXYL caches these audio outputs. In production, we’re seeing 90%+ TTS cache hit rates, which means the majority of audio responses are served in milliseconds at near-zero marginal cost rather than being regenerated each time.

No platform tax on compliance. For healthcare deployments requiring data sovereignty — which is essentially all healthcare deployments in India — on-premise deployment means your patient data never leaves your network. You’re not paying a cloud platform’s HIPAA compliance add-on. The compliance is architectural.

Why Self-Hosted Voice AI Makes Sense for Indian Healthcare and Enterprises

Indian enterprises face a set of constraints that the global voice AI market was simply not designed for. The language diversity alone — with 22 scheduled languages and hundreds of dialects — means that English-first platforms will always be a second-best fit. But the economics compound this problem further.

At the exchange rate of roughly ₹84 to the dollar, even “cheap” Western voice AI platforms cost ₹6–₹12 per minute at realistic usage levels. A hospital processing 1,000 minutes per month pays ₹6,000–₹12,000 monthly just in AI platform costs, before any telephony or infrastructure. At ₹0.83/min, the same hospital pays ₹830. That’s not a 20% saving. It’s a 10x cost reduction.

For government agencies, the calculus is even sharper. Public sector organisations cannot route citizen data through US-based cloud infrastructure. Data residency is a legal requirement, not a preference. Self-hosted AI voice infrastructure deployed on-premise is the only viable path — and it also happens to be dramatically cheaper.

Call centres processing millions of minutes annually are looking at the difference between a voice AI deployment that costs ₹83 lakhs per year versus ₹6–₹10 crores for the equivalent cloud platform usage. At that scale, the business case for self-hosted is not a conversation — it’s arithmetic.

Is Self-Hosted Voice AI Difficult to Deploy?

This is the question I get asked most often, and it deserves a direct answer. VEXYL is distributed as binary executables and Docker images. Organisations with existing Asterisk or FreePBX infrastructure can integrate via the AudioSocket protocol without replacing their telephony stack. The platform runs on standard Ubuntu servers, manages up to 20–50 concurrent calls with PM2 clustering, and uses Redis for session management.

In my experience, a technical team comfortable with Linux and basic networking can be processing live calls within a day. That’s not a marketing claim — it’s what this production healthcare deployment looked like in practice. The complexity of managing multiple AI providers is abstracted by VEXYL’s modular provider architecture, which allows you to switch STT, LLM, or TTS providers without code changes.

The honest caveat: self-hosted requires ongoing operational ownership. You’re responsible for uptime, updates, and infrastructure maintenance. For organisations that want fully managed infrastructure, cloud platforms like Vapi or Retell remain valid options — but you should go in knowing what that convenience costs.

What’s Next for VEXYL Cost Optimisation?

The ₹0.83/min benchmark is production-validated, but it’s not the floor. Several optimisations on the roadmap should push costs lower still:

1. LLM response caching for structured conversations. Healthcare IVR flows are highly predictable — the same questions get asked and answered thousands of times. Caching LLM responses for identical inputs could reduce LLM costs by 40–60% on structured use cases.
2. Pre-warmed greetings for outbound calling. Generating TTS audio before the call connects eliminates the first-response latency and removes that TTS cost from the real-time path entirely.
3. Multi-provider cost routing. Dynamically selecting the cheapest provider for each request based on real-time pricing and availability — similar to how data centres use spot pricing for compute.

The goal is sub-₹0.50/min for structured healthcare and government workflows within the next two quarters. I believe that’s achievable without sacrificing the Indian language quality that makes this deployment actually useful to patients.

The Bottom Line

The numbers from this deployment are not a projection or a whitepaper estimate. They’re 19 days of live production data from a working hospital deployment in Kerala. ₹0.83 per minute. 88.5% savings versus Vapi. Flat, predictable costs. Real Malayalam conversations with real patients.

If your organisation is evaluating voice AI and currently paying cloud platform rates, I’d encourage you to run the numbers against your actual call volume. For most Indian enterprises, the difference between cloud and self-hosted isn’t a rounding error — it’s the difference between a pilot project and a viable, scalable deployment.

VEXYL is open to pilot deployments for healthcare providers, government agencies, and call centres. We’ll show you what the cost analytics dashboard looks like for your own traffic — not a demo environment.

Originally published at vexyl.ai.

This guide explains the three common ways to connect Asterisk to AI, looks at the open-source projects people find when they search GitHub, and lists the questions that matter once real customers start calling. It is written for teams in India who care about Indian languages, telecom costs in rupees and keeping call data on their own servers.

How an Asterisk AI voice agent works

Every working setup has the same five parts, whatever the project calls them:

1. Audio transport. Asterisk hands the call audio to an external process and accepts audio back.
2. Speech-to-text (STT). The caller’s speech becomes text, ideally streamed while they talk.
3. The AI brain. An LLM, a workflow tool or your own API decides the reply, often calling tools such as a booking system.
4. Text-to-speech (TTS). The reply becomes audio in the right language and voice.
5. Turn-taking. Voice activity detection, barge-in (the caller interrupting the bot) and silence handling.

The transport is the Asterisk-specific part. The other four are the same problems any voice agent has to solve, and they decide most of the quality you hear on a call.

Three ways to connect Asterisk to AI

1. AudioSocket

AudioSocket is an Asterisk application that opens a plain TCP connection for each call and streams raw audio both ways. You add a few lines to the dialplan and point it at your AI service. It is simple to reason about: one call, one socket, 8 kHz audio in and out.

A typical dialplan looks like this:

exten => _X.,1,Answer() same => n,Set(SESSION_UUID=${TRIM(${SHELL(uuidgen)})}) same => n,AudioSocket(${SESSION_UUID},127.0.0.1:8080) same => n,Hangup()

The first argument must be a proper UUID, which is why the dialplan generates one instead of using ${UNIQUEID}. You can also post caller details (number, language, which bot to use) to the AI service before the AudioSocket line, so the agent knows who is calling from the first second. The full walkthrough is in the Asterisk AudioSocket setup docs.

2. ARI with external media

The Asterisk REST Interface (ARI) lets an external application control calls: answer, bridge, play, record and transfer. Combined with an external media channel, it streams audio over RTP to your app. ARI gives you more call control than AudioSocket, but you write and maintain more code, and you need to handle RTP timing and codecs yourself.

3. SIP trunk to an AI gateway

The third option treats the AI as another SIP endpoint. Asterisk routes a call over a SIP trunk to an AI gateway, exactly as it would to a branch office or a carrier. This keeps the dialplan change tiny and works well when the AI runs on a different server or when you also have non-Asterisk PBXs to connect.

Open-source Asterisk AI projects on GitHub

Search for “asterisk ai agent github” and a few names come up repeatedly. Projects move quickly, so check each repository’s current README, licence and last commit before you depend on it. As of late 2026, these are the ones worth knowing about:

• AVA (AI Voice Agent for Asterisk). An MIT-licensed project that integrates with Asterisk and FreePBX. Its README describes AudioSocket/RTP transport and ARI control, with pluggable providers.
• ictinnovations asterisk-ai-voice-agent. A self-hosted Python sidecar that bridges an AudioSocket call to an LLM, with barge-in and 20 ms frame pacing according to its listing. It describes itself as an early release.
• Agent Voice Response. A community-built, Docker-based stack that connects to Asterisk via AudioSocket, discussed on the Asterisk community forum.
• General frameworks. Pipecat and LiveKit Agents are open-source voice agent frameworks rather than Asterisk projects. You can connect them through SIP, but you assemble the telephony side yourself.

These projects are good for learning and for prototypes. The gap usually shows up later: Indian language support, per-call configuration, transfers to a human, cost tracking, and someone to call when a carrier changes something at 11 pm.

What to check before production

Whichever route you take, test these on real calls from real phones, not from a softphone on your office Wi-Fi.

Indian language recognition on 8 kHz audio

Phone audio is narrowband. A model that scores well on clean 16 kHz recordings can stumble on a caller in a moving auto-rickshaw. Test Hindi, your regional languages and Hinglish or Manglish code-mixing with your actual callers. Providers built for Indian languages, such as Sarvam, often do better here than English-first engines.

Streaming vs batch STT

Streaming recognisers return text while the caller is still speaking. Batch recognisers wait for silence, then send the whole utterance. Batch is simpler, but it adds a noticeable pause on every turn. The speech recognition provider guide lists which providers stream and which do not.

Barge-in and silence

Callers interrupt. If the bot keeps talking over them, the call feels robotic within seconds. Check that the agent stops speaking when the caller starts, and that it handles long silences without hanging up too early.

Transfers to a human

Decide how a call leaves the AI. With AudioSocket, a transfer usually means an HTTP call back to something that moves the Asterisk channel, so the agent needs the channel name. With SIP, a REFER to a queue or extension does the job.

Where the audio goes

Under India’s DPDP Act, a recording of a caller’s voice is personal data. Know which providers receive the audio and text, where they process it, and how long they keep it. Self-hosted STT and TTS remove most of that question.

How VEXYL fits

The VEXYL Voice Gateway sits between Asterisk and the AI. It accepts calls over AudioSocket (the default mode) or over a SIP trunk, and it also documents FreePBX and IP-PBX brands such as Panasonic, Avaya IP Office, Grandstream and Bitvoice.

What it adds over a DIY project:

• Provider choice. STT from Sarvam, Google, Deepgram, Groq, Gemini or the self-hosted VEXYL-STT. TTS from Sarvam, Google, Gemini, ElevenLabs, Smallest AI, Deepgram or Azure. You bring your own API keys.
• Per-call settings. Post the caller’s number, language and bot before the call starts, or override the voice and recogniser for a single call.
• Any brain. Connect LiteBot (which works with Groq, OpenAI, Gemini, Sarvam, Ollama and others), an n8n webhook, or your own HTTP endpoint. A realtime speech-to-speech mode is also supported.
• Transfers. AudioSocket calls are transferred through your webhook; SIP calls with a REFER.
• Self-hosting. It runs as a single binary or in Docker on your own server, so call audio does not have to leave your network if you also use self-hosted speech.

If you have already tried an open-source project and want a broader look at running everything on your own hardware, read our guide to a self-hosted AI voice agent. FreePBX users can follow the FreePBX AI receptionist walkthrough, and there is an older but still useful Asterisk gateway setup guide on this blog.

FAQ

Can Asterisk run an AI voice agent without a cloud platform?

Yes. Asterisk handles the call, and the AI pieces can all run on your own servers: a self-hosted STT model, a local LLM through something like Ollama, and a self-hosted TTS model. Expect to need decent hardware for the LLM, and test response times carefully.

Which Asterisk version do I need for AudioSocket?

AudioSocket is a module that ships with newer Asterisk releases. The VEXYL docs ask for Asterisk 16 or later with the AudioSocket application loaded. Run module show like audiosocket on your server to confirm it is available.

Is AudioSocket better than ARI for AI?

For most voice agents, AudioSocket is easier because the dialplan change is small and the audio format is fixed. ARI is better when your application needs to control the whole call itself, such as complex bridging or recording logic.

Does this work with FreePBX?

Yes. FreePBX is built on Asterisk, so you add the AudioSocket context to extensions_custom.conf and route an inbound number or IVR option to it.

How much does an Asterisk AI call cost in India?

It depends on your STT, TTS and LLM providers and how long callers talk. One live hospital deployment in India came to about ₹0.83 per minute of AI call time, which shows the order of magnitude when you pick providers carefully.

Try it on your own Asterisk

The Developer edition of the VEXYL Voice Gateway is free, self-hosted and handles 4 concurrent calls, which is enough to wire it to a test extension this week. If you want help sizing a larger rollout, talk to the VEXYL team.

Originally published at vexyl.ai.

VEXYL-STT is a free, self-hosted speech-to-text server wrapping AI4Bharat’s IndicConformer 600M multilingual model. It supports 14 Indian languages, runs on CPU (no GPU required for typical workloads), and exposes both a WebSocket streaming API and a REST batch API on a single port. Apache 2.0 licensed. No API keys. No per-minute billing. Your audio never leaves your server.

What Is VEXYL-STT and Why Did We Build It?

VEXYL-STT started as an internal component of the VEXYL AI Voice Gateway — our enterprise platform that bridges Asterisk/FreePBX telephony infrastructure with modern AI services. We needed a self-hosted STT option for deployments where cloud APIs are not viable: hospitals with patient audio data, government agencies with data sovereignty requirements, and enterprises that simply can’t afford ₹1–3 per minute of STT billing at scale.

We chose AI4Bharat’s IndicConformer 600M multilingual model as the backbone because, frankly, nothing else comes close for Indian languages. It’s the most capable open-source Indian ASR model available, trained on thousands of hours of Indic language audio across 22 official languages. We extracted the STT component, wrapped it in a production-ready server, and are now releasing it so the community can use it independently.

Which Indian Languages Does VEXYL-STT Support?

The current release supports 14 Indian languages out of the box:

Switch between languages at the session level — no server restart needed. Just pass the language code in your WebSocket start message or batch API request. If you pass an unknown code, it defaults to Malayalam (ml).

How Does the API Work?

VEXYL-STT runs two modes on a single port — you don’t need to run separate services for streaming vs. batch use cases.

Real-Time WebSocket Streaming

Connect over WebSocket, send raw 16kHz 16-bit mono PCM audio as binary frames, and receive JSON transcripts in real time. The server uses energy-based VAD to detect speech boundaries and fires a final message when an utterance completes. This is the right mode for telephony integrations, browser mic capture, and any scenario where you’re streaming live audio.

// Connect and start a Malayalam session
ws.send(JSON.stringify({ type: "start", lang: "ml-IN", session_id: "abc123" }));

// Stream raw PCM audio as binary frames
ws.send(pcmBuffer);

// Receive transcript
// {"type":"final","text":"ഡോക്ടറെ കാണണം","lang":"ml-IN","latency_ms":280}

Batch REST API

For file-based transcription or systems with their own VAD (including our own VEXYL Voice Gateway which uses Silero VAD v5), the batch API accepts WAV, MP3, FLAC, OGG, and M4A files up to 25MB / 5 minutes. Submit a file, poll for status, retrieve the transcript. Supported formats cover almost everything you’ll encounter in the field.

curl -X POST http://localhost:8091/batch/transcribe \
  -H "X-API-Key: your-secret" \
  -F "file=@patient-recording.wav" \
  -F "language_code=ml-IN"

# Response: {"job_id":"batch_a1b2c3","status":"queued","audio_duration":4.52}

How Do I Get It Running in 5 Minutes?

The quickest path is the automated setup script. You’ll need Python 3.10+, a HuggingFace account with access approved for the gated IndicConformer model, and about 3GB of free disk space for the model weights.

1. Request access to the model at huggingface.co/ai4bharat/indic-conformer-600m-multilingual (approval is typically quick)
2. Clone the repo: git clone https://github.com/vexyl-ai/vexyl-stt
3. Run ./setup.sh — it handles venv creation, dependency installation, HuggingFace authentication, and model download
4. Start the server: ./run.sh
5. Open test.html in your browser to test live transcription from your microphone

For Docker deployments, pass your HuggingFace token as a build arg and the model gets baked into the image:

docker build --build-arg HF_TOKEN=$HF_TOKEN -t vexyl-stt .
docker run -p 8080:8080 -e VEXYL_STT_API_KEY=mysecret vexyl-stt

One thing worth knowing before you start: the HuggingFace gated model approval is required. If you hit a 401 or 403, run huggingface-cli whoami and verify your token has read access and that your access request has been approved. This catches most people out.

How Does VEXYL-STT Compare to Cloud STT for Indian Languages?

I’ll be direct: if you’re doing low-volume transcription with no data sovereignty concerns, Sarvam AI’s API is excellent and I’d use it. But if you’re running at scale, building in healthcare, or deploying for government — the economics and compliance picture shifts dramatically in favour of self-hosting.

The latency numbers above are from our production deployment at a tertiary care hospital in Kerala processing Malayalam patient interactions. On the same VPS, local WebSocket transcription consistently returns under 350ms. Cloud providers add network round-trip time on top of model inference — which matters a lot when you’re building a real-time voice assistant.

CTC vs RNNT: Which Decoding Mode Should I Use?

The IndicConformer model supports both CTC and RNNT decoding. VEXYL-STT defaults to CTC because it’s faster — typically 30–50% lower latency on CPU. RNNT is more accurate, especially for longer utterances and noisy telephony audio, but adds latency.

My recommendation: start with CTC (VEXYL_STT_DECODE=ctc), measure your accuracy on real audio, and switch to RNNT (VEXYL_STT_DECODE=rnnt) if accuracy is the limiting factor rather than latency. For most patient interaction use cases we’ve tested, CTC accuracy is sufficient and the latency advantage is worth it.

Production Deployment Tips

A few things we learned running this in production that aren’t obvious from the docs:

• Use PM2 to manage the process. The repo includes PM2 setup instructions. Set --restart-delay=3000 so a crash doesn’t hammer the model reload loop.
• The VAD silence threshold (0.015) may need tuning for telephony audio. PSTN audio often has different gain characteristics than microphone audio. If you’re getting premature cut-offs or missed utterances, adjust the SILENCE_THRESHOLD constant in vexyl_stt_server.py.
• CPU is fine up to ~10 concurrent streams. At that point, look at GPU acceleration. Swap to the CUDA PyTorch build and set VEXYL_STT_DEVICE=auto.
• For Asterisk use cases, bring your own VAD. The VEXYL Voice Gateway uses Silero VAD v5 which is far more accurate than energy-threshold VAD. Use the batch REST API with pre-segmented speech rather than relying on the built-in WebSocket VAD for telephony.

Get Involved

VEXYL-STT is actively maintained and we welcome contributions — new language support, performance improvements, integration examples, and documentation. Star the repo if it’s useful, open an issue if you find a bug, and submit a PR if you’ve built something on top of it.

If you’re looking to go further — full telephony AI with Asterisk, multi-language IVR, voice bots for healthcare or enterprise — that’s what the VEXYL AI Voice Gateway is built for. It’s a self-hosted platform where you bring your own provider keys, with support for Indian languages over PSTN and SIP.

Originally published at vexyl.ai.