# Authentication Source: https://docs.gladia.io/api-reference/authentication Use your API key to authenticate your calls ### Signing Up You will first need to create your account. Sign-up to [app.gladia.io](https://app.gladia.io/?utm_campaign=197609689-Website\&utm_source=documentation\&utm_content=sign-up). You can sign-up through Google and more sign-up methods will be available in the near feature. ### Get your API key Now that you signed up, login to app.gladia.io and go to the [API keys section](https://app.gladia.io/apikeys). We should have already created a default key for you. You can use this one or create your own. Gladia offers 10 Hours of free audio transcription per month if you want to test the service! With your API key, you're now ready to use Gladia APIs. ## Authenticate your API calls Once you got your API key, pass it in the header `x-gladia-key`, like this: ```bash theme={"system"} curl --request GET \ --url https://api.gladia.io/v2/transcription \ --header 'x-gladia-key: YOUR_GLADIA_API_KEY' ``` The header authentication method ensure that you are able to use Gladia in any environment or any cloud-based third party service. This is the **only** authentication method you'll need to use all the API endpoints. Next, we'll see what's the general pre-recorded flow and how to manage it on your side. # Introduction Source: https://docs.gladia.io/api-reference/index Technical documentation for Gladia API If you're not looking for technical API reference documentation, you can check our [Quickstart guide](/chapters/introduction/getting-started). **Gladia API** aims to provide a complete set of resources alongside with its **state-of-the-art API** features to makes the integration as easy as possible, like **webhooks event**, **callback urls** and **audio intelligence** to transform your transcriptions from raw data to usable knowledge, allowing you to focus on your product. The API is structured following the **REST** standard and JSON input/output whenever it's possible, with a user-friendly experience and ensure a **low effort** integration process. Let's get started with learning how to use [Authentication](/api-reference/authentication) with your API calls. # Audio chunk acknowledge (ack) Source: https://docs.gladia.io/api-reference/v2/live/callback/audio-chunk-ack Payload definition for the [`audio_chunk`](/api-reference/v2/live/action/audio-chunk) acknowledgment. # End recording Source: https://docs.gladia.io/api-reference/v2/live/callback/end-recording Payload definition for the callback event `live.end_recording`. # End session Source: https://docs.gladia.io/api-reference/v2/live/callback/end-session Payload definition for the callback event `live.end_session`. # Named Entity Recognition Source: https://docs.gladia.io/api-reference/v2/live/callback/named-entity-recognition Payload definition for the callback event `live.named_entity_recognition`. # Chapterization Source: https://docs.gladia.io/api-reference/v2/live/callback/post-chapterization Payload definition for the callback event `live.post_chapterization`. # Final transcript Source: https://docs.gladia.io/api-reference/v2/live/callback/post-final-transcript Payload definition for the callback event `live.post_final_transcript`. # Summarization Source: https://docs.gladia.io/api-reference/v2/live/callback/post-summarization Payload definition for the callback event `live.post_summarization`. # Post Transcript Source: https://docs.gladia.io/api-reference/v2/live/callback/post-transcript Payload definition for the callback event `live.post_transcript`. # Sentiment Analysis Source: https://docs.gladia.io/api-reference/v2/live/callback/sentiment-analysis Payload definition for the callback event `live.sentiment_analysis`. # Speech End Source: https://docs.gladia.io/api-reference/v2/live/callback/speech-end Payload definition for the callback event `live.speech_end`. # Speech Start Source: https://docs.gladia.io/api-reference/v2/live/callback/speech-start Payload definition for the callback event `live.speech_start`. # Start recording Source: https://docs.gladia.io/api-reference/v2/live/callback/start-recording Payload definition for the callback event `live.start_recording`. # Start session Source: https://docs.gladia.io/api-reference/v2/live/callback/start-session Payload definition for the callback event `live.start_session`. # Stop recording acknowledge (ack) Source: https://docs.gladia.io/api-reference/v2/live/callback/stop-recording-ack Payload definition for the [`stop_recording`](/api-reference/v2/live/action/stop-recording) acknowledgment. # Transcript Source: https://docs.gladia.io/api-reference/v2/live/callback/transcript Payload definition for the callback event `live.transcript`. # Translation Source: https://docs.gladia.io/api-reference/v2/live/callback/translation Payload definition for the callback event `live.translation`. # Delete transcription Source: https://docs.gladia.io/api-reference/v2/live/delete delete /v2/live/{id} Delete a live transcription and all its data (audio file, transcription). # Get result Source: https://docs.gladia.io/api-reference/v2/live/get GET /v2/live/{id} Get live transcription's status, parameters and result. # Download audio file Source: https://docs.gladia.io/api-reference/v2/live/get-audio GET /v2/live/{id}/file Download the audio file recorded during a live transcription. # Initiate a session Source: https://docs.gladia.io/api-reference/v2/live/init POST /v2/live Initiate a live transcription WebSocket session. Use the returned WebSocket url to connect to the WebSocket and start sending audio chunks. Use the returned `id` and the [GET /v2/live/:id](/api-reference/v2/live/get) endpoint to obtain the status and results. * **Security**: Generate the WebSocket URL on your backend and keep your API key private. The init call returns a connectable URL and a session `id` that you can safely pass to web, iOS, or Android clients without exposing credentials in the app. * **Lower infrastructure load**: The secure URL is generated on your backend, the client can connect directly to Gladia's WebSocket server without a pass-through on your side, saving your own resources. * **Resilient reconnection and session continuity**: If the WebSocket disconnects (which can happen on unreliable networks), the session created by the init call lets the client reconnect without losing context. Traditional flows that open a socket first typically force a brand‑new session on disconnect, dropping in‑progress state. # List transcriptions Source: https://docs.gladia.io/api-reference/v2/live/list GET /v2/live List all the live transcriptions matching the parameters. # End recording Source: https://docs.gladia.io/api-reference/v2/live/webhook/end-recording Payload definition for the webhook event `live.end_recording`. # End session Source: https://docs.gladia.io/api-reference/v2/live/webhook/end-session Payload definition for the webhook event `live.end_session`. # Start recording Source: https://docs.gladia.io/api-reference/v2/live/webhook/start-recording Payload definition for the webhook event `live.start_recording`. # Start session Source: https://docs.gladia.io/api-reference/v2/live/webhook/start-session Payload definition for the webhook event `live.start_session`. # Live WebSocket Source: https://docs.gladia.io/api-reference/v2/live/websocket WebSocket to send audio chunks and receive events. # Error Source: https://docs.gladia.io/api-reference/v2/pre-recorded/callback/error Payload definition for the callback event `transcription.error`. # Success Source: https://docs.gladia.io/api-reference/v2/pre-recorded/callback/success Payload definition for the callback event `transcription.success`. # Delete transcription Source: https://docs.gladia.io/api-reference/v2/pre-recorded/delete delete /v2/pre-recorded/{id} Delete a pre-recorded transcription and all its data (audio file, transcription). # Get result Source: https://docs.gladia.io/api-reference/v2/pre-recorded/get GET /v2/pre-recorded/{id} Get pre-recorded transcription's status, parameters and result. # Download audio file Source: https://docs.gladia.io/api-reference/v2/pre-recorded/get-audio GET /v2/pre-recorded/{id}/file Download the audio file used on a pre-recorded transcription. # Initiate a transcription Source: https://docs.gladia.io/api-reference/v2/pre-recorded/init POST /v2/pre-recorded Initiate a pre-recorded transcription job. Use the returned `id` and the [GET /v2/pre-recorded/:id](/api-reference/v2/pre-recorded/get) endpoint to obtain the results. Pass `model` to choose the transcription model: **`"solaria-3"`** — our latest model: highest accuracy on European real-world audio. * **Async (pre-recorded) only** — not available for live transcription. * **Languages:** English, French, German, Spanish, Italian * **Single language only** — pass exactly one language in `language_config.languages` (no code switching). * All Audio Intelligence add-ons available. **`"solaria-1"`** — our generalist model: maximum language coverage across any domain. * Available for async and live. * Code switching and multi-language configuration (100+ languages covered) * All Audio Intelligence add-ons available. If omitted, the API uses the default model. (Solaria-1) # List transcriptions Source: https://docs.gladia.io/api-reference/v2/pre-recorded/list GET /v2/pre-recorded List all the pre-recorded transcriptions matching the parameters. # Created Source: https://docs.gladia.io/api-reference/v2/pre-recorded/webhook/created Payload definition for the webhook event `transcription.created`. # Error Source: https://docs.gladia.io/api-reference/v2/pre-recorded/webhook/error Payload definition for the webhook event `transcription.error`. # Success Source: https://docs.gladia.io/api-reference/v2/pre-recorded/webhook/success Payload definition for the webhook event `transcription.success`. # Delete transcription Source: https://docs.gladia.io/api-reference/v2/transcription/delete delete /v2/transcription/{id} (Deprecated) Prefer the more specific [pre-recorded endpoint](/api-reference/v2/pre-recorded/delete).

Delete a transcription and all its data (audio file, transcription). # Get result Source: https://docs.gladia.io/api-reference/v2/transcription/get GET /v2/transcription/{id} (Deprecated) Prefer the more specific [pre-recorded endpoint](/api-reference/v2/pre-recorded/get).

Get transcription's status, parameters and result. # Download audio file Source: https://docs.gladia.io/api-reference/v2/transcription/get-audio GET /v2/transcription/{id}/file (Deprecated) Prefer the more specific [pre-recorded endpoint](/api-reference/v2/pre-recorded/get-audio).

Download the audio file used on a transcription. # Initiate a transcription Source: https://docs.gladia.io/api-reference/v2/transcription/init POST /v2/transcription (Deprecated) Prefer the more specific [pre-recorded endpoint](/api-reference/v2/pre-recorded/init).

Initiate a pre-recorded transcription job. Use the returned id and the [GET /v2/transcription/:id](/api-reference/v2/transcription/get) endpoint to obtain the results. # List transcriptions Source: https://docs.gladia.io/api-reference/v2/transcription/list GET /v2/transcription (Deprecated) Prefer the more specific [pre-recorded endpoint](/api-reference/v2/pre-recorded/list).

List all the transcriptions matching the parameters. # Upload a file Source: https://docs.gladia.io/api-reference/v2/upload/audio-file post /v2/upload Upload a file for use in a pre-recorded job. # Audio-to-LLM Source: https://docs.gladia.io/chapters/audio-intelligence/audio-to-llm Run your own prompts on a pre-recorded transcript with an LLM - summaries, Q&A, extraction, and more. Pre-recorded **Audio-to-LLM** runs once the transcription is generated. You provide **one or more prompts**; each prompt is executed against the **transcript text** from the same job using the configured model, yielding **one LLM response per prompt**. Use it to extract action items, answer questions about the recording, or run any text analysis you express in natural language. Unlike the built-in [Summarization](/chapters/audio-intelligence/summarization) feature — which produces a fixed-format summary — Audio-to-LLM lets you write **your own instructions**: ask for a summary in the exact format, tone, and level of detail your product needs, or combine a summary with other analyses (action items, compliance checks) in a single request. ## Usage 1. Include `audio_to_llm: true` and an `audio_to_llm_config` object (at minimum, a `prompts` array) in your [pre-recorded transcription request](/chapters/pre-recorded-stt/quickstart). 2. Gladia transcribes the audio, along with any other audio-intelligence options you enabled on that request. 3. Each prompt is run on the resulting transcript via the LLM. 4. The API returns **one result object per prompt** (same order as `prompts`), each containing the original `prompt` and the model `response`. Audio-to-LLM sends **plain transcript text** to the model. Raw audio and other fields from the transcription response are **not** added to the LLM prompt context. ## Model selection By default the model used to execute your prompts is **[GPT 5.4 Nano](https://openrouter.ai/openai/gpt-5.4-nano)** (`openai/gpt-5.4-nano`), a fast option suited to high-volume summaries and extraction. The `model` can be customized when you need stronger reasoning, richer analysis, longer outputs, or behavior that fits a specific model. You can use **any model listed on [OpenRouter](https://openrouter.ai/models)** by setting the `model` key. Prices reflect the public OpenRouter rate plus a platform fee added by Gladia. ## Example A single prompt is enough to get started (you can omit `model` to use the default): ```json Pre-recorded theme={"system"} { "audio_to_llm": true, "audio_to_llm_config": { "prompts": [ "Summarize the transcript in three bullet points." ] } } ``` Example response shape for one prompt: ```json Pre-recorded theme={"system"} { "success": true, "is_empty": false, "results": [ { "success": true, "is_empty": false, "results": { "prompt": "Summarize the transcript in three bullet points.", "response": "- Intro and context\n- Main discussion\n- Conclusion and next steps" }, "exec_time": 1.4122809978485107, "error": null } ], "exec_time": 4.521103805541992, "error": null } ``` ## Example: post-meeting workflow For a **post-meeting** pass, you might ask for bullet takeaways, a short summary, and follow-up actions for the next meeting: ```json Pre-recorded theme={"system"} { "audio_to_llm": true, "audio_to_llm_config": { "model": "openai/gpt-5.4", "prompts": [ "Summarize the meeting as bullet points: main topics, decisions, and open questions.", "Give a concise paragraph summarizing what this meeting was about and the outcome.", "List action items and follow-ups to prepare for the next meeting; include owners if they were mentioned." ] } } ``` With this configuration, your output might look like this: ```json Pre-recorded theme={"system"} { "success": true, "is_empty": false, "results": [ { "success": true, "is_empty": false, "results": { "prompt": "Summarize the meeting as bullet points: main topics, decisions, and open questions.", "response": "- **Roadmap Q2**: Team aligned on shipping the billing integration first.\n- **Decision**: Weekly sync moved to Tuesday.\n- **Open question**: Whether to support SSO in v1 is still TBD." }, "exec_time": 1.7726809978485107, "error": null }, { "success": true, "is_empty": false, "results": { "prompt": "Give a concise paragraph summarizing what this meeting was about and the outcome.", "response": "The group reviewed Q2 priorities, agreed to prioritize billing, and rescheduled the standing meeting. SSO scope was left for a follow-up once design signs off." }, "exec_time": 1.5122809978485107, "error": null }, { "success": true, "is_empty": false, "results": { "prompt": "List action items and follow-ups to prepare for the next meeting; include owners if they were mentioned.", "response": "- **Alex**: Finalize SSO requirements doc by Friday.\n- **Jamie**: Share billing API cutover checklist with the team.\n- **Everyone**: Review the updated roadmap draft before next sync." }, "exec_time": 1.8932809978258485, "error": null } ], "exec_time": 6.267103805541992, "error": null } ``` ## Response shape * Top-level `results` is an **array** with **one entry per prompt**, in the **same order** as `audio_to_llm_config.prompts`. * Each entry includes `success`, optional `error`, timing fields, and nested `results.prompt` / `results.response` with the LLM output for that prompt. ## Pricing The input provided to the LLM is the full transcription. All prices are per 1M tokens and include platform fees. | Model | `model` config | Context Window | Input | Output | | ------------------------------------------------------------------------------------- | ------------------------------- | -------------- | ------ | ------- | | [OpenAI: GPT-5.4 Nano](https://openrouter.ai/openai/gpt-5.4-nano) | `openai/gpt-5.4-nano` | 400k | \$0.26 | \$1.76 | | [OpenAI: GPT-5.4](https://openrouter.ai/openai/gpt-5.4) | `openai/gpt-5.4` | 1.1M | \$3.25 | \$19.50 | | [Anthropic: Claude Opus 4.7](https://openrouter.ai/anthropic/claude-opus-4.7) | `anthropic/claude-opus-4.7` | 1M | \$6.50 | \$32.50 | | [Google: Gemini 3.1 Pro Preview](https://openrouter.ai/google/gemini-3.1-pro-preview) | `google/gemini-3.1-pro-preview` | 1M | \$2.60 | \$15.60 | | [xAI: Grok 4.20](https://openrouter.ai/x-ai/grok-4.20) | `x-ai/grok-4.20` | 2M | \$2.60 | \$7.80 | | [Meta: Llama 4 Maverick](https://openrouter.ai/meta-llama/llama-4-maverick) | `meta-llama/llama-4-maverick` | 1M | \$0.20 | \$0.78 | # Chapterization Source: https://docs.gladia.io/chapters/audio-intelligence/chapterization The Chapterization model segments the audio into distinct chapters, each with a descriptive headline and summaries Pre-recorded Live This feature is in **Alpha** state.
Breaking changes may still be introduced to this API, but advance notice will be sent. We're looking for feedback to improve this feature, [share yours on Discord](https://discord.gg/T22a4ETUQp). The chapterization model segments the audio into logical chapters based on the audio and content, and makes it easier to navigate long audios such as meeting recordings. Each chapter will contain its start and end time, as well as a summary, headline, bottom line "gist" and keywords. ## Usage Enable chapterization by setting the appropriate flag: ```json Pre-recorded theme={"system"} { "chapterization": true } ``` ```json Live theme={"system"} { "post_processing": { "chapterization": true }, "messages_config": { "receive_post_processing_events": true } } ``` ## Result The transcription result will contain a `"chapterization"` key with the output of the model: ```json Pre-recorded theme={"system"} { "transcription":{...}, "chapterization": { "success": true, "is_empty": false, "results": [ { "summary": "In a world where minimalism is valued, yet excess is desired, hope for the future remains. The past predicts the present, which is yet to be determined.", "headline": "Headline: \"Embracing Hope: The Interconnectedness of Past, Present, and Future\"", "gist": "Embracing Hope: Past, Present, Future Interconnected", "keywords": [ "Split infinity", "less is more", "too much", "hope", "present" ], "start": 0.0, "end": 19.83977, }], "exec_time": 5.078396797180176, "error": null } } ``` ```json Live theme={"system"} { "session_id": "550e8400-e29b-41d4-a716-446655440000", "created_at": "2025-09-19T12:46:00Z", "type": "post_chapterization", "error": null, "data": { "results": [ { "headline": "Project kickoff", "start": 0, "end": 60, "keywords": [ "timeline", "owner" ] } ] } } ``` # Custom metadata Source: https://docs.gladia.io/chapters/audio-intelligence/custom-metadata Attach metadata to transcription jobs for filtering and retrieval Pre-recorded Live You can add metadata to your transcription using the `custom_metadata` property. This helps recognize your transcription when fetching results, and enables filtering via the list endpoints. * **Pre-recorded**: add `custom_metadata` to your POST request to `/v2/pre-recorded`, then filter via GET `/v2/pre-recorded` or fetch via GET `/v2/pre-recorded/:id`. * **Live**: add `custom_metadata` to your session configuration, then filter via the [GET `/v2/live` list endpoint](/api-reference/v2/live/list) or fetch via the [GET `/v2/live/:id` endpoint](/api-reference/v2/live/get). For example: ```json theme={"system"} "custom_metadata": { "internalUserId": 2348739875894375, "paymentMethod": { "last4Digits": 4576 }, "internalUserName": "Spencer" } ``` Now, you'll be able to filter transcription jobs based on their metadata: ```bash Pre-recorded theme={"system"} https://api.gladia.io/v2/pre-recorded?custom_metadata={"internalUserId": "2348739875894375"} ``` ```bash Live theme={"system"} https://api.gladia.io/v2/live?custom_metadata={"internalUserId": "2348739875894375"} ``` or: ```bash Pre-recorded theme={"system"} https://api.gladia.io/v2/pre-recorded?custom_metadata={"paymentMethod": {"last4Digits": 4576}, "internalUserName": "Spencer"} ``` ```bash Live theme={"system"} https://api.gladia.io/v2/live?custom_metadata={"paymentMethod": {"last4Digits": 4576}, "internalUserName": "Spencer"} ``` `custom_metadata` cannot be longer than 2000 characters when stringified. # Custom spelling Source: https://docs.gladia.io/chapters/audio-intelligence/custom-spelling Normalize spelling variants to your preferred forms Pre-recorded Live As Speech-to-text models are trained on general vocabulary, under-represented words such as brand names, proper nouns, or domain-specific terms are often transcribed incorrectly. Custom Spelling is a post-processing operation that applies literal matching between the correct word and the pronunciations entries. When there is a literal match, the transcribed text is replaced with your term. If the word comes out garbled or replaced by something phonetically similar (e.g. **"le vin"** instead of **"Levain"**), use **[Custom vocabulary](/chapters/audio-intelligence/custom-vocabulary)** instead. Custom vocabulary matches on phonemes, not literal text. ## How it works Gladia runs custom spelling on the **transcript text** after transcription: 1. Gladia scans the output for strings listed in your dictionary **values**. 2. When a variant is found, it is replaced with the corresponding **key**. 3. Each entry supplies: * **Key** — the spelling to write (case-sensitive). * **Values** — variant strings to find (case-insensitive; can be multiple words). Custom spelling is **precise but strict**: Gladia replaces only strings listed in your dictionary and leaves everything else unchanged. ### Example: name "Gorish" If the model outputs **"gaurish"** or **"ghorish"**, Gladia replaces them with **"Gorish"** when they appear in your dictionary: ```json theme={"system"} "Gorish": ["ghorish", "gaurish", "gaureish", "geurish", "go rich"] ``` Custom spelling is not based on phoneme-matching but **literal matching** so make sure to list every spelling carefully as missing variants are never inferred. ## When to use custom vocabulary vs. custom spelling Use **[Custom spelling](/chapters/audio-intelligence/custom-spelling)** when the model outputs a recognizable but wrong form. It applies **literal string matching** on variants you list (e.g. **"data-science"** → **"Data Science"**). **List every close variant** the model might output. Use **[Custom vocabulary](/chapters/audio-intelligence/custom-vocabulary)** when the model outputs garbled or sound-alike text. It applies **phoneme-based matching** on entries you define (e.g. **"le vin"** / **"levine"** → **"Levain"**). **Add pronunciations** for each spelling the model might produce. | | Custom spelling | Custom vocabulary | | --------------- | -------------------------------------------- | --------------------------------------- | | **Matches on** | Exact text in the transcript | How words sound | | **Best for** | Wrong spelling, punctuation, formatting | Phonetically similar mis-transcriptions | | **You provide** | All the words that the model outputs wrongly | `value`, `pronunciations`, `intensity` | **Rule of thumb:** start with a transcription run *without* any custom vocabulary. Look at what the output actually says. If the word appears but is just misspelled, custom spelling is the simpler and safer fix. If the word is completely garbled, that's when custom vocabulary is the right tool. ## Example configuration ```json Pre-recorded theme={"system"} { "custom_spelling": true, "custom_spelling_config": { "spelling_dictionary": { "Gorish": ["ghorish", "gaurish", "gaureish", "geurish", "go rich"], "Data Science": ["data-science", "data science"], ".": ["period", "full stop"], "SQL": ["sequel"] } } } ``` ```json Live theme={"system"} { "realtime_processing": { "custom_spelling": true, "custom_spelling_config": { "spelling_dictionary": { "Gorish": ["ghorish", "gaurish", "gaureish", "geurish", "go rich"], "Data Science": ["data-science", "data science"], ".": ["period", "full stop"], "SQL": ["sequel"] } } } } ``` ## Parameter reference Map of preferred spellings (**keys**) to variant strings the model might output (**values**). The correct word to transcribe. This parameter is **case-sensitive** so `"Gorish"` and `"gorish"` are different keys. Variant strings to search for in the transcript. Those strings are **case-insensitive.** and can be multiple words (e.g. `"full stop"`). ## Tuning tips * **Collect variants from real transcripts** — run without custom spelling first, then add keys and values from what the model actually outputs. * **Match key capitalization** to how the word should appear in the final transcript. * **List phonetically different strings separately** — custom spelling will not group them the way custom vocabulary does. * **Move garbled or sound-alike output to [custom vocabulary](/chapters/audio-intelligence/custom-vocabulary)** when listing every variant becomes impractical. ## Recommended workflow 1. **Transcribe without custom spelling** and note misspelled terms. 2. **Route each term:** recognizable but wrong spelling → custom spelling; garbled or phonetically wrong → custom vocabulary. 3. **Build the dictionary** — correct form as the key, every variant you have seen as values. 4. **Transcribe again** — confirm replacements and check that nothing else was changed unexpectedly. 5. **Refine:** add new variants as they appear in production audio. # Custom vocabulary Source: https://docs.gladia.io/chapters/audio-intelligence/custom-vocabulary Improve recognition of domain-specific words and phrases Pre-recorded Live As Speech-to-text models are trained on general vocabulary, under-represented words such as brand names, proper nouns, or domain-specific terms are often transcribed incorrectly. Custom Vocabulary is a post-processing operation that compares **phonemes** between the transcript and your pronunciations entries. When the phonetic match is close enough, the transcribed text is replaced with your term. If you already know which *text* variants the model produces and only need to normalize spelling, use **[Custom spelling](/chapters/audio-intelligence/custom-spelling)** instead. Custom spelling is not based on phonemes but literal matching. ## How it works Custom vocabulary operates at a **text level** and is based on **phoneme similarity**. Once the transcription is generated, Gladia converts both the transcribed words and your vocabulary entries into phonemes, then compares them. The `intensity` controls how aggressively the model applies replacements: a higher intensity means the model will replace words more readily (wider phoneme matching), while a lower intensity requires a closer phoneme match before a replacement is made. The `pronunciations` field lets you provide **plain-text alternative spellings that reflect how the word actually sounds** in speech. These are *not* phonetic notation. Just write the word the way someone might naively spell it based on how it sounds. Gladia converts these strings to phonemes internally. For example, if your term is "Nietzsche", you might add `["Niche", "Neechee"]` as pronunciations. This widens the phoneme net without having to raise the intensity (which would increase false positives across the board). ## When to use custom vocabulary vs. custom spelling Use **[Custom spelling](/chapters/audio-intelligence/custom-spelling)** when the model outputs a recognizable but wrong form. It applies **literal string matching** on variants you list (e.g. **"data-science"** → **"Data Science"**). **List every close variant** the model might output. Use **[Custom vocabulary](/chapters/audio-intelligence/custom-vocabulary)** when the model outputs garbled or sound-alike text. It applies **phoneme-based matching** on entries you define (e.g. **"le vin"** / **"levine"** → **"Levain"**). **Add pronunciations** for each spelling the model might produce. | | Custom spelling | Custom vocabulary | | --------------- | -------------------------------------------- | --------------------------------------- | | **Matches on** | Exact text in the transcript | How words sound | | **Best for** | Wrong spelling, punctuation, formatting | Phonetically similar mis-transcriptions | | **You provide** | All the words that the model outputs wrongly | `value`, `pronunciations`, `intensity` | **Rule of thumb:** start with a transcription run *without* any custom vocabulary. Look at what the output actually says. If the word appears but is just misspelled, custom spelling is the simpler and safer fix. If the word is completely garbled, that's when custom vocabulary is the right tool. ## Example configuration ```json Pre-recorded theme={"system"} { "audio_url": "YOUR_AUDIO_URL", "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": [ "Gladia", {"value": "Solaria"}, { "value": "Salesforce", "pronunciations": ["sell force", "sale forces"], "intensity": 0.5, "language": "en" }, ], "default_intensity": 0.4 } } ``` ```json Live theme={"system"} { "realtime_processing": { "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": [ "Gladia", {"value": "Solaria"}, { "value": "Salesforce", "pronunciations": ["sell force", "sale forces"], "intensity": 0.5, "language": "en" }, ], "default_intensity": 0.4 } } } ``` ## Parameter reference The correct word you want to be transcribed. Words with different spellings the word might be mis-spelled or mis-transcribed. Per-entry intensity, we suggest **0.4–0.6** as value. Inherits `default_intensity` when omitted. Language used for phoneme comparison (defaults to the transcription language). Set this when a term is pronounced in a different language than the rest of the audio. Global intensity for entries. We suggest **0.4–0.6** raise if terms are missed, lower if unrelated words get replaced. ## Tuning tips * **Start at `default_intensity` 0.4** and adjust per entry only when needed. * **Add `pronunciations` before raising `intensity`** — variants narrow what can match without loosening every comparison. * **Keep lists focused** — every transcribed word is compared against every entry; long lists increase false positives. * **Move stable misspellings to [custom spelling](/chapters/audio-intelligence/custom-spelling)** when the model already outputs a recognizable (but wrong) form. ## Recommended workflow 1. **Transcribe without custom vocabulary** and note mis-transcribed terms. 2. **Route each term:** garbled or phonetically wrong output → custom vocabulary; recognizable but misspelled → custom spelling. 3. **Add entries** with `pronunciations` and `default_intensity` around **0.4–0.6**. 4. **Transcribe again** — confirm targets appear and scan for false positives. 5. **Refine:** lower `intensity`, tighten `pronunciations`, or move stubborn terms to custom spelling. # Named Entity Recognition Source: https://docs.gladia.io/chapters/audio-intelligence/named-entity-recognition The Named Entity Recognition model automatically identifies and categorizes key information in the audio. Pre-recorded Live This feature is in **Alpha** state.
Breaking changes may still be introduced to this API, but an advanced notice will be sent. We're looking for feedback to improve this feature, [share yours on Discord](https://discord.gg/T22a4ETUQp). **Named Entity Recognition** (also known as **Entity Detection**) detects and categorizes key information in the audio. ## Usage To enable named entity recognition simply set the appropriate flag: ```json Pre-recorded theme={"system"} { "named_entity_recognition": true } ``` ```json Live theme={"system"} { "realtime_processing": { "named_entity_recognition": true }, "messages_config": { "receive_realtime_processing_events": true } } ``` ## Result The transcription result will contain a `"named_entity_recognition"` key with the output of the model: ```json Pre-recorded theme={"system"} { "transcription":{...} "named_entity_recognition": { "success": true, "is_empty": false, "results": [ { "entity_type": "EMAIL_ADDRESS", "text": "privacy@gladia.io", "start": 123.4, "end": 124.5 }, { "entity_type": "AGE", "text": "27 years old", "start": 234.7, "end": 235.6 } ], "exec_time": 1.7726809978485107, "error": null } } ``` ```json Live theme={"system"} { "session_id": "550e8400-e29b-41d4-a716-446655440000", "created_at": "2025-09-19T12:34:30Z", "type": "named_entity_recognition", "error": null, "data": { "utterance_id": "utt_002", "utterance": { "text": "meeting with Alice at 3pm", "start": 7, "end": 10.5 }, "results": [ { "entity_type": "PERSON", "text": "Alice", "start": 2, "end": 2.5 }, { "entity_type": "TIME", "text": "3pm", "start": 3, "end": 3.3 } ] } } ``` # Supported Regulations Gladia helps you cover the following regulations for your business: * [EU General Data Protection Regulation (GDPR)](https://gdpr.eu/eu-gdpr-personal-data/) * [California Privacy Rights Act (CPRA)](https://iapp.org/news/a/new-categories-new-rights-the-cpras-opt-out-provision-for-sensitive-data/) * [USA Health Insurance Portability and Accountability Act (HIPAA Safe Harbor)](https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html) * [Quebec Privacy Act (law 25)](https://iapp.org/news/a/quebecs-bill-64-the-first-of-many-privacy-modernization-bills-in-canada/) * [Japan Act on the Protection of Personal Information (APPI)](https://www.dlapiperdataprotection.com/index.html?t=definitions\&c=JP) To automatically redact PII in your transcript, see [PII Redaction feature](/chapters/audio-intelligence/pii-redaction) # Supported Entities | Entity Type | Description | Example | Information Type | Regulatory Compliance | Note | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ACCOUNT\_NUMBER | Customer account or membership identification number | Policy No. 10042992; Member ID: HZ-5235-001 | PII | | Note: Full support for English; Multilingual support in progress | | AGE | Numbers associated with an individual’s age | 27 years old; 18 months old | PII | GDPR,
HIPAA,
Quebec Privacy Act,
APPI | When given in years, only the number is flagged, but both number and time unit are flagged when given in other units like months or weeks
Also includes age ranges:
29-35 years old; 18+; A man in his forties | | DATE | Specific calendar dates, which can include days of the week, dates, months, or years.

See also: DATE\_INTERVAL, DOB | Friday, Dec. 18, 2002; Dated: 02/03/97 | PII | HIPAA,
Quebec Privacy Act | If no calendar date is specified, days of the week are not flagged:
Your appointment is on Monday
Indexical terms are not flagged:
yesterday; tomorrow | | DATE\_INTERVAL | Broader time periods, including date ranges, months, seasons, years, and decades

See also: DATE, DOB | 2020-2021; 5-9 May; January 1984 | PII | HIPAA | | | DOB | Dates of birth

See also: DATE, DATE\_INTERVAL | Born: March 7, 1961 | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | DRIVER\_LICENSE | Driver's permit numbers

See also: VEHICLE\_ID | DL# 134711-320 | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | Includes International Driving Permits (IDP) and Pilot’s licenses | | DURATION | Periods of time, specified as a number and a unit of time | 8 months; 2 years | PII | | Full support for English; Multilingual support in progress | | EMAIL\_ADDRESS | Email addresses | | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | EVENT | Names of events or holidays | Olympics; Yom Kippur | PII | | | | FILENAME | Names of computer files, including the extension or filepath | Taxes/2012/brad-tax-returns.pdf | PII | | | | GENDER\_SEXUALITY | Terms indicating gender identity or sexual orientation, including slang terms | female; bisexual; trans | PII | CPRA,
GDPR,
GDPR Sensitive,
APPI Sensitive | | | HEALTHCARE\_NUMBER | Healthcare numbers and health plan beneficiary numbers | Policy No.: 5584-486-674-YM | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | Includes medical record numbers, health insurance policy/account numbers, and member IDs, for example, German Sozialversicherungsnummer (also used as SSN), Philippine PhilHealth ID number, Ukrainian VHI number | | IP\_ADDRESS | Internet IP address, including IPv4 and IPv6 formats | 192.168.0.1
2001:db8:0:0:0:8a2e::7334 | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | LANGUAGE | Names of natural languages | Korean; French | PII | GDPR,
GDPR Sensitive,
APPI Sensitive | | | LOCATION | Metaclass for any named location reference; See subclasses below | Eritrea; Lake Victoria | PII | GDPR,
HIPAA,
APPI | May co-occur with ORGANIZATION when the context refers explicitly to the organization’s location
The patient was transferred to Northwest General Hospital | | LOCATION\_ADDRESS | Full or partial physical mailing addresses, which can include: building name or number, street, city, county, state, country, zip code | 25/300 Adelaide T., Perth WA 6000, Aus.
145 Windsor St.
Mail to: Kollwitzstr 13, 10405, Berlin | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | LOCATION\_CITY | Municipality names, including villages, towns, and cities | Toronto; Berlin; Denpasar | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | LOCATION\_COORDINATE | Geographic positions referred to using latitude, longitude, and/or elevation coordinates | We’re at 40.748440 and -73.984559 | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | LOCATION\_COUNTRY | Country names | Canada; Namibia | PII | GDPR,
APPI | | | LOCATION\_STATE | State, province, territory, or prefecture names | Ontario; Arkansas; Ich lebe in NRW | PII | GDPR,
APPI | | | LOCATION\_ZIP | Zip codes (including Zip+4), postcodes, or postal codes | 90210; B2N 3E3 | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | Optimized for various English-speaking locales (Australia, Canada, United Kingdom, United States), as well as international equivalents | | MARITAL\_STATUS | Terms indicating marital status | single; common-law; ex-wife; married | PII | APPI Sensitive | | | MONEY | Names and/or amounts of currency | 15 pesos; \$94.50 | PII | | | | NAME | Names of individuals, not including personal titles such as ‘Mrs.’ or ‘Mr.’ | Dwayne Johnson; Mr. Khanna | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | NAME\_FAMILY | Names indicating a person’s family or community; often a last name in Western cultures and first name in Eastern cultures | François Truffaut; Ozu Yasujirō | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | NAME\_GIVEN | Names given to an individual, usually at birth; often first / middle names in Western cultures and middle / last names in Eastern cultures | François Truffaut; Ozu Yasujirō | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | NAME\_MEDICAL\_PROFESSIONAL | Full names, including professional titles and certifications, of medical professional, such as doctors and nurses | Attending physician: Dr. Kay Martinez, MD | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | NUMERICAL\_PII | Numerical PII (including alphanumeric strings) that doesn't fall under other categories. See also a section below on international variants as some of them are mapped to this category, for example, Belgian BTW nummer or European VAT number. | | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | Includes the following: numbers in the medical field, such as device serial numbers, POS codes, NPI numbers, etc.; computer numbers like MAC addresses, cookie IDs, VPNs, error codes, access codes, message IDs, etc.; business-related numbers like DUNS numbers, company registration numbers, provider IDs, etc.; numbers related to purchasing, like order IDs, transaction numbers, confirmation numbers, tracking numbers, etc.; also numbers assigned to various forms of IDs, files, documents, proceedings, invoices, claim IDs, record IDs, etc. | | OCCUPATION | Job titles or professions | professor; actors; engineer; CPA | PII | Quebec Privacy Act,
APPI | | | ORGANIZATION | Names of organizations or departments within an organization | BHP; McDonald's; LAPD | PII | Quebec Privacy Act,
APPI | May co-occur with LOCATION when the context refers explicitly to the organization’s location
Donations can be brought to Royal Canadian Legion Branch 43 | | ORGANIZATION\_MEDICAL\_FACILITY | Names of medical facilities, such as hospitals, clinics, pharmacies, etc. | Northwest General Hospital; Union Family Health Clinic | PII | Quebec Privacy Act,
APPI | | | ORIGIN | Terms indicating nationality, ethnicity, or provenance | Canadian; Sri Lankan | PII | CPRA,
GDPR,
GDPR Sensitive,
Quebec Privacy Act,
APPI Sensitive | | | PASSPORT\_NUMBER | Passport numbers, issued by any country | PA4568332; NU3C6L86S12 | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | PASSWORD | Account passwords, PINs, access keys, or verification answers | 27%alfalfa; temp1234
My mother's maiden name is Smith | PII | CPRA,
APPI | | | PHONE\_NUMBER | Telephone or fax numbers | +4917643476050 | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | PHYSICAL\_ATTRIBUTE | Distinctive bodily attributes, including terms indicating race | I'm 190cm tall; He belongs to the Black students’ association | PII | CPRA,
GDPR,
GDPR Sensitive,
APPI Sensitive | | | POLITICAL\_AFFILIATION | Terms referring to a political party, movement, or ideology | liberal; Republican | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | RELIGION | Terms indicating religious affiliation | Hindu; Presbyterian | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | SSN | Social Security Numbers or international equivalent government identification numbers | 078-05-1120; \*\*\*-\*\*\*-3256 | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | Includes, for example, Australian TFN, Belgian NISS, British NIN, Canadian SIN, Dutch BSN, German Sozialversicherungsnummer (also used as a healthcare number, see: HEALTHCARE\_NUMBER), French INSEE, Indian Aadhaar, Italian TIN, Philippine SSS, Spanish NUSS, Ukrainian TIN, and Mexican NSS formats. Flags mentions of complete numbers as well as the last four digits only. | | TIME | Expressions indicating clock times | 19:37:28; 10pm EST | PII | | | | URL | Internet addresses | [www.private-ai.com](http://www.private-ai.com/) | PII | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act | | | USERNAME | Usernames, login names, or handles | privateairocks; @\_PrivateAI | PII | CPRA,
GDPR,
APPI | | | VEHICLE\_ID | Vehicle identification numbers (VINs), vehicle serial numbers, and license plate numbers

See also: DRIVER\_LICENSE | 5FNRL38918B111818; BIF7547 | PII | CPRA,
GDPR,
HIPAA,
APPI | | | ZODIAC\_SIGN | Names of Zodiac signs | Aries; Taurus | PII | | | | BLOOD\_TYPE | Blood types | She's type AB positive | PHI | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act | | | CONDITION | Names of medical conditions, diseases, syndromes, deficits, disorders | chronic fatigue syndrome; arrhythmia; depression | PHI | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI Sensitive | | | DOSE | Medically prescribed quantity of a medication | limit intake to 700 mg/day | PHI | | | | DRUG | Medications, vitamins, and supplements | advil; Acetaminophen; Panadol | PHI | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI Sensitive | | | INJURY | Bodily injuries, including mutations, miscarriages, and dislocations | I broke my arm; I have a sprained wrist | PHI | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI Sensitive | | | MEDICAL\_PROCESS | Medical processes, including treatments, procedures, and tests | heart surgery; CT scan | PHI | CPRA,
DPR,
HIPAA,
Quebec Privacy Act,
APPI Sensitive | | | STATISTICS | Medical statistics | 18% of patients | PHI | HIPAA,
Quebec Privacy Act | | | BANK\_ACCOUNT | Bank account numbers and international equivalents, such as IBAN | Acct. No.: 012345-67 | | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | CREDIT\_CARD | Credit card numbers | 0123 0123 0123 0123
\*\*\*\* \*\*\*\* \*\*\*\*4252 | | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | Includes debit, ATM, Direct Debit, PrePay, Charge Cards, and support for cards that do not have 16 digits such as American Express or China UnionPay cards. Flags mentions of complete numbers as well as the last four digits only. | | CREDIT\_CARD\_EXPIRATION | Expiration date of a credit card | Expires: July 2023; Exp: 02/28 | | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | | | CVV | 3- or 4-digit card verification codes and equivalents | CVV: 080 | | CPRA,
GDPR,
HIPAA,
Quebec Privacy Act,
APPI | Includes institution-specific variants:

American Express: CID (card ID), CVD (card verification data) CSC / 3CSC (card security code)
China UnionPay: CVN (card validation number)
CIBC Mastercard: SPC (signature panel code)
Discover: CID (card ID), CVD (card verification data)
ELO (Brazil): CVE (Elo verification code)
JCB (Japan Credit Bureau): CAV (card authentication value)
Mastercard: CVC (card validation code)
VISA: CVV (card verification value) | | ROUTING\_NUMBER | Routing number associated with a bank or financial institution | 012345678 | | | Includes international equivalents: Canadian & British sort codes, Australian BSB numbers, Indian Financial System Codes, Branch/transit numbers, Institution numbers, and Swift codes | | CORPORATE\_ACTION | Any action a company takes that could affect its stock value or its shareholders | Bridge Investment Group LLC (later renamed Bridge Investment Group Holdings LLC); We’ve merged two neighboring retail locations | | | Only available with our English models. | | FINANCIAL\_METRIC | Financial metrics or financial ratios are quantitative indicators of a company’s financial health | adjusted earnings per share declined year-over-year; Online sales slow as UK shoppers rein in Christmas spending | | | Only available with our English models. | | MEDICAL\_CODE | Codes belonging to medical classification systems such as SNOMED, ICD-10, NDC, etc. | 1981-03-11T04:11:32-03:00 Forearm sprain SNOMED-CT 70704007;
R74.8 Abnormal levels of other serum enzymes | PHI | CPRA,
GDPR,
GDPR Sensitive,
HIPAA,
Quebec Privacy Act,
APPI Sensitive | Only available with our English models. | | PRODUCT | Names or model numbers of items made by an organization; includes intangible products like software and games, as well as other services | iPhone; Toyota Camry | | | Only available with our English models. | | TREND | A description of the “quality” or the direction in which a financial measurement is going | reflecting the accelerating shift of off-line to online; amid rising costs and shrinking profits | | | Only available with our English models. | # PII Redaction Source: https://docs.gladia.io/chapters/audio-intelligence/pii-redaction Automatically detect and redact personally identifiable information in pre-recorded transcripts Pre-recorded **PII Redaction** automatically detects and replaces sensitive entities (names, emails, addresses, etc.) in your transcript output. This feature is only available for **pre-recorded** transcription. \ \ Handling audio data often involves processing conversations that contain personally identifiable information such as names, phone numbers, or financial details. Redacting PII helps you comply with **privacy regulations** like GDPR and CCPA/CPRA, protect your users' sensitive data, and reduce the risk of data breaches when storing or sharing transcripts. ## Usage Add `"pii_redaction": true` to your request to redact all detected PII in the transcript. Sensitive entities will be replaced with markers in the output. ```json Pre-recorded theme={"system"} { "audio_url": "YOUR_AUDIO_URL", "pii_redaction": true } ``` ## Optional configuration You can customize the behavior with `pii_redaction_config`: Preset or list of PII entity types to redact (e.g. `["GDPR"]`). See [Supported entity types](#supported-entity-types) for available presets. How to replace detected PII: * **`MASK`**: Each character replaced by a mask (e.g. "John Smith" → `#### #####`) * **`MARKER`**: Placeholder labels like `[NAME_1]`, `[EMAIL_1]`. Same entity will have same ID. ## Example body ```json Pre-recorded theme={"system"} { "audio_url": "YOUR_AUDIO_URL", "pii_redaction": true, "pii_redaction_config": { "entity_types": ["GDPR"], "processed_text_type": "MARKER" } } ``` ## Example output **Without PII redaction (raw transcript):** > Hi, I'm calling about the order for John Smith. Can you confirm the delivery to [john.smith@company.com](mailto:john.smith@company.com)? Yes, John Smith placed it yesterday. **With PII redaction (`processed_text_type="MASK"`):** > Hi, I'm calling about the order for #### #####. Can you confirm the delivery to ######################? Yes, #### ##### placed it yesterday. **With PII redaction (`processed_text_type="MARKER"`):** > Hi, I'm calling about the order for \[NAME\_1]. Can you confirm the delivery to \[EMAIL\_1]? Yes, \[NAME\_1] placed it yesterday. The same entity mentioned multiple times receives the **same marker ID** (e.g. "John Smith" becomes \[NAME\_1] both times), so you can track references across the transcript while keeping sensitive data redacted. \ This consistency is also useful for downstream tasks using LLMs, which can reason about entities (e.g. "the person in \[NAME\_1]") without ever seeing the raw PII. ## Supported entity types When using `entity_types`, you can pass a preset group or a list of specific types. ### Preset groups Personal data entities covered by the EU GDPR. Includes: `AGE`, `DRIVER_LICENSE`, `DOB`, `EMAIL_ADDRESS`, `HEALTHCARE_NUMBER`, `IP_ADDRESS`, `LANGUAGE`, `LOCATION`, `LOCATION_ADDRESS`, `LOCATION_ADDRESS_STREET`, `LOCATION_CITY`, `LOCATION_COORDINATE`, `LOCATION_COUNTRY`, `LOCATION_STATE`, `LOCATION_ZIP`, `NAME`, `NAME_FAMILY`, `NAME_GIVEN`, `NAME_MEDICAL_PROFESSIONAL`, `NUMERICAL_PII`, `PASSPORT_NUMBER`, `PHONE_NUMBER`, `SSN`, `URL`, `USERNAME`, `VEHICLE_ID`, `BANK_ACCOUNT`, `CREDIT_CARD`, `CREDIT_CARD_EXPIRATION`, `CVV`, `BLOOD_TYPE`, `CONDITION`, `DRUG`, `INJURY`, `MEDICAL_PROCESS` Sensitive personal data as defined by [GDPR Article 9](https://gdpr-info.eu/art-9-gdpr/). Includes: `GENDER`, `LANGUAGE`, `ORIGIN`, `PHYSICAL_ATTRIBUTE`, `POLITICAL_AFFILIATION`, `RELIGION`, `SEXUALITY` De-identification standard for US healthcare data under HIPAA. Includes: `ACCOUNT_NUMBER`, `AGE`, `DATE`, `DATE_INTERVAL`, `DOB`, `DRIVER_LICENSE`, `EMAIL_ADDRESS`, `HEALTHCARE_NUMBER`, `IP_ADDRESS`, `LOCATION`, `LOCATION_ADDRESS`, `LOCATION_ADDRESS_STREET`, `LOCATION_CITY`, `LOCATION_COORDINATE`, `LOCATION_ZIP`, `NAME`, `NAME_FAMILY`, `NAME_GIVEN`, `NAME_MEDICAL_PROFESSIONAL`, `NUMERICAL_PII`, `PASSPORT_NUMBER`, `PHONE_NUMBER`, `SSN`, `URL`, `VEHICLE_ID`, `BANK_ACCOUNT`, `CREDIT_CARD`, `CREDIT_CARD_EXPIRATION`, `CVV` Medical and health-related information. Includes: `BLOOD_TYPE`, `CONDITION`, `DOSE`, `DRUG`, `INJURY`, `MEDICAL_PROCESS`, `STATISTICS` PCI-specific financial entities only. Includes: `BANK_ACCOUNT`, `CREDIT_CARD`, `CREDIT_CARD_EXPIRATION`, `CVV`, `ROUTING_NUMBER` Personal data entities covered by the California Privacy Rights Act. Includes: `DOB`, `DRIVER_LICENSE`, `EMAIL_ADDRESS`, `GENDER`, `HEALTHCARE_NUMBER`, `IP_ADDRESS`, `LOCATION_ADDRESS`, `LOCATION_ADDRESS_STREET`, `LOCATION_CITY`, `LOCATION_COORDINATE`, `LOCATION_ZIP`, `NAME`, `NAME_FAMILY`, `NAME_GIVEN`, `NAME_MEDICAL_PROFESSIONAL`, `NUMERICAL_PII`, `ORIGIN`, `PASSPORT_NUMBER`, `PASSWORD`, `PHONE_NUMBER`, `PHYSICAL_ATTRIBUTE`, `POLITICAL_AFFILIATION`, `RELIGION`, `SEXUALITY`, `SSN`, `URL`, `USERNAME`, `VEHICLE_ID`, `BANK_ACCOUNT`, `CREDIT_CARD`, `CREDIT_CARD_EXPIRATION`, `CVV`, `BLOOD_TYPE`, `CONDITION`, `DRUG`, `INJURY`, `MEDICAL_PROCESS` Personal data entities covered by Japan's APPI. Includes: `AGE`, `DOB`, `DRIVER_LICENSE`, `EMAIL_ADDRESS`, `HEALTHCARE_NUMBER`, `IP_ADDRESS`, `LOCATION`, `LOCATION_ADDRESS`, `LOCATION_ADDRESS_STREET`, `LOCATION_CITY`, `LOCATION_COORDINATE`, `LOCATION_COUNTRY`, `LOCATION_STATE`, `LOCATION_ZIP`, `NAME`, `NAME_FAMILY`, `NAME_GIVEN`, `NAME_MEDICAL_PROFESSIONAL`, `NUMERICAL_PII`, `OCCUPATION`, `ORGANIZATION`, `ORGANIZATION_MEDICAL_FACILITY`, `PASSPORT_NUMBER`, `PASSWORD`, `PHONE_NUMBER`, `SSN`, `VEHICLE_ID`, `BANK_ACCOUNT`, `CREDIT_CARD`, `CREDIT_CARD_EXPIRATION`, `CVV`, `DRUG` Sensitive data subset under Japan's APPI. Includes: `CONDITION`, `DRUG`, `GENDER`, `INJURY`, `LANGUAGE`, `MARITAL_STATUS`, `MEDICAL_PROCESS`, `ORIGIN`, `PHYSICAL_ATTRIBUTE`, `POLITICAL_AFFILIATION`, `RELIGION`, `SEXUALITY` Personal data entities covered by Quebec's Privacy Act / Law 25. Includes: `AGE`, `DATE`, `DATE_INTERVAL`, `DOB`, `DRIVER_LICENSE`, `EMAIL_ADDRESS`, `HEALTHCARE_NUMBER`, `IP_ADDRESS`, `LOCATION_ADDRESS`, `LOCATION_ADDRESS_STREET`, `LOCATION_CITY`, `LOCATION_COORDINATE`, `LOCATION_ZIP`, `NAME`, `NAME_FAMILY`, `NAME_GIVEN`, `NAME_MEDICAL_PROFESSIONAL`, `NUMERICAL_PII`, `OCCUPATION`, `ORGANIZATION`, `ORGANIZATION_MEDICAL_FACILITY`, `ORIGIN`, `PASSPORT_NUMBER`, `PHONE_NUMBER`, `POLITICAL_AFFILIATION`, `RELIGION`, `SSN`, `BANK_ACCOUNT`, `CREDIT_CARD`, `CREDIT_CARD_EXPIRATION`, `CVV`, `BLOOD_TYPE`, `CONDITION`, `DRUG`, `INJURY`, `MEDICAL_PROCESS`, `STATISTICS` Core set of the most commonly used PII entity types across all regulations. Business and corporate data entities. Includes: `ACCOUNT_NUMBER`, `DATE`, `DATE_INTERVAL`, `EMAIL_ADDRESS`, `FILENAME`, `LOCATION`, `LOCATION_ADDRESS`, `LOCATION_ADDRESS_STREET`, `LOCATION_CITY`, `LOCATION_COUNTRY`, `LOCATION_STATE`, `LOCATION_ZIP`, `MONEY`, `NUMERICAL_PII`, `OCCUPATION`, `ORGANIZATION`, `PASSWORD`, `TIME`, `URL`, `VEHICLE_ID`, `BANK_ACCOUNT`, `CREDIT_CARD`, `CREDIT_CARD_EXPIRATION`, `CVV`, `ROUTING_NUMBER`, `DRUG`, `MEDICAL_PROCESS` Entities that can identify individuals within a local context. Numerical PII entities, excluding PCI-specific types. Includes: `ACCOUNT_NUMBER`, `NUMERICAL_PII`, `SSN`, etc. Excludes: `CREDIT_CARD`, `CVV`, `BANK_ACCOUNT`, `ROUTING_NUMBER` ### Individual entity types #### Core PII | Entity | What it catches | | ------------------ | --------------------------------------------- | | `NAME` | Full person names | | `EMAIL_ADDRESS` | Email addresses | | `PHONE_NUMBER` | Phone/fax numbers | | `LOCATION_ADDRESS` | Full mailing addresses | | `DATE` | Specific dates | | `DOB` | Dates of birth | | `SSN` | Social security numbers (+ intl. equivalents) | | `PASSPORT_NUMBER` | Passport numbers | | `DRIVER_LICENSE` | Driver's license numbers | | `IP_ADDRESS` | IPv4 / IPv6 addresses | | `URL` | Web addresses | | `USERNAME` | Logins, handles | #### Financial / PCI | Entity | What it catches | | -------------- | ---------------------------------- | | `CREDIT_CARD` | Credit card numbers (incl. last 4) | | `BANK_ACCOUNT` | Bank accounts, IBAN | | `CVV` | Card verification codes | #### Sensitive / GDPR Article 9 | Entity | What it catches | | ----------------------- | ---------------------- | | `ORIGIN` | Nationality, ethnicity | | `RELIGION` | Religious affiliation | | `POLITICAL_AFFILIATION` | Political opinions | | `SEXUALITY` | Sexual orientation | | `PHYSICAL_ATTRIBUTE` | Race, physical traits | #### Healthcare (HIPAA / GDPR) | Entity | What it catches | | ------------------- | --------------------------------------- | | `CONDITION` | Medical conditions, diseases | | `DRUG` | Medications, supplements | | `HEALTHCARE_NUMBER` | Health plan IDs, medical record numbers | | `MEDICAL_PROCESS` | Treatments, procedures | # Sentiment and Emotion Analysis Source: https://docs.gladia.io/chapters/audio-intelligence/sentiment-analysis Extract sentiments and emotions from the transcript. Pre-recorded Live The sentiment and emotion analysis model analyzes the transcript, detecting the general sentiment which is conveyed in each sentence (positive, neutral or negative) as well as any emotion that is being expressed. ## Usage To enable sentiment and emotion analysis simply set the appropriate flag: ```json Pre-recorded theme={"system"} { "sentiment_analysis": true } ``` ```json Live theme={"system"} { "realtime_processing": { "sentiment_analysis": true }, "messages_config": { "receive_realtime_processing_events": true } } ``` ## Result Your transcription result will contain a `sentiment_analysis` key which will contain an array of all sentences in the audio, and for each one the conveyed sentiment and expressed emotion. See possible values below. When `diarization` is enabled, the sentiment analysis output will contain the `speaker` as well, allowing you to analyze each speaker separately. ```json Pre-recorded theme={"system"} { "transcription":{...}, "sentiment_analysis": { "success": true, "is_empty": false, "results": [ { "text": "Jonathan, it says you are trained in technology.", "sentiment": "neutral", "emotion": "neutral", "start": 0.45158000000000004, "end": 2.364, "channel": 0, "speaker": 0, }, { "text": "That's very good.", "sentiment": "positive", "emotion": "positive_surprise", "start": 2.54438, "end": 3.5432300000000003, "channel": 0, "speaker": 0, } ], "exec_time": 1.127103805541992, "error": null } } ``` ```json Live theme={"system"} ``` ## Possible values ### Sentiments * positive * negative * neutral * mixed * unknown ### Emotions * adoration * amusement * anger * awe * confusion * contempt * contentment * desire * disappointment * disgust * distress * ecstatic * elation * embarrassment * fear * interest * pain * realization * relief * sadness * negative\_surprise * positive\_surprise * sympathy * triumph * neutral # Speaker Diarization Source: https://docs.gladia.io/chapters/audio-intelligence/speaker-diarization Detect speakers and understand who said what. Pre-recorded Speaker diarization is the process of detecting multiple speakers in an audio, and understanding which parts of the transcription each speaker said. ## Enabling diarization Diarization is enabled by sending the `diarization` parameter in the transcription request: ```json Pre-recorded theme={"system"} { "audio_url": "", "diarization": true } ``` ## Response When diarization is enabled, each utterance will contain a `speaker` field, whose value is an index representing the speaker. Speakers will be assigned indexes by **order of appearance** (i.e. the 1st speaker will be speaker 0, the 2nd speaker 1, etc). ```json Pre-recorded theme={"system"} { "transcription": { "utterances": [ { "words": [...], "text": "it says you are trained in technology.", "language": "en", "start": 0.7334100000000001, "end": 2.364, "confidence": 0.8914285714285715, "channel": 0, "speaker": 0, }, ... ] } } ``` ## Improving diarization accuracy You can improve the accuracy of the diarization by providing the model with hints regarding the expected number or lower/upper bounds ofspeakers using the `diarization_config.num_of_speakers`, `diarization_config.min_speakers` and `diarization_config.max_speakers` parameters respectively. **Important:** These parameters are hints, not hard constraints. The actual number of speakers detected by the model may not comply with the provided parameters. | Key | Type | Description | | --------------------------------------- | ------ | ---------------------------------------------------------------------------------------------------- | | `diarization_config.number_of_speakers` | number | Guiding number of speakers - instructs the model to detect an exact number of speakers in the audio. | | `diarization_config.min_speakers` | number | Instructs the model to detect no less than this number of speakers in the audio. | | `diarization_config.max_speakers` | number | Causes the model to detect no more than this number of speakers in the audio. | # Export subtitles (SRT/VTT) Source: https://docs.gladia.io/chapters/audio-intelligence/subtitles Generate caption files directly from transcripts Pre-recorded You can export completed transcripts in both SRT and VTT formats for subtitles and captions. You can use the `subtitles` feature alongside `translation`. You'll have subtitles in the original language, and in each targeted translation language. ```json Pre-recorded theme={"system"} { "audio_url": "YOUR_AUDIO_URL", "subtitles": true, "subtitles_config": { "formats": ["srt", "vtt"], "minimum_duration": 1, "maximum_duration": 5, "maximum_characters_per_row": 42, "maximum_rows_per_caption": 2, "style": "compliance" } } ``` The `subtitles_config` object supports: The format of the subtitles. The minimum duration of the subtitles in seconds (min 0). The maximum duration of the subtitles in seconds (min 1, max 30). The maximum number of characters per row (min 1). The maximum number of rows per caption (min 1, max 5). The style of the subtitles ("default" or "compliance"). The JSON response adds a `subtitles` array with items like: ```json Pre-recorded theme={"system"} { "format": "srt", "subtitles": "1\n00:00:00,210 --> 00:00:04,711....." } ``` # Summarization Source: https://docs.gladia.io/chapters/audio-intelligence/summarization Retrieve important information from your audio files Pre-recorded Live The **Summarization** model generates a summary of your transcript. You can choose one of our summary types to customize the summarization based on your preference. **3 summarization types** are available: * `general` : A regular summary of the transcription * `concise` : A shorter summary for quick overview * `bullet_points` : Retrieve the key points in a list If no `summarization_config` is provided, `general` type will be used by default. ### Notes on options * **general**: Balanced summary for most use cases; good readability and coverage. * **concise**: Shorter output for quick overviews or previews; fewer details. * **bullet\_points**: Lists key takeaways; ideal for action items, meeting notes, or highlights. ## Usage To enable summarization simply set the `"summarization"` parameter to true ```json Pre-recorded theme={"system"} { "summarization": true, "summarization_config": { "type": "concise" } } ``` ```json Live theme={"system"} { "post_processing": { "summarization": true, "summarization_config": { "type": "concise" } }, "messages_config": { "receive_post_processing_events": true } } ``` ## Result The transcription result will contain a `"summarization"` key with the output of the model: ```json Pre-recorded theme={"system"} { "transcription": {...}, "summarization": { "success": true, "is_empty": false, "results": "This transcription suggests that...", "exec_time": 1.5126123428344727, "error": null } } ``` You'll find the summarization of your audio under the `results` key. # Translation Source: https://docs.gladia.io/chapters/audio-intelligence/translation Translate your transcriptions & subtitles Pre-recorded Live The **Translation** model generates translations of your transcriptions to one or more targeted languages. If [subtitles](/chapters/audio-intelligence/subtitles) and/or [sentences](/chapters/pre-recorded-stt/features/sentences) are enabled, the translations will also include translated results for them. You can translate your transcription to **multiple languages** in a single API call. The list of the languages covered by the Translation feature are listed in [Supported Languages](/chapters/language/supported-languages). *** **2 translation models** are available: * `base` : Fast, cover most use cases * `enhanced` : Slower, but higher quality and with context awareness ## Quickstart To enable translation, set `translation` to `true` on your request, and add a `translation_config` object : ```json Pre-recorded theme={"system"} { "translation": true, "translation_config": { "target_languages": [ "fr" ], "model": "base", "match_original_utterances": true, "lipsync": true, "context_adaptation": true, "context": "", "informal": false } } ``` ```json Live theme={"system"} { "realtime_processing": { "translation": true, "translation_config": { "target_languages": [ "fr" ], "model": "base", "match_original_utterances": true, "lipsync": true, "context_adaptation": true, "context": "", "informal": false } }, "messages_config": { "receive_realtime_processing_events": true } } ``` ### Translation configuration fields Target language codes for translation output. See the list of supported language codes in [>Supported Languages](/chapters/language/supported-languages). Specifies the translation model to be used. Keep translated segments aligned with source segmentation. Use `true` for subtitles/dubbing; set `false` for a more natural flow in the target language. * When **true**, the system attempts to match the translated segments (utterances, sentences) to the timing and structure of the original detected speech segments. * When **false**, the translation might be more fluid or natural-sounding in the target language but could deviate from the original utterance segmentation. Controls alignment with visual cues, specifically lip movements. When enabled (default), uses an advanced lip synchronization algorithm that aligns translated output with speaker's lip movements using timestamps from lip activity. \ This enhances viewing experience for dubbed content but may occasionally merge distinct words into single objects to achieve better visual sync. Set to `false` if strict word-for-word mapping is required over visual timing synchronization. Enable context-aware translation. When `true`, the model leverages extra context and style preferences for better accuracy. Turn off for purely literal translations. Additional context to improve terminology, proper nouns, or disambiguation. Effective with `context_adaptation: true`. \ **Example**: `"Medical consultation between doctor and patient discussing cardiology"` Prefer informal register when available; useful for chatty UX or youth audiences. Especially relevant for languages with formal/informal distinctions (e.g., French "tu/vous", German "du/Sie", Spanish "tú/usted", Dutch "U/jij"). ## Result The transcription result will contain a `"translation"` key with the output of the model: ```json Pre-recorded theme={"system"} { "transcription":{...}, "translation": { success: true, is_empty: false, results: [ { words: [ { word: "Diviser", start: 0.20043, end: 0.7008000000000001, confidence: 1 }, { word: "l'infini", start: 0.9009500000000001, end: 1.5614400000000002, confidence: 1 }, ... ], languages: ["fr"], full_transcript: "Diviser l'infini dans un temps où moins est plus...", utterances: [Array], // Also translated error: null }, { words: [ { word: "Dividir", start: 0.20043, end: 0.7008000000000001, confidence: 1 }, { word: "la infinidad", start: 0.9009500000000001, end: 1.5614400000000002, confidence: 1 }, ... ], languages: ["es"], full_transcript: "Dividir la infinidad en un tiempo en que menos es más...", utterances: [Array], // Also translated error: null } ], exec_time: 0.6475496292114258, error: null } } ``` ```json Live theme={"system"} { "session_id": "550e8400-e29b-41d4-a716-446655440000", "created_at": "2025-09-19T12:34:20Z", "type": "translation", "error": null, "data": { "utterance_id": "utt_001", "utterance": { "text": "buenos días", "language": "es", "start": 4.2, "end": 6.1 }, "original_language": "es", "target_language": "en", "translated_utterance": { "text": "good morning", "language": "en", "start": 4.2, "end": 6.1 } } } ``` If you enabled the `subtitles` generation, those will also benefits from the translation model. ## Best practices * Set `target_languages` to only the languages you need. * Use `enhanced` with `context_adaptation` for high-accuracy, domain-heavy content. * Provide a meaningful `context` to improve terminology and named entities. * Keep `match_original_utterances: true` for subtitles; set to `false` for a more natural flow. * Pair with [language detection](/chapters/language/language-detection) and [code switching](/chapters/language/code-switching) when source language may vary. # Gladia CLI Source: https://docs.gladia.io/chapters/developer-tools/gladia-cli Transcribe audio from your terminal with one install, one command, and no boilerplate Transcribe audio from your terminal. One install, one command, no boilerplate. The Gladia CLI wraps the [Gladia speech-to-text API](/chapters/pre-recorded-stt/quickstart) for terminals, shell scripts, and pipelines — 100+ languages, speaker diarization, and code switching in a single command. Get your API key from the [Gladia dashboard](https://app.gladia.io/apikeys) before running your first transcription. ## Why use the CLI * **One command away** — Install the CLI, set your API key, run `gladia transcribe`. * **Flexible output** — Get plain text, JSON, SRT, or VTT straight to stdout. * **Full feature set** — Speaker diarization, multilingual configuration, and model selection (`solaria-1`, `solaria-3`). * **Files or URLs** — Transcribe a local recording or a remote link, with no download step. * **Pipe-friendly** — Machine-readable output for shell scripts and CI pipelines. ## Install ```bash theme={"system"} # macOS & Linux curl -fsSL https://github.com/gladiaio/gladia-cli/releases/latest/download/install.sh | sh # Windows (PowerShell) powershell -c "irm https://github.com/gladiaio/gladia-cli/releases/latest/download/install.ps1 | iex" ``` Other platforms and binaries are available on [GitHub releases](https://github.com/gladiaio/gladia-cli/releases). ## Quick start ```bash theme={"system"} gladia auth set your_key gladia transcribe meeting.wav # transcript to stdout gladia transcribe podcast.mp3 -o srt # subtitles instead gladia transcribe call.wav --diarize # label who spoke when gladia languages # list supported language codes ``` ### API key setup Provide your API key in one of three ways (checked in this order): ```bash theme={"system"} export GLADIA_API_KEY=your_key # 1. environment variable gladia auth set your_key # 2. saved to ~/.gladia (mode 0600) gladia transcribe meeting.wav --gladia-key your_key # 3. per-command flag ``` ## Everyday examples ```bash theme={"system"} # Transcribe a local file or a remote URL gladia transcribe meeting.wav gladia transcribe https://example.com/audio.mp3 -o json # Narrow language detection to a shortlist gladia transcribe podcast.mp3 --language en,fr,de # Mixed-language audio: re-detect on every utterance gladia transcribe mixed.mp3 --code-switching --language en,fr # Who spoke when, as subtitles gladia transcribe call.wav --diarize -o srt # Pick a model gladia transcribe podcast.mp3 --model solaria-3 --language en # Machine-readable output in a pipeline gladia transcribe interview.mp3 -o json | jq '.transcription' ``` ## Commands | Command | Description | | -------------------------- | ----------------------------------------------------------- | | `transcribe ` | Transcribe an audio file or URL | | `auth set ` | Save API key to `~/.gladia` | | `languages` | List supported ISO 639-1 codes | | `completion ` | Generate shell tab completion (bash, zsh, fish, powershell) | ## Output formats Use `-o` or `--output` to control how the transcript is returned: | Format | Description | | ----------- | -------------------------- | | `text` | Plain transcript (default) | | `json` | Structured JSON transcript | | `json-full` | Full API response as JSON | | `srt` | SubRip subtitles | | `vtt` | WebVTT subtitles | ## Transcription options | Flag | Default | Description | | -------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `-o`, `--output` | `text` | Output format: `text`, `json`, `json-full`, `srt`, `vtt` | | `--language` | — | Expected language(s), comma-separated (`en` or `en,fr,de`); narrows detection | | `--cs`, `--code-switching` | off | Re-detect language on each utterance (mixed-language audio; `solaria-1` only) | | `--diarize` | off | Identify speakers in the transcript | | `--model` | — | STT model: `solaria-1` or `solaria-3`. Solaria-3 accepts at most one `--language` (`en`, `fr`, `de`, `es`, or `it`) and does not support code switching. | | `-v`, `--verbose` | off | Show progress while polling | **Global flag** (any command): `--gladia-key` — API key if not set in the environment or `~/.gladia`. ### Language | Goal | What to run | | ------------------- | -------------------------------------------------- | | Auto-detect | `gladia transcribe ` | | Constrain detection | `--language en,fr,de` | | Code switching | `--code-switching` (+ optional `--language` hints) | * **`--language`** limits which language(s) Gladia considers. A comma-separated list is a hint list, not per-utterance switching. * **`--code-switching`** turns on per-utterance language detection. Add `--language` to restrict which languages may appear. Not available with `solaria-3`. ### Diarization Use `--diarize` when you need **who spoke when**. Speaker labels are included in the output (for example, `Speaker 0: …`). ```bash theme={"system"} gladia transcribe meeting.wav --diarize gladia transcribe panel.mp3 --diarize -o srt ``` ## Shell completion When you install via `install.sh` or `install.ps1`, the script offers to configure tab completion for your shell. To skip the prompt (for example in CI), set `GLADIA_NO_COMPLETION_PROMPT=1`. You can also set up completion manually: ```bash theme={"system"} # bash source <(gladia completion bash) # zsh mkdir -p ~/.zsh/completions gladia completion zsh > ~/.zsh/completions/_gladia # fish mkdir -p ~/.config/fish/completions gladia completion fish > ~/.config/fish/completions/gladia.fish ``` Run `gladia completion --help` for full instructions per shell. ## Next steps Create an API key on the Gladia dashboard Source code, issues, and releases Learn more about Gladia transcription features # Attendee Source: https://docs.gladia.io/chapters/integrations/attendee How to integrate Gladia with the Attendee Meeting Bot API ## What is Attendee? [Attendee](https://attendee.dev/) is an open‑source, universal Meeting Bot API that can join Zoom, Google Meet, and Microsoft Teams calls, capture per‑participant audio, and route it to a transcription provider in real time. By connecting Gladia STT, you get accurate, multilingual transcription with perfect speaker diarization for your meeting recordings without building any of the bot or RTC plumbing yourself. Looking for API details and options? See the Attendee docs: [docs.attendee.dev](https://docs.attendee.dev/) and the Gladia docs [docs.gladia.io](https://docs.gladia.io/). Attendee is open source on GitHub: [attendee-labs/attendee](https://github.com/attendee-labs/attendee). ## Benefits of integration * **Single API for multiple platforms**: Join and transcribe meetings from Zoom, Google Meet, and Microsoft Teams through one consistent API * **Open source**: Self‑host Attendee or use the hosted version at [app.attendee.dev](https://app.attendee.dev/) * **Per‑participant audio**: Third‑party transcription uses separate audio streams for perfect speaker diarization on Zoom and Google Meet * **Multilingual**: Gladia supports automatic language detection and code switching for speakers who change languages mid‑sentence * **Real‑time webhooks**: Receive `transcript.update` events as utterances are produced ## What you can build * Meeting assistants that take notes, summaries, and action items * LLM agents that join calls and understand participants in real time ## Setup instructions ### Prerequisites 1. An Attendee account (hosted at [app.attendee.dev](https://app.attendee.dev/) or self‑hosted) with an API key 2. A Gladia account with an API key ### Add your Gladia API key in Attendee Attendee calls Gladia with an API key that you provide: 1. [Sign up for a Gladia account](https://app.gladia.io/auth/signup) if you don't have one 2. Copy your key from the Gladia dashboard: [app.gladia.io/apikeys](https://app.gladia.io/apikeys) 3. In Attendee, go to **Settings → Credentials** and find the **Gladia Credentials** card, then click **Add Credentials** Gladia Credentials card in the Attendee Settings → Credentials page 4. Paste your Gladia API key and click **Save Credentials** Add Gladia Credentials dialog with the API Key field ## Quickstart via API When creating a bot, select Gladia as the transcription provider in the `transcription_settings` object of the [Create Bot request](https://docs.attendee.dev/api-reference/tag/bots/post/api/v1/bots): ```bash theme={"system"} curl -X POST "https://app.attendee.dev/api/v1/bots" \ -H "Authorization: Token " \ -H "Content-Type: application/json" \ -d '{ "meeting_url": "https://meet.google.com/xyz-abc-def", "bot_name": "My Bot", "transcription_settings": { "gladia": {} } }' ``` ### Gladia provider options Set Gladia‑specific options inside the `gladia` object: | Parameter | Type | Description | | -------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `enable_code_switching` | boolean | Transcribe meetings where speakers switch languages mid‑sentence | | `code_switching_languages` | array of strings | The languages to consider when code switching is enabled. See [supported languages](https://docs.gladia.io/chapters/limits-and-specifications/languages) | Example with code switching enabled: ```bash theme={"system"} curl -X POST "https://app.attendee.dev/api/v1/bots" \ -H "Authorization: Token " \ -H "Content-Type: application/json" \ -d '{ "meeting_url": "https://meet.google.com/xyz-abc-def", "bot_name": "My Bot", "transcription_settings": { "gladia": { "enable_code_switching": true, "code_switching_languages": ["en", "fr", "es"] } } }' ``` ## Fetching transcripts Receive utterances in real time by enabling the `transcript.update` webhook trigger in **Settings → Webhooks**, or pull the transcript during and after the meeting from the [`/transcript` endpoint](https://docs.attendee.dev/api-reference/tag/bots/get/api/v1/bots/object_id/transcript). ## Next steps Get your API key on the Gladia dashboard Explore the Meeting Bot API and transcription settings Self‑host the open‑source Meeting Bot API # Composio Source: https://docs.gladia.io/chapters/integrations/composio How to use Gladia with Composio ## What is Composio? Composio is a platform that connects AI agents and LLMs to external tools and services. With the Gladia toolkit on Composio, you can integrate Gladia's transcription capabilities directly into your AI agent workflows, enabling automated audio processing pipelines without writing custom integration code. Ready to connect Gladia in Composio? Visit the Gladia toolkit page: [composio.dev/toolkits/gladia](https://composio.dev/toolkits/gladia). ## What you can build * AI agents that automatically transcribe audio files * Automated workflows that process meeting recordings * LLM pipelines with speech-to-text as an input step * Multi-tool AI agents combining transcription with other services ## Next steps Get your API key on the Gladia dashboard Set up the Gladia toolkit in Composio Learn more about the platform # Livekit Source: https://docs.gladia.io/chapters/integrations/livekit How to integrate Gladia with LiveKit Agents ## What is LiveKit Agents? LiveKit Agents is a framework for building real‑time, voice‑enabled AI applications that connect to LiveKit rooms. It lets you compose speech‑to‑text (STT), large language models (LLMs), and text‑to‑speech (TTS) to create interactive agents that can hear, think, and speak in low‑latency pipelines. With the Gladia STT integration, your agents benefit from accurate, fast transcription. Looking for the full API and options? See the official LiveKit Gladia STT docs: [docs.livekit.io/agents/integrations/stt/gladia](https://docs.livekit.io/agents/integrations/stt/gladia/). ## What you can build * Real‑time voice assistants in LiveKit rooms * Live captions for webinars and meetings * Customer‑support and IVR voice bots * In‑app guidance with speech interfaces ## Quickstart using the sample repo Use our end‑to‑end sample that wires LiveKit Agents with Gladia STT : [gladia-samples/integrations-examples/livekit-agent](https://github.com/gladiaio/gladia-samples/tree/main/integrations-examples/livekit-agent) ## Next steps Get your API key on the Gladia dashboard Browse and run the full sample Explore service options and parameters # Make Source: https://docs.gladia.io/chapters/integrations/make How to use Gladia with Make ## What is Make? Make (formerly Integromat) is a visual automation platform that lets you design, build, and automate workflows by connecting apps and services together. With the Gladia community integration on Make, you can incorporate speech-to-text into your automated scenarios without writing any code. Ready to connect Gladia in Make? Visit the Gladia integration page: [make.com/en/integrations/gladia-community](https://www.make.com/en/integrations/gladia-community). ## What you can build * Automated transcription pipelines triggered by new files or events * Workflows that route transcription results to databases, CRMs, or messaging apps * Multi-step scenarios combining transcription with translation or summarization * Scheduled batch processing of audio recordings ## Next steps Get your API key on the Gladia dashboard Browse available modules and actions Learn more about the platform # Meeting BaaS Source: https://docs.gladia.io/chapters/integrations/meeting-baas How to integrate Gladia with Meeting BaaS ## What is Meeting BaaS? Meeting BaaS provides a unified API for Google Meet, Zoom, and Microsoft Teams so you can record and transcribe meetings through one service. By connecting Gladia STT, you leverage Gladia’s accurate, feature‑rich transcription for your meeting recordings without building the capture and RTC plumbing yourself. Looking for API details and options? See the Meeting BaaS docs: [docs.meetingbaas.com](https://docs.meetingbaas.com/) and the Gladia docs [docs.gladia.io](https://docs.gladia.io/). ## Benefits of integration * **Single API for Multiple Platforms**: Record and transcribe meetings from Google Meet, Zoom, and Microsoft Teams through one consistent API * **Advanced Transcription**: Utilize Gladia's accurate and feature-rich transcription service * **Instant Availability**: Get video recordings and transcriptions immediately after meetings * **Custom Branding**: Display your user's branding with custom names and chat messages * **GDPR Compliance**: Meeting BaaS provides a DPA and focuses on data minimization ## What you can build * Use the generated transcripts in your applications * Set up webhook notifications for transcription events * Explore LLM summaries and metadata extraction features * Build custom interfaces using the provided transcription data ## Quickstart via API There are two main ways to use Gladia with Meeting BaaS. ### Create a bot with Gladia as the STT provider When creating a bot to join a meeting, specify Gladia as the speech‑to‑text provider: ```bash theme={"system"} curl -X POST "https://api.meetingbaas.com/bots" \ -H "x-meeting-baas-api-key: " \ -H "Content-Type: application/json" \ -d '{ "meeting_url": "https://meet.google.com/xyz-abc", "speech_to_text": "Gladia", "webhook_url": "https://yourapp.com/webhook" }' ``` ### Retranscribe an existing recording with Gladia For existing recordings, use the Retranscribe Bot API to process audio with Gladia: ```bash theme={"system"} curl -X POST "https://api.meetingbaas.com/bots/retranscribe" \ -H "x-meeting-baas-api-key: " \ -H "Content-Type: application/json" \ -d '{ "bot_uuid": "string", "speech_to_text": { "api_key": "", "provider": "Gladia" }, "webhook_url": "string" }' ``` ## Setup instructions ### Prerequisites 1. A Meeting BaaS account with an API key 2. A Gladia account with an API key ### Integration steps 1. **Get your Gladia API key** * [Sign up for a Gladia account](https://app.gladia.io/auth/signin) if you don't have one * Navigate to your API keys section and copy your key 2. **Create a Meeting BaaS bot with Gladia** * Use the API endpoint `/bots` with your Meeting BaaS API key * Set `"speech_to_text": "Gladia"` in your request body as shown in the example above 3. **Retranscribe with Gladia** * Use the `/bots/retranscribe` endpoint * Provide your Gladia API key in the `speech_to_text.api_key` field * Set `"provider": "Gladia"` to specify Gladia as the transcription service ## Response handling Upon successful integration, Meeting BaaS will return a response with status code 200 or 202, containing details about the transcription job. ## What's next? After integrating with Meeting BaaS, you can: * Use the generated transcripts in your applications * Set up webhook notifications for transcription events * Explore additional Meeting BaaS features like LLM summaries and metadata extraction * Build custom interfaces using the provided transcription data Get your API key on the Gladia dashboard Explore service options and API details Learn about STT options and parameters # n8n Source: https://docs.gladia.io/chapters/integrations/n8n How to use Gladia with n8n ## What is n8n? n8n is an open-source workflow automation platform that lets you connect apps, APIs, and services through a visual node-based editor. The official Gladia community node for n8n makes it easy to add speech-to-text transcription to any workflow — supporting audio from a URL or directly from binary data piped between nodes. Install the Gladia node directly from your n8n instance: go to **Settings → Community Nodes**, search for `@gladiaio/n8n-nodes-gladia`, and click Install. Community node installation requires a self-hosted n8n instance. ## What you can build * Transcription pipelines triggered by new audio files in cloud storage * Meeting recording workflows that produce transcripts, summaries, and subtitles automatically * Multi-step automations combining transcription with translation, sentiment analysis, or NER * Real-time webhook-driven flows that transcribe and route audio to Slack, Notion, or any downstream app ## Quickstart ### Prerequisites * A self-hosted n8n instance (v1.0.0 or later) * A [Gladia API key](https://app.gladia.io/apikeys) ### Installation Install the node from the n8n UI or via npm: ```bash theme={"system"} npm install @gladiaio/n8n-nodes-gladia ``` Then restart your n8n instance. ### Configure credentials In n8n, add a new **Gladia API** credential and paste your API key from the [Gladia dashboard](https://app.gladia.io/apikeys). ### Transcribe audio from a URL 1. Add a **Gladia** node to your workflow. 2. Set **Audio Source** to `URL` and provide a publicly accessible audio link. 3. Enable **Wait for Completion** to receive the full transcript immediately, or disable it to get the transcription ID for asynchronous polling. ### Transcribe binary audio data 1. Pipe binary data from an upstream node (e.g. an HTTP Request or Read Binary File node). 2. Set **Audio Source** to `Binary Data` and specify the binary field name. 3. The Gladia node will upload the file and return the transcript automatically. ## Supported features The Gladia n8n node exposes the full Gladia feature set: * 100+ language detection and code-switching * Speaker diarization and segmentation * Sentiment analysis and named entity recognition (NER) * Subtitle generation (SRT, VTT) * Summarization and multilingual translation * Custom vocabulary support * Configurable polling intervals for async jobs ## Next steps Get your API key on the Gladia dashboard View the package and release notes Learn more about the platform # Pipecat Source: https://docs.gladia.io/chapters/integrations/pipecat How to integrate Gladia with Pipecat ## What is Pipecat? Pipecat is an open‑source framework for building realtime, voice‑first AI agents. It orchestrates audio input/output, speech‑to‑text (STT), large language models (LLMs), and text‑to‑speech (TTS) into one low‑latency pipeline you can run on the server and connect to from the browser or native apps. With Pipecat’s Gladia STT service, you can plug Gladia’s accurate, low‑latency transcription directly into your agent. Looking for the full API and options? See the official Pipecat Gladia STT docs: [docs.pipecat.ai/server/services/stt/gladia](https://docs.pipecat.ai/server/services/stt/gladia). ## What you can build * Voice assistants that understand users in real time * Live captions and meeting companions * Contact‑center bots and IVRs * In‑app help widgets with speech interfaces ## Quickstart using the sample repo Use our end‑to‑end sample that wires Pipecat with Gladia STT : [gladia-samples/integrations-examples/pipecat-bot](https://github.com/gladiaio/gladia-samples/tree/main/integrations-examples/pipecat-bot) ## Next steps Get your API key on the Gladia dashboard Browse and run the full sample Explore service options and parameters # Recall Source: https://docs.gladia.io/chapters/integrations/recall How to use Gladia with Recall.ai Meeting Bot ## What is Recall? Recall.ai provides a unified Meeting Bot API that can join Zoom, Google Meet, Microsoft Teams, Cisco Webex, and Slack calls, stream audio in real time, and route it to providers for transcription and intelligence. With Gladia STT connected in Recall.ai, you get accurate, low‑latency transcription for your calls without building any of the bot or RTC plumbing yourself. Looking to connect Gladia in Recall.ai now? Open the Gladia page in the Recall.ai dashboard: [us-east-1.recall.ai/dashboard/transcription/gladia](https://us-east-1.recall.ai/dashboard/transcription/gladia). If you use a different region, switch the region in the URL. ## What you can build * Meeting assistants that take notes, summaries, and action items * Live captions for meetings, webinars, and trainings * Contact‑center analytics and QA from customer calls * LLM agents that join calls and understand participants in real time ## Quick setup via the dashboard 1. Go to the Recall.ai Gladia page: [https://us-east-1.recall.ai/dashboard/transcription/gladia](https://us-east-1.recall.ai/dashboard/transcription/gladia) 2. Paste your Gladia API key from the Gladia dashboard: [https://app.gladia.io/apikeys](https://app.gladia.io/apikeys) 3. Click “Save credentials.” Recall.ai will use your Gladia key for transcription. ## Next steps Get your API key on the Gladia dashboard Open the dashboard page and save your key Learn more about the platform and features # SDK Source: https://docs.gladia.io/chapters/integrations/sdk Use the SDK to integrate Gladia into your project To integrate Gladia into your project faster, and use all of our best practices for transcribing audio, you can use one of our SDKs. Want to start building realtime transcription applications with Gladia? Check out our [live transcription guide](/chapters/live-stt/quickstart). Use the JavaScript SDK to integrate Gladia into your project Use the Python SDK to integrate Gladia into your project # Twilio Source: https://docs.gladia.io/chapters/integrations/twilio How to integrate Gladia with Twilio Media Streams ## What is Twilio Twilio is a cloud platform that lets you add voice, messaging, and video to your apps with simple APIs. Instead of worrying about telecom infrastructure, you can focus on your product. In this guide, we'll use Twilio for voice with Gladia Speech-to-Text. ## Twilio Media Streams Twilio Programmable Voice lets you make and receive calls. Media Streams can send real-time audio from a live call to your server over WebSocket. When connected to Gladia's low-latency Speech-to-Text (STT), you get instant transcription. This powers real-time voice agents, call analytics, compliance tools, and live captions. Want the full API reference? Check the official Twilio Media Streams docs:\ [www.twilio.com/docs/voice/twiml/stream](https://www.twilio.com/docs/voice/twiml/stream). ## What you can build * Real-time transcription and analytics * IVR and call center bots * Compliance and QA monitoring * Live captions on phone calls ## Quickstart Try our sample project that connects Twilio Media Streams with Gladia STT:\ [gladia-samples/integrations-examples/twilio](https://github.com/gladiaio/gladia-samples/tree/main/integrations-examples/twilio) ## Next steps Get your API key on the Gladia dashboard See and run the complete sample Learn about streaming audio from Twilio # Vapi Source: https://docs.gladia.io/chapters/integrations/vapi # VideoSDK Source: https://docs.gladia.io/chapters/integrations/videosdk How to integrate Gladia with VideoSDK AI Agents ## What is VideoSDK? VideoSDK is a platform that provides real-time video, audio, and interactive live streaming infrastructure. Its AI Agents framework lets you build voice-first AI agents that can join video calls, listen to participants, and respond in real time. With the Gladia STT plugin, your VideoSDK AI agents get accurate, low-latency transcription out of the box. Looking for the full API and options? See the official VideoSDK Gladia STT plugin docs: [docs.videosdk.live/ai\_agents/plugins/stt/gladia](https://docs.videosdk.live/ai_agents/plugins/stt/gladia). ## What you can build * Real-time voice AI agents in video calls * Live captions and transcription for meetings * Customer-support bots with speech understanding * Interactive voice assistants in live streaming ## Next steps Get your API key on the Gladia dashboard Explore the plugin options and parameters Learn more about the platform # Zapier Source: https://docs.gladia.io/chapters/integrations/zapier How to use Gladia with Zapier ## What is Zapier? Zapier is a no-code automation platform that connects thousands of apps together. With the Gladia integration on Zapier, you can build automated workflows (called Zaps) that trigger transcription jobs, process results, and route them to other apps — all without writing any code. Ready to connect Gladia in Zapier? Visit the Gladia integration page: [zapier.com/apps/gladia-ca229109/integrations](https://zapier.com/apps/gladia-ca229109/integrations). ## What you can build * Automated transcription of audio files uploaded to cloud storage * Workflows that send transcription results to Slack, email, or Notion * Meeting recording pipelines that trigger on calendar events * Content repurposing workflows from audio to text ## Next steps Get your API key on the Gladia dashboard Browse available triggers and actions Learn more about the platform # Getting started Source: https://docs.gladia.io/chapters/introduction/getting-started Setup your Gladia account and start using the most reliable state-of-the-art Speech-To-Text API ### Signing Up You will first need to create your account. Sign-up to [app.gladia.io](https://app.gladia.io/?utm_campaign=197609689-Website\&utm_source=documentation\&utm_content=sign-up). You can sign-up through Google and more sign-up methods will be available in the near feature. ### Get your API key Now that you signed up, login to app.gladia.io and go to the [API keys section](https://app.gladia.io/apikeys). We should have already created a default key for you. You can use this one or create your own. Gladia offers 10 Hours of free audio transcription per month if you want to test the service! With your API key, you're now ready to use Gladia APIs. ### With our SDK / API Want to build your integration yourself ? Use our SDK or our API directly: Quickstart with Real-Time transcription using Gladia. Quickstart with Asynchronous transcription using Gladia. ### With one of our partners If you want to build fast with Gladia, you can use one of our integration partners: For built-in best practices Framework for building voice AI agents For real-time voice infrastructure Platform for hosted voice AI agents For meeting recorder / agents For meeting recorder / agents Open-source meeting bot API To use Gladia with telephony For adding video/audio calls to your app For connecting agents to external tools For no-code automation For no-code automation For workflow automation ## Next steps Now that you tested Gladia's basic transcription, you might want to extract data, enhance, translate or format your audio transcriptions. On top of its **Speech-to-text API**, Gladia provides a whole set of tools that you might want to use for your particular use cases like: # Welcome to Gladia Source: https://docs.gladia.io/chapters/introduction/index The most reliable state-of-the-art Speech-To-Text API provider Gladia is a state-of-the-art audio transcription and intelligence provider. We offer a simple API to get transcription from your audios and videos, in both Real-time and asynchronous ways, with audio intelligence tools to extract, analyze and understand data from your audios. Getting started with Gladia Speech-To-Text API. Quickstart with Real-Time transcription using Gladia. Quickstart with Asynchronous transcription using Gladia. Use the power of LLMs directly on your audios without a third party service. Test gladia quickly on our playground. Visit our technical documentation for more details. # Our integration partners If you want to build fast with Gladia, you can use one of our integration partners: For built-in best practices Framework for building voice AI agents For real-time voice infrastructure Platform for hosted voice AI agents For meeting recorder / agents For meeting recorder / agents Open-source meeting bot API To use Gladia with telephony For adding video/audio calls to your app For connecting agents to external tools For no-code automation For no-code automation For workflow automation *** # Playground Source: https://docs.gladia.io/chapters/introduction/playground Use the Gladia app to transcribe your audio [Gladia's playground](https://app.gladia.io/?utm_campaign=197609689-Website\&utm_source=documentation\&utm_content=sign-up) is a convenient way to test our Speech-To-Text API. On the playground you are able to transcribe remote audio files through URL, and also upload your local audio files, alongside with live audio transcription. Choose your audio source (stream from you microphone, or upload a local file) Then proceed to the next step. You'll be able to select some of the options Gladia API provide for your transcription. \ For this example, we want to detect the named entity (like email adresses, phone numbers, etc.), so we turned on **named entity recognition**. Only a few features of Gladia API are available on the playground. For more advanced testing, check [our API documentation](/api-reference/) instead. You can talk to Gladia by clicking on the "Start transcribing" button, and you'll be able to see the transcription of your voice in the "Transcription" tab. Text in italic in the transcription represents [partials transcripts](/chapters/live-stt/features#partial-transcripts). You can see an already formatted and readable results in the default "Transcription" tab, and you'll also find the result in JSON format (the one you'd get with an API call). # Code switching Source: https://docs.gladia.io/chapters/language/code-switching Transcribe conversations that mix multiple languages in real time or pre-recorded Code switching handles conversations where speakers switch languages mid-utterance or across turns. Gladia can detect and transcribe the active language dynamically, and annotate results with the detected languages code. For best accuracy and latency, provide a small set of expected languages. Avoid listing dozens of similar languages. ## How to use code switching ? Enable code switching in the session configuration: ```json theme={"system"} { "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "language_config": { "languages": ["en", "de", "fr"], "code_switching": true } } ``` When enabled, transcript messages include the detected `language` per utterance and per word. Use the [`transcript` message](/api-reference/v2/live/message/transcript) schema for details. **Do not enable `code_switching` with an empty `languages` list.** When no languages are specified, the language detector evaluates every utterance against 100+ supported languages, leading to frequent misdetections - especially between similar-sounding languages. ## Recommendations * Limit the number of languages to those you expect. * Combine with [custom vocabulary](/chapters/pre-recorded-stt/features/custom-vocabulary) per language for domain terms. # Automatic language detection Source: https://docs.gladia.io/chapters/language/language-detection How to let Gladia detect the spoken language automatically Gladia can automatically detect the spoken language from audio, either within a candidate set you provide or across all supported languages. Provide a small list of likely languages when possible. It improves accuracy and reduces latency. ## How to use language detection ? You can either omit `language_config.languages` to search across all supported languages, or pass a constrained list: ```json Detect across all languages theme={"system"} { "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "language_config": { "languages": [], "code_switching": false } } ``` ```json Detect within a set of languages theme={"system"} { "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "language_config": { "languages": ["en", "de", "fr"], "code_switching": false } } ``` The detected `language` is returned on each utterance and can vary over time if you also enable code switching. ## Tips * Use supported languages to pick valid ISO 639-1 codes. * Combine with `messages_config.receive_partial_transcripts` in real time to get immediate feedback. * For mixed-language conversations, enable code switching. # Supported languages Source: https://docs.gladia.io/chapters/language/supported-languages All languages supported by Gladia with ISO 639-1 codes and capabilities Not sure which language is spoken ? See [Automatic language detection](/chapters/language/language-detection). For conversations mixing languages, see [Code switching](/chapters/language/code-switching). In our APIs and results, languages use 2-letter ISO 639-1 codes. When no 639-1 code exists, we use the corresponding ISO 639-3 three-letter code (e.g. `haw` for Hawaiian). Use the 2-letter code in language\_config.languages (e.g. \["en", "fr"]). | Language | Code | Auto-discovery
and code-switch | Translation | | :------------: | :--: | :---------------------------------: | :---------: | | Afrikaans | af | Yes | Yes | | Albanian | sq | Yes | Yes | | Amharic | am | Yes | Yes | | Arabic | ar | Yes | Yes | | Armenian | hy | Yes | Yes | | Assamese | as | Yes | Yes | | Azerbaijani | az | Yes | Yes | | Bashkir | ba | Yes | Yes | | Basque | eu | Yes | Yes | | Belarusian | be | Yes | Yes | | Bengali | bn | Yes | Yes | | Bosnian | bs | Yes | Yes | | Breton | br | Yes | Yes | | Bulgarian | bg | Yes | Yes | | Catalan | ca | Yes | Yes | | Chinese | zh | Yes | Yes | | Croatian | hr | Yes | Yes | | Czech | cs | Yes | Yes | | Danish | da | Yes | Yes | | Dutch | nl | Yes | Yes | | English | en | Yes | Yes | | Estonian | et | Yes | Yes | | Faroese | fo | Yes | Yes | | Finnish | fi | Yes | Yes | | French | fr | Yes | Yes | | Galician | gl | Yes | Yes | | Georgian | ka | Yes | Yes | | German | de | Yes | Yes | | Greek | el | Yes | Yes | | Gujarati | gu | Yes | Yes | | Haitian Creole | ht | Yes | Yes | | Hausa | ha | Yes | Yes | | Hawaiian | haw | Yes | Yes | | Hebrew | he | Yes | Yes | | Hindi | hi | Yes | Yes | | Hungarian | hu | Yes | Yes | | Icelandic | is | Yes | Yes | | Indonesian | id | Yes | Yes | | Italian | it | Yes | Yes | | Japanese | ja | Yes | Yes | | Javanese | jw | Yes | Yes | | Kannada | kn | Yes | Yes | | Kazakh | kk | Yes | Yes | | Khmer | km | Yes | Yes | | Korean | ko | Yes | Yes | | Lao | lo | Yes | Yes | | Latin | la | Yes | Yes | | Latvian | lv | Yes | Yes | | Lingala | ln | Yes | Yes | | Lithuanian | lt | Yes | Yes | | Luxembourgish | lb | Yes | Yes | | Macedonian | mk | Yes | Yes | | Malagasy | mg | Yes | Yes | | Malay | ms | Yes | Yes | | Malayalam | ml | Yes | Yes | | Maltese | mt | Yes | Yes | | Maori | mi | Yes | Yes | | Marathi | mr | Yes | Yes | | Mongolian | mn | Yes | Yes | | Myanmar | my | Yes | Yes | | Nepali | ne | Yes | Yes | | Norwegian | no | Yes | Yes | | Nynorsk | nn | Yes | Yes | | Occitan | oc | Yes | Yes | | Pashto | ps | Yes | Yes | | Persian | fa | Yes | Yes | | Polish | pl | Yes | Yes | | Portuguese | pt | Yes | Yes | | Punjabi | pa | Yes | Yes | | Romanian | ro | Yes | Yes | | Russian | ru | Yes | Yes | | Sanskrit | sa | Yes | Yes | | Serbian | sr | Yes | Yes | | Shona | sn | Yes | Yes | | Sindhi | sd | Yes | Yes | | Sinhala | si | Yes | Yes | | Slovak | sk | Yes | Yes | | Slovenian | sl | Yes | Yes | | Somali | so | Yes | Yes | | Spanish | es | Yes | Yes | | Sundanese | su | Yes | Yes | | Swahili | sw | Yes | Yes | | Swedish | sv | Yes | Yes | | Tagalog | tl | Yes | Yes | | Tajik | tg | Yes | Yes | | Tamil | ta | Yes | Yes | | Tatar | tt | Yes | Yes | | Telugu | te | Yes | Yes | | Thai | th | Yes | Yes | | Tibetan | bo | Yes | Yes | | Turkish | tr | Yes | Yes | | Turkmen | tk | Yes | Yes | | Ukrainian | uk | Yes | Yes | | Urdu | ur | Yes | Yes | | Uzbek | uz | Yes | Yes | | Vietnamese | vi | Yes | Yes | | Welsh | cy | Yes | Yes | | Wolof | wo | No | Yes | | Yiddish | yi | Yes | Yes | | Yoruba | yo | Yes | Yes | # Concurrency and Rate limits Source: https://docs.gladia.io/chapters/limits-and-specifications/concurrency Rate limiting and transcriptions concurrency In order to provide a smooth experience and optimal performance for all users, Gladia accounts can have up to **3** differents limits for **Free**, **Paid** & **Enterprise** users : Need a higher concurrency limit? The value listed below are default values. We can provide high concurrencies based on your needs, so feel free to [contact us](https://www.gladia.io/contact). | Plan type | Usage limit
(per month) | Max Transcriptions in concurrency
(pre-recorded) | Max Transcriptions in concurrency (Live) | | -------------- | ------------------------------ | ------------------------------------------------------ | ---------------------------------------- | | **Enterprise** | Unlimited | On demand | On demand | | **Paid** | Unlimited | 25 | 30 | | **Free** | 10 Hours | 3 | 1 | * **Usage limit** : (**free-tier** users only) This determines the number of hours a user can transcribe a given month. This is a limitation for free tier users only. * **Concurrency** : (depending on free/paid tier) This refers to the maximum number of transcription (pre-recorder or real-time) that a user can process at the same time. For asynchronous transcriptions, Paid plan users can queue up to 300 requests, but only will still have 25 max processed concurrently. * **Realtime session duration** : (all plans) A single realtime (live) transcription session cannot exceed **3 hours**. After 3 hours, the session will be terminated. For longer events, start a new session when approaching the limit. * **API level rate limit** : (same for every user) Which is the number of API calls that a user can make within a particular time frame. This is to ensure that a single user or malicious actor doesn't affect the performance of the API for all the other users. # Data retention Source: https://docs.gladia.io/chapters/limits-and-specifications/data-retention Data retention policies and zero data behavior To provide transcription services, Gladia processes several types of data: * **Audio input**: Audio files or audio streams provided for transcription * **Transcription output**: Text, timestamps, words, utterances * **API Metadata**: Request IDs, timestamps, processing status * **Logs**: Operational logs for system reliability The duration for which your data is stored depends on your plan type. You have two main options for data retention: * **Standard data retention:** Your data (such as audio files, transcripts, and metadata) remains accessible for a set number of days, up to a maximum allowed by your plan. The minimum retention value is 0, which means your data is deleted within 24 hours. The maximum and default value is 12 months. * **Zero data retention:** Data storage is minimized at all stages, avoiding temporary storage whenever possible. All data is deleted immediately after processing. Only Enterprise users are eligible for custom data retention and the zero data retention option. To enable usage tracking, Gladia retains essential API metadata: request ID, timestamp, processing status and audio duration. Immutable logs are also maintained, for a limited period, to ensure service quality and reliability. ## Zero Data Retention behavior When Zero Data Retention is enabled, Gladia processes data ephemerally; no data is stored at rest. * **No audio files are stored**: Files cannot be retrieved through the API or in the [Gladia's playground](https://app.gladia.io/?utm_campaign=197609689-Website\&utm_source=documentation\&utm_content=sign-up). File upload is also disabled; the asynchronous API must use an external audio file url, such as S3 presigned url. * **No transcripts are stored**: Transcription results cannot be retrieved through the API and are not visible in the [Gladia's playground](https://app.gladia.io/?utm_campaign=197609689-Website\&utm_source=documentation\&utm_content=sign-up). * **No metadata retrieval**: Transcription API calls, audio duration, and other metadata cannot be retrieved through the API or in the [Gladia's playground](https://app.gladia.io/?utm_campaign=197609689-Website\&utm_source=documentation\&utm_content=sign-up). * **Transcription results delivered only via callbacks**: The only way to receive transcription results is through callbacks; they cannot be retrieved by any other means. Once the result is delivered, the audio, transcript, and metadata cannot be accessed. # Multiple channels Source: https://docs.gladia.io/chapters/limits-and-specifications/multiple-channels Transcribe multi-channel audio and tag utterances by channel Gladia supports multi-channel audio for both pre-recorded files and live streams. Each utterance in the result includes a `channel` key corresponding to the source channel. ## Pre-recorded If your audio file has multiple distinct channels, Gladia will transcribe them automatically. Sending an audio with two different channels (with different content) will be billed as two audios. If your audio has multiple channels with the same content, it will only be billed once. ## Live transcription For live audio streams, specify the channel count in your [init request](/api-reference/v2/live/init): ```json theme={"system"} { "channels": 2 } ``` Gladia's real-time API will automatically split the channels and transcribe them separately. Transcribing an audio stream with multiple channels is billed per channel. For example, an audio stream with 2 channels will be billed as double the audio duration, even if the channels are identical. For a detailed guide on how to merge multiple audio tracks into a single multi-channel stream and send it over a WebSocket, see the [Sending multiple audio tracks over a single WebSocket](/chapters/live-stt/quickstart#merging-multiple-audio-tracks-into-one-multi-channel-websocket) section. # Supported files & duration Source: https://docs.gladia.io/chapters/limits-and-specifications/supported-formats Media files limitations We support almost all types of audio or video files with a tradeoff to be taken into account between the transfer time of specific formats that can generate big files and the time to convert the original format to the target one (WAV pcm 16KHz little-endian). You can find an estimate of the conversion times in the table below. ## Gladia API current limitations Those limits will be gradually lifted to ensure the full stability and performance of the service for everyone. * **Audio length (pre-recorded)**: The maximum length of audio that can be transcribed in a single request is currently 135 minutes. Attempts to transcribe longer audio files will result in an error. We support up to 4h15 audio length for enterprise plans. * **Realtime session duration**: For [live (realtime) transcription](/chapters/live-stt/quickstart), a single WebSocket session cannot exceed **3 hours**. The session will be terminated after 3 hours; for longer events, start a new session before reaching the limit. * **File size**: Audio files must not exceed 1000 MB in size. Larger files will not be accepted by the API. ### Splitting oversize audio files For audio files that are near or exceed the limitations on length and size, it is recommended to split them into smaller chunks of \~60 minutes each. This approach not only adheres to the API constraints but also generally yields better transcription results. Tools for Splitting Audio Files: * **FFMPEG** : FFMPEG is a versatile command-line tool that can be used to manipulate audio and video files. It is a popular choice for splitting long audio files. * **ffmpeg-python** : For Python users, ffmpeg-python is a wrapper around FFMPEG that provides a more Pythonic interface for interacting with FFMPEG. * **prism-media** for Node.js : Node.js users can use prism-media for manipulating media files, including splitting audio files. * **fluent-ffmpeg** for Node.js : Another option for Node.js users is fluent-ffmpeg, which offers a simpler and more fluent API for handling media files. Following these best practices will help you avoid issues due to limitations and maximize the quality of the transcriptions you obtain from the Audio Transcription API. ## Supported audio formats | Source Format | Mime Type | Audio/Video | | ------------- | --------------- | ----------- | | aac | audio/aac | Audio | | ac3 | audio/ac3 | Audio | | eac3 | audio/eac3 | Audio | | flac | audio/flac | Audio | | m4a | audio/mp4 | Audio | | mp2 | audio/mpeg | Audio | | mp3 | audio/mpeg | Audio | | ogg | application/ogg | Audio | | opus | audio/opus | Audio | | wav | audio/wav | Audio | ## Supported video formats | Source Format | Mime Type | Audio/Video | | ------------- | ---------------- | ----------- | | 3g2 | video/3gpp2 | Video | | 3gp | video/3gpp | Video | | avi | video/x-msvideo | Video | | flv | video/x-flv | Video | | m4v | video/x-m4v | Video | | matroska | video/x-matroska | Audio/Video | | mov | video/quicktime | Video | | mp4 | video/mp4 | Audio/Video | | wmv | video/x-ms-wmv | Video | ## Supported online video services | Platform | Audio/Video Support | Stage | | -------------- | ------------------- | -------- | | TikTok | Video | Released | | Instagram | Video | Released | | Facebook | Video | Released | | Vimeo | Video | Released | | Dailymotion | Video | Released | | LinkedIn | Video | Released | | Sharechat | Video | Released | | Likee | Video | Released | | TikTok (Beta) | Video | Beta | | Twitter (Beta) | Video | Beta | ## Conversion time | Source Format | Mime Type | Audio/Video | Estimated File Size (1 Hour) | Estimated Conversion Time (1 Hour) | | ------------- | ---------------- | ----------- | ---------------------------- | ---------------------------------- | | 3g2 | video/3gpp2 | Video | \~300 MB | \~30 seconds | | 3gp | video/3gpp | Video | \~300 MB | \~40 seconds | | aac | audio/aac | Audio | \~60 MB | \~36 seconds | | ac3 | audio/ac3 | Audio | \~215 MB | \~42 seconds | | avi | video/x-msvideo | Video | \~800 MB | \~1 minute | | eac3 | audio/eac3 | Audio | \~215 MB | \~32 seconds | | flac | audio/flac | Audio | \~260 MB | \~46 seconds | | flv | video/x-flv | Video | \~400 MB | \~40 seconds | | m4a | audio/m4a | Audio | \~60 MB | \~26 seconds | | x-m4a | audio/x-m4a | Audio | \~60 MB | \~26 seconds | | m4v | video/x-m4v | Video | \~800 MB | \~1 minute | | matroska | video/x-matroska | Audio/Video | \~800 MB | \~1 minute | | mov | video/quicktime | Video | \~800 MB | \~1 minute | | mp2 | audio/mpeg | Audio | \~120 MB | \~42 seconds | | mp3 | audio/mpeg | Audio | \~120 MB | \~37 seconds | | mp4 | video/mp4 | Audio/Video | \~800 MB | \~1 minute | | ogg | application/ogg | Audio | \~60 MB | \~1 minute | | opus | audio/opus | Audio | \~30 MB | \~1 minute | | wav | audio/wav | Audio | \~510 MB | N/A | | wmv | video/x-ms-wmv | Video | \~800 MB | \~1 minute | # Audio Intelligence Source: https://docs.gladia.io/chapters/live-stt/audio-intelligence Turn speech into actionable insights with translation, summarization, entities, chapters, sentiment and more Audio intelligence turns raw speech into structured, useful data on top of transcription. Once the words are captured, these features help you understand, organize, and act on the content — from instant translation to key-point summaries, entity detection, chapter markers, or emotions analysis. Translate transcripts and subtitles into multiple languages in one request. Boost recognition accuracy for specific terms. Generate concise summaries or bullet points for quick understanding. Detect and categorize key entities like people, organizations, dates, and more. Segment long audio into chapters with headlines and summaries for easy navigation. Understand the tone and emotions expressed across the transcript. # Endpointing Source: https://docs.gladia.io/chapters/live-stt/features/endpointing What's endpointing and how it works Endpointing is the mechanism Gladia uses in live transcription to decide when a speaker has "finished" an utterance, so the API can close that utterance and emit a final transcript segment. In practice, endpointing answers the question: "How much silence should we wait before we consider the sentence (or turn) complete?" ### Why endpointing matters Endpointing is one of the main knobs that controls the tradeoff between: * **Latency (speed)**: how quickly you get final utterances * **Completeness**: whether you avoid cutting someone off mid-thought * **Chunking quality**: whether utterances align well with natural turns or sentences Lower endpointing values feel "snappier" (great for voice agents), while higher values tend to produce cleaner, more complete segments (great for meetings and lectures). ### How it works conceptually During a live session, Gladia continuously analyzes the incoming audio stream and: 1. Detects speech activity on each channel (voice activity detection) 2. Groups speech into an "utterance" while speech is ongoing 3. When it observes silence lasting at least endpointing seconds, it considers the utterance finished and closes it (finalizes it). 4. The AI model is then used to transcribe the final result of the utterance. 5. If speech never pauses long enough, Gladia still has a safety mechanism to close the utterance (*maximum\_duration\_without\_endpointing*, see next section) You can also subscribe to speech activity messages to know when speech [starts](https://docs.gladia.io/api-reference/v2/live/callback/speech-start) and [ends](https://docs.gladia.io/api-reference/v2/live/callback/speech-end) (useful to drive UI or agent turn-taking) ### The 2 key parameters **endpointing (seconds)** \ Definition: the duration of silence that closes the current utterance. * Default: 0.05 * Range: 0.01 to 10 Effect: * Smaller value = closes utterances faster, but can split sentences if the speaker hesitates briefly. * Larger value = waits longer before finalizing, which improves segment completeness but increases latency. **maximum\_duration\_without\_endpointing (seconds)** Definition: maximum amount of time Gladia will keep an utterance open without detecting endpointing silence. If that limit is reached, the utterance is considered finished anyway. * Default: 5 * Range: 5 to 60 Why it exists: it prevents extremely long, never-ending utterances (for example: constant background noise, a speaker who never pauses, or long monologues), which is important for downstream UX and processing stability. # Partial transcripts Source: https://docs.gladia.io/chapters/live-stt/features/partial-transcripts Low-latency streaming transcripts before the final result Partial transcripts provide a low-latency streaming transcription as words are spoken, offering immediate insights before the final, high-accuracy transcript is ready. To enable partial transcripts, add the `receive_partial_transcripts` property to the `messages_config` object: ```json theme={"system"} { "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "language_config": { "languages": ["en"], "code_switching": false }, "messages_config": { "receive_partial_transcripts": true, "receive_final_transcripts": true } } ``` With this configuration, you will receive both partial transcripts as they are generated and the final, most accurate version of each utterance. To reduce the total response time and create a more fluid user experience, partial transcripts use a faster, smaller model than the one used for final transcripts, trading a small amount of accuracy for a large gain in latency. Partial transcripts accuracy deteriorates when multiple languages and/or code switching are enabled. For best results, limit the number of languages. When `receive_partial_transcripts` is `true`, the real-time API will send transcript messages for both intermediate and final results. To distinguish between them, the message payload includes the [`is_final`](/api-reference/v2/live/message/transcript#schema-data-is-final) boolean field. * `"is_final": false`: The message contains a partial transcript, which is subject to change. * `"is_final": true`: The message contains the final, most accurate transcript for an utterance. This transcript will not change. In the same utterance, the partial and final transcripts share the same `data.id`. # Live transcription - Migration guide from V1 to V2 Source: https://docs.gladia.io/chapters/live-stt/migration-from-v1 Migrate to the latest version of Gladia's real-time speech-to-text API Live transcription V2 is the latest real-time speech-to-text API from Gladia. It offers more features and has significant improvements in latency compared to V1. Here is a guide on how to migrate to V2, so you can start enjoying all the benefits. Please make sure you migrate sooner rather than later as we're looking to remove support for V1 sometime in the future. Before we do so however, we'll of course reach out to those of you who are still on V1. ## Initiating the connection to the WebSocket In V1, you always connect to the same WebSocket URL (wss\://api.gladia.io/audio/text/audio-transcription) and send your configuration through the WebSocket connection. In V2, you first generate a unique WebSocket URL with a call to our [POST /v2/live endpoint](/api-reference/v2/live/init), and then connect to it. This URL contains a token that is unique to your live session. You'll be able to resume your session in case of a lost connection, or give the URL to a web client without exposing your Gladia API key. ```javascript JavaScript theme={"system"} import WebSocket from 'ws'; const socket = new WebSocket('wss://api.gladia.io/audio/text/audio-transcription'); socket.addEventListener("open", function() { // Send configuration socket.send(JSON.stringify({ 'x_gladia_key': 'YOUR_GLADIA_API_KEY', // ...config properties })) // Start sending audio chunks }); ``` ```javascript JavaScript theme={"system"} import WebSocket from 'ws'; const response = await fetch('https://api.gladia.io/v2/live', { method: 'POST', headers: { 'Content-Type': 'application/json', 'x-gladia-key': '', }, body: JSON.stringify({ // ...config properties }), }); if (!response.ok) { // Look at the error message // It might be a configuration issue console.error(`${response.status}: ${(await response.text()) || response.statusText}`); process.exit(response.status); } const {url} = await response.json(); const socket = new WebSocket(url); socket.addEventListener("open", function() { // Start sending audio chunks }); ``` ## Configuration With V2 offering more features, the configuration comes with some changes. You'll find the full configuration definition in the [POST /v2/live API reference page](/api-reference/v2/live/init). Here, we'll show you how to migrate your V1 configuration object to the V2 one. ### Audio encoding `encoding`, `bit_depth` and `sample_rate` are still present in V2, but with less options for now. As `wav` is the same `encoding` as `wav/pcm`, V2 has dropped support for `wav` and defaults to `wav/pcm`. `amb`, `mp3`, `flac`, `ogg/vorbis`, `opus`, `sphere` and `amr-nb` are no longer supported. `bit_depth` option `64` is no longer supported. If you're using an unsupported `encoding` or `bit_depth`, please contact us with your use case. In the mean time, keep using V1. ### Model Only one model is supported in V2 for now, so omit the property `model`. ### End-pointing and maximum audio duration `endpointing` is now declared in seconds instead of milliseconds. `maximum_audio_duration` has been renamed to `maximum_duration_without_endpointing`. ```json theme={"system"} { "endpointing": 800, "maximum_audio_duration": 10 } ``` ```json theme={"system"} { "endpointing": 0.8, "maximum_duration_without_endpointing": 10 } ``` ### Language #### Automatic single language Automatic single language behavior is the default in both V1 and V2, so you can just omit those parameters from your configuration. ```json theme={"system"} { "language_behaviour": "automatic single language" } ``` ```json theme={"system"} { "language_config": { "languages": [], "code_switching": false } } ``` #### Automatic multiple languages ```json theme={"system"} { "language_behaviour": "automatic multiple languages" } ``` ```json theme={"system"} { "language_config": { "languages": [], // You can now specify the expected languages in V2 as guidance to improve accuracy and latency "code_switching": true } } ``` #### Manual Languages are now specified with a 2-letter code, as in the API for asynchronous speech-to-text.
See [this page](/chapters/language/supported-languages) for a complete list of codes. ```json theme={"system"} { "language_behaviour": "manual", "language": "english" } ``` ```json theme={"system"} { "language_config": { "languages": ["en"], "code_switching": false } } ``` ### Frames format You can send audio chunk as bytes or base64 and we'll detect the format automatically. The parameter `frames_format` is no longer present. ### Audio enhancer `audio_enhancer` has been moved into the `pre_processing` object. ```json theme={"system"} { "audio_enhancer": true } ``` ```json theme={"system"} { "pre_processing": { "audio_enhancer": true } } ``` ### Word timestamps `word_timestamps` has been renamed to `words_accurate_timestamps` and moved into the `realtime_processing` object. ```json theme={"system"} { "word_timestamps": true } ``` ```json theme={"system"} { "realtime_processing": { "words_accurate_timestamps": true } } ``` ### Other properties `prosody`, `reinject_context` and `transcription_hint` are not supported for now. They may return in another form in the future. ### Full config migration sample ```json theme={"system"} { "encoding": "wav", "bit_depth": 8, "sample_rate": 48000, "model": "accurate", "endpointing": 800, "maximum_audio_duration": 10, "language_behaviour": "manual", "language": "english", "audio_enhancer": true, "word_timestamps": true } ``` ```json theme={"system"} { "encoding": "wav/pcm", "bit_depth": 8, "sample_rate": 48000, "endpointing": 0.8, "maximum_duration_without_endpointing": 10, "language_config": { "languages": ["en"] } "pre_processing": { "audio_enhancer": true }, "realtime_processing": { "words_accurate_timestamps": true } } ``` ## Send audio chunks If you were sending chunks as bytes, nothing has changed. If you were sending them as base64, the format of the JSON messages changed in V2. See the [API reference](/api-reference/v2/live/action/audio-chunk) for the full format. ```json theme={"system"} { "frames": "" } ``` ```json theme={"system"} { "type": "audio_chunk", "data": { "chunk": "" } } ``` ## Transcription message In V1, we only send two kinds of messages through WebSocket: * the "connected" message * the "transcript" messages In V2, we send more: * lifecycle event messages * acknowledgment messages * add-on messages * post-processing messages * ... To read a transcription message in V1, you verify that the `type` field is `"final"` and/or the `transcription` field is not empty.
In V2, you should confirm that the `type` field is `transcript` and that `data.is_final` is `true`. Below are examples of transcript messages in V1 and V2, so you can see the differences. See the [API reference](/api-reference/v2/live/message/transcript) for the full format. ```json theme={"system"} { "event": "transcript", "request_id": "G-3abade39", "type": "final", "transcription": " Hello world", "time_begin": 1.4376875, "time_end": 2.4696875, "confidence": 0.65, "language": "en", "utterances": [ { "transcription": " Hello world", "time_begin": 1.4376875, "time_end": 2.4696875, "language": "en", "confidence": 0.65, "stable": true, "id": 0 } ], "inference_time": 0.843909502029419, "duration": 2.5845000000000002 } ``` ```json theme={"system"} { "type": "transcript", "session_id": "de0a341d-c69f-4e15-a649-7b3f49e211f0", "created_at": "2024-10-10T14:35:28.387Z", "data": { "id": "00_00000000", "is_final": true, "utterance": { "text": " Hello world", "start": 0.188, "end": 1.284, "language": "en", "confidence": 1, "channel": 0, "words": [ { "word": " Hello", "start": 0.188, "end": 0.735, "confidence": 1 }, { "word": " world", "start": 0.736, "end": 1.284, "confidence": 1 } ] } } } ``` If you're not interested in new messages and simply want the ones from the V1 API, you can always configure what kind of messages you want when calling the [POST /v2/live endpoint](/api-reference/v2/live/init) to initiate the session. With the following configuration, you will only receive final transcript messages: ```json theme={"system"} { "messages_config": { "receive_partial_transcripts": false, "receive_final_transcripts": true, "receive_speech_events": false, "receive_pre_processing_events": false, "receive_realtime_processing_events": false, "receive_post_processing_events": false, "receive_acknowledgments": false, "receive_lifecycle_events": false } } ``` ## End the live session The format of this message also changed. See the [API reference](/api-reference/v2/live/action/stop-recording) for the full format. ```json theme={"system"} { "event": "terminate" } ``` ```json theme={"system"} { "type": "stop_recording" } ``` # Quickstart Source: https://docs.gladia.io/chapters/live-stt/quickstart How to transcribe live audio with Gladia's Real-time speech-to-text (STT) API Live transcription supports **`"solaria-1"` only**. The SDK simplifies real-time speech-to-text integration by abstracting the underlying API. Designed for developers, it offers: * Effortless implementation with minimal code to write. * Built-in resilience with automatic error handling (e.g., reconnection on network drops) ensures uninterrupted transcription. No need to manually manage retries or state recovery. ## Install the SDK ```sh JavaScript theme={"system"} npm install @gladiaio/sdk ``` ```sh Python theme={"system"} # Using pip pip install gladiaio-sdk # Using uv uv add gladiaio-sdk ``` ```javascript JavaScript theme={"system"} import { GladiaClient } from "@gladiaio/sdk"; ``` ```python Python theme={"system"} from gladiaio_sdk import ( GladiaClient, LiveV2InitRequest, LiveV2LanguageConfig, LiveV2MessagesConfig, LiveV2WebSocketMessage, LiveV2InitResponse, LiveV2EndedMessage, ) ``` ## Initiate your real-time session First, call the [ endpoint](/api-reference/v2/live/init) and pass your configuration. It's important to correctly define the properties `encoding`, `sample_rate`, `bit_depth` and `channels` as we need them to parse your audio chunks. ```typescript JavaScript theme={"system"} const gladiaClient = new GladiaClient({ apiKey: , }); const gladiaConfig = { model: "solaria-1", encoding: 'wav/pcm', sample_rate: 16000, bit_depth: 16, channels: 1, language_config: { languages: ["fr"], code_switching: false, }, }; const liveSession = gladiaClient.liveV2().startSession(gladiaConfig); ``` ```python Python theme={"system"} # Our Python SDK supports sync/threaded and asyncio versions. gladia_client = GladiaClient(api_key="") # sync/threaded version live_client = gladia_client.live_v2() # asyncio version live_client = gladia_client.live_v2_async() init_request = LiveV2InitRequest( model="solaria-1", encoding="wav/pcm", sample_rate=16000, bit_depth=16, channels=1, language_config=LiveV2LanguageConfig(languages=["fr"], code_switching=False), messages_config=LiveV2MessagesConfig(receive_partial_transcripts=True), ) live_session = live_client.start_session(init_request) ``` * **Security**: Generate the WebSocket URL on your backend and keep your API key private. The init call returns a connectable URL and a session `id` that you can safely pass to web, iOS, or Android clients without exposing credentials in the app. * **Lower infrastructure load**: The secure URL is generated on your backend, the client can connect directly to Gladia's WebSocket server without a pass-through on your side, saving your own resources. * **Resilient reconnection and session continuity**: If the WebSocket disconnects (which can happen on unreliable networks), the session created by the init call lets the client reconnect without losing context. Traditional flows that open a socket first typically force a brand‑new session on disconnect, dropping in‑progress state. ## Connect to the WebSocket Now that you've initiated the session and have the URL, you can connect to the WebSocket using your preferred language/framework. Here's an example in JavaScript: ```typescript JavaScript theme={"system"} liveSession.on("message", (message) => { // Handle messages from the API }); liveSession.on("started", (message) => { // Handle start session message }); liveSession.on("ended", (message) => { // Handle end session message }); liveSession.on("error", (message) => { // Handle error message }); ``` ```python Python theme={"system"} from gladiaio_sdk import ( LiveV2WebSocketMessage, LiveV2InitResponse, LiveV2EndedMessage, ) @live_session.on("message") def on_message(message: LiveV2WebSocketMessage) -> None: # Handle messages from the API pass @live_session.on("error") def on_error(error: Exception) -> None: # Handle error message print(f"Live session error: {error}") @live_session.once("started") def on_started(_response: LiveV2InitResponse): # Handle start session print("Session started. Listening…") @live_session.once("ended") def on_ended(_ended: LiveV2EndedMessage): # Handle end session print("Session ended.") ``` ## Send audio chunks You can now start sending us your audio chunks through the WebSocket: ```typescript JavaScript theme={"system"} liveSession.sendAudio(audioChunk) ``` ```python Python theme={"system"} live_session.send_audio(audio_chunk) ``` A single realtime transcription session cannot exceed **3 hours**. For longer events, start a new session before reaching the limit. See [Concurrency and rate limits](/chapters/limits-and-specifications/concurrency) and [Supported files & duration](/chapters/limits-and-specifications/supported-formats) for details. ## Read messages During the whole session, we will send various messages through the WebSocket, the callback URL or webhooks. You can specify which kind of messages you want to receive in the initial configuration. See [`messages_config`](/api-reference/v2/live/init) for WebSocket messages and [`callback_config`](/api-reference/v2/live/init) for callback messages. Here's an example of how to read a [`transcript`](/api-reference/v2/live/message/transcript) message received through a WebSocket: ```javascript JavaScript theme={"system"} liveSession.on("message", (message) => { if (message.type === 'transcript' && message.data.is_final) { console.log(`${message.data.id}: ${message.data.utterance.text}`) }); ``` ```python Python theme={"system"} @live_session.on("message") def on_message(message: LiveV2WebSocketMessage) -> None: if getattr(message, "type", None) == "transcript": data = getattr(message, "data", None) if not data: return is_final = bool(getattr(data, "is_final", False)) utterance = getattr(data, "utterance", None) text = getattr(utterance, "text", "") if utterance else "" if is_final and text: print(text.strip()) ``` **Need low-latency partial results?** Enable [partial transcripts](/chapters/live-stt/features/partial-transcripts) by setting `messages_config.receive_partial_transcripts: true`. Use the `is_final` property to distinguish between partial and final transcript messages. ## Stop the recording Once you're done, send us the `stop_recording` message. We will process remaining audio chunks and start the post-processing phase, in which we put together the final audio file and results with the add-ons you requested. You'll receive a message at every step of the process in the WebSocket, or in the callback if configured. Once the post-processing is done, the WebSocket is closed with a code 1000. ```javascript JavaScript theme={"system"} liveSession.stopRecording() ``` ```python Python theme={"system"} live_session.stop_recording() ``` ## Get the final results If you want to get the complete result, you can call the [`GET /v2/live/:id` endpoint](/api-reference/v2/live/get) with the `id` you received from the initial request. ```javascript JavaScript theme={"system"} const response = await fetch(`https://api.gladia.io/v2/live/${sessionId}`, { method: 'GET', headers: { 'x-gladia-key': '', }, }); if (!response.ok) { // Look at the error message // It might be a configuration issue console.error(`${response.status}: ${(await response.text()) || response.statusText}`) return; } const result = await response.json(); console.log(result) ``` ```python Python theme={"system"} import os import requests session_id = "" api_key = os.environ.get("GLADIA_API_KEY") or "" response = requests.get( f"https://api.gladia.io/v2/live/{session_id}", headers={"x-gladia-key": api_key}, ) if not response.ok: print(f"{response.status_code}: {response.text or response.reason}") else: print(response.json()) ``` ```bash cURL theme={"system"} curl --request GET \ --url https://api.gladia.io/v2/live/ID_OF_THE_SESSION \ --header 'x-gladia-key: ' ``` ## Initiate your real-time session First, call the [ endpoint](/api-reference/v2/live/init) and pass your configuration. It's important to correctly define the properties `encoding`, `sample_rate`, `bit_depth` and `channels` as we need them to parse your audio chunks. ```javascript JavaScript theme={"system"} const response = await fetch("https://api.gladia.io/v2/live", { method: "POST", headers: { "Content-Type": "application/json", "x-gladia-key": "", }, body: JSON.stringify({ encoding: "wav/pcm", sample_rate: 16000, bit_depth: 16, channels: 1, }), }); if (!response.ok) { // Look at the error message // It might be a configuration issue console.error( `${response.status}: ${(await response.text()) || response.statusText}` ); process.exit(response.status); } const { id, url } = await response.json(); ``` ```bash cURL theme={"system"} curl --request POST \ --url https://api.gladia.io/v2/live \ --header 'Content-Type: application/json' \ --header 'x-gladia-key: YOUR_GLADIA_API_KEY' \ --data '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1 } ' ``` You'll receive a response with a WebSocket URL to connect to. If you loose connection, you can reconnect to that same URL and resume where you left off. Here's an example of a response: * **Security**: Generate the WebSocket URL on your backend and keep your API key private. The init call returns a connectable URL and a session `id` that you can safely pass to web, iOS, or Android clients without exposing credentials in the app. * **Lower infrastructure load**: The secure URL is generated on your backend, the client can connect directly to Gladia's WebSocket server without a pass-through on your side, saving your own resources. * **Resilient reconnection and session continuity**: If the WebSocket disconnects (which can happen on unreliable networks), the session created by the init call lets the client reconnect without losing context. Traditional flows that open a socket first typically force a brand‑new session on disconnect, dropping in‑progress state. ```json theme={"system"} { "id": "636c70f6-92c1-4026-a8b6-0dfe3ecf826f", "url": "wss://api.gladia.io/v2/live?token=636c70f6-92c1-4026-a8b6-0dfe3ecf826f" } ``` ## Connect to the WebSocket Now that you've initiated the session and have the URL, you can connect to the WebSocket using your preferred language/framework. Here's an example in JavaScript: ```javascript JavaScript theme={"system"} import WebSocket from "ws"; const socket = new WebSocket(url); socket.addEventListener("open", function () { // Connection is opened. You can start sending audio chunks. }); socket.addEventListener("error", function (error) { // An error occurred during the connection. // Check the error to understand why }); socket.addEventListener("close", function ({ code, reason }) { // The connection has been closed // If the "code" is equal to 1000, it means we closed intentionally the connection (after the end of the session for example). // Otherwise, you can reconnect to the same url. }); socket.addEventListener("message", function (event) { // All the messages we are sending are in JSON format const message = JSON.parse(event.data.toString()); console.log(message); }); ``` ## Send audio chunks You can now start sending us your audio chunks through the WebSocket. You can send them directly as binary, or in JSON by encoding your chunk in base64, like this: ```javascript JavaScript theme={"system"} // as binary socket.send(buffer); // as json socket.send( JSON.stringify({ type: "audio_chunk", data: { chunk: buffer.toString("base64"), }, }) ); ``` A single realtime transcription session cannot exceed **3 hours**. For longer events, start a new session before reaching the limit. See [Concurrency and rate limits](/chapters/limits-and-specifications/concurrency) and [Supported files & duration](/chapters/limits-and-specifications/supported-formats) for details. ## Read messages During the whole session, we will send various messages through the WebSocket, the callback URL or webhooks. You can specify which kind of messages you want to receive in the initial configuration. See [`messages_config`](/api-reference/v2/live/init) for WebSocket messages and [`callback_config`](/api-reference/v2/live/init) for callback messages. Here's an example of how to read a [`transcript`](/api-reference/v2/live/message/transcript) message received through a WebSocket: ```javascript JavaScript theme={"system"} socket.addEventListener("message", function(event) { // All the messages we are sending are in JSON format const message = JSON.parse(event.data.toString()); if (message.type === 'transcript' && message.data.is_final) { console.log(`${message.data.id}: ${message.data.utterance.text}`) } }); ``` **Need low-latency partial results?** Enable [partial transcripts](/chapters/live-stt/features/partial-transcripts) by setting `messages_config.receive_partial_transcripts: true`. Use the `is_final` property to distinguish between partial and final transcript messages. ## Sending multiple audio tracks in real-time If you have multiple audio sources (like different participants in a conversation) that you need to transcribe simultaneously, you can merge these separate audio tracks into a single multi-channel audio stream and send it over one WebSocket connection. ### Merging multiple audio tracks into one multi-channel WebSocket This approach allows you to consolidate multiple audio tracks from different participants into a single WebSocket connection while maintaining the ability to identify each speaker through their dedicated channel. Benefits: * Reduce the number of WebSocket connections from multiple to just one * Maintain speaker identity through channel mapping * Simplify synchronization of audio streams from multiple participants * Reduce network overhead and connection management complexity #### Creating a multi-channel audio stream To combine multiple audio tracks into a single multi-channel stream, you need to interleave the audio samples. Here's a TypeScript function that merges multiple audio buffers into a single multi-channel buffer: ```typescript TypeScript theme={"system"} export function interleaveAudio(channelsData: Buffer[], bitDepth = 16): Buffer { const nbChannels = channelsData.length; if (nbChannels === 1) { return channelsData[0]; } const bytesPerSample = bitDepth / 8; const samplesPerChannel = channelsData[0].byteLength / bytesPerSample; const audio = Buffer.alloc(nbChannels * samplesPerChannel * bytesPerSample); for (let i = 0; i < samplesPerChannel; i++) { for (let j = 0; j < nbChannels; j++) { const sample = channelsData[j].subarray( i * bytesPerSample, (i + 1) * bytesPerSample ); audio.set(sample, (i * nbChannels + j) * bytesPerSample); } } return audio; } ``` #### Example use case Consider a scenario with three participants in a room: Sami, Maxime, and Mark. Instead of opening three separate WebSocket connections (one for each participant), you can merge their audio tracks and send them over a single WebSocket: 1. Collect audio buffers from each participant 2. Merge them into a single multi-channel audio stream using the `interleaveAudio` function 3. Specify the number of channels in your API configuration (3 in this case) 4. Send the combined audio over a single WebSocket ```typescript TypeScript theme={"system"} // Collect audio buffers from each participant const samiAudio = getSamiAudioBuffer(); const maximeAudio = getMaximeAudioBuffer(); const markAudio = getMarkAudioBuffer(); // Merge into a multi-channel audio // Channel ordering: [0]=Sami, [1]=Maxime, [2]=Mark const channelsData = [samiAudio, maximeAudio, markAudio]; const mergedAudio = interleaveAudio(channelsData, 16); // 16-bit depth // Initialize a single WebSocket session with multi-channel config const response = await fetch("https://api.gladia.io/v2/live", { method: "POST", headers: { "Content-Type": "application/json", "x-gladia-key": "", }, body: JSON.stringify({ encoding: "wav/pcm", sample_rate: 16000, bit_depth: 16, channels: 3, // Specify the number of channels }), }); const { url } = await response.json(); const socket = new WebSocket(url); // Send the merged audio over a single WebSocket socket.addEventListener("open", function () { socket.send(mergedAudio); }); ``` #### Understanding the response When you send a multi-channel audio stream to Gladia, the channel order is preserved in the transcription results. Each transcription message will include a `channel` field that indicates which audio channel (and thus which participant) the transcription belongs to: ```json theme={"system"} { "type": "transcript", "session_id": "de70f43f-3041-46e0-892c-8e7f53800a22", "created_at": "2025-04-09T08:44:16.471Z", "data": { "id": "00_00000000", "utterance": { "text": "Hello, I'm Sami. I'm the first speaker", "start": 0.188, "end": 2.852, "language": "en", "channel": 0 // This indicates the first channel (Sami) } } } ``` ```json theme={"system"} { "type": "transcript", "session_id": "de70f43f-3041-46e0-892c-8e7f53800a22", "created_at": "2025-04-09T08:44:19.693Z", "data": { "id": "01_00000000", "utterance": { "text": "And this is Maxime, nice to meet you, I am the second speaker.", "start": 3.468, "end": 6.132, "language": "en", "channel": 1 // This indicates the second channel (Maxime) } } } ``` ```json theme={"system"} { "type": "transcript", "session_id": "a587386c-8755-4c67-ad67-d2c304eb8a49", "created_at": "2025-04-09T08:56:16.370Z", "data": { "id": "00_00000002", "utterance": { "text": "And this is Mark", "start": 8.614, "end": 10.574, "language": "en", "channel": 2 // This indicates the third channel (Mark) } } } ``` The channel numbers directly correspond to the order in which you added the audio tracks to the `channelsData` array: * Channel 0 → Sami (first in the array) * Channel 1 → Maxime (second in the array) * Channel 2 → Mark (third in the array) Remember to keep track of channel assignments in your application to properly attribute transcriptions to the correct participants. As mentioned in the [Multiple channels](/chapters/limits-and-specifications/multiple-channels) section, transcribing a multi-channel audio stream will be billed based on the total duration multiplied by the number of channels. ## Read messages During the whole session, we will send various messages through the WebSocket, the callback URL or webhooks. You can specify which kind of messages you want to receive in the initial configuration. See for WebSocket messages and for callback messages. Here's an example of how to read a message received through a WebSocket: ```javascript JavaScript theme={"system"} socket.addEventListener("message", function (event) { // All the messages we are sending are in JSON format const message = JSON.parse(event.data.toString()); if (message.type === "transcript" && message.data.is_final) { console.log(`${message.data.id}: ${message.data.utterance.text}`); } }); ``` ## Stop the recording Once you're done, send us the `stop_recording` message. We will process remaining audio chunks and start the post-processing phase, in which we put together the final audio file and results with the add-ons you requested. You'll receive a message at every step of the process in the WebSocket, or in the callback if configured. Once the post-processing is done, the WebSocket is closed with a code 1000. ```javascript JavaScript theme={"system"} socket.send( JSON.stringify({ type: "stop_recording", }) ); ``` Instead of sending the `stop_recording` message, you can also close the WebSocket with the code 1000. We will still do the post-processing in background and send you the messages through the callback you defined. ```javascript JavaScript theme={"system"} socket.close(1000); ``` ## Get the final results If you want to get the complete result, you can call the [`GET /v2/live/:id` endpoint](/api-reference/v2/live/get) with the `id` you received from the initial request. ```javascript JavaScript theme={"system"} const response = await fetch(`https://api.gladia.io/v2/live/${id}`, { method: "GET", headers: { "x-gladia-key": "", }, }); if (!response.ok) { // Look at the error message // It might be a configuration issue console.error( `${response.status}: ${(await response.text()) || response.statusText}` ); return; } const result = await response.json(); console.log(result); ``` ```bash cURL theme={"system"} curl --request GET \ --url https://api.gladia.io/v2/live/ID_OF_THE_SESSION \ --header 'x-gladia-key: YOUR_GLADIA_API_KEY' ``` Want to know more about a specific feature? Check out our [Features chapter](/chapters/live-stt/features) for more details. ## Full code sample You can find complete code samples in our Github repository: # Recommended Parameters by Use Case Source: https://docs.gladia.io/chapters/live-stt/recommended-parameters Best parameter configurations for realtime transcription depending on your application (Voice Agents, Meeting Recorders, Call Centers, Subtitles). The right parameter configuration can make a significant difference in transcription quality and latency for realtime use cases. This guide covers recommended starting points for common scenarios and highlights pitfalls that frequently trip up new integrations. These recommendations apply to the **[Realtime API](/chapters/live-stt/quickstart)** and are passed during session initialization. They are starting points — tune them to match your specific needs. *** ## Language Configuration One of the most common configuration mistakes is misunderstanding how `language_config` works. Choosing the right setup avoids unnecessary detection overhead and improves accuracy. **When to set an explicit language:** * You **know** the language of the audio ahead of time. * The audio is **monolingual** (single language throughout). * You want the **fastest, most accurate** results. ```json theme={"system"} { "language_config": { "languages": ["en"], "code_switching": false } } ``` **When to use auto-detection:** * You process audio in **many different languages** and don't know which one beforehand. * You want Gladia to pick the language automatically. ```json theme={"system"} { "language_config": { "languages": [], "code_switching": false } } ``` When `code_switching` is `false` and no language is set, the language is detected on the **first utterance** and reused for the rest of the session or file. If the beginning of your audio contains silence, music, or a different language than the main content, this can lead to incorrect detection for the whole transcription. Even when using auto-detection, pass a **small list of likely languages** in `languages` to constrain the search. This improves both accuracy and processing time. *** ## Code Switching Code switching (`language_config.code_switching: true`) lets Gladia detect and transcribe **multiple languages** within the same audio, re-evaluating the language on each utterance. **When to enable it:** * Speakers **switch languages** mid-conversation (e.g. bilingual meetings, multilingual customer support). * You need the detected `language` returned **per utterance**. **When NOT to enable it:** * The audio is in a **single language** — code switching adds unnecessary processing and can introduce misdetections. * You've set **exactly one language** in `languages` — in that case `code_switching` is ignored anyway. ```json theme={"system"} { "language_config": { "languages": ["en", "fr", "es"], "code_switching": true } } ``` **Do not enable `code_switching` with an empty `languages` list.** When no languages are specified, the language detector evaluates every utterance against 100+ supported languages, which leads to frequent misdetections — especially between similar-sounding languages. Always provide a short list of languages you **actually expect** in the audio. *** ## Custom Vocabulary [Custom vocabulary](/chapters/audio-intelligence/custom-vocabulary) is a post-transcription replacement based on **phoneme similarity**. It's essential for domain-specific terms that speech models frequently mis-transcribe. **Best practices:** * **Always provide both** the `custom_vocabulary` flag and a `custom_vocabulary_config`. * **Add pronunciations** to provide all the close spelling variants. You can use Automatic Phonemic Transcriber (IPA) in order to check if all the different spellings are covered. * **Keep `intensity` moderate** (0.4-0.6). High values increase false positives where unrelated words get replaced. * **Set `language`** on individual vocabulary entries when your audio is multilingual and a term is pronounced differently depending on the language. ```json Pre-recorded theme={"system"} { "audio_url": "YOUR_AUDIO_URL", "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": [ "Kubernetes", { "value": "Gladia", "pronunciations": ["Glad", "Gladio"], "intensity": 0.5 }, { "value": "PostgreSQL", "pronunciations": ["Postgres Q L", "Post gress"], "intensity": 0.4 } ], "default_intensity": 0.5 } } ``` ```json Live theme={"system"} { "realtime_processing": { "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": [ "Kubernetes", { "value": "Gladia", "pronunciations": ["Glad", "Gladio"], "intensity": 0.5 }, { "value": "PostgreSQL", "pronunciations": ["Postgres Q L", "Post gress"], "intensity": 0.4 } ], "default_intensity": 0.5 } } } ``` *** ## Voice Agents For callbots, customer-service assistants, or voice-driven chatbots the top priority is **low latency**. The agent must react quickly to user speech, even if sentence boundaries are not perfectly formed. | Parameter | Recommended value | Why | | --------------------------------------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `endpointing` | `0.05` - `0.1` | Closes utterances fast, keeping turn-taking snappy. See [Endpointing](/chapters/live-stt/features/endpointing). | | `maximum_duration_without_endpointing` | `15` | Prevents very long utterances from staying open without cutting off the conversation. | | `messages_config.receive_partial_transcripts` | `true` | Enables interim results so the agent can start processing early. Use the `speech_stop` event to know when the user has finished speaking. See [Partial transcripts](/chapters/live-stt/features/partial-transcripts). | | `realtime_processing.custom_vocabulary` | `true` | Add product names and action keywords so the agent can react accurately. | This setup is optimized for **fast turn-taking**. If utterances get cut off mid-sentence, raise `endpointing` slightly. *** ## Meeting Recorders For apps that record and transcribe meetings in real time — team stand-ups, board sessions, 1-on-1s — the goal is to produce a **structured, speaker-attributed live transcript** that can feed downstream features like summarization or live note-taking. | Parameter | Recommended value | Why | | --------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `endpointing` | `0.3` - `0.5` | Lets speakers finish their sentences before closing an utterance. See [Endpointing](/chapters/live-stt/features/endpointing). | | `maximum_duration_without_endpointing` | `15` | Prevents very long utterances in case a speaker doesn't pause. | | `messages_config.receive_partial_transcripts` | `true` | Feeds live captions to the UI while waiting for final results. See [Partial transcripts](/chapters/live-stt/features/partial-transcripts). | | `language_config.languages` | Set explicitly | Meeting language is almost always known in advance — setting it avoids detection overhead. | | `realtime_processing.custom_vocabulary` | `true` | Add company-specific terms, project names, and participant names for better accuracy. | **Diarization vs. multi-channel:** if each speaker is on a **separate audio channel** (e.g. a, use the `channel` field on each utterance to identify who is speaking — diarization is not needed. See [Multiple channels](/chapters/limits-and-specifications/multiple-channels) If all speakers share a **single audio channel**, enable `diarization` to separate the speakers. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). *** ## Call Centers For live phone calls the priorities are **speaker identification** and **fast, accurate transcription** despite variable audio quality (telephony codecs, background noise, cross-talk). | Parameter | Recommended value | Why | | --------------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------------------------------- | | `endpointing` | `0.2` - `0.4` | Keeps turn-taking responsive without cutting off mid-sentence. See [Endpointing](/chapters/live-stt/features/endpointing). | | `maximum_duration_without_endpointing` | `15` | Prevents very long utterances in monologue-style segments. | | `language_config.languages` | Set explicitly (e.g. `["en"]`) | Call center audio typically has a known language. Setting it avoids detection errors on noisy recordings. | | `realtime_processing.custom_vocabulary` | `true` | Add product names, plan names, and internal terminology. | **Diarization vs. multi-channel:** if each speaker is on a **separate audio channel** (e.g. a, use the `channel` field on each utterance to identify who is speaking — diarization is not needed. See [Multiple channels](/chapters/limits-and-specifications/multiple-channels) If all speakers share a **single audio channel**, enable `diarization` to separate the speakers. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). For calls with more than two participants (e.g. conference bridges), use `diarization_config.min_speakers` / `max_speakers` instead of `number_of_speakers` to give the model a flexible range. *** ## Subtitles & Captioning When providing live subtitles, the goal is to **sync text with the speaker in real time**. The right balance between speed and segment quality depends on whether captions are displayed live or post-produced. | Parameter | Recommended value | Why | | --------------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------- | | `endpointing` | `0.3` (live) / `0.8` (post-production) | Lower values keep captions close to the speaker; higher values produce cleaner subtitle blocks. | | `maximum_duration_without_endpointing` | `5` | Prevents excessively long subtitle segments that are hard to read on screen. | | `messages_config.receive_partial_transcripts` | `true` | Shows words as they are spoken, then refines them when the final result arrives. | | `language_config.languages` | Set explicitly | Avoids detection lag when the broadcast language is known. | For post-production subtitles generated from a recording, consider using the [Pre-recorded API](/chapters/pre-recorded-stt/quickstart) with the dedicated [subtitles feature](/chapters/audio-intelligence/subtitles) instead — it produces SRT/VTT files with fine-grained timing controls. # Quick migration guide: realtime STT from AssemblyAI to Gladia Source: https://docs.gladia.io/chapters/migrations/from-assembly Guide to seamlessly switch your realtime WebSocket transcription from AssemblyAI's SDK to Gladia's SDK. This step-by-step guide shows how to switch your realtime transcription from AssemblyAI to Gladia with minimal code changes. It highlights equivalences, subtle differences, and drop‑in replacements so you can migrate quickly and confidently, without any regressions. ## Step-by-step guide ### Install the SDK Install the official SDKs to enable realtime streaming. That's all you need to get started in Python or TypeScript. *For AssemblyAI :* ```bash Python theme={"system"} pip install assemblyai ``` ```bash Typescript theme={"system"} npm i assemblyai ``` *For Gladia:* ```bash Python theme={"system"} pip install gladiaio-sdk ``` ```bash Typescript theme={"system"} npm i @gladiaio/sdk ``` ### Initialize the client Create and authenticate the client that manages your live connection. The shape is the same idea across providers—just swap the client and key. *For AssemblyAI :* ```python Python theme={"system"} from assemblyai.streaming.v3 import StreamingClient, StreamingClientOptions api_key = "" assemblyClient = StreamingClient( StreamingClientOptions( api_key=api_key, api_host="streaming.assemblyai.com", ) ) ``` ```typescript Typescript theme={"system"} import { AssemblyAI } from "assemblyai"; const assemblyClient = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY!, }); ``` *For Gladia :* ```python Python theme={"system"} from gladiaio_sdk import GladiaClient gladia_client = GladiaClient(api_key="") ``` ```typescript Typescript theme={"system"} import { GladiaClient } from "@gladiaio/sdk"; const gladiaClient = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, }); ``` ### Configure the session Choose the model, audio format, and language options your app needs. Most parameters map one‑to‑one, so your existing settings carry over naturally. #### AssemblyAI to Gladia parameter mapping | AssemblyAI | Gladia | Notes / Example | | :---------------: | :---------------------------------------------------: | --------------------------------------------------------------------------------------------------------------------- | | `model` | `model` | Choose the latest Gladia model ("solaria-1"). | | `encoding` | `encoding` | Match the actual [audio format](/api-reference/v2/live/init#body-encoding) (e.g., `linear16` ↔ `wav/pcm`). | | | `bit_depth` | Choose the [bit depth](/api-reference/v2/live/init#body-bit-depth) value from your audio | | `sample_rate` | `sample_rate` | Same unit (Hz). | | `channels` | `channels` | Same meaning. | | `interim_results` | `messages_config.receive_partial_transcripts` | Set `true` to receive [partials messages](/chapters/live-stt/features/partial-transcripts). | | `endpointing` | `endpointing`; `maximum_duration_without_endpointing` | Port thresholds and consider a hard cap. | | `language` | `language_config.languages` (+ `code_switching`) | Pass one or more languages; enable switching when [multiple languages](/chapters/language/code-switching) are spoken. | ```json Gladia config example theme={"system"} { "model": "solaria-1", "encoding": "wav/pcm", "bit_depth": 16, "sample_rate": 16000, "channels": 1, "language_config": { "languages": ["en"], "code_switching": false }, "messages_config": { "receive_partial_transcripts": true, "receive_final_transcripts": true }, "endpointing": 0.8, "maximum_duration_without_endpointing": 30, "realtime_processing": { "custom_vocabulary": false, "custom_spelling": false } } ``` See the full schema in the [live init reference](/api-reference/v2/live/init). ### Start the transcription session Open a live transcription session using your configuration. The flow is the same as with AssemblyAI: establish the WebSocket and get ready to stream audio. *For AssemblyAI :* ```python Python theme={"system"} from assemblyai.streaming.v3 import StreamingParameters assemblyClient.connect( StreamingParameters( sample_rate=16000, format_turns=True, ) ) ``` ```typescript Typescript theme={"system"} const assemblySession = assemblyClient.streaming.transcriber({ sampleRate: 16_000, formatTurns: true, }); await assemblySession.connect(); ``` *For Gladia :* ```python Python theme={"system"} gladia_session = gladia_client.live_v2().start_session(gladia_config) ``` ```typescript Typescript theme={"system"} const gladiaSession = gladiaClient.liveV2().startSession(gladiaConfig); ``` ### Send audio chunks Stream audio frames to the session as they are produced. Both SDKs accept small chunks continuously—keep your existing chunking logic. *For AssemblyAI :* ```python Python theme={"system"} assemblyClient.stream(audio_chunk) ``` ```typescript Typescript theme={"system"} assemblySession.stream(audioChunk); ``` *For Gladia :* ```python Python theme={"system"} gladia_session.send_audio(audio_chunk) ``` ```typescript Typescript theme={"system"} gladiaSession.sendAudio(audioChunk); ``` ### Read transcription messages After audio is flowing, subscribe to transcript and lifecycle events. The mapping below shows how to translate AssemblyAI listeners to Gladia in a single place. Event mapping from AssemblyAI to Gladia: * `Transcript` → listen to Gladia `message` and branch on `message.data.is_final` to separate partial vs final results. * `Open`/`Close`/`Error` → map to Gladia `started`/`ended`/`error`. In practice, subscribe once to `message` and use the `is_final` flag instead of wiring separate listeners—less boilerplate, same control. To receive partials, enable `messages_config.receive_partial_transcripts: true` in your init config. *For AssemblyAI :* ```python Python theme={"system"} from typing import Type from assemblyai.streaming.v3 import ( BeginEvent, StreamingClient, StreamingError, StreamingEvents, StreamingSessionParameters, TerminationEvent, TurnEvent, ) def on_begin(self: Type[StreamingClient], event: BeginEvent): print(f"Session started: {event.id}") def on_turn(self: Type[StreamingClient], event: TurnEvent): print(f"{event.transcript} ({event.end_of_turn})") if event.end_of_turn and not event.turn_is_formatted: params = StreamingSessionParameters( format_turns=True, ) self.set_params(params) def on_terminated(self: Type[StreamingClient], event: TerminationEvent): print( f"Session terminated: {event.audio_duration_seconds} seconds of audio processed" ) def on_error(self: Type[StreamingClient], error: StreamingError): print(f"Error occurred: {error}") client.on(StreamingEvents.Begin, on_begin) client.on(StreamingEvents.Turn, on_turn) client.on(StreamingEvents.Termination, on_terminated) client.on(StreamingEvents.Error, on_error) ``` ```typescript Typescript theme={"system"} assemblySession.on("open", ({ id }) => { console.log(`Session opened with ID: ${id}`); }); assemblySession.on("error", (error) => { console.error("Error:", error); }); assemblySession.on("close", (code, reason) => console.log("Session closed:", code, reason), ); assemblySession.on("turn", (turn) => { if (!turn.transcript) { return; } console.log("Turn:", turn.transcript); }); ``` *For Gladia :* ```python Python theme={"system"} from gladiaio_sdk import ( LiveV2WebSocketMessage, LiveV2InitResponse, LiveV2EndedMessage, ) @live_session.on("message") def on_message(message: LiveV2WebSocketMessage): # Partial and final transcripts are delivered here # filter them with message.data.is_final field print(message) @live_session.once("started") def on_started(_response: LiveV2InitResponse): print("Session started. Listening...") @live_session.once("ended") def on_ended(_ended: LiveV2EndedMessage): print("Session ended.") @live_session.on("error") def on_error(error: Exception): print(f"Error: {error}") ``` ```typescript Typescript theme={"system"} gladiaSession.on("message", (message) => { // Partial and final transcripts are delivered here // filter them with message.data.is_final field console.log(message); }); gladiaSession.on("started", (info) => { console.log("Start session", info); }); gladiaSession.on("ended", (info) => { console.log("End session", info); }); gladiaSession.on("error", (err) => { console.error("Error", err); }); ``` # Quick migration guide: realtime STT from Deepgram to Gladia Source: https://docs.gladia.io/chapters/migrations/from-deepgram Guide to seamlessly switch your realtime WebSocket transcription from Deepgram's SDK to Gladia's SDK. This step-by-step guide shows how to switch your realtime transcription from Deepgram to Gladia with minimal code changes. It highlights equivalences, subtle differences, and drop‑in replacements so you can migrate quickly and confidently, without any regressions. ## Step-by-step guide ### Install the SDK Install the official SDKs to enable realtime streaming. That's all you need to get started in Python or TypeScript. *For Deepgram :* ```bash Python theme={"system"} pip install deepgram-sdk ``` ```bash Typescript theme={"system"} npm i @deepgram/sdk ``` *For Gladia:* ```bash Python theme={"system"} pip install gladiaio-sdk ``` ```bash Typescript theme={"system"} npm i @gladiaio/sdk ``` ### Initialize the client Create and authenticate the client that manages your live connection. The shape is the same idea across providers—just swap the client and key. *For Deepgram :* ```python Python theme={"system"} from deepgram import DeepgramClient deepgram_client = DeepgramClient(api_key="") ``` ```typescript Typescript theme={"system"} import { createClient } from "@deepgram/sdk"; const deepgramClient = createClient(process.env.DEEPGRAM_API_KEY); ``` *For Gladia :* ```python Python theme={"system"} from gladiaio_sdk import GladiaClient gladia_client = GladiaClient(api_key="") ``` ```typescript Typescript theme={"system"} import { GladiaClient } from "@gladiaio/sdk"; const gladiaClient = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, }); ``` ### Configure the session Choose the model, audio format, and language options your app needs. Most parameters map one‑to‑one, so your existing settings carry over naturally. #### Deepgram to Gladia parameter mapping | Deepgram | Gladia | Notes / Example | | :---------------: | :---------------------------------------------------: | --------------------------------------------------------------------------------------------------------------------- | | `model` | `model` | Choose the latest Gladia model ("solaria-1"). | | `encoding` | `encoding` | Match the actual [audio format](/api-reference/v2/live/init#body-encoding) (e.g., `linear16` ↔ `wav/pcm`). | | | `bit_depth` | Choose the [bit depth](/api-reference/v2/live/init#body-bit-depth) value from your audio | | `sample_rate` | `sample_rate` | Same unit (Hz). | | `channels` | `channels` | Same meaning. | | `interim_results` | `messages_config.receive_partial_transcripts` | Set `true` to receive [partials messages](/chapters/live-stt/features/partial-transcripts). | | `endpointing` | `endpointing`; `maximum_duration_without_endpointing` | Port thresholds and consider a hard cap. | | `language` | `language_config.languages` (+ `code_switching`) | Pass one or more languages; enable switching when [multiple languages](/chapters/language/code-switching) are spoken. | ```json Gladia config example theme={"system"} { "model": "solaria-1", "encoding": "wav/pcm", "bit_depth": 16, "sample_rate": 16000, "channels": 1, "language_config": { "languages": ["en"], "code_switching": false }, "messages_config": { "receive_partial_transcripts": true, "receive_final_transcripts": true }, "endpointing": 0.8, "maximum_duration_without_endpointing": 30, "realtime_processing": { "custom_vocabulary": false, "custom_spelling": false } } ``` See the full schema in the [live init reference](/api-reference/v2/live/init). ### Start the transcription session Open a live transcription session using your configuration. The flow is the same as with Deepgram: establish the WebSocket and get ready to stream audio. *For Deepgram :* ```python Python theme={"system"} with deepgram_client.listen.v1.connect(deepgram_config) as deepgram_connection: # Further code to add event handlers deepgram_connection.start_listening() ``` ```typescript Typescript theme={"system"} const deepgramConnection = deepgramClient.listen.live(deepgramConfig); ``` *For Gladia :* ```python Python theme={"system"} gladia_session = gladia_client.live_v2().start_session(gladia_config) ``` ```typescript Typescript theme={"system"} const gladiaSession = gladiaClient.liveV2().startSession(gladiaConfig); ``` ### Send audio chunks Stream audio frames to the session as they are produced. Both SDKs accept small chunks continuously—keep your existing chunking logic. *For Deepgram :* ```python Python theme={"system"} deepgram_connection.send(audio_chunk) ``` ```typescript Typescript theme={"system"} deepgramConnection.send(audioChunk); ``` *For Gladia :* ```python Python theme={"system"} gladia_session.send_audio(audio_chunk) ``` ```typescript Typescript theme={"system"} gladiaSession.sendAudio(audioChunk); ``` ### Read transcription messages After audio is flowing, subscribe to transcript and lifecycle events. The mapping below shows how to translate Deepgram listeners to Gladia in a single place. Event mapping from Deepgram to Gladia: * `Transcript` → listen to Gladia `message` and branch on `message.data.is_final` to separate partial vs final results. * `Open`/`Close`/`Error` → map to Gladia `started`/`ended`/`error`. In practice, subscribe once to `message` and use the `is_final` flag instead of wiring separate listeners—less boilerplate, same control. To receive partials, enable `messages_config.receive_partial_transcripts: true` in your init config. *For Deepgram :* ```python Python theme={"system"} from deepgram.core.events import EventType def on_open(_): print("Deepgram connection opened") def on_message(data, **kwargs): # Print the top alternative transcript if present print(data["channel"]["alternatives"][0]) def on_close(data, **kwargs): print("Deepgram connection closed") def on_error(error, **kwargs): print("Error:", error) deepgram_connection.on(EventType.OPEN, on_open) deepgram_connection.on(EventType.MESSAGE, on_message) deepgram_connection.on(EventType.CLOSE, on_close) deepgram_connection.on(EventType.ERROR, on_error) ``` ```typescript Typescript theme={"system"} deepgramConnection.on(LiveTranscriptionEvents.Open, () => { deepgramConnection.on(LiveTranscriptionEvents.Transcript, (data) => { console.log(data.channel.alternatives[0].transcript); }); deepgramConnection.on(LiveTranscriptionEvents.Error, (err) => { console.error(err); }); }); ``` *For Gladia :* ```python Python theme={"system"} from gladiaio_sdk import ( LiveV2WebSocketMessage, LiveV2InitResponse, LiveV2EndedMessage, ) @live_session.on("message") def on_message(message: LiveV2WebSocketMessage): # Partial and final transcripts are delivered here # filter them with message.data.is_final field print(message) @live_session.once("started") def on_started(_response: LiveV2InitResponse): print("Session started. Listening...") @live_session.once("ended") def on_ended(_ended: LiveV2EndedMessage): print("Session ended.") @live_session.on("error") def on_error(error: Exception): print(f"Error: {error}") ``` ```typescript Typescript theme={"system"} gladiaSession.on("message", (message) => { // Partial and final transcripts are delivered here // filter them with message.data.is_final field console.log(message); }); gladiaSession.on("started", (info) => { console.log("Start session", info); }); gladiaSession.on("ended", (info) => { console.log("End session", info); }); gladiaSession.on("error", (err) => { console.error("Error", err); }); ``` # Audio Intelligence Source: https://docs.gladia.io/chapters/pre-recorded-stt/audio-intelligence Turn speech into actionable insights with translation, summarization, entities, chapters, sentiment and more Audio intelligence turns raw speech into structured, useful data on top of transcription. Once the words are captured, these features help you understand, organize, and act on the content — from instant translation to key-point summaries, entity detection, chapter markers, emotions, and even custom LLM prompts. Detect speakers and understand who said what, and when. Translate transcripts and subtitles into multiple languages in one request. Automatically redact names, emails, vehicle IDs, and other PII in pre-recorded transcripts. Generate ready-to-use subtitle files in SRT or VTT formats. Boost recognition accuracy for specific terms. Generate concise summaries or bullet points for quick understanding. Detect and categorize key entities like people, organizations, dates, and more. Segment long audio into chapters with headlines and summaries for easy navigation. Understand the tone and emotions expressed across the transcript. Ask custom questions and run prompts on your audio like an assistant would. # Benchmarking Source: https://docs.gladia.io/chapters/pre-recorded-stt/benchmarking A practical guide to benchmarking speech-to-text accuracy — from defining goals to choosing datasets, normalizing transcripts, computing WER, and interpreting results. Benchmarking speech-to-text systems is easy to get wrong. Small methodology changes can produce large swings in reported quality, which makes comparisons misleading. ## Benchmarking at a glance Decide what "good" means for your product before comparing systems. Normalize both references and predictions before computing WER. Measure substitutions, deletions, and insertions on normalized text. Benchmark on audio that matches your real traffic and target users. Look beyond one average score and inspect meaningful slices. ## 0. Define your evaluation goal Before comparing providers and models, the first step is to define which aspects of performance matter most for your use case. Below are examples of performance aspects that would be more weighted for domain applications of speech to text: * Accuracy on noisy backgrounds: for contact centers, telephony, and field recordings. * Speaker diarization quality: for meeting assistants and multi-speaker calls. * Named entity accuracy: for workflows that extract people, organizations, phone numbers, or addresses. * Domain-specific vocabulary handling: for medical, legal, or financial transcription. * Timestamp accuracy: for media workflows that need readable, well-timed captions. * Filler-word handling: for agentic workflows . Those choices shape every downstream decision: which dataset to use, which normalization rules to apply, and which metrics to report. If your benchmark does not reflect your real traffic, the result will not tell you much about production performance. ## 1. Normalize transcripts before computing WER Normalization removes surface-form differences (casing, abbreviations, numeric rendering) so you compare apples to apples when judging transcription output. | Reference | Prediction | Why raw WER is wrong | | ----------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------ | | `It's $50` | `it is fifty dollars` | Contraction and currency formatting differ, but the semantic content is the same. | | `Meet at Point 14` | `meet at point fourteen` | The normalization should preserve the numbered entity instead of collapsing it into an unrelated form. | | `Mr. Smith joined at 3:00 PM` | `mister smith joined at 3 pm` | Honorific and timestamp formatting differ, but the transcript content is equivalent. | One common limitation is "Whisper-style normalization" (OpenAI, 2022): implemented in packages like [`whisper-normalizer`](https://pypi.org/project/whisper-normalizer/). It does not affect numbers, and applies aggressive lowercasing and punctuation stripping. Gladia's recommended approach is [`gladia-normalization`](https://github.com/gladiaio/normalization), our open-source library designed for transcript evaluation: * `It's $50` -> `it is 50 dollars` * `Meet at Point 14` -> `meet at point 14` * `Mr. Smith joined at 3:00 PM` -> `mister smith joined at 3 pm` Open-source transcript normalization library used before WER computation. ```python theme={"system"} from normalization import load_pipeline pipeline = load_pipeline("gladia-3", language="en") reference = "Meet at Point 14. It's $50 at 3:00 PM." prediction = "meet at point fourteen it is fifty dollars at 3 pm" normalized_reference = pipeline.normalize(reference) normalized_prediction = pipeline.normalize(prediction) ``` Always apply the same normalization pipeline to both the reference transcript and every hypothesis output you compare. Changing the normalization rules between references invalidates the results. ## 2. Compute WER correctly Word Error Rate measures the edit distance between a reference transcript and a predicted transcript at the word level. The standard formula is: ```text theme={"system"} WER = (S + D + I) / N ``` Where: * `S` = substitutions * `D` = deletions * `I` = insertions * `N` = number of words in the reference transcript Lower is better. In practice: 1. Prepare a reference transcript for each audio sample. 2. Run each provider on the exact same audio. 3. Normalize both the reference and each prediction with the same pipeline. 4. Compute WER on the normalized outputs. 5. Aggregate results across the full dataset. Do not compute WER on raw transcripts if providers format numbers, punctuation, abbreviations, or casing differently. That mostly measures formatting conventions, not recognition quality. Inspect your reference transcripts carefully before computing WER. If a reference contains text that is not actually present in the audio, for example an intro such as "this audio is a recording of...", it can make WER look much worse across all providers. ## 3. Choose a representative dataset Start from your [evaluation goal](#1-define-your-evaluation-goal): the right dataset depends on the use case and traffic shape you want to measure. A good benchmark dataset should look as close as possible to your real production audio. If the audio in the benchmark does not match what you actually process, the results will not tell you much. When choosing your dataset, make sure it matches your real audio on: * Language: the target language, accents, and whether speakers switch languages. * Audio quality: telephony, browser microphone, studio recordings, noisy field audio, overlapping speech, or compressed audio. * Topics: medical, operational, legal, financial, customer support, or any other domain you care about. * Important words: numbers, names, acronyms, product names, addresses, or domain-specific terminology. * Interaction style: single-speaker dictation, calls, meetings, interviews, or long-form recordings. Use transcripts that are strong enough to serve as ground truth. When possible, combine public datasets for comparability with private in-domain datasets that reflect your real traffic. Typical failure cases: * Benchmarking call-center audio with clean podcast recordings overestimates real-world performance. * Benchmarking English-only speech does not capture code-switching traffic. * Benchmarking short clips can hide failures that appear on long recordings with multiple speakers. Your favorite LLM with internet access can be very effective at finding public datasets that match your use case. For a broader methodology view, see [this benchmark guide](/chapters/pre-recorded-stt/benchmarking), especially the evaluation-goal section above when mapping use cases to dataset types. ## 4. Interpret results carefully Do not stop at a single WER number. Review: * overall average WER * median WER and spread across files * breakdowns by language, domain, or audio condition * failure modes on proper nouns, acronyms, and numbers * whether differences are consistent or concentrated in a few hard samples Two systems can post similar average WER while failing on different error classes. Separate statistically meaningful gaps from noise introduced by dataset composition or normalization choices. If two systems are close, inspect actual transcript examples before drawing strong conclusions. ## Common pitfalls * Comparing providers on different datasets * Using low-quality or inconsistent ground truth * Treating punctuation and formatting differences as recognition errors * Drawing conclusions from too few samples * Reporting one average score without any slice analysis * Not inspecting the reference transcript: if it contains text not present in the audio, for example an intro like "this audio is a recording of...", it will inflate WER across all providers * Not experimenting with provider configurations: for example, using Gladia's [custom vocabulary](/chapters/audio-intelligence/custom-vocabulary) to improve proper noun accuracy, then comparing against the ground truth # Sentences Source: https://docs.gladia.io/chapters/pre-recorded-stt/features/sentences Semantic sentence segmentation for more readable results In addition to getting the transcription split by utterances, you can request semantic sentence segmentation for more readable results. You can get translated sentences by enabling both `sentences` and `translation`! You'll receive sentences output for the original transcript, and each `translation` result will also contain sentences in the translated language. ```json request data theme={"system"} { "sentences": true } ``` The result will contain a `sentences` key (in addition to `utterances`): ```json theme={"system"} "sentences": { "success": true, "is_empty": false, "results": [ { "sentence": "Amy, it says you are trained in technology.", "start": 0.4681999999999999, "end": 2.45525, "words": [...], "confidence": 0.95, "language": "en", "speaker": 0, "channel": 0 } ] } ``` # Pre-recorded - Migration guide from V1 to V2 Source: https://docs.gladia.io/chapters/pre-recorded-stt/migration-from-v1 Migrate to the latest version of Gladia's pre-recorded speech-to-text API ## General flow changes In the first version of Gladia API, to get your transcription through an HTTP call, you had to send everything (audio file/url, parameters, etc) in a single HTTP call, and then keep the connection open until you get your result. This was not ideal for many scenarios that could lead to longer waiting time to get your results, or in case of connection errors, not getting your results at all despite the transcription being successful. In V2, we addressed this by decomposing the process in multiple steps, and have merged both audio & video endpoints: This step is **optional** if you are already working with **audio URLs**. If you're working with audio or video files, you'll need to upload it first using our `/upload` endpoint with `multipart/form-data` content-type since Gladia `/v2/pre-recorded` endpoint only accept audio URLs now. More information about this step [in the API Reference](/api-reference/v2/upload/audio-file) ```bash theme={"system"} curl --request POST \ --url https://api.gladia.io/v2/upload \ --header 'Content-Type: multipart/form-data' \ --header 'x-gladia-key: YOUR_GLADIA_API_TOKEN' \ --form audio=@/path/to/your/audio/conversation.wav ``` Example response : ```json theme={"system"} { "audio_url": "https://api.gladia.io/file/636c70f6-92c1-4026-a8b6-0dfe3ecf826f", "audio_metadata": { "id": "636c70f6-92c1-4026-a8b6-0dfe3ecf826f", "filename": "conversation.wav", "extension": "wav", "size": 99515383, "audio_duration": 4146.468542, "number_of_channels": 2 } } ``` We will now proceed to the next steps using the returned `audio_url`. We'll now make the transcription request to Gladia's API. Instead of `/audio/text/audio-transcription` now we'll use `/v2/pre-recorded` Since `/v2/pre-recorded` does not accept any `audio` file, `Content-Type` is not `multipart/form-data` anymore, but `application/json`. More information about this step [in the API Reference](/api-reference/v2/pre-recorded/init) ```bash V1 (deprecated) theme={"system"} curl --request POST \ --url https://api.gladia.io/audio/text/audio-transcription/ \ # Content-Type is multipart/form-data, v2 is application/json --header 'Content-Type: multipart/form-data' \ --header 'x-gladia-key: YOUR_GLADIA_API_TOKEN' \ --form audio_url=http://youraudiofileurl.com/conversation.wav \ # Diarization toggle & config --form toggle_diarization=true \ --form diarization_min_speakers=1 \ --form diarization_max_speakers=5 \ --form diarization_num_speakers=3 \ # Output format --form output_format=srt \ # Language behaviour --form 'language_behaviour=automatic single language' \ # Translation --form toggle_direct_translate=true \ --form target_translation_language=french ``` ```bash V2 theme={"system"} curl --request POST \ --url https://api.gladia.io/v2/pre-recorded \ # Content-type is now application/json --header 'Content-Type: application/json' \ --header 'x-gladia-key: YOUR_GLADIA_API_TOKEN' \ --data '{ "audio_url": "https://api.gladia.io/file/636c70f6-92c1-4026-a8b6-0dfe3ecf826f", "diarization": true, "diarization_config": { "number_of_speakers": 3, "min_speakers": 1, "max_speakers": 5 }, "translation": true, "translation_config": { "model": "base", "target_languages": ["fr", "en"], "match_original_utterances": true, "lipsync": true, "context_adaptation": true, "context": "Technical discussion about software development", "informal": false }, "subtitles": true, "subtitles_config": { "formats": ["srt", "vtt"] }, "detect_language": true, "enable_code_switching": false } ' ``` * **Old V1** : The HTTP connection is kept opened until you get your transcription result, and there's no third step. * **New V2** : You get an instant response from the request with an `id` and a `result_url`.\ The `id` is your transcription ID that you will use to get your transcription result once it's done. You don't have to keep any HTTP connection open on your side.\ `result_url` is returned for convenience. This is a pre-built url with your transcription id in it that you can use to get your result in the next step. As on V1 you get the transcription results in the previous step, this step is only relevant for V2. You can get your transcription results in **3 different ways**: ```javascript JavaScript theme={"system"} import { GladiaClient } from '@gladiaio/sdk'; const gladiaClient = new GladiaClient({ apiKey: 'YOUR_GLADIA_API_KEY', }); // Use job.id from createUntyped(...) const result = await gladiaClient.preRecorded().poll("YOUR_TRANSCRIPTION_JOB_ID"); console.log(result.result?.transcription?.full_transcript ?? ""); ``` ```python Python theme={"system"} from gladiaio_sdk import GladiaClient gladia_client = GladiaClient(api_key="YOUR_GLADIA_API_KEY").prerecorded() # Use job.id from gladia_client.create(...) result = gladia_client.poll("YOUR_TRANSCRIPTION_JOB_ID") print(result.result.transcription.full_transcript) ``` You can use `create_and_poll()` in Python or `createAndPollUntyped()` in JavaScript to **create** and **poll** in one call with the same job payload. The methods implemented in the sdk are automatically polling until success or errors. To get the result with the cURL, you'll just have to GET continuously on the given `result_url` until the status of your transcription is `done`. You can get more information on the different transcriptions status by checking directly the [API Reference](/api-reference/). You can configure webhooks at [https://app.gladia.io/webhooks](https://app.gladia.io/webhooks) to be notified when your transcriptions are done. Once a transcription is done, a `POST` request will be made to the endpoint you configured. The request body is a JSON object containing the transcription `id` that you can use to retrieve your result with [our API](/api-reference/v2/pre-recorded/get).\ For the full body definition, check [our API definition](/api-reference/v2/pre-recorded/webhook/success). Callback are HTTP calls that you can use to get notified when your transcripts are ready. Instead of polling and keeping your server busy and maintaining work, you can use the `callback` feature to receive the result to a specified endpoint: ```json theme={"system"} { "audio_url": "YOUR_AUDIO_URL", "callback": true, "callback_config": { "url": "https://yourserverurl.com/your/callback/endpoint/", "method": "POST" } } ``` Once the transcription is done, a request will be made to the url you provided in `callback_config.url` using the HTTP method you provided in `callback_config.method`. Allowed methods are `POST` and `PUT` with the default being `POST`. The request body is a JSON object containing the transcription `id` and an `event` property that tells you if it's a [success](/api-reference/v2/pre-recorded/callback/success) or an [error](/api-reference/v2/pre-recorded/callback/error). ## Transcription Input & Output changes In addition to the transcription flow changes, input & output also changed. To get the exhaustive documentation of the **V2 input/output**, please refer to the [API Reference](/api-reference/) part of the documentation. ### Input changes The most efficient way to get the new inputs list is to check the [API Reference](/api-reference/). But here's a quick recap table about the most used parameters changes : | V1 | V2 | | -------------------- | ------------------------------------------------------ | | `toggle_diarization` | `diarization` | | `language_behaviour` | `detect_language`, `enable_code_switching`, `language` | | `output_format` | `subtitles` + `subtitles_config` | | `webhook_url` | `callback_url` | ### Output changes Here is a general changelog for the output part of the transcription's core features: ```json V1 (deprecated) theme={"system"} { //"prediction" do not exist anymore, equivalent on V2 would be result.transcription.utterances "prediction": [ { "words": [ { "word": "Split", "time_begin": 0.21001999999999998, // v2 -> utterances[i].words[n].start "time_end": 0.69015, // v2 -> utterances[i].words[n].end "confidence": 1 }, { "word": " infinity", "time_begin": 0.91021, "time_end": 1.55038, "confidence": 0.95 }, // More words ... ], "language": "en", "transcription": "Split infinity in a time when less is more,", // v2 -> utterances[i].text "confidence": 0.84, "time_begin": 0.21001999999999998, // v2 -> utterances[i].start "time_end": 4.71123, // v2 -> utterances[i].end "speaker": 0, // Not present if diarization is not enabled "channel": "channel_0" // not a string anymore but integer (channel: 0) }, // More transcriptions... ], // "prediction_raw" do not exist anymore "prediction_raw": { "metadata": { // v2-> result.metadata // Most of those metadata are not present anymore "audio_conversion_time": 1.0006206035614014, "vad_time": 15.870725154876709, "audio_preprocessing_time": 0.8548638820648193, "language_discovery_time": 0.8840088844299316, "inference_time": 4.302580833435059, "translation_time": 0.0000059604644775390625, "formatting_time": 0.00007605552673339844, "total_transcription_time": 22.91288471221924, // -> result.metadata.transcription_time "provided_file_metadata": { "channels": 1, // -> result.metadata.number_of_channels "sample_rate": 44100, "duration": 20.555465, // -> result.metadata.audio_duration "nb_channels": 1, "sample_width": 1, "number_similar_channels": 0, "original_file_type": "audio" }, "nb_silent_channels": -1, "total_speech_duration": 0, "summarization_time": 0, "chapterization_time": 0 }, //"prediction_raw.transcription" do not exist anymore "transcription": [ { "words": [ { "word": "Split", "time_begin": 0.21001999999999998, "time_end": 0.69015, "confidence": 1 }, { "word": " infinity", "time_begin": 0.91021, "time_end": 1.55038, "confidence": 0.95 }, // More words ... ], "language": "en", "transcription": "Split infinity in a time when less is more,", "confidence": 0.84, "time_begin": 0.21001999999999998, "time_end": 4.71123, "speaker": 0, "channel": "channel_0" }, // More transcriptions... ] } } ``` ```json V2 theme={"system"} { "id": "0777a4fc-0b7e-40fb-9d37-690bdd83c9c6", "request_id": "G-0777a4fc", "status": "done", "created_at": "2024-01-03T11:11:03.872Z", "completed_at": "2024-01-03T11:11:14.962Z", "result": { "metadata": { "audio_duration": 20.555465, "number_of_channels": 1, "transcription_time": 11.09, "billing_time": 20.555465 }, "transcription": { "languages": [ "en" ], "full_transcript": "Split infinity in a time when less is more, where too much is never enough. There is always hope for the future. The future can be read from the past. The past foreshadows the present, and the present hasn't been written yet.", "utterances": [ { "text": "Split infinity in a time when less is more,", "language": "en", "start": 0.21001999999999998, "end": 4.71123, "channel": 0, "speaker": 0, "words": [ { "word": "Split", "start": 0.21001999999999998, "end": 0.69015, "confidence": 1 }, { "word": " infinity", "start": 0.91021, "end": 1.55038, "confidence": 0.95 }, // More words ... ] }, // More utterances of same shape... ]} } } ``` To dive deeper into the V2 version of the API, please take a look at those next: * [API Reference](/api-reference/) * [Speech Recognition](/chapters/pre-recorded-stt/features) * [Translation model](/chapters/audio-intelligence/translation) * [Summarization model](/chapters/audio-intelligence/summarization) # Quickstart Source: https://docs.gladia.io/chapters/pre-recorded-stt/quickstart How to transcribe pre-recorded audio with Gladia's speech-to-text (STT) API The SDK simplifies pre-recorded speech-to-text by abstracting upload, job creation, and result retrieval. Designed for developers, it offers: * A `transcribe()` for an end-to-end flow * Individual steps when you need control over each step. Pass `model` to choose the transcription model: **`"solaria-3"`** — our latest model: highest accuracy on European real-world audio. * **Async (pre-recorded) only** — not available for live transcription. * **Languages:** English, French, German, Spanish, Italian * **Single language only** — pass exactly one language in `language_config.languages` (no code switching). * All Audio Intelligence add-ons available. **`"solaria-1"`** — our generalist model: maximum language coverage across any domain. * Available for async and live. * Code switching and multi-language configuration (100+ languages covered) * All Audio Intelligence add-ons available. If omitted, the API uses the default model. (Solaria-1) ## Install the SDK ```sh JavaScript theme={"system"} npm install @gladiaio/sdk ``` ```sh Python theme={"system"} # Using pip pip install gladiaio-sdk # Using uv uv add gladiaio-sdk ``` ## Transcribe in one call End-to-end transcription — from upload to result in one call. Pass in a local file, binary data, or a remote URL — and let the method handle the rest. ```javascript JavaScript theme={"system"} import { GladiaClient } from "@gladiaio/sdk"; const gladiaClient = new GladiaClient({ apiKey: "YOUR_GLADIA_API_KEY" }); const transcription = await gladiaClient .preRecorded() .transcribe("YOUR_AUDIO_URL_OR_LOCAL_PATH"); ``` ```python Python theme={"system"} from gladiaio_sdk import GladiaClient gladia_client = GladiaClient(api_key="YOUR_GLADIA_API_KEY").prerecorded() transcription = gladia_client.transcribe("YOUR_AUDIO_URL_OR_LOCAL_PATH") ``` With customizable features: ```javascript JavaScript theme={"system"} import { GladiaClient } from "@gladiaio/sdk"; const gladiaClient = new GladiaClient({ apiKey: "YOUR_GLADIA_API_KEY" }); const transcription = await gladiaClient.preRecorded().transcribe( "YOUR_AUDIO_URL_OR_LOCAL_PATH", { model: "solaria-3", language_config: { languages: ["fr"], }, custom_vocabulary: true, custom_vocabulary_config: { vocabulary: ["Gladia", "Solaria", "Salesforce"], }, } ); ``` ```python Python theme={"system"} from gladiaio_sdk import GladiaClient gladia_client = GladiaClient(api_key="YOUR_GLADIA_API_KEY").prerecorded() transcription = gladia_client.transcribe( "YOUR_AUDIO_URL_OR_LOCAL_PATH", { "model": "solaria-3", "language_config": { "languages": ["fr"], }, "custom_vocabulary": True, "custom_vocabulary_config": { "vocabulary": ["Gladia", "Solaria", "Salesforce"], }, }, ) ``` With `"solaria-3"`, set **one language** in `language_config.languages` — for example `["fr"]`. Do not pass multiple languages or enable code switching. Want to go further? See [Audio Intelligence](/chapters/pre-recorded-stt/audio-intelligence) for add-ons like: * [Speaker diarization](/chapters/audio-intelligence/speaker-diarization): separate the speakers across the conversation * [Translation](/chapters/audio-intelligence/translation): translate the transcript into one of our 100 target languages. * [PII redaction](/chapters/audio-intelligence/pii-redaction): detect and anonymize sensitive entities (ex: GDPR-related) * [Sentiment analysis](/chapters/audio-intelligence/sentiment-analysis): extract the main sentiment and up to 25 emotions ## Individual steps The building blocks behind `transcribe()` — upload audio, create a job, then retrieve the result when you need finer control over the flow. ### Upload your audio Upload a local file and pass the returned `audio_url` to the next step. ```javascript JavaScript theme={"system"} import { GladiaClient } from "@gladiaio/sdk"; const gladiaClient = new GladiaClient({ apiKey: "YOUR_GLADIA_API_KEY" }); const uploadResponse = await gladiaClient.preRecorded().uploadFile("YOUR_LOCAL_PATH"); ``` ```python Python theme={"system"} from gladiaio_sdk import GladiaClient gladia_client = GladiaClient(api_key="YOUR_GLADIA_API_KEY").prerecorded() upload_response = gladia_client.upload_file("YOUR_LOCAL_PATH") ``` Example response: ```json theme={"system"} { "audio_url": "https://api.gladia.io/file/636c70f6-92c1-4026-a8b6-0dfe3ecf826f", "audio_metadata": { "id": "636c70f6-92c1-4026-a8b6-0dfe3ecf826f", "filename": "your_audio_file.mp3", "extension": "mp3", "size": 99515383, "audio_duration": 4146.468542, "number_of_channels": 2 } } ``` ### Create a transcription job Pass the `audio_url` from the previous step along with your transcription options. ```javascript JavaScript theme={"system"} import { GladiaClient } from "@gladiaio/sdk"; const gladiaClient = new GladiaClient({ apiKey: "YOUR_GLADIA_API_KEY" }); const job = await gladiaClient.preRecorded().createUntyped({ audio_url: "YOUR_AUDIO_URL", model: "solaria-3", language_config: { languages: ["fr"], }, custom_vocabulary: true, custom_vocabulary_config: { vocabulary: ["Gladia", "Solaria", "Salesforce"], }, }); ``` ```python Python theme={"system"} from gladiaio_sdk import GladiaClient gladia_client = GladiaClient(api_key="YOUR_GLADIA_API_KEY").prerecorded() job = gladia_client.create( { "audio_url": "YOUR_AUDIO_URL", "model": "solaria-3", "language_config": { "languages": ["fr"], }, "custom_vocabulary": True, "custom_vocabulary_config": { "vocabulary": ["Gladia", "Solaria", "Salesforce"], }, } ) ``` ### Get the transcription result You can get your transcription results in **3 different ways**: ```javascript JavaScript theme={"system"} import { GladiaClient } from '@gladiaio/sdk'; const gladiaClient = new GladiaClient({ apiKey: 'YOUR_GLADIA_API_KEY', }); // Use job.id from createUntyped(...) const result = await gladiaClient.preRecorded().poll("YOUR_TRANSCRIPTION_JOB_ID"); console.log(result.result?.transcription?.full_transcript ?? ""); ``` ```python Python theme={"system"} from gladiaio_sdk import GladiaClient gladia_client = GladiaClient(api_key="YOUR_GLADIA_API_KEY").prerecorded() # Use job.id from gladia_client.create(...) result = gladia_client.poll("YOUR_TRANSCRIPTION_JOB_ID") print(result.result.transcription.full_transcript) ``` You can use `create_and_poll()` in Python or `createAndPollUntyped()` in JavaScript to **create** and **poll** in one call with the same job payload. The methods implemented in the sdk are automatically polling until success or errors. To get the result with the cURL, you'll just have to GET continuously on the given `result_url` until the status of your transcription is `done`. You can get more information on the different transcriptions status by checking directly the [API Reference](/api-reference/). You can configure webhooks at [https://app.gladia.io/webhooks](https://app.gladia.io/webhooks) to be notified when your transcriptions are done. Once a transcription is done, a `POST` request will be made to the endpoint you configured. The request body is a JSON object containing the transcription `id` that you can use to retrieve your result with [our API](/api-reference/v2/pre-recorded/get).\ For the full body definition, check [our API definition](/api-reference/v2/pre-recorded/webhook/success). Callback are HTTP calls that you can use to get notified when your transcripts are ready. Instead of polling and keeping your server busy and maintaining work, you can use the `callback` feature to receive the result to a specified endpoint: ```json theme={"system"} { "audio_url": "YOUR_AUDIO_URL", "callback": true, "callback_config": { "url": "https://yourserverurl.com/your/callback/endpoint/", "method": "POST" } } ``` Once the transcription is done, a request will be made to the url you provided in `callback_config.url` using the HTTP method you provided in `callback_config.method`. Allowed methods are `POST` and `PUT` with the default being `POST`. The request body is a JSON object containing the transcription `id` and an `event` property that tells you if it's a [success](/api-reference/v2/pre-recorded/callback/success) or an [error](/api-reference/v2/pre-recorded/callback/error). For file size, duration, and concurrency limits, see [Supported files & duration](/chapters/limits-and-specifications/supported-formats) and [Concurrency and rate limits](/chapters/limits-and-specifications/concurrency). Pass `model` to choose the transcription model: **`"solaria-3"`** — our latest model: highest accuracy on European real-world audio. * **Async (pre-recorded) only** — not available for live transcription. * **Languages:** English, French, German, Spanish, Italian * **Single language only** — pass exactly one language in `language_config.languages` (no code switching). * All Audio Intelligence add-ons available. **`"solaria-1"`** — our generalist model: maximum language coverage across any domain. * Available for async and live. * Code switching and multi-language configuration (100+ languages covered) * All Audio Intelligence add-ons available. If omitted, the API uses the default model. (Solaria-1) ## Individual steps Upload audio, create a transcription job, then poll until the job is done (or use webhooks or a callback URL). ### Upload your audio Call the [upload endpoint](/api-reference/v2/upload/audio-file) with multipart form data. Use the returned `audio_url` when creating a transcription job. ```bash cURL theme={"system"} curl --request POST \ --url https://api.gladia.io/v2/upload \ --header 'Content-Type: multipart/form-data' \ --header 'x-gladia-key: YOUR_GLADIA_API_KEY' \ --form audio=@/path/to/your/audio/your_audio_file.mp3 ``` Example response: ```json theme={"system"} { "audio_url": "https://api.gladia.io/file/636c70f6-92c1-4026-a8b6-0dfe3ecf826f", "audio_metadata": { "id": "636c70f6-92c1-4026-a8b6-0dfe3ecf826f", "filename": "your_audio_file.mp3", "extension": "mp3", "size": 99515383, "audio_duration": 4146.468542, "number_of_channels": 2 } } ``` ### Create a transcription job POST to [`/v2/pre-recorded`](/api-reference/v2/pre-recorded/init) with your `audio_url` and options. ```javascript JavaScript theme={"system"} const response = await fetch("https://api.gladia.io/v2/pre-recorded", { method: "POST", headers: { "Content-Type": "application/json", "x-gladia-key": "", }, body: JSON.stringify({ audio_url: "YOUR_AUDIO_URL", model: "solaria-3", language_config: { languages: ["fr"], }, diarization: true, diarization_config: { number_of_speakers: 3, min_speakers: 1, max_speakers: 5, }, translation: true, translation_config: { model: "base", target_languages: ["fr", "en"], context_adaptation: true, context: "Business meeting discussing quarterly results", informal: false, }, subtitles: true, subtitles_config: { formats: ["srt", "vtt"], }, }), }); if (!response.ok) { console.error( `${response.status}: ${(await response.text()) || response.statusText}` ); process.exit(response.status); } const { id, result_url } = await response.json(); ``` ```bash cURL theme={"system"} curl --request POST \ --url https://api.gladia.io/v2/pre-recorded \ --header 'Content-Type: application/json' \ --header 'x-gladia-key: YOUR_GLADIA_API_KEY' \ --data '{ "audio_url": "YOUR_AUDIO_URL", "model": "solaria-3", "language_config": { "languages": ["fr"] }, "diarization": true, "diarization_config": { "number_of_speakers": 3, "min_speakers": 1, "max_speakers": 5 }, "translation": true, "translation_config": { "model": "base", "target_languages": ["fr", "en"], "context_adaptation": true, "context": "Business meeting discussing quarterly results", "informal": false }, "subtitles": true, "subtitles_config": { "formats": ["srt", "vtt"] } }' ``` ### Get the transcription result Poll [`GET /v2/pre-recorded/:id`](/api-reference/v2/pre-recorded/get) (or the `result_url` from the create response) until the job status is `done`. ```javascript JavaScript theme={"system"} const response = await fetch( `https://api.gladia.io/v2/pre-recorded/${id}`, { method: "GET", headers: { "x-gladia-key": "", }, } ); if (!response.ok) { console.error( `${response.status}: ${(await response.text()) || response.statusText}` ); return; } const result = await response.json(); console.log(result); ``` Instead of polling, configure [webhooks](https://app.gladia.io/webhooks) or set `callback` and `callback_config` on the job — see the [init](/api-reference/v2/pre-recorded/init) reference. The **Using our SDKs** tab also documents polling helpers, webhooks, and callbacks together. Want to know more about a specific feature? Check out our [Features chapter](/chapters/pre-recorded-stt/features) for more details. ## Full code sample You can find complete code samples in our Github repository: # Recommended Parameters by Use Case Source: https://docs.gladia.io/chapters/pre-recorded-stt/recommended-parameters Best parameter configurations for pre-recorded transcription depending on your application (Meeting Recorders, Call Centers, Podcasts, Subtitles, Multilingual Content). The right parameter configuration can significantly impact transcription quality for pre-recorded audio. This guide covers recommended starting points for common scenarios and highlights pitfalls that frequently trip up new integrations. These recommendations apply to the **[Pre-recorded API](/chapters/pre-recorded-stt/quickstart)** and are passed in the `POST /v2/pre-recorded` request body. They are starting points — tune them to match your specific needs. *** ## Language Configuration One of the most common configuration mistakes is misunderstanding how `language_config` works. Choosing the right setup avoids unnecessary detection overhead and improves accuracy. **When to set an explicit language:** * You **know** the language of the audio ahead of time. * The audio is **monolingual** (single language throughout). * You want the **fastest, most accurate** results. ```json theme={"system"} { "language_config": { "languages": ["en"], "code_switching": false } } ``` **When to use auto-detection:** * You process audio in **many different languages** and don't know which one beforehand. * You want Gladia to pick the language automatically. ```json theme={"system"} { "language_config": { "languages": [], "code_switching": false } } ``` When `code_switching` is `false` and no language is set, the language is detected on the **first utterance** and reused for the rest of the session or file. If the beginning of your audio contains silence, music, or a different language than the main content, this can lead to incorrect detection for the whole transcription. Even when using auto-detection, pass a **small list of likely languages** in `languages` to constrain the search. This improves both accuracy and processing time. *** ## Code Switching Code switching (`language_config.code_switching: true`) lets Gladia detect and transcribe **multiple languages** within the same audio, re-evaluating the language on each utterance. **When to enable it:** * Speakers **switch languages** mid-conversation (e.g. bilingual meetings, multilingual customer support). * You need the detected `language` returned **per utterance**. **When NOT to enable it:** * The audio is in a **single language** — code switching adds unnecessary processing and can introduce misdetections. * You've set **exactly one language** in `languages` — in that case `code_switching` is ignored anyway. ```json theme={"system"} { "language_config": { "languages": ["en", "fr", "es"], "code_switching": true } } ``` **Do not enable `code_switching` with an empty `languages` list.** When no languages are specified, the language detector evaluates every utterance against 100+ supported languages, which leads to frequent misdetections — especially between similar-sounding languages. Always provide a short list of languages you **actually expect** in the audio. *** ## Custom Vocabulary [Custom vocabulary](/chapters/audio-intelligence/custom-vocabulary) is a post-transcription replacement based on **phoneme similarity**. It's essential for domain-specific terms that speech models frequently mis-transcribe. **Best practices:** * **Always provide both** the `custom_vocabulary` flag and a `custom_vocabulary_config`. * **Add pronunciations** to provide all the close spelling variants. You can use Automatic Phonemic Transcriber (IPA) in order to check if all the different spellings are covered. * **Keep `intensity` moderate** (0.4-0.6). High values increase false positives where unrelated words get replaced. * **Set `language`** on individual vocabulary entries when your audio is multilingual and a term is pronounced differently depending on the language. ```json Pre-recorded theme={"system"} { "audio_url": "YOUR_AUDIO_URL", "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": [ "Kubernetes", { "value": "Gladia", "pronunciations": ["Glad", "Gladio"], "intensity": 0.5 }, { "value": "PostgreSQL", "pronunciations": ["Postgres Q L", "Post gress"], "intensity": 0.4 } ], "default_intensity": 0.5 } } ``` ```json Live theme={"system"} { "realtime_processing": { "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": [ "Kubernetes", { "value": "Gladia", "pronunciations": ["Glad", "Gladio"], "intensity": 0.5 }, { "value": "PostgreSQL", "pronunciations": ["Postgres Q L", "Post gress"], "intensity": 0.4 } ], "default_intensity": 0.5 } } } ``` *** ## Meeting Recorders For apps that record and process meetings — team stand-ups, board sessions, 1-on-1s — the goal is to produce **structured, actionable meeting notes** with clear speaker attribution. Meetings typically have a known set of participants and benefit heavily from post-processing features like summarization. | Parameter | Recommended value | Why | | -------------------------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `diarization` | `true` | Attributes speech to each participant. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). | | `diarization_config.min_speakers` / `max_speakers` | Set a range (e.g. `2`-`10`) | Meeting size varies — a range lets the model adapt without over- or under-splitting speakers. | | `summarization` | `true` | Generates a summary for quick review. Use `bullet_points` type for action-item style output. See [Summarization](/chapters/audio-intelligence/summarization). | | `named_entity_recognition` | `true` | Surfaces people, organizations, dates, and other key entities mentioned during the meeting. See [NER](/chapters/audio-intelligence/named-entity-recognition). | | `sentences` | `true` | Produces well-segmented, readable output suitable for meeting minutes. See [Sentences](/chapters/pre-recorded-stt/features/sentences). | | `language_config.languages` | Set explicitly | Meeting language is almost always known in advance — setting it avoids detection overhead. | | `custom_vocabulary` | `true` | Add company-specific terms, project names, and participant names for better accuracy. | **Diarization vs. multi-channel:** if each speaker is on a **separate audio channel** (e.g. a, use the `channel` field on each utterance to identify who is speaking — diarization is not needed. See [Multiple channels](/chapters/limits-and-specifications/multiple-channels) If all speakers share a **single audio channel**, enable `diarization` to separate the speakers. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). *** ## Call Centers For recorded phone calls the priorities are **speaker identification** and **accurate transcription** despite variable audio quality (telephony codecs, background noise, cross-talk). | Parameter | Recommended value | Why | | --------------------------------------- | ------------------------------ | --------------------------------------------------------------------------------------------------------------------------- | | `language_config.languages` | Set explicitly (e.g. `["en"]`) | Call center audio typically has a known language. Setting it avoids detection errors on noisy recordings. | | `diarization` | `true` | Separates agent and customer speech. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). | | `diarization_config.number_of_speakers` | `2` | Most calls have exactly two participants — giving this hint improves speaker assignment accuracy. | | `custom_vocabulary` | `true` | Add product names, plan names, and internal terminology. | | `summarization` | `true` | Automatically generates a summary for agent wrap-up notes. See [Summarization](/chapters/audio-intelligence/summarization). | **Diarization vs. multi-channel:** if each speaker is on a **separate audio channel** (e.g. a, use the `channel` field on each utterance to identify who is speaking — diarization is not needed. See [Multiple channels](/chapters/limits-and-specifications/multiple-channels) If all speakers share a **single audio channel**, enable `diarization` to separate the speakers. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). *** ## Podcasts & Interviews For long-form audio with multiple speakers the focus is on **readability** and **correct speaker attribution**. Transcripts are often repurposed as articles or show notes, so segment quality matters. | Parameter | Recommended value | Why | | -------------------------------------------------- | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | `diarization` | `true` | Essential for multi-speaker content. | | `diarization_config.min_speakers` / `max_speakers` | Set a range (e.g. `2`-`4`) | Provides a flexible hint when the exact count varies across episodes. | | `sentences` | `true` | Produces well-segmented, readable output suitable for publishing. See [Sentences](/chapters/pre-recorded-stt/features/sentences). | | `custom_vocabulary` | `true` | Add recurring guest names, show-specific terms, and brand names. | | `language_config.languages` | Set explicitly | Podcast language is almost always known in advance. | *** ## Subtitles & Captioning When generating subtitle files from pre-recorded content, tune the formatting parameters for the best viewing experience. Gladia produces SRT and VTT files directly — no post-processing needed. See [Subtitles](/chapters/audio-intelligence/subtitles) for the full parameter reference. | Parameter | Recommended value | Why | | --------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | | `subtitles` | `true` | Enables subtitle generation. | | `subtitles_config.formats` | `["srt", "vtt"]` | Generate both formats to cover different players and platforms. | | `subtitles_config.maximum_characters_per_row` | `42` | Standard broadcast limit for readability. | | `subtitles_config.maximum_rows_per_caption` | `2` | Keeps captions compact on screen. | | `subtitles_config.style` | `"compliance"` | Uses stricter formatting rules suited for broadcast or accessibility requirements. | | `translation` | `true` (if needed) | When enabled, subtitles are automatically generated for each target language. See [Translation](/chapters/audio-intelligence/translation). | For live captions streamed in real time, use the [Realtime API](/chapters/live-stt/quickstart) with partial transcripts instead — see the [Live recommended parameters](/chapters/live-stt/recommended-parameters#subtitles--captioning) guide. *** ## Multilingual Content For content with mixed languages — conferences, multilingual media, interviews with speakers from different countries — combine language detection with code switching. | Parameter | Recommended value | Why | | -------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------- | | `language_config.languages` | List of expected languages (e.g. `["en", "fr", "de"]`) | Constrain to 3-5 expected languages for best accuracy. | | `language_config.code_switching` | `true` | Detects language shifts across utterances. See [Code switching](/chapters/language/code-switching). | | `custom_vocabulary` | `true` | Add terms for each language with appropriate `language` tags on each entry. | Do not enable `code_switching` with an empty `languages` list. The detector would evaluate every utterance against 100+ languages, leading to frequent misdetections — especially between similar-sounding languages.