Speech Recognition

The core functionality of the Gladia API is its Speech Recognition model, designed to convert spoken language into written text. This serves as the basis for all Gladia API offerings.

Additional capabilities, like Speaker Diarization, Summarization, Translation, Custom Prompts and more can be integrated seamlessly into the transcription process by including extra parameters in the transcription request.

​

Sending a transcription request

async function makeFetchRequest(url: string, options: any) {
  const response = await fetch(url, options);
  return response.json();
}

async function pollForResult(resultUrl: string, headers: any) {
  while (true) {
    console.log("Polling for results...");
    const pollResponse = await makeFetchRequest(resultUrl, { headers });

    if (pollResponse.status === "done") {
      console.log("- Transcription done: \n ");
      console.log(pollResponse.result.transcription.full_transcript);
      break;
    } else {
      console.log("Transcription status : ", pollResponse.status);
      await new Promise((resolve) => setTimeout(resolve, 1000));
    }
  }
}

async function startTranscription() {
  const gladiaKey = "YOUR_GLADIA_API_TOKEN";
  const requestData = {
    audio_url:
      "YOUR_AUDIO_URL",
  };
  const gladiaUrl = "https://api.gladia.io/v2/transcription/";
  const headers = {
    "x-gladia-key": gladiaKey,
    "Content-Type": "application/json",
  };

  console.log("- Sending initial request to Gladia API...");
  const initialResponse = await makeFetchRequest(gladiaUrl, {
    method: "POST",
    headers,
    body: JSON.stringify(requestData),
  });

  console.log("Initial response with Transcription ID :", initialResponse);

  if (initialResponse.result_url) {
    await pollForResult(initialResponse.result_url, headers);
  }
}

startTranscription();

After that your audio have been processed, here’s an example output of what you should get :

Split infinity in a time when less is more, where too much is never enough. There is always hope for the future. The future can be read from the past. The past foreshadows the present, and the present hasn't been written yet.`

​

Getting the result of a request

You can get your transcription results in 3 different ways:

Polling

Once you post your transcription request, you get a transcription id and a pre-built result_url for convenience. To get the result with this method, you’ll just have to GET continuously on the given result_url until the status of your transcription is done.

You can get more information on the different transcriptions status by checking directly the API Reference.

Webhook

You can configure webhooks at https://app.gladia.io/account to be notified when your transcriptions are done.

Once a transcription is done, a POST request will be made to the endpoint you configured. The request body is a JSON object containing the transcription id that you can use to retrieve your result with our API.
For the full body definition, check our API definition.

Callback URL

Callback are HTTP calls that you can use to get notified when your transcripts are ready.

Instead of polling and keeping your server busy and maintaining work, you can use the callback_url feature. By providing a callback_url on your transcription request body payload, we will send the results right there once your transcription is done.

{
  "audio_url": "YOUR_AUDIO_URL",
  "callback_url": "https://yourserverurl.com/your/callback/endpoint/"
}

Once the transcription is done, a POST request will be made to the callback_url you provided.
The request body is a JSON object containing the transcription id and an event property that tells you if it’s a success or an error.

​

Transcription job status

The transcription status can have different values :

​

Word-level timestamps

Instead of just getting utterances start and end timestamps, Gladia Speech-to-text API provides by default the Word-level timestamps feature. It lets you know the exact timestamp for each word and give you a more precise transcription. This feature is particularly useful for detailed analysis, as it allows you to pinpoint the exact moment each word is spoken, facilitating a more accurate synchronization with audio or video files.

Under each utterance, you’ll find a words property like this:

// other properties...
"utterances": [
    {
      "words": [
        {
          "word": "Split",
          "start": 0.21001999999999998,
          "end": 0.69015,
          "confidence": 1
        },
        {
          "word": " infinity",
          "start": 0.91021,
          "end": 1.55038,
          "confidence": 0.95
        },
        ...
      ]
    }
  ]

​

Sentences

In addition to getting the transcription split by utterances, you can request to semantically segment the transcription to sentences, providing a more human readable result.

{
  "sentences": true
}

The result will contain a sentences key (in addition to utterances):

		"sentences": {
			"success": true,
			"is_empty": false,
			"results": [
				{
					"sentence": "Amy, it says you are trained in technology.",
					"start": 0.4681999999999999,
					"end": 2.45525,
					"words": [...],
					"confidence": 0.95,
					"language": "en",
					"speaker": 0,
					"channel": 0
				},
				{
					"sentence": "That's very good.",
					"start": 2.51546,
					"end": 3.5992999999999995,
					"words": [...],
					"confidence": 0.96,
					"language": "en",
					"speaker": 0,
					"channel": 0
				},
        ...
      ]
    }

​

Language behaviour

If you know the audio used in the language, you should use Manual, otherwise, Automatic Language Detection.

{
  "audio_url": "YOUR_AUDIO_URL",
  "detect_language": false
}

{
  "audio_url": "YOUR_AUDIO_URL",
  "enable_code_switching": true
}

{
  "audio_url": "YOUR_AUDIO_URL",
  "detect_language": false,
  "language": "fr"
}

​

Export SRT or VTT caption files

You can export completed transcripts in both SRT and VTT format, which can be used for subtitles and captions in videos.

{
  "audio_url": "YOUR_AUDIO_URL",
  "subtitles": true,
  "subtitles_config": {
    "formats": ["srt", "vtt"]
  }
}

The JSON response will include a new property subtitles which is an array of every formats you requested. With the given example, subtitles will contains 2 items of shape:

{
  "format": "srt", //format name
  "subtitles": "1\n00:00:00,210 --> 00:00:04,711....." // subtitles
}

​

Context prompt

• Context prompt : If you know the context of the audio you’re sending, you can provide it in the context_prompt.

{
  "audio_url": "YOUR_AUDIO_URL",
  "context_prompt": "A conversation between Sansa Stark and Peter Baelish from the Game of Thrones series.",
}

​

Custom vocabulary

• Custom vocabulary : To enhance the precision of transcription, especially for words or phrases that recur often in your audio file, you can utilize the custom_vocabulary feature in the transcription configuration settings.
  The custom vocabulary has a global limit of 10k characters.

{
  "audio_url": "YOUR_AUDIO_URL",
  "custom_vocabulary": ["westeros", "stark", "night's watch"]
}

​

Dual-channel or Multiple channels transcription

If you have multiples channels in your audio file with different content each, Gladia API automatically transcribe them. In the transcription result, you will get for each utterances a channel key corresponding to the channels the transcription came from.

​

Adding custom metadata

You can add metadata to your transcription using the custom_metadata input during your POST request on /v2/transcription endpoint. This will allow you to recognize your transcription when you get its data from the GET /v2/transcription/{id} endpoint, but more important, it will allow you to use it as a filter in the GET /v2/transcription list endpoint. For example, you can add the following when asking for a transcription:

"custom_metadata": {
    "internalUserId": 2348739875894375,
    "paymentMethod": {
        "last4Digits": 4576
     },
     "internalUserName": "Spencer"
}

And then, use the following GET request to filter results like:

https://api.gladia.io/v2/transcription?custom_metadata={"internalUserId": "2348739875894375"}

or

https://api.gladia.io/v2/transcription?custom_metadata={"paymentMethod": {"last4Digits": 4576}, "internalUserName": "Spencer"}