For AI agents: a documentation index is available at /llms.txt. Markdown versions of all pages can be requested by appending `.md` to the URL, or by setting the `Accept` header to `text/markdown`.
The returned JSON will include a new property called translations, which contains a list of translated text for each target language requested (using the same ISO language codes as for transcription).
The punctuation marks which the client is prepared to accept in transcription output, or the special value 'all' (the default). Unsupported marks are ignored. This value is used to guide the transcription process.
metadata.transcription_config.diarization
"none" | "speaker" | "channel"
optional
Specify whether speaker or channel labels are added to the transcript.
The default is none.
none: no speaker or channel labels are added.
speaker: speaker attribution is performed based on acoustic matching;
all input channels are mixed into a single stream for processing.
channel: multiple input channels are processed individually and collated
into a single transcript.
Transcript labels to use when using collating separate input channels.
metadata.transcription_config.enable_entities
boolean
optional
Include additional 'entity' objects in the transcription results (e.g. dates, numbers) and their original spoken form. These entities are interleaved with other types of results. The concatenation of these words is represented as a single entity with the concatenated written form present in the 'content' field. The entities contain a 'spoken_form' field, which can be used in place of the corresponding 'word' type results, in case a spoken form is preferred to a written form. They also contain a 'written_form', which can be used instead of the entity, if you want a breakdown of the words without spaces. They can still contain non-breaking spaces and other special whitespace characters, as they are considered part of the word for the formatting output. In case of a written_form, the individual word times are estimated and might not be accurate if the order of the words in the written form does not correspond to the order they were actually spoken (such as 'one hundred million dollars' and '$100 million').
metadata.transcription_config.max_delay_mode
"fixed" | "flexible"
optional
Whether or not to enable flexible endpointing and allow the entity to continue to be spoken.
Controls the lower limit of audio volume at which speech and audio events will be transcribed. If the volume limit is very low, then most sound will be passed to the speech recognition engine. Higher numbers will cut out increasing amounts of sound.
If true, words that are identified as disfluencies will be removed from the transcript. If false (default), they are tagged in the transcript as 'disfluency'.
An array of objects defining custom replacements. Each replacement contains a pair of strings: the text to find from and the text to replace it with to.
If true, the algorithm will prefer to stay with the current active speaker if it is a close enough match, even if other speakers may be closer. This is useful for cases where we can flip incorrectly between similar speakers during a single speaker section.
Controls how sensitive the algorithm is in terms of keeping similar speakers separate, as opposed to combining them into a single speaker. Higher values will typically lead to more speakers, as the degree of difference between speakers in order to allow them to remain distinct will be lower. A lower value for this parameter will conversely guide the algorithm towards being less sensitive in terms of retaining similar speakers, and as such may lead to fewer speakers overall. The default is 0.5.
Use this option to provide speaker labels linked to their speaker identifiers. When passed, the transcription system will tag spoken words in the transcript with the provided speaker labels whenever any of the specified speakers is detected in the audio. A maximum of 50 speakers identifiers across all speakers can be provided.
List of errors that occurred in the auto chapters stage.
metadata.auto_chapters_errors[].type
"auto_chapters_failed" | "unsupported_language"
optional
metadata.auto_chapters_errors[].message
string
optional
Human readable error message
output_config
OutputConfig
optional
metadata.output_config.srt_overrides
object
optional
Parameters that override default values of srt conversion. max_line_length: sets maximum count of characters per subtitle line including white space. max_lines: sets maximum count of lines in a subtitle section.
Orchestrator version in PEP 440 Format or set to 'version_not_found' as default.
results
RecognitionResult[]
An ASR job output item. The primary item types are word and punctuation. Other item types may be present, for example to provide semantic information of different forms.
channel
string
optional
start_time
number<float>
end_time
number<float>
volume
number<float>≥ 0, ≤ 100
optional
An indication of the volume of audio across the time period the word was spoken.
is_eos
boolean
optional
Whether the punctuation mark is an end of sentence character. Only applies to punctuation marks.
type
"word" | "punctuation" | "entity"
New types of items may appear without being requested; unrecognized item types can be ignored.
written_form
WrittenFormRecognitionResult[]
optional
A WrittenFormRecognitionResult describes a simple object which consists solely of 'word' type entries with a start and end time. It can occur only inside the written_form property of a full RecognitionResult'
results[].written_form[].alternatives
RecognitionAlternative[]
List of possible job output item values, ordered by likelihood.
What kind of object this is. See #/Definitions/RecognitionResult for definitions of the enums.
spoken_form
SpokenFormRecognitionResult[]
optional
A SpokenFormRecognitionResult describes a simple object which consists solely of 'word' or 'punctuation' type entries with a start and end time. It can occur only inside the spoken_form property of a full 'RecognitionResult'
results[].spoken_form[].alternatives
RecognitionAlternative[]
List of possible job output item values, ordered by likelihood.
What kind of object this is. See #/Definitions/RecognitionResult for definitions of the enums.
alternatives
RecognitionAlternative[]
optional
List of possible job output item values, ordered by likelihood.
results[].alternatives[].content
string
results[].alternatives[].confidence
number<float>
results[].alternatives[].language
string
results[].alternatives[].display
RecognitionDisplay
optional
results[].alternatives[].display.direction
"ltr" | "rtl"
results[].alternatives[].speaker
string
optional
results[].alternatives[].tags
string[]
optional
attaches_to
enum"previous""next""both""none"
optional
Attachment direction of the punctuation mark. This only applies to punctuation marks. This information can be used to produce a well-formed text representation by placing the word_delimiter from language_pack_info on the correct side of the punctuation mark.
speakers
SpeakersResultItem[]
optional
List of unique speaker identifiers detected in the transcript.
label
string≥ 1 characters
Speaker label.
speaker_identifiers
Array<string<bytes>>≥ 1 itemsunique items
translations
Record<string, TranslationSentence[]>
optional
Translations of the transcript into other languages. It is a map of ISO language codes to arrays of translated sentences. Configured using translation_config.
[key: string]
TranslationSentence[]
optional
translations[key: string][].start_time
number<float>
optional
translations[key: string][].end_time
number<float>
optional
translations[key: string][].content
string
optional
translations[key: string][].speaker
string
optional
translations[key: string][].channel
string
optional
summary
SummarizationResult
optional
Summary of the transcript, configured using summarization_config.
content
string
optional
sentiment_analysis
SentimentAnalysisResult
optional
The main object that holds sentiment analysis data.
sentiment_analysis
object
optional
Holds the detailed sentiment analysis information.
sentiment_analysis.sentiment_analysis.segments
SentimentSegment[]
optional
An array of objects that represent a segment of text and its associated sentiment.
Realtime provides a stream of translation messages per language requested. Translation messages will arrive after transcription messages, but won't delay transcription. Realtime translations have the following schema:
Partial translations typically correspond to unfinished sentences and have lower latency than final translations. By default, only final translations are produced. Enable partials using the enable_partials property in translation_config for the session. For example:
Final translations are the most accurate and complete translations, usually at the end of a sentence. These translations are considered final and will not be updated afterwards. For example:
Python client example to translate a file for batch.
from speechmatics.models import ConnectionSettings from speechmatics.batch_client import BatchClient from httpx import HTTPStatusError API_KEY ="YOUR_API_KEY" PATH_TO_FILE ="example.wav" LANGUAGE ="en"# Transcription language TRANSLATION_LANGUAGES =["es","de"] settings = ConnectionSettings( url="https://eu1.asr.api.speechmatics.com/v2", auth_token=API_KEY, ) # Define transcription parameters conf ={ "type":"transcription", "transcription_config":{ "language": LANGUAGE, "model":"enhanced" }, "translation_config":{ "target_languages":TRANSLATION_LANGUAGES } } # Open the client using a context manager with BatchClient(settings)as client: try: job_id = client.submit_job( audio=PATH_TO_FILE, transcription_config=conf, ) print(f'job {job_id} submitted successfully, waiting for transcript') # Note that in production, you should set up notifications instead of polling. # Notifications are described here: https://docs.speechmatics.com/features-other/notifications transcript = client.wait_for_completion(job_id, transcription_format='json-v2') for language in TRANSLATION_LANGUAGES: # Print the translation for each language from the JSON print(f"Translation for {language}") translation ="" for translated_segment in transcript["translations"][language]: translation += translated_segment["content"]+" " print(translation) except HTTPStatusError as e: if e.response.status_code ==401: print('Invalid API key - Check your API_KEY at the top of the code!') elif e.response.status_code ==400: print(e.response.json()['detail']) else: raise e
Python client example to translate a file in real-time, see here for more examples of Realtime Transcription
import speechmatics from httpx import HTTPStatusError API_KEY ="YOUR_API_KEY" PATH_TO_FILE ="example.wav" LANGUAGE ="en"# Transcription language TRANSLATION_LANGUAGES =["es","de"] CONNECTION_URL =f"wss://eu.rt.speechmatics.com/v2/{LANGUAGE}" # Create a transcription client ws = speechmatics.client.WebsocketClient( speechmatics.models.ConnectionSettings( url=CONNECTION_URL, auth_token=API_KEY, ) ) # Define an event handler to print the translations defprint_translation(msg): msg_type="Final" if msg['message']=="AddPartialTranslation": msg_type="Partial" language = msg['language']# language for translation message translations =[] for translation_segment in msg['results']: translations.append(translation_segment['content']) translation =" ".join(translations).strip() print(f"{msg_type} translation for {language}: {translation}") # Register the event handler for partial translation ws.add_event_handler( event_name=speechmatics.models.ServerMessageType.AddPartialTranslation, event_handler=print_translation, ) # Register the event handler for full translation ws.add_event_handler( event_name=speechmatics.models.ServerMessageType.AddTranslation, event_handler=print_translation, ) settings = speechmatics.models.AudioSettings() # Define transcription parameters with translation # Full list of parameters described here: https://speechmatics.github.io/speechmatics-python/models translation_config = speechmatics.models.RTTranslationConfig( target_languages=TRANSLATION_LANGUAGES, #enable_partials=True # Optional argument to provide translation of partial sentences ) transcription_config = speechmatics.models.TranscriptionConfig( model="enhanced", language=LANGUAGE, translation_config=translation_config ) print("Starting transcription (type Ctrl-C to stop):") withopen(PATH_TO_FILE,'rb')as fd: try: ws.run_synchronously(fd, transcription_config, settings) except KeyboardInterrupt: print("\nTranscription stopped.") except HTTPStatusError as e: if e.response.status_code ==401: print('Invalid API key - Check your API_KEY at the top of the code!') else: raise e