Custom Dictionary Cache

Transcription:Real-TimeDeployments:Virtual Appliance

The Speechmatics Real-Time Virtual Appliance includes a cache mechanism for custom dictionaries. By using this cache mechanism, the Appliance is able to reduce the time needed for processing a Custom Dictionary before starting the recognition of an audio stream.

Once the Real-Time Virtual Appliance has started a recognition session with a given Custom Dictionary, any subsequent streams with the identical Custom Dictionary sent to the Appliance by any client will benefit from a reduced setup time. Transcription requests using an already cached Custom Dictionary will start recognition in less than 3 seconds if the Custom Dictionary is below our recommended limit of 1000 words.

info

V1 API When using V1 API the setup time spent processing a cached entry is about 3 seconds longer compared to V2 API.

Overview

Size available

The size available for storing Custom Dictionary Cache entries depends on the variant of the Real-Time Virtual Appliance.

Variant	Cache Space
nano	100MB
mini	150MB
midi	200MB
maxi	250MB
plus	300MB

Size of cache entries

The entry size varies depending on the amount of information included in the Custom Dictionary. For guidance, the table below displays some values for different custom dictionaries.

Custom Dictionary	Cache Used Bytes	Cache Entry Size
None (empty cache)	14KiB	NA
1000 words	120KiB	106KiB
1000 words + 1 sounds like each	188KiB	174KiB

Size of empty cache

The Custom Dictionary cache needs to keep certain metadata files in order to function. For this reason the used_bytes reported will never be 0, even if the cache is not storing any entry. The amount of storage used by an empty cache is typically around 14KiB.

Cache life cycle

When a Custom Dictionary is used for a transcription, it is automatically cached by the Real-Time Virtual Appliance. This allows for a reduced setup time on any further transcriptions using the same Custom Dictionary. Custom dictionary cache is persisted on disk, making it available between reboots. If there is no space left in the cache for a new entry, entries are deleted in order of when they were last used when submitting a job.

Cache entries are ready to be consumed by any subsequent transcription request, from any client, after a RecognitionStarted message is emitted by the Real-Time Appliance. For this reason, transcription requests made in parallel, each with the same Custom Dictionary that is new to the Appliance, won’t benefit from a reduced setup time.

A Custom Dictionary could be cached beforehand by requesting a transcription for an empty audio file. After receiving the RecognitionStarted message from the Appliance, other requests using the same Custom Dictionary will benefit from a reduced startup time.

Administering the Cache

When the Real-Time Virtual Appliance is started for the first time the cache will be empty. The Management API allows you to retrieve cache usage data and to purge the cache contents.

View Cache Usage

Cache usage reports the maximum cache size and the used number of bytes in the cache.

In order to retrieve usage statistics, send a GET request to the /v1/management/cache endpoint:

curl -L -X GET http://${APPLIANCE_HOST}:8080/v1/management/cache \
-H 'Accept: application/json' \
| jq

Here is an example response:

{
  "total_bytes": "105188352",
  "used_bytes": "192512"
}

Purge Cache Contents

It is possible to remove all contents in the cache.

In order to purge the cache contents, send a DELETE request to the /v1/management/cache endpoint:

curl -L -X DELETE http://${APPLIANCE_HOST}:8080/v1/management/cache \
-H 'Accept: application/json' \
| jq

Here is an example response:

{
  "confirmation": "Custom dictionary cache purged successfully",
  "usage": {
    "total_bytes": "105188352",
    "used_bytes": "14336"
  }
}