Audio Files in Data > Explore

Encord supports the curation and organization of your audio data. You can import your audio data using the UI, from cloud storage (with a JSON file), and using Encord’s SDK.

Data > Explore Audio Support

Data > Explore supports the following audio formats:

Support	Description
Codec	.mp3, .wav, .flac, .eac3
Container	.mp4, .m4a
Sample rate	8k to 96k
Bit depth	16bit to 32bit

Quick Tour

Use the Audio tab to natively view all the audio files available in your Folders. From the Audio tab you can sort and filter all of your audio files based on Audio Quality Metrics. If only audio files are in your Folders, no Audio tab displays. Your audio files display in the Index Explorer. If there are a mix of file types (images, videos, and audio files), tabs display for the various file types (Video, Frames, Audio).

• Audio Quality Metrics for sorting and filtering your audio files.

• Waveform playback controls to move through audio files.

Use Audio Quality Metrics

Use Audio Quality Metrics to sort and filter your audio files.

Sort

Filter

Audio Quality Metrics

Data > Explore supports filtering your audio files using the following audio quality metrics:

• Bit depth: Size of each sample (int) in bits.
• Duration: The duration of the audio file.
• Number of channels: Number of audio channels in an audio file.
• Random value: A random number assigned to each audio file.
• Samples per second: The number of samples per second.

Transcription for Audio Files

First update the custom metadata schema to include a text (formally long_string) data type for your transcripts. Then register data with the appropriately named metadata field. When you display that field, using the Display control section of the app, a resizable text field displays, so transcripts can be previewed directly under the relevant audio waveform.

Step 1: Add Transcript Key to Schema

Before adding transcripts to your audio files, make sure you add a transcript key with long_string specified as the type to your custom metadata schema.

Add Transcript Key to Schema

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "<file-path-to-ssh-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Retrieve the current client metadata schema
current_schema = user_client.get_client_metadata_schema()

# Define the new metadata to append
new_metadata = {'<my-transcript>': 'long_string'}

# Update the current schema with the new metadata
updated_schema = {**current_schema, **new_metadata}

# Set the updated client metadata schema
user_client.set_client_metadata_schema_from_dict(updated_schema)

# Optionally, print the updated schema to verify the changes
print(updated_schema)

Step 2: Import Transcript

After updating your custom metadata schema, you can import your transcripts to your audio files.

Import your Transcript

from encord import EncordUserClient

# Authentication
SSH_PATH = "<file-path-to-ssh-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    "<data-unit-id-1>": {"transcript-latin": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."},
    "<data-unit-id-1>": {"transcript-en-us": "It is important to take care of the patient, to be followed by the patient, but it will happen at such a time that there is a lot of work and pain. For to come to the smallest detail, no one should practice any kind of work unless he derives some benefit from it. Do not be angry with the pain in the reprimand in the pleasure he wants to be a hair from the pain in the hope that there is no breeding. Unless they are blinded by lust, they do not come forth; they are in fault who abandon their duties and soften their hearts, that is, their labors.
"},
    "<data-unit-id-2>": {"transcript-en-gb": "To be, or not to be, that is the question."},
    "<data-unit-id-3>": {"transcript-en-ca": "You won't hear this from any 1935 comic."}
}

# Use the Bundle context manager
with Bundle() as bundle:
    # Update the storage items based on the dictionary
    for item_uuid, metadata_update in updates.items():
        item = user_client.get_storage_item(item_uuid=item_uuid)

        # Make a copy of the current metadata and update it with the new metadata
        curr_metadata = item.client_metadata.copy()
        curr_metadata.update(metadata_update)

        # Update the item with the new metadata and bundle
        item.update(client_metadata=curr_metadata, bundle=bundle)

Step 3: View/Filter in Index

In Index, you can filter and view the transcripts applied to your audio files.

View with audio files

To view transcripts:

1. Click Display.

2. Click the field under Card details. A menu appears.

3. Select your transcript under Custom metadata. Transcripts appear under your audio files.

Filter using transcripts

To filter using transcripts:

1. Click Filter.

2. Click Add filter. A menu appears.

3. Select Custom metadata. A field appears.

4. Select your transcripts from the field.

5. Type the text you want to search for.

6. Click Display.

7. Click the field under Card details. A menu appears.

8. Select your transcript under Custom metadata. Transcripts appear under your audio files with the filtered text highlighted.

Long Audio Files

When audio files in your cloud storage are longer than an hour, the Label Editor may take extra time to load their waveforms. To significantly reduce load times, you can generate a JSON file containing the waveform data and store it alongside your audio file.

Locally imported data CANNOT have long audio performance improved using this method. Improving long audio file performance is ONLY supported for audio files stored in cloud storage and accessed by Encord through cloud storage.

We STRONGLY recommend generating a waveform file for any audio files that are an hour or longer in duration.

1. Install audiowaveform.

2. Use the following command to generate a JSON file of your waveform:

The name of the JSON file must exactly match the name of your audio file. Only the file extensions differ.

command

audiowaveform -i ./input.mp3 -o ./waveform-output.json

audiowaveform -i "/Users/chris/Documents/awesome-audio-file-01.mp3" \
  -o "/Users/chris/Documents/awesome-audio-file-01.json"

3. Upload the JSON file to the location of your audio file.

The JSON file MUST reside in the same cloud storage location as your audio file.

For example:

• Audio File Path in Bucket : “/my-bucket/all-audio/my-favorite-song.mp3”
• Audio Waveform File Path in Bucket: “/my-bucket/all-audio/my-favorite-song.json”

You can upload the JSON file during audio file import or any time after.