Metadata Schema

Before importing your custom metadata, we recommend importing a metadata schema. Schemas help Encord validate your custom metadata and instruct Index and Active how to display it.

To manage custom metadata across multiple teams, we recommend using namespaced keys in your schema. This helps avoid conflicts—for example, one team might use video.description, while another uses audio.description or TeamName.MetadataKey. It keeps things organized and clear across departments.

Benefits of Using a Metadata Schema

Using a metadata schema provides several benefits:

• Validation: Ensures that all custom metadata conforms to predefined data types, reducing errors during data registration and processing.
• Consistency: Maintains uniformity in data types across different datasets and projects, which simplifies data management and analysis.
• Filtering and Sorting: Enhances the ability to filter and sort data efficiently in the Encord platform, enabling more accurate and quick data retrieval.

---

Metadata Schema Table

• Metadata Schema keys support letters (a-z, A-Z), numbers (0-9), and blank spaces ( ), hyphens (-), underscores (_), and periods (.). Metadata schema keys are case sensitive.

• If you are unsure about the type to assign to a metadata key, we recommend using varchar as a versatile default.

Use .add_scalar() to add a scalar key to your metadata schema.

Scalar Key	Description	Display Benefits
boolean	Binary data type with values “true” or “false”.	Filtering by binary values
datetime	ISO 8601 formatted date and time.	Filtering by time and date
number	Numeric data type supporting float values.	Filtering by numeric values
uuid	UUIDv4 formatted unique identifier for a data unit.	Filtering by customer specified unique identifier
varchar	Textual data type. Formally string. string can be used as an alias for varchar, but we STRONGLY RECOMMEND that you use varchar.	Filtering by string.
text	Text data with unlimited length (example: transcripts for audio). Formally long_string. long_string can be used as an alias for text, but we STRONGLY RECOMMEND that you use text.	Storing and filtering large amounts of text.

Use add_enum and add_enum_options to add an enum and enum options to your metadata schema.

Key	Description	Display Benefits
enum	Enumerated type with predefined set of values.	Facilitates categorical filtering and data validation

Use add_embedding to add an embedding to your metadata schema.

Key	Description	Display Benefits
embedding	1 to 4096 for Index. 1 to 2000 for Active.	Filtering by embeddings, similarity search, 2D scatter plot visualization (Coming Soon)

Use add_geospatial to add a new geospatial type to the metadata schema.

Key	Description	Display Benefits
geospatial	Geospatial data that contains latitude and longitude information. Latitude values are between +90 (North) to -90 (South). Longitude values are between +180 (West) to -180 (East).	Filtering data by using a geospatial map.

• Geospatial custom metadata is supported only in Index.

• Geospatial custom metadata can be applied to all data unit types and on individual frames in videos.

Incorrectly specifying a data type in the schema can cause errors when filtering in Index or Active. To fix this, check your schema, correct any issues, re-import it, and re-sync your Active project.

Deprecated

While still supported, we STRONGLY RECOMMEND that you use the latest metadata schema instructions.

ENUM Name	String Representation	Description	Display Benefits
NUMBER	number	Numeric data type supporting float values	Filtering by numeric values
STRING	string	Textual data type	Filtering by string
BOOLEAN	boolean	Binary data type with values “true” or “false”	Filtering by binary values
DATETIME	datetime	ISO 8601 formatted date and time	Filtering by time and date
ENUM	enum	Enumerated type with predefined set of values	Facilitates categorical filtering and data validation
EMBEDDING	embedding	1 to 4096 for Index. 1 to 2000 for Active.	Filtering by embeddings, similarity search, 2D scatter plot visualization (Coming Soon)
LONG_STRING	long_string	Text data with unlimited length (example: transcripts for audio)	Storing and filtering large amounts of text

Incorrectly specifying a data type in the schema can cause errors when filtering your data in Index or Active. If you encounter errors while filtering, verify your schema is correct. If your schema has errors, correct the errors, re-import the schema, and then re-sync your Active Project.

---

Import Your Metadata Schema to Encord

Import schema template

# Import dependencies
from encord import EncordUserClient
from encord.metadata_schema import MetadataSchema

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Create the schema
metadata_schema = user_client.metadata_schema()

# Add various metadata fields
metadata_schema.add_scalar("metadata_1", data_type="boolean")
metadata_schema.add_scalar("metadata_2", data_type="datetime")
metadata_schema.add_scalar("metadata_3", data_type="number")
metadata_schema.add_scalar("metadata_4", data_type="uuid")
metadata_schema.add_scalar("metadata_5", data_type="varchar")
metadata_schema.add_scalar("metadata_6", data_type="text")

# Add an enum field
metadata_schema.add_enum("my-enum", values=["enum-value-01", "enum-value-02", "enum-value-03"])

# Add embedding fields
metadata_schema.add_embedding('my-test-active-embedding', size=512)
metadata_schema.add_embedding('my-test-index-embedding', size=<values-from-1-to-4096>)

# Add geospatial
metadata_schema.add_geospatial('geo-point')

# Save the schema
metadata_schema.save()

# Print the schema for verification
print(metadata_schema)

# Import dependencies
from encord import EncordUserClient
from encord.metadata_schema import MetadataSchema

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Create the schema
metadata_schema = user_client.metadata_schema()

# Add various metadata fields
metadata_schema.add_scalar("G2G", data_type="boolean")
metadata_schema.add_scalar("Date", data_type="datetime")
metadata_schema.add_scalar("Cost", data_type="number")
metadata_schema.add_scalar("Priority", data_type="number")
metadata_schema.add_scalar("ID-Sys-01", data_type="uuid")
metadata_schema.add_scalar("ID-Sys-02", data_type="uuid")
metadata_schema.add_scalar("Description", data_type="varchar")
metadata_schema.add_scalar("Location", data_type="varchar")
metadata_schema.add_scalar("Address", data_type="varchar")
metadata_schema.add_scalar("Translation en-gb", data_type="text")
metadata_schema.add_scalar("Translation en-ca", data_type="text")
metadata_schema.add_scalar("Translation en-us", data_type="text")
metadata_schema.add_scalar("Translation zh-tw", data_type="text")
metadata_schema.add_scalar("Translation zh-hk", data_type="text")
metadata_schema.add_scalar("Translation zh-cn", data_type="text")

# Add an enum field
metadata_schema.add_enum("ready", values=["ripe", "partially-ripe", "unripe"])
metadata_schema.add_enum("Fruit", values=["Blueberries", "Cherries", "Apples", "Kiwi"])

# Add embedding fields
metadata_schema.add_embedding("active-embedding-01", size=512)
metadata_schema.add_embedding("active-embedding-02", size=512)
metadata_schema.add_embedding("index-embedding-100", size=100)
metadata_schema.add_embedding("index-embedding-1000", size=1000)

# Add geospatial
metadata_schema.add_geospatial("Coordinates")

# Save the schema
metadata_schema.save()

# Print the schema for verification
print(metadata_schema)

---

Deprecated

This is supported, as is, we STRONGLY RECOMMEND that you use the most current method of creating your custom metadata schema.

Import schema template

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

user_client.set_client_metadata_schema_from_dict({'metadata_1': 'data type', 'metadata_2': 'data type', 'metadata_3': 'data type'})

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

user_client.set_client_metadata_schema_from_dict({'captured_at': 'datetime', 'city': 'datetime', 'dark': 'boolean'})

---

Verify Your Schema

After importing your schema to Encord we recommend that you verify that the import is successful. Run the following code to verify your metadata schema imported and that the schema is correct.

Verify schema template

# Import dependencies
from encord import EncordUserClient
from encord.metadata_schema import MetadataSchema

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Create the schema
metadata_schema = user_client.metadata_schema()

# Print the schema for verification
print(metadata_schema)

# Import dependencies
from encord import EncordUserClient
from encord.metadata_schema import MetadataSchema

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Create the schema
metadata_schema = user_client.metadata_schema()

# Print the schema for verification
print(metadata_schema)

---

Deprecated

This is supported, as is, we STRONGLY RECOMMEND that you use the most current method of creating your custom metadata schema.

Verify your schema template

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

schema = user_client.get_client_metadata_schema()

print(schema)

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

schema = user_client.get_client_metadata_schema()

print(schema)

---

Edit Schema Keys

You can change the data type of schema keys using the .set_scalar() method. The example below shows how to update the data type for multiple metadata fields.

Restore Schema Key

# Import dependencies
from encord import EncordUserClient
from encord.metadata_schema import MetadataSchema

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Get your metadata schema
metadata_schema = user_client.metadata_schema()

# Edit various metadata fields
metadata_schema.set_scalar("metadata_1", data_type="number")
metadata_schema.set_scalar("metadata_2", data_type="boolean")
metadata_schema.set_scalar("metadata_3", data_type="boolean")

# Print the schema for verification
print(metadata_schema)

---

Delete Schema Keys

Schema key options cannot be deleted. Instead, we recommend creating new schema key options to meet your needs and phasing out any that are no longer required.

You can delete schema keys using the .delete() method.

There are two types of deletion: hard delete and soft delete. A hard delete permanently removes the key, making it impossible to restore. A soft delete allows you to restore the key later using the .restore_key() method.

The following examples show hard delete and soft deletion of a schema key called Fruit.

Soft Delete

# Import dependencies
from encord import EncordUserClient
from encord.metadata_schema import MetadataSchema

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Get your metadata schema
metadata_schema = user_client.metadata_schema()

metadata_schema.delete_key(k: "Fruit", hard: False)

# Print the schema for verification
print(metadata_schema)

# Import dependencies
from encord import EncordUserClient
from encord.metadata_schema import MetadataSchema

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Get your metadata schema
metadata_schema = user_client.metadata_schema()

metadata_schema.delete_key(k: "Fruit", hard: True)

# Print the schema for verification
print(metadata_schema)

Restore Schema Keys

Keys that have been soft deleted can be restored using the .restore_key() method. The following example restores a schema key called Fruit.

Restore Schema Key

# Import dependencies
from encord import EncordUserClient
from encord.metadata_schema import MetadataSchema

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Get your metadata schema
metadata_schema = user_client.metadata_schema()

metadata_schema.restore_key(k: "Fruit")

# Print the schema for verification
print(metadata_schema)