Import Labels/Annotations to Consensus Branches

Learn how to import your labels/annotations to a Consensus branch.

Overview

To upload labels to a Consensus Project, you need to create a Consensus branch.

ALL Consensus branches start with encord-.

Supported Label/Annotation Formats

Encord Format (Recommended)

• Supports multi-level nested classifications (radio, checklist, or free-form text) under objects or classifications.
• Handles all object types and classification.

COCO Format

Does not supports multiple levels of nested classifications (radio, checklist, or free-form text) under tools or classifications.

Confidence Score

You can include confidence scores when uploading labels/annotations. Encord automatically calculates model metrics based on your label and prediction sets and assigned confidence scores.

Label Branches

When importing label/annotation sets into a Consensus Project, they are added as branches to individual label rows on your data units (images, videos, audio). Each data unit has the following:

• A MAIN branch for ground truth annotations or pre-labels.
• Optional Consensus branches and Prediction branches for different label/annotation or prediction sets.

You can export main branch labels from the UI. Consensus labels and model predictions can only be exported using the SDK.

Import Labels/Annotations to Consensus Projects

Import your labels to a Project in Annotate. Encord currently supports importing labels from the Encord format and from COCO.

Import Encord-Format Labels

Use branch_name to create a Consensus label branch in label_rows_v2 for a data unit.

• branch_name MUST start with encord- for Consensus branches.
• branch_name supports alphanumeric characters (a-z, A-Z, 0-9) and is case sensitive.
• branch_name supports the following special characters: hyphens (-), underscores (_), and periods (.).

Bounding Box

Example 1

Imports a single bounding box (Cherry) to a single image (cherry_001.png).

Example 2:

Imports three instances (tracking an object across three sequential frames: 103, 104, and 105) of a bounding box (Cherry) to a video (Cherries_video.mp4).

Example 3

Imports three bounding boxes (Cherry) to a single image (cherry_001.png).

Example 4:

Imports three instances (tracking 3 different objects across three frames: object 1 - frames 103, 104, 105, object 2 - frames 206, 207, 208, and object 3 - frames 313, 315, 317) of three bounding boxes (Cherry) to a video (Cherries_video.mp4).

Rotatable Bounding Box

Example 1

Imports a single rotatable bounding box (Other type of fruit) to a single image (apple_001.png).

Example 2

Imports three instances (tracking an object across three sequential frames: 120, 121, and 122) of a bounding box (Other type of fruit) to a video (Cherries_video.mp4).

Example 3

Imports three rotatable bounding boxes (Other type of fruit) to a single image (apple_001.png).

Example 4:

Imports three instances (tracking 3 different objects across three frames: object 1 - frames 120, 121, 122, object 2 - frames 222, 224, 226, and object 3 - frames 321, 323, 325) of three rotatable bounding boxes (Other type of fruit) to a video (Cherries_video.mp4).

Polygons

Example 1

Imports a single polygon (Persimmon) to a single image (persimmon_001.jpg).

Example 2

Imports three instances (tracking an object across three sequential frames: 143, 144, and 145) of a polygon (Persimmon) to a video (Cherries_video.mp4).

Example 3

Imports three polygons (Persimmon) to a single image (persimmon_001.jpg).

Example 4:

Imports three instances (tracking 3 different objects across three frames: object 1 - frames 153, 154, 155, object 2 - frames 242, 244, 246, and object 3 - frames 343, 345, 347) of three polygons (Persimmon) to a video (Cherries_video.mp4).

Polyline

Example 1

Imports a single polyline (Branch) to a single image (persimmon_001.jpg).

Example 2

Imports three instances (tracking an object across three sequential frames: 146, 147, and 148) of a polygon (Branch) to a video (Cherries_video.mp4).

Example 3

Imports three polylines (Branch) to a single image (persimmon_001.jpg).

Example 4:

Imports three instances (tracking 3 different objects across three frames: object 1 - frames 246, 247, 248, object 2 - frames 346, 347, 348, and object 3 - frames 446, 447, 448) of three polylines (Branch) to a video (Cherries_video.mp4).

Keypoint

Example 1

Imports a single keypoint (Pedicel) to a single image (blueberry_003.png).

Example 2

Imports three instances (tracking an object across three sequential frames: 143, 144, and 145) of a keypoint (Pedicel) to a video (Blueberries_video.mp4).

Example 3

Imports three keypoints (Pedicel) to a single image (blueberry_003.png).

Example 4:

Imports three instances (tracking 3 different objects across three frames: object 1 - frames 143, 144, 145, object 2 - frames 242, 244, 246, and object 3 - frames 343, 345, 347) of three keypoints (Pedicel) to a video (Blueberries_video.mp4).

Bitmask

Example 1:

Imports a single bitmask (Blueberry) to a single image (blueberry_003.jpg). For simplicity, the bitmask covers the entire image (image dimensions: 1254x836).

Example 2:

Imports three instances (tracking an object across three sequential frames: 156, 157, and 159) of a bitmask (Blueberry) to a video (Blueberries_video.mp4). For simplicity, the bitmask covers the entire frame (video dimensions: 1920x1080).

Example 3:

Imports three bitmasks (Blueberry) to a single image (blueberry_003.jpg). For simplicity, the bitmasks cover the entire image (image dimensions: 1254x836).

Example 4:

Imports three instances (tracking 3 different objects across three frames: object 1 - frames 156, 157, 158, object 2 - frames 256, 258, 259, and object 3 - frames 355, 357, 359) of three bitmasks (Blueberry) to a video (Blueberries_video.mp4). For simplicity, the bitmasks cover the entire frame (video dimensions: 1920x1080).

Object Primitives

Before you can import Object Primitive labels into Encord, the Object Primitive Template MUST exist in Encord. Use the UI to create the Object Primitive Template so you can visually inspect the Object Primitive.

Import Object Primitive labels

Example 1

Imports a single object primitive (Ontology object = Strawberry Object Primitive name = Triangle) to a single image (strawberries_10.jpg).

Example 2

Imports three instances (tracking an object across three sequential frames: 163, 164, and 165) of a object primitive (Ontology object = Strawberry Object Primitive name = Triangle) to a video (Cherries_video.mp4).

Example 3

Imports three object primitives (Ontology object = Strawberry Object Primitive name = Triangle) to a single image (strawberries_10.jpg).

Example 4

Imports three instances (tracking 3 different objects across three frames: object 1 - frames 173, 174, 175, object 2 - frames 183, 184, 185, and object 3 - frames 193, 194, 195) of three object primitives (Ontology object = Strawberry Object Primitive name = Triangle) to a video (Cherries_video.mp4).

Radio Button

Example 1:

Imports a radio button classification (Blueberry or Cherry?) to a single image (blueberry_003.jpg).

Example 2:

Imports a radio button classification (Blueberry or Cherry?) across a range of sequential frames: 193 to 197) to a video (Blueberries_video.mp4).

Checklist

Example 1:

Imports a checklist classification (Many types of fruit?) to a single image (apple_003.jpg). The selected items from the list are apple and kiwi.

Example 2:

Imports a checklist classification (Many types of fruit?) across a range of sequential frames: 193 to 197) to a video (Blueberries_video.mp4). The selected items from the list are apple and kiwi.

This simple example imports a bounding box model to all data units in the Consensus branch.

Store Labels Boilerplate

# Import dependencies
import os
from encord import EncordUserClient, Project
from encord.objects import LabelRowV2, Object, OntologyStructure, ObjectInstance

from encord.objects.coordinates import BoundingBoxCoordinates, RotatableBoundingBoxCoordinates, PolygonCoordinates, PolylineCoordinates, PointCoordinate, BitmaskCoordinates


# Configuration
SSH_PATH = "file-path-to-your-ssh-key"
PROJECT_HASH = "unique-id-for-project"

# Specify a label_rows_v2 branch name for your predictions.
CONSENSUS_BRANCH_NAME = "encord-<name-of-your-consensus-branch>"

assert SSH_PATH is not None, "SSH path cannot be None"
assert PROJECT_HASH is not None, "Project hash cannot be None"

# Authenticate with Encord
user_client = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Access the project and prepare the branch for predictions
project = user_client.get_project(PROJECT_HASH)
consensus_branch_rows = project.list_label_rows_v2(branch_name=CONSENSUS_BRANCH_NAME)

if len(consensus_branch_rows) > 0:
  print("Branch is:", consensus_branch_rows[0].branch_name)

ontology_object = project.ontology_structure.objects[0]
bundle = project.create_bundle()
for row in consensus_branch_rows:
    row.initialise_labels(bundle=bundle)
bundle.execute()

for row in consensus_branch_rows:
    inst = ontology_object.create_instance()
    inst.set_for_frames(
        coordinates=BoundingBoxCoordinates(
            height=0.8,
            width=0.8,
            top_left_x=0.1,
            top_left_y=0.1,
        ),
        # Add the bounding box to the first frame
        frames=0,
        # There are multiple additional fields that can be set optionally:
        manual_annotation=False,
    )
    row.add_object_instance(inst)
    row.save(bundle=bundle)
bundle.execute()

Import COCO Labels to Consensus Branches

The following code imports COCO labels as predictions for Active.

For more information on importing COCO labels into Encord, refer to our documentation.

Replace the following:

• <private_key_path> with the file path to your SSH private key.

• encord-<name-of-your-consensus-branch> with the name of your label branch.

• <project_hash> with the Project ID for your Project.

• COCOimportfile.json with the full path of the COCO file containing the predictions you want to import.

COCO Label import as Predictions

import json
from pathlib import Path
from encord.utilities.coco.datastructure import FrameIndex
from encord import EncordUserClient
from encord.exceptions import OntologyError

# Authenticate client
SSH_PATH = "file-path-to-your-ssh-key"

# Specify a Project to import your predictions to. This Project must already exist in Encord.
PROJECT_HASH = "unique-id-for-project"

# Specify a label_rows_v2 branch name for your labels.
CONSENSUS_BRANCH_NAME = "encord-<name-of-your-consensus-branch>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Replace with your project hash
project = user_client.get_project(PROJECT_HASH)

# Load the COCO annotations JSON file
# Replace 'COCOimportfile.json' with the full path to your COCO file
coco_file = Path("COCOimportfile.json")
labels_dict = json.loads(coco_file.read_text())

# Build a mapping from COCO category IDs to the feature hashes in your Encord Ontology. 
category_id_to_feature_hash = {}
ont_struct = project.ontology_structure
for coco_category in labels_dict["categories"]:
    try:
        ont_obj = ont_struct.get_child_by_title(coco_category["name"])
        category_id_to_feature_hash[coco_category["id"]] = ont_obj.feature_node_hash
    except OntologyError:
        print(f"Could not match {coco_category['name']} in the Ontology. Import will crash if these are present.")

# Build a mapping from COCO image IDs to Encord frame indices
# This is only applicable for images, image groups, image sequences, videos, and DICOM series
image_id_to_frame_index = {}
data_title_to_label_row = {lr.data_title: lr for lr in project.list_label_rows_v2(branch_name=CONSENSUS_BRANCH_NAME)}
for img in labels_dict['images']:
    if "video_title" in img.keys():
        lr = data_title_to_label_row[img["video_title"]]
        frame_num = int(img["file_name"].split('/')[-1].split(".")[0])
    else:
        lr = data_title_to_label_row[img['image_title']]
        frame_num = 0

    # Creates a mapping between the COCO image IDs and the corresponding frame indices in Encord
    # In this example, the target frame is 0 because the files in the sample project are single images
    image_id_to_frame_index[img['id']] = FrameIndex(lr.data_hash, frame=frame_num)

# Import the COCO labels into Encord
project.import_coco_labels(
    labels_dict,
    category_id_to_feature_hash,
    image_id_to_frame_index,
    branch_name=CONSENSUS_BRANCH_NAME,
)

Verify Label Import

After importing your labels, verify that your labels imported.

The following code returns all labels and predictions on all branches.

# Import dependencies
from encord import EncordUserClient
import json

SSH_PATH = "file-path-of-your-ssh-key"
PROJECT_HASH = "unique-id-for-your-project"

# Instantiate client. Replace \<private_key_path> with the path to the file containing your private key.
user_client = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Specify Project. Replace \<project_hash> with the hash of the Project you want to export labels for.
project = user_client.get_project(PROJECT_HASH)

# Downloads a local copy of all the labels
# Without the include_all_label_branches flag only the MAIN branch labels export
label_rows = project.list_label_rows_v2(include_all_label_branches=True) 

for label_row in label_rows:
    # Here we have the label row for the branch, but without labels themselves downloaded
    print(f"Title: {label_row.data_title}, branch: {label_row.branch_name}")

    # And now we download the label content itself (bounding boxes and stuff)
    label_row.initialise_labels()

    # Print essential label information for all objects
    for object_instance in label_row.get_object_instances():
        print (f"objectHash: {object_instance.object_hash}")
        print (f"Object name: {object_instance.object_name}")
        print (f"featureHash: {object_instance.feature_hash}")
        print (f"uid: {object_instance.ontology_item.uid}")
        print (f"Object color: {object_instance.ontology_item.color}")
        print (f"Ontology shape: {object_instance.ontology_item.shape}")

        # Print the frame number and the location of the object on the frame
        for annotation in object_instance.get_annotations():
            print(f"Frame {annotation.frame} -> {annotation.coordinates}")

        # Print all attributes 
        for attribute in object_instance.ontology_item.attributes:
            print (attribute, object_instance)

    # Print all essential classification information
    for classification_instance in label_row.get_classification_instances():
        print (f"classificationHash: {classification_instance.classification_hash}")
        print (f"Classification name: {classification_instance.classification_name}")
        print (f"featureHash: {classification_instance.feature_hash}")
        print (f"Classification answer: {classification_instance.get_answer().value}")
        print (f"Classification answer hash: {classification_instance.get_answer().feature_node_hash}")


        # Print the frame number(s) that a classification appears on
        for annotation in classification_instance.get_annotations():
            print(f"Classification appears on frame: {annotation.frame}")