Scene SDK Workflows

The Scene SDK (encord.beta.scene) is the recommended way to register point clouds and synchronized sensor streams in Encord. It lets you build Scenes from point cloud, image, camera, and frame-of-reference streams without writing raw Scene JSON.

Use this guide for the workflow. Use the Scene SDK reference for method-level details.

Build and Upload a Scene

Build the Scene with SceneBuilder, then register it into Files & Folders with StorageFolder.upload_scene.

Build and Upload a Scene

from uuid import UUID

from encord.beta.scene import SceneBuilder, identity_pose, intrinsics_pinhole
from encord.storage import StorageFolder


def build_scene() -> SceneBuilder:
    scene = SceneBuilder()

    # Anchor the sensor streams to the moving platform.
    ego = scene.add_for_stream("ego")
    ego.add_pose(identity_pose(), timestamp=0)

    # Matching timestamps make the point cloud and image appear together.
    scene.add_pcd_stream("lidar", frame_of_reference=ego).add_pcd(
        uri="s3://my-bucket/scenes/scene-001/lidar/000000.pcd",
        timestamp=0,
    )

    # Inline camera parameters create a camera stream named "front/camera".
    scene.add_image_stream(
        "front",
        width=1920,
        height=1080,
        intrinsics=intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
        frame_of_reference=ego,
    ).add_image(
        uri="s3://my-bucket/scenes/scene-001/front/000000.jpg",
        timestamp=0,
    )

    return scene


def register_scene(storage_folder: StorageFolder, integration_id: str) -> UUID:
    scene = build_scene()

    # Pass the SceneBuilder itself. upload_scene validates and serializes it internally.
    return storage_folder.upload_scene(
        scene=scene,
        title="scene-001",
        integration_id=integration_id,
        client_metadata={"source": "sample"},
    )

upload_scene returns the new Scene item UUID. Add that item to a Dataset before creating an annotation Project.

Add Multi-frame Sensor Streams

Create stream builders once, then append events. Use the same timestamp for events that should appear together in the editor.

Multi-frame Scene Streams

from encord.beta.scene import Direction, SceneBuilder, intrinsics_pinhole, translation_only


def build_multiframe_scene(frame_count: int = 3) -> SceneBuilder:
    scene = SceneBuilder()
    scene.set_world_convention(x=Direction.FORWARD, y=Direction.LEFT, z=Direction.UP)
    scene.set_camera_convention(x=Direction.RIGHT, y=Direction.DOWN, z=Direction.FORWARD)

    ego = scene.add_for_stream("ego")
    lidar = scene.add_pcd_stream(
        "lidar",
        frame_of_reference=ego,
        pose=translation_only(x=0.0, y=0.0, z=1.8),
    )
    front = scene.add_image_stream(
        "front",
        width=1920,
        height=1080,
        intrinsics=intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
        frame_of_reference=ego,
        pose=translation_only(x=1.2, y=0.0, z=1.5),
    )

    for frame in range(frame_count):
        ego.add_pose(translation_only(x=frame * 0.5, y=0.0, z=0.0), timestamp=frame)
        lidar.add_pcd(
            uri=f"s3://my-bucket/scenes/scene-001/lidar/{frame:06d}.pcd",
            timestamp=frame,
        )
        front.add_image(
            uri=f"s3://my-bucket/scenes/scene-001/front/{frame:06d}.jpg",
            timestamp=frame,
        )

    return scene

Use integer timestamps for frame-indexed data. Use sensor timestamps when streams are sampled at different rates.

Coordinate Conventions

Set coordinate conventions in the SDK when your data uses known world and camera axes. For example, driving datasets often use a world convention such as forward, left, up and camera axes such as right, down, forward.

Use SceneBuilder.set_world_convention and SceneBuilder.set_camera_convention with Direction values. The world and camera conventions must have matching handedness.

Scene View Settings

SceneViewSettings holds per-Scene rendering settings that the label editor applies when the Scene is opened from the Editor or Explorer view. Unset fields fall through to the annotator’s own preferences.

• point_cloud_colouring — one of SceneHeightColouring, SceneProvidedColouring, SceneSolidColouring, SceneImageColouring, or SceneSensorColouring. SceneHeightColouring takes color_height_bounds, the minimum and maximum heights used for the color gradient.
• point_radius — radius used to render point cloud points, from 0 to 100.
• render_image_outside_base_range — Render the full image captured by the camera. Useful for wide-angle and fisheye lenses.
• radius_indicators — a list of SceneRadiusIndicator, each with the frame_of_reference_id at its center, a radius in scene units, and a hex color.
• radius_filter_enabled — whether point clouds are filtered to the configured radius indicators.

Assign the settings to SceneBuilder.settings before upload, patch a registered Scene with StorageItem.update(scene_view_settings=...), or read them from SceneReader.view_settings:

Configure Scene View Settings

from uuid import UUID

from encord import EncordUserClient
from encord.beta.scene import (
    SceneBuilder,
    SceneHeightColouring,
    SceneProvidedColouring,
    SceneRadiusIndicator,
    SceneReader,
    SceneViewSettings,
    identity_pose,
)
from encord.orm.storage import DataUploadItems
from encord.storage import StorageFolder, StorageItem


# --- Configuration ---
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"
FOLDER_ID = "00000000-0000-0000-0000-000000000000"
TARGET_FOLDER_ID = "00000000-0000-0000-0000-000000000000"
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"


def register_scene_with_view_settings(
    storage_folder: StorageFolder,
    integration_id: str,
) -> UUID:
    """Create and upload a Scene with initial view settings."""

    scene = SceneBuilder()

    ego = scene.add_for_stream("ego")
    ego.add_pose(identity_pose(), timestamp=0)

    scene.add_pcd_stream(
        "lidar",
        frame_of_reference=ego,
    ).add_pcd(
        uri="s3://my-bucket/scenes/scene-001/lidar/000000.pcd",
        timestamp=0,
    )

    scene.settings = SceneViewSettings(
        point_cloud_colouring=SceneHeightColouring(
            color_height_bounds=(-1.5, 4.0),
        ),
        point_radius=2.0,
        radius_filter_enabled=True,
        radius_indicators=[
            SceneRadiusIndicator(
                frame_of_reference_id="ego",
                radius=3,
                color="#FF8A00",
            ),
            SceneRadiusIndicator(
                frame_of_reference_id="root",
                radius=4,
                color="#3366FF80",
            ),
        ],
    )

    return storage_folder.upload_scene(
        scene=scene,
        title="scene-001",
        integration_id=integration_id,
    )


def update_scene_view_settings(scene_item: StorageItem) -> None:
    """Update the persisted view settings of an existing Scene."""

    reader = SceneReader(scene_item)

    current_settings = reader.view_settings or SceneViewSettings()

    current_settings.point_radius = 5.0

    scene_item.update(
        scene_view_settings=current_settings,
    )


def copy_scene_with_new_view_settings(
    scene_item: StorageItem,
    target_storage_folder: StorageFolder,
    *,
    integration_id: str,
    title: str,
) -> UUID:
    """Copy a Scene with different view settings."""

    reader = SceneReader(scene_item)

    new_settings = reader.view_settings or SceneViewSettings()

    new_settings.point_cloud_colouring = SceneProvidedColouring(
        display_point_intensity=False,
        srgb_colors=True,
    )

    reader.view_settings = new_settings

    # to_upload_payload() includes the Scene's view settings.
    upload_payload = reader.to_upload_payload(title=title)

    return target_storage_folder.add_private_data_to_folder_start(
        integration_id=integration_id,
        private_files=DataUploadItems(
            scenes=[upload_payload],
        ),
    )


# --- Connect to Encord ---

user_client: EncordUserClient = (
    EncordUserClient.create_with_ssh_private_key(
        ssh_private_key_path=SSH_PATH,
        # For US platform users, use:
        # domain="https://api.us.encord.com",
        domain="https://api.encord.com",
    )
)

folder = user_client.get_storage_folder(FOLDER_ID)
target_folder = user_client.get_storage_folder(TARGET_FOLDER_ID)


# --- Create a Scene with view settings ---

scene_uuid = register_scene_with_view_settings(
    folder,
    INTEGRATION_ID,
)

scene_item = user_client.get_storage_item(scene_uuid)


# --- Read the Scene's view settings ---

reader = SceneReader(scene_item)
print(f"Initial view settings: {reader.view_settings}")


# --- Update the stored Scene's view settings ---

update_scene_view_settings(scene_item)

reader = SceneReader(scene_item)
print(f"Updated view settings: {reader.view_settings}")


# --- Copy the Scene with different view settings ---

copy_job_uuid = copy_scene_with_new_view_settings(
    scene_item,
    target_folder,
    integration_id=INTEGRATION_ID,
    title="scene-001-copy",
)

print(f"Started copy job with UUID: {copy_job_uuid}")

frame_of_reference_id must be "root" for the world frame, or the name of a frame-of-reference stream in the same Scene.

Read and Copy Registered Scenes

SceneReader fetches a registered Scene and returns signed URLs for its constituent files. It can also convert a registered Scene back into a DataUploadScene for migration or copying.

Read and Copy a Scene

from uuid import UUID

from encord.beta.scene import SceneReader
from encord.orm.storage import DataUploadItems, StorageItemType
from encord.storage import StorageFolder, StorageItem


def find_scene(storage_folder: StorageFolder, title: str) -> StorageItem:
    scene_items = storage_folder.list_items(
        search=title,
        item_types=[StorageItemType.SCENE],
        page_size=1,
    )
    scene_item = next(iter(scene_items), None)
    if scene_item is None:
        raise ValueError(f"Scene with title '{title}' not found.")
    return scene_item


def get_lidar_signed_url(scene_item: StorageItem, timestamp: float = 0) -> str:
    scene = SceneReader(scene_item).read()
    lidar = scene.get_stream("lidar", kind="point_cloud")
    return lidar.get_event(timestamp=timestamp).signed_url


def get_image_signed_urls(scene_item: StorageItem, timestamp: float = 0) -> list[tuple[str, str]]:
    scene = SceneReader(scene_item).read()
    return [(stream_id, event.signed_url) for stream_id, event in scene.get_images_at_timestamp(timestamp)]


def copy_scene_to_folder(
    scene_item: StorageItem,
    target_storage_folder: StorageFolder,
    *,
    integration_id: str,
    title: str,
    source_uri_prefix: str,
    target_uri_prefix: str,
) -> UUID:
    payload = SceneReader(scene_item).to_upload_payload(
        title=title,
        uri_mapper=lambda uri: uri.replace(source_uri_prefix, target_uri_prefix),
    )

    return target_storage_folder.add_private_data_to_folder_start(
        integration_id=integration_id,
        private_files=DataUploadItems(scenes=[payload]),
    )

Use find_stream and find_event when a missing stream or timestamp is acceptable. Use get_stream and get_event when missing data should fail fast.

Camera Distortion Models

The supported distortion models are defined in encord.beta.scene.intrinsics.

SDK API	Model type	Distortion coefficients	Use when
intrinsics_pinhole(...)	pinhole	None	Use for calibrated pinhole cameras with no distortion.
intrinsics_radial(...)	radial	k1, k2, k3	Use when calibration only supplies radial coefficients.
intrinsics_plumb_bob(...)	plumb_bob	k1, k2, k3, t1, t2	Use for Brown-Conrady calibration with radial and tangential coefficients.
intrinsics_fisheye(...)	fisheye	k1, k2, k3, k4	Use for fisheye lenses calibrated with four coefficients.
intrinsics_rational_polynomial(...)	rational_polynomial	k1, k2, k3, k4, k5, k6, t1, t2	Use for OpenCV rational-polynomial calibration.
intrinsics_simple(..., model="division")	division	k	Use for one-parameter division-model calibration.
intrinsics_simple(..., model="ucm")	ucm	xi, k1, k2, k3	Use for unified camera model calibration.
intrinsics_cylindrical(...)	cylindrical	None	Use for cylindrical camera projections.

Camera Distortion Models

from encord.beta.scene import (
    SimpleIntrinsics,
    intrinsics_cylindrical,
    intrinsics_fisheye,
    intrinsics_pinhole,
    intrinsics_plumb_bob,
    intrinsics_radial,
    intrinsics_rational_polynomial,
    intrinsics_simple,
)


def build_distortion_model_examples() -> dict[str, SimpleIntrinsics]:
    return {
        "pinhole": intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        "radial": intrinsics_radial(fx=1000, fy=1000, ox=960, oy=540, k1=0.1, k2=-0.01, k3=0.001),
        "plumb_bob": intrinsics_plumb_bob(
            fx=1000,
            fy=1000,
            ox=960,
            oy=540,
            k1=0.1,
            k2=-0.01,
            k3=0.001,
            t1=0.0001,
            t2=-0.0001,
        ),
        "fisheye": intrinsics_fisheye(
            fx=1000,
            fy=1000,
            ox=960,
            oy=540,
            k1=0.1,
            k2=-0.01,
            k3=0.001,
            k4=-0.0001,
        ),
        "rational_polynomial": intrinsics_rational_polynomial(
            fx=1000,
            fy=1000,
            ox=960,
            oy=540,
            k1=0.1,
            k2=-0.01,
            k3=0.001,
            k4=-0.0001,
            k5=0.00001,
            k6=-0.000001,
            t1=0.0001,
            t2=-0.0001,
        ),
        "division": intrinsics_simple(fx=1000, fy=1000, ox=960, oy=540, model="division", k=0.01),
        "ucm": intrinsics_simple(fx=1000, fy=1000, ox=960, oy=540, model="ucm", xi=0.5, k1=0.1, k2=0, k3=0),
        "cylindrical": intrinsics_cylindrical(fx=1000, fy=1000, ox=960, oy=540),
    }

For most workflows, use intrinsics_pinhole or a dedicated distortion constructor. Use intrinsics_advanced only when you already have full calibration matrices and need to supply them directly.

Validation Checklist

Before handing Scenes to annotators:

• Confirm every point cloud and image URI is accessible by the integration used for registration.
• Keep stream names stable and descriptive, such as lidar, front, rear, or ego.
• Use integer timestamps for frame-indexed data unless you need sensor-time alignment.
• Confirm every SceneRadiusIndicator names "root" or a frame-of-reference stream the Scene defines. Upload fails otherwise.
• Open at least one registered Scene in the editor to verify orientation, calibration, and frame navigation.

Create and Register Scenes

Learn the Scene data model and registration choices.