SAM 2 Segmentation
Availability
SAM 2 supports labeling the following Ontology shapes.Ontology Shape | Supported? |
---|---|
Polygon | ✅ |
Bounding box | ✅ |
Bitmask | ✅ |
Rotatable bounding box | ❌ |
Keypoint | ❌ |
Polyline | ❌ |
Object Primitive | ❌ |
Modality | Supported |
---|---|
Single Image | ✅ |
Image Group | ✅ |
Image Sequence | ✅ |
Video | ✅ |
Audio | ❌ |
Text / HTML | ❌ |
Document | ❌ |
New Instances
Use the Shift + A keyboard shortcut to toggle SAM mode. Enabling SAM mode automatically activates SAM 2 whenever the class is selected, so there’s no need to click the wand icon separately.
- Click the wand icon on Ontology class you want to label with.
- Click, or click-and-drag the cursor across the part of the image you want to label. The SAM popup appears.
- Left-click parts of the image to include them in the label.
- Right-click parts of the selection to exclude them from the label.
- Click Save or press ENTER
Existing Instances
Videos and image sequences can show a single object instance in multiple frames.- Navigate to the frame you want to create the label.
- Click the Toggle auto-annotation button or use the SHIFT + A keyboard shortcut.
SAM mode remains active when switching to a different annotation class. Deactivate SAM 2 by clicking the Toggle auto-annotation button, or using the SHIFT + A keyboard shortcut again.
- Click, or click-and-drag the cursor across the part of the image you want to label. The SAM popup appears.
- Left-click parts of the image to include them in the label.
- Right-click parts of the selection to exclude them from the label.
- Click Save or press ENTER
SAM 2 Tracking
SAM 2 supports tracking objects forwards and backwards across multiple frames.SAM 2 tracking is computationally intensive and performance might be slower than other labeling actions.
Availability
The following table shows the Ontology shapes for which SAM 2 forwards and backwards tracking can be used.Ontology shape | SAM 2 Video Tracking |
---|---|
Bounding box | ✅ |
Rotatable bounding box | ✅ |
Polygon | ✅ |
Polyline | ✅ |
Object Primitive | ✅ |
Keypoint | ✅ |
Bitmask | ✅ |
Modalities | SAM 2 Video Tracking |
---|---|
Images | ❌ |
Videos | ✅ |
Image Groups | ✅ |
Image Sequences | ✅ |
DICOM | ✅ (Forwards only) |
Audio | ❌ |
Text / HTML | ❌ |
Documents | ❌ |
Track While Creating Labels (Bounding Boxes, Polygons, Bitmasks)
Before you begin, set your tracking range in the Editor Settings. By default, object tracking is configured to run for 30 frames, meaning SAM 2 tracking tracks the object forwards or backwards across 30 consecutive frames.- Click the wand icon next to the class you want to label, or press Shift + A to toggle SAM mode.
- Click the instance you want to label. Adjust your label until the correct region is selected by SAM 2 if needed.
- Specify the number of frames you want to track the instance across.
- Click Backwards or Forwards to start the SAM 2 tracking algorithm.
You can also use keyboard shortcuts to initiate SAM 2 tracking:
- Tracking forwards: SHIFT + T
- Tracking backwards: SHIFT + T + OPTION
- Click Save when tracking completes to save the generated labels.
Track Existing Instances
Method 1:The Automated labeling feature supports tracking forwards through data units. Specify frames after the existing frame to track.
- Click Automated labeling.
- Expand the Tracking and interpolation section.
- Select the instances you want to track.
- Make sure the Tracking method is selected.
- Set the tracking annotation interval. This defines how many consecutive frames an instance can be missing before the tracking algorithm stops.
- Specify the frame range you want to track the selected instances across.
- Click Run tracking to begin the process.
Method 2:
Before you begin, set your default tracking range in the Editor Settings. By default, object tracking is configured to run for 30 frames, meaning SAM 2 tracking tracks the object forwards or backwards across 30 consecutive frames.
- Right-click the instance label you want to start tracking.
-
Click Track forwards or Track backwards to run SAM 2 tracking for 30 consecutive frames.
You can also use keyboard shortcuts to initiate SAM 2 tracking:
- Tracking forwards: SHIFT + T
- Tracking backwards: SHIFT + T + OPTION
Tracking Multiple Objects
To track multiple objects of any shape:- Click one of the objects you want to track.
- Hold Shift on your keyboard and select all the other objects you want to track.
- Right-click on one of the selected objects and click Track forwards all or Track backwards all.
You can also use keyboard shortcuts to initiate SAM 2 tracking on multiple object instances:
- Track forwards all: SHIFT + T
- Track backwards all: SHIFT + T + OPTION