Import/Register Cloud Data - UI - Cloud-synced - Images and Videos

Why do this?

Quick way to get going with Encord using cloud data.

If you are a little more tech savvy, register your cloud data using Cloud-synced Folders.

If you intend to use Encord at scale,

Pros and Cons

Pros	Cons
Simple way to get data into Encord Able to sync your cloud data with Encord easily	Requires a little bit of technical knowledge to set integrations No data management or curation (custom metadata needs to be imported separately)

Import/Register Data

We’re going to register our dataset of fruit by connecting the cloud storage directly to a Cloud-synced Folder. The dataset is a mixture of images and videos of the following types of fruit:

• Apples
• Bananas
• Blueberries
• Cherries
• Kiwi
• Persimmons
• Strawberries

1. Create Integration

   Select your cloud provider.

   AWS Integration

   GCP Integration

   Azure Integration

   OTC Integration

   Wasabi Integration

   Oracle Integration

   Direct Access Integration

2. Download Data

   Download and extract the contents of Fruit-images-videos.zip file.

3. Import Data to Cloud Storage

   Import the contents of Fruit-images-videos.zip into your cloud storage.

4. Create Cloud-synced Folder

   Syncing the data registers the data in Encord. You data stays in your cloud storage.

   

   1. Go to Index > Files.

   2. Click New folder > Cloud-synced folder. The New Cloud-synced folder dialog appears.

   3. Provide the following:

      • Title: E2E - Fruit - Images and Videos - Cloud-synced Folders.
      • Description: OPTIONAL - Provide a meaningful description for the Cloud-synced folder.
      • Select your integration: Select the integration to use from the drop down.
      • Storage path: Specify the storage/file path to your cloud storage. For example: gs://encord-gcp-bucket/CloudSync/ or s3://encord-aws-bucket/CloudSync.

   4. Click Test to verify that Encord can communicate with your cloud storage.

   5. Click Create. The page for the new Cloud-synced folder appears.

   Find Storage Path

   Finding the Storage path for your folder or object varies across Cloud Storage platforms.

   AWS

   

   GCP

   

5. Sync Data Between Encord and Cloud Storage

   1. Go to Index > Files > E2E - Fruit - Images and Videos - Cloud-synced Folders. The Cloud-synced folder page appears.

      

   2. Click Initiate sync. The sync between the folder and your cloud storage begins.

6. Re-encode Videos

   We strongly recommend re-encoding any imported videos with issues. Re-encoding your videos ensures the best performance when annotating your data.

   

7. Create Project

   Once all the videos are re-encoded you are ready to create an Annotate Project. Once you create a Project and you and your team are ready to begin annotating the data.

   Performing ANY of these tasks results in creating the Project E2E Fruit Project to annotate.

   • The Ontology is the same for ALL tasks.
   • The Workflow varies depending on the task.

   

   Create Project - UI

   Create Projects (Ontologies, Workflows, and Projects) using the Encord UI.

   

   Create Project - SDK

   Create Projects (Ontologies, Workflows, and Projects) using the SDK.

   

   Create Consensus Project - UI

   Create Consensus Projects (Ontologies, Workflows, and Projects) using the UI.

   

   Create Consensus Project - SDK

   Create Consensus Projects (Ontologies, Workflows, and Projects) using the UI.

   

   Create Project - Import Labels

   Create Projects (Ontologies, Workflows, and Projects) for use with your own labels.

   

   Create Project - Import Predictions

   Create Projects (Ontologies, Workflows, and Projects) for use with your own model predictions.

   

   Create Project - Task Agents

   Create Projects (Ontologies, Workflows, and Projects) for use with Task Agents.