Text / HTML Annotation

This guide assumes your text files are stored in AWS. To learn how to create integrations with other cloud providers click here. To learn how to upload files locally click here.

STEP 1: Register Your Text Files

1. Set Up AWS

   Before you can do anything with the Encord platform and cloud storage, you need to configure your cloud storage to work with Encord. Once the integration between Encord and your cloud storage is complete, you can then use your data in Encord.

   In order to integrate with AWS S3, you need to:

   1. Create a permission policy for your resources that allows appropriate access to Encord.
   2. Create a role for Encord and attach the policy so that Encord can access those resources.
   3. Activate Cross-origin resource sharing which allows Encord to access those resources from a web browser.
   4. Test the integration to make sure it works.

   See our AWS integration documentation for a detailed explanation of setting up AWS to work with Encord.

   You have the following options to integrate AWS and Encord:

   Use AWS UI

   Use Terraform

2. Create AWS Integration in Encord

   Create an S3 bucket to store your files if you haven’t already. Your S3 bucket permissions should be set to be blocking all public access.

   In the Integrations section of the Encord platform, click +New integration to create a new integration.

   

   Select AWS S3 at the top of the chooser.

   

   It is essential you do not close this tab or window until you have finished the whole integration process. If you use the AWS UI for integration, we advise opening the AWS console in a separate tab.

   See our AWS integration documentation for a detailed explanation of how to set up the AWS integration.
3. Create JSON file for Registration

   See our documentation here for more comprehensive information

   Create a JSON file based on the templates provided below.

   The title field is optional. If omitted, the file path and file name are used as the default title. For example, if the file is located at https://encord-solutions-bucket.s3.eu-west-2.amazonaws.com/path/to/my/bucket/website.html, the title defaults to /path/to/my/bucket/website.html.

   Key or Flag	Required?	Default value
   ”objectUrl”	Yes
   ”title”	No	The file’s path + title
   ”clientMetadata”	No

   Keys / Flags that are not required can be omitted from the JSON file entirely.

   Standard integration

   {
   "text": [
       {
       "objectUrl": "/path/to/my/bucket/website1.html",
       "clientMetadata": {"htmlType": "Single Extension"}
       },
       {
       "objectUrl": "/path/to/my/bucket/website2.html",
       "clientMetadata": {"htmlType": "Raw"}
       }
     ]
   }
   

4. Create a Folder to Store Your Audio Files

   All files in Encord must be stored within folders. Therefore, you need to create a folder before registering any data with Encord. To create a folder:

   1. Navigate to Files under the Index heading in the Encord platform.
   2. Click the + New folder button to create a new folder. Select the type of folder you want to create.

   

   3. Give the folder a meaningful name and description.

   4. Click Create to create the folder. The folder is listed in Files.

5. Upload Your Text files to Encord

   To learn how to upload local files click here

   To ensure smoother uploads and faster completion times, and avoid hitting absolute file limits, we recommend adding smaller batches of data. Limit uploads to 100 videos or up to 1,000 images at a time. You can also create multiple Datasets, all of which can be linked to a single Project. Familiarize yourself with our limits and best practices for data import/registration before adding data to Encord.

   1. Navigate to Files section of Index in the Encord platform.
   2. Click into a Folder.
   3. Click + Upload files. A dialog appears.

   

   4. Click Import from cloud data.

   

   We recommend turning on the Ignore individual file errors feature. This ensures that individual file errors do not lead to the whole upload process being aborted.

   6. Click Add JSON or CSV files to add a JSON or CSV file specifying cloud data that is to be added.

STEP 2: Set Up Your Project

1. Create a Dataset

   1. Click the New dataset button in the Datasets section in Annotate.

   

   2. Give your Dataset a meaningful title and description. A clear title and description keeps your data organized.

   Toggle Looking to create a mirrored dataset? to create a Mirrored Dataset.

   

   3. Click Create dataset to create the Dataset.

   Attach files

   You can add data to Datasets from the Files or the Datasets screens.

   To ensure smoother uploads and faster completion times, and avoid hitting absolute file limits, we recommend adding smaller batches of data. Limit uploads to 100 videos or up to 1,000 images at a time. You can also create multiple Datasets, all of which can be linked to a single Project. Familiarize yourself with our limits and best practices for data import/registration before adding data to Encord.

   1. Navigate to the Datasets section under the Annotate heading.
   2. Click the Dataset you want to attach data to.
   3. Click +Attach existing files.

   If the files you want have not been uploaded into Encord yet, click +Upload files to upload new files.

   

   4. Select the folders containing the files you want to attach to the Dataset. To select individual files, double-click a folder to see its contents, and select the files you want to add to the Dataset.

   5. Click Attach data to attach the selected files to the Dataset.

2. Create an Ontology

   We recommend learning about Ontology structure before creating Ontologies.

   Ensure that Ontologies used for text annotations only include Classifications and Text Region objects

   1. Click the New ontology button in the Ontologies section to create a new Ontology.

   

   2. Give your Ontology a meaningful title and description. A clear title and description keeps your Ontologies organized.

   3. Click Next to continue.

   

   4. Define your Ontology structure. Text files are labeled using the Text Region object type and Classifications.

   To add a Text Region object:

   

   1. Click + Add object to create a new object.
   2. Give the object a name, for example “Region of Interest”.
   3. Select the Text Region object type.
   4. Optionally, configure the object to add any attributes.
   5. Optionally, enable the Required toggle to mark the object as Required.
   6. Repeat these steps for as many Text Region objects as necessary.

   Configure Objects:

   Objects can be configured to add attributes.

   

   1. Click the arrow button next to the object to start configuring.
   2. Click + Add attribute to add an attribute.
   3. Select the attribute type: Text, radio button, or checklist.
   4. Give the attribute a name, for example “Why is this region of interest?” could be a text attribute.
   5. If your attribute is a radio button or a checklist click Add option and add options.

   

   6. Optionally mark the object as Required and / or Dynamic.

   7. Click the Back to parent button to return the Ontology creation view.

   To add a Classification:

   

   1. Click + Add classification to create a new classification.
   2. Give the classification a name. For example, “HTML Type”.
   3. Optionally, configure the classification to change the classification type. The default classification type is a text field.
   4. Optionally, enable the Required toggle to mark the classification as Required.
   5. Repeat these steps for as many classifications as necessary.

   Configure classifications:

   Classifications can be configured to change the classification type, and to add classification options to radio buttons and check lists.

   1. Click the arrow icon next to an object to configure the classification.

   

   2. Click the classification type to change the classification type. The default classification type is a text field.
   3. Click Add option to add an option if you have selected a radio button or check list classification.
   4. Enter a name for the classification option. For example, the classification “HTML Type” can have the options “Raw”, and “Single Extension”.
   5. Click the Back to parent button to return the Ontology creation view.
3. Create a Project

   1. In the Encord platform, select Projects under Annotate.
   2. Click the + New annotation project button to create a new Project.

   

   3. Give the Project a meaningful title and description.

   

   If you are part of an Organization, an optional Project tags drop-down is visible. Project tags are useful for categorizing and finding your Projects. Select as many tags as are relevant for your Project.

   4. Click the Attach ontology button.

   5. Select the Ontology you created previously from the list using the Select button.

   

   6. Click OK to attach the Ontology to the Project.

   7. Click the Attach datasets button.

   8. Select the Dataset you created previously from the list using the Attach button.

   

   9. Click OK to attach the Dataset(s) to the Project.

   

   10. Click the Load from template button to use a Workflow template.

   

   11. Select the template you want to use and click Load template.

   12. Click Create project to finish creating the Project.

   

STEP 3: Label Your Text Files

Text

Labeling Text Files

Text

Text Region objects must be selected at the top of the editor. Only classifications are visible in the left side bar

1. Highlight the text to label.
2. Click a Text Region label type at the top of the Editor (or use the hotkey) to apply the label.

Labeling JSON Files

JSON

Text Region objects must be selected at the top of the editor. Only classifications are visible in the left side bar

1. Highlight the text to label.
2. Click a Text Region label type at the top of the Editor (or use the hotkey) to apply the label.

Labeling XML Files

XML

Text Region objects must be selected at the top of the editor. Only classifications are visible in the left side bar

1. Highlight the text to label.
2. Click a Text Region label type at the top of the Editor (or use the hotkey) to apply the label.

HTML

Encord supports both raw HTML files and single-extension HTML files. The key difference is that single-extension HTML files include all the necessary elements to render the webpage, such as CSS and JavaScript.

Labeling HTML - Single Extension

HTML - Single Extension

1. Click a Text Region label type (or use the hotkey)
2. Highlight the text you want to apply the label to.

Labeling HTML - Raw

HTML - Raw

1. Click a Text Region label type (or use the hotkey)
2. Highlight the text you want to apply the label to.

Using Relation Text Attributes - Text and HTML

Relation text attributes link an object to another object. In images or videos this could be linking a chicken’s body to its wing. In PDFs or text documents (text or HTML files) this could be linking words with measurements, or linking words with other words.

Your Ontology must MUST contain text attributes with the Relation feature enabled if you want to use relation text attributes.

While annotating, you can create a relationship between any object with a relation text attribute and any other object. After you create a relation between objects a workflow arrow points from the object with the relation text attribute to the other object.