Merlin - BETA

Merlin is Encord’s agentic layer. Use natural language in Encord or a connected AI tool to create projects, define ontologies, check analytics, and more.

Merlin is currently in BETA.

What Merlin can do

Once connected, your AI tools can take actions on your behalf across the Encord platform:

Category	What you can ask
Projects	List your Projects, create new annotation Projects with a chosen Ontology, Dataset (or storage folder), and Workflow, attach an Ontology to an empty Project, and change a Project’s status
Ontologies	List, create, read, and edit Ontologies — objects, classifications, and nested attributes
Datasets	Browse and search your Datasets
Folders & storage	List and create storage folders (including cloud-synced folders), list your cloud integrations, and link a folder to an existing Project
Workflows	List Workflow templates; read and edit a Project’s live Workflow graph; view live per-stage task counts; list the tasks in a stage; set or randomize task priorities
Editor layouts	Configure, read back, and update custom annotation editor layouts for evaluation, rating, and scoring tasks
Analytics	Query task throughput, time spent, per-task metrics, and collaborator performance; label quality and label/comment activity broken down by ontology class; plus organization-wide summaries
Documentation	Search the Encord documentation and get grounded, cited answers to “how do I…” questions
Workspaces	List the Workspaces you can access and choose which one to work in
Feedback	File a feature request or bug report directly from a Merlin conversation

Use Merlin in Encord

When Merlin is enabled for your organization, you can start a conversation from several places in Encord:

• Enter a question in the Merlin box on the home page, or select a suggested prompt.
• Select Merlin near the bottom of the navigation sidebar. On the new-chat page, write your own question or choose a starter prompt. The chat sidebar lets you start another chat or return to a previous conversation.
• Open global search and select Ask Merlin. If you entered a search term, Merlin opens a conversation and searches for it across your Projects, data, and documentation.

You can continue the conversation in Merlin after entering from the home page or global search.

Prerequisites

• An Encord account (cloud). Merlin uses OAuth via WorkOS and is not available on on-premises or air-gapped deployments.
• To use Merlin outside Encord, a supported MCP client (see Connect your AI tools below).

Chat with Merlin in Encord

Select Merlin in the Encord navigation, then select New chat or open a previous conversation. Enter a question in the composer, or select one of the suggested questions to begin.

Type @ in the composer to search for and mention a Project, Ontology, Dataset, or storage folder that you can access. For Projects, Ontologies, and Datasets, the picker includes all resources of that type in the Workspace if your Workspace role lets you list or manage them all; otherwise, it includes only resources that you have joined. You can search for folders at any level of the folder hierarchy. Folder results show their parent path so you can distinguish folders with the same name; hidden folders do not appear. For example, ask Merlin to link @Folder to @Project or list the files in @Folder, then select each resource from the picker.

Resources without a name appear in the picker and conversation as Unnamed project, Unnamed ontology, Unnamed dataset, or Unnamed folder.

You can switch chats or navigate to another page while Merlin is responding. The response continues in the background, and you can return to the conversation to see it streaming or completed. You can also start responses in multiple chats and let them run at the same time. Reloading or closing the browser tab ends in-progress responses. Selecting Stop or deleting a conversation also ends its in-progress response.

The icon beside a chat can also indicate that Merlin is waiting for your approval or that the chat contains an unsent draft. Hover over the icon to see its status. Each chat shows one icon: working takes precedence over waiting for approval, which takes precedence over an unsent draft. Approval status appears only for chats opened or streamed in the same browser tab.

Merlin saves unsent text in your browser, so you can switch chats, navigate to another page, or reload and return to your draft. A pencil icon marks conversations with unsent text and also appears beside New chat when its composer contains a draft. Text entered in the home-page composer is also available when you open a new chat. Sending the question clears the draft. You can switch chats or navigate to another page while Merlin is responding. The response continues in the background, and you can return to the conversation to see it streaming or completed. You can also start responses in multiple chats and let them run at the same time. Reloading or closing the browser tab ends in-progress responses. Selecting Stop or deleting a conversation also ends its in-progress response.

Merlin saves unsent text in your browser, so you can switch chats, navigate to another page, or reload and return to your draft. The Chats list marks conversations with unsent text as Draft. Text entered in the home-page composer is also available when you open a new chat. Sending the question clears the draft.

The Merlin navigation item is visible only when Merlin is enabled for your organization.

Connect your AI tools

Choose your region’s endpoint. Merlin is available in two regions. Use the URL that matches where your Encord organization’s data is hosted:

• EU: https://mcp.encord.com/mcp
• US: https://mcp.us.encord.com/mcp

Claude Code

Run the following command in your terminal:

EU

claude mcp add --transport http encord-merlin https://mcp.encord.com/mcp

The first time you call a Merlin tool, Claude Code opens a browser to sign in with your Encord account.

Claude Desktop

For Team/Enterprise plans:

1. An organization admin must first add the connector in Admin settings > Connectors with the URL https://mcp.encord.com/mcp or https://mcp.us.encord.com/mcp.
2. Members then add the same connector under Settings > Connectors.
3. Restart Claude Desktop for the changes to take effect.

For Individual plans:

Add the server to your claude_desktop_config.json:

EU

{
  "mcpServers": {
    "encord-merlin": {
      "type": "http",
      "url": "https://mcp.encord.com/mcp"
    }
  }
}

Claude Web

For Team/Enterprise plans:

1. An organization admin must first add the connector in Admin settings > Connectors with the URL https://mcp.encord.com/mcp or https://mcp.us.encord.com/mcp.
2. Members then add the same connector under Settings > Connectors.
3. Refresh claude.ai for the changes to take effect.

For Individual plans:

1. Go to claude.ai/customize/connectors.
2. Click Add custom connector.
3. Paste the server URL above and confirm: https://mcp.encord.com/mcp or https://mcp.us.encord.com/mcp.
4. Complete the browser sign-in to authorize access.

Cursor

Click here to configure automatically to add Merlin to Cursor.

Or add the following to your Cursor MCP configuration:

EU

{
  "mcpServers": {
    "encord-merlin": {
      "type": "http",
      "url": "https://mcp.encord.com/mcp"
    }
  }
}

Codex

Run the following commands in your terminal:

EU

codex mcp add encord --url https://mcp.encord.com/mcp
codex mcp login encord

Other

Merlin uses the standard MCP streamable-HTTP transport. Point any MCP-compatible client to:

https://mcp.encord.com/mcp

The client runs an OAuth authorization flow against WorkOS on first use. Every WorkOS-backed login method (SSO, email, etc.) is supported.

Authentication

Merlin authenticates using OAuth. When your AI tools first call a Merlin tool, they open a browser window and prompt you to sign in with your Encord credentials. The session is managed by WorkOS. Merlin itself never stores your credentials.

You must sign in as an Encord user whose account was created (or backfilled) after Merlin launched. If your sign-in succeeds but tool calls are rejected, contact your workspace admin.

Org-level MFA enforcement is not yet supported for Merlin. Requests from organizations with mandatory MFA will be rejected.

Merlin with Workspaces

If you belong to only one Encord Workspace, Merlin automatically uses that Workspace. No additional setup is required.

If you belong to multiple Workspaces, Merlin needs to know which Workspace you want to work in before it can perform actions for you.

The first time you use Merlin, your AI tools may ask you to choose which Workspace to work in. Your AI tool then works within that Workspace for the rest of the conversation, until you ask it to switch.

To switch Workspaces later, simply tell your AI tools which Workspace you would like to use. Merlin lists your available Workspaces and confirms your selection, and your AI tool works within that Workspace from then on.

Example:

“Show me my available Workspaces.”

“Switch my active Workspace to Data Operations.”

Example prompts

Once connected, you can talk to your AI tools naturally. A few examples:

Setting up a new project end-to-end:

“Create an ontology for vehicle detection with bounding box classes for car, truck, and motorcycle. Then create a project called ‘Vehicle QA’ using that ontology and my existing dataset ‘Street Footage’.”

Checking project health:

“How many tasks were completed in my ‘Pedestrian Detection’ project last month, and who were the top three annotators by output?”

Exploring what’s available:

“List all my projects that are currently active and show me which ones I’m an admin on.”

Configuring an evaluation workflow:

“Set up a custom editor layout on project ‘RLHF Round 3’ that shows the media and a preference rating question side by side.”

Removing upload-order bias:

“Randomize the task order in my ‘Pairwise Model Comparison’ project so annotators aren’t biased by the order files were uploaded in.”

Checking queue state before reassigning work:

“How many tasks are currently sitting in the Review stage of my ‘Pedestrian Detection’ project?”

Available tools reference

Projects

Tool	Description
create_project_tool	Create an annotation project. Accepts a title, description, ontology, one or more datasets (or a storage folder), and an optional workflow template. Returns a direct link to the new project.
list_projects_tool	List your projects, most recently created first. Supports title search and pagination (50 per page).
set_project_ontology_tool	Attach an ontology to a project that does not yet have one. Switching the ontology of a project that already defines objects or classifications is not supported, to avoid orphaning existing labels. Requires admin or team-manager access.
update_project_status_tool	Change a project’s status — for example marking it in progress, paused, completed, cancelled, or archived. Requires admin or team-manager access.

Ontologies

Tool	Description
create_ontology_tool	Create a label taxonomy. Define objects (things to localize, with a shape) and classifications (frame-level questions). Attributes support radio, checklist, text, and numeric types, with radio options supporting up to three levels of nesting. If the Project will use a custom editor layout with an object table, keep object attributes to one level. The object table only renders an object’s top-level attributes as columns.
get_ontology_tool	Read an ontology’s current taxonomy so it can be edited. Call this before update_ontology_tool, modify the returned tree, then pass it back — keeping each node’s featureNodeHash unchanged so edits are recognized as the same node.
update_ontology_tool	Edit an existing Ontology. Add, modify, or remove objects, classifications, and attributes, instead of only creating new ones.
list_ontologies_tool	List your Ontologies, most recently created first, with cursor-based pagination and optional title search.

Datasets

Tool	Description
list_datasets_tool	List your datasets with optional title/description search and pagination.

Folders

Tool	Description
list_folders_tool	List your storage folders, with optional title search.
create_folder_tool	Create a new storage folder.
create_cloud_synced_folder_tool	Create a storage folder mirrored from a connected cloud storage location (S3, GCS, or Azure) and start an initial scan. Set auto_sync to keep it in sync as the bucket changes. Files appear over time, so the folder may look empty briefly after creation.
list_cloud_integrations_tool	List the cloud storage integrations you can access. Use this to find the integration_hash for create_cloud_synced_folder_tool.
link_folder_to_project_tool	Attach a storage folder to an existing project as a dataset — useful when data needs to be added after the project was created, rather than at creation time.

Workflows

Tool	Description
list_workflow_templates_tool	List all workflow templates, including Encord’s built-in templates and your saved custom templates. Use this to find the right template UUID before creating a project that needs a non-default review process.
get_project_workflow_tool	Read a Project’s live Workflow graph and settings.
set_project_workflow_tool	Edit and save a Project’s Workflow graph. Add, remove, or rewire stages after the Project has already been created, rather than only choosing a template up front.
get_project_workflow_task_counts_tool	Show how many tasks currently sit in each Workflow stage (annotate, review, agent, consensus, complete, and so on). Respects role-based access: managers see counts for every stage, while annotators and reviewers only see counts for stages they can work, capped at the annotator queue limit.
list_workflow_stage_tasks_tool	List the tasks currently in a specific Workflow stage.
set_task_priorities_tool	Set or randomize task priority within a workflow stage. Useful for removing upload-order bias when tasks are otherwise presented in a fixed sequence, such as pairwise comparison projects.

Editor layouts

Tool	Description
get_project_layout_features_tool	List the ontology classifications a custom editor layout can reference for a given project. Call this before set_project_layout_tool to get the valid schema_feature_uuid values.
get_project_layout_tool	Read back a Project’s current custom editor layout.
set_project_layout_tool	Apply a custom annotation editor layout to a project. Supports both classifications and ontology objects, with a visibleWhen vocabulary for conditionally showing or hiding elements based on another field’s answer. Useful for evaluation, preference rating, and rubric-scoring tasks where the editor functions as a structured form.

Custom editor layouts that include an object table only render an object’s first level of attributes as columns. Nested attributes (attributes under a radio or checklist option) do not appear. When you ask Merlin to create an Ontology for a Project that will use a custom layout, ask it to keep object attributes to a single level.

Switching a project’s ontology resets any custom editor layout already applied to that project, since the layout’s field references no longer match.

Analytics

Project-scoped analytics tools are available to project admins and team managers only. Organization-wide analytics tools are available to organization admins, with organisation_summary_tool and organisation_active_projects_tool self-scoping to your own data if you are not an admin. Time windows default to the last 30 days and cannot exceed 90 days per call — ask your AI tool to chain multiple 90-day windows together for longer ranges.

Tool	Description
project_throughput_tool	Count task actions (completions, approvals, rejections, skips) over a time window. Optional group_by splits results by user, day, or workflow stage.
project_time_summary_tool	Total time spent on a project over a time window, broken down by workflow stage, workflow stage type, user, or UTC day. Stage and user groups use stable IDs; their label values contain the latest recorded stage titles or user emails in the selected data. Each workflow-stage result also includes the latest recorded stage type.
project_task_metrics_tool	Per-task metrics — one row per task with its media length, time spent, and label create/edit/delete counts, ranked by time spent and paginated (50 per page).
project_collaborator_performance_tool	Rank collaborators by task counts, rejection rate, label activity, and time — for either the annotation or review stage.
project_label_quality_tool	Per-ontology-class label quality — submitted, approved, rejected, deleted, and edited counts plus rejection rate, sorted worst-first.
project_label_actions_tool	Day-by-day label activity — created, approved, rejected, and deleted label counts over the window.
project_comment_thread_actions_tool	Day-by-day comment-thread (issue) activity — threads created, resolved, reopened, re-rejected, and comments added.
project_comment_tag_actions_tool	Day-by-day breakdown of comment (issue) tags raised, including those created via label rejection.
project_comment_tag_by_class_tool	Comment (issue) tags broken down per ontology class — which classes attract which kinds of issues.
organisation_summary_tool	Organization-wide KPIs over a time window (tasks submitted, label instances created/deleted, time spent, files uploaded, and more). With organization-wide analytics access, the file upload count covers the selected time window; if the count cannot be retrieved, Merlin omits it from reported figures. Without organization-wide analytics access, the tool returns a placeholder zero that Merlin treats as unavailable at your access level, not as zero uploads.
organisation_active_projects_tool	The organization’s most active Projects, ranked by time spent.
organisation_active_collaborators_tool	The organization’s most active collaborators, ranked by time spent.

Documentation

Tool	Description
search_encord_docs_tool	Search the Encord documentation and get a grounded answer with source links for “how do I…” and product-capability questions about the platform, SDK, and APIs. Use it for how-to questions, not for querying your own projects or data.

Feedback

Tool	Description
feedback_tool	File a feature request or bug report from within a Merlin conversation. Captures your original prompt alongside the report to help the Encord team triage it.

Workspaces

Tool	Description
list_workspaces_tool	List the Encord Workspaces you have access to. To work in a specific Workspace, your AI tool passes that Workspace’s identifier with each subsequent tool call — relevant only for users who belong to more than one Workspace.

Projects created via Merlin now start in the same active state as projects created through the web app, instead of a transient “setup” state.