Dependencies & Specs

Dependencies

When writing agents, you often rely on common resources, whether data for running your agent or for recording the output. Encord Agents use dependency injection to declaratively acquire these resources, so you can focus on developing your agent instead of writing boilerplate to set them up manually.

What is Dependency Injection?

Dependency injection means your code (in this case, your path operation functions) can declare what it needs to function — its “dependencies.” The system (e.g., FastAPI or encord-agents) then takes care of providing your code with those dependencies (“injecting” them).

We follow the same pattern as FastAPI Dependencies, where you can find more examples.

Injecting Dependencies

When defining your agents, you can inject essential dependencies, such as the path to the underlying asset or frame iterators. You can also add custom dependencies if needed.

To inject dependencies, type-annotate your agent function variables using the Depends class. In the example below, replace {module} with the type of agent you’re building:

from typing_extensions import Annotated

from encord.core.dependencies import Depends
# or from fastapi import Depends # if you are building a fastapi app

from encord.{module}.dependencies import dep_single_frame

def my_agent(frame: Annotated[np.ndarray, Depends(dep_single_frame)]):
    # the frame becomes available here.

See the references section for available dependencies.

Custom Dependencies

To add a custom dependency:

def my_custom_dependency(label_row: LabelRowV2) -> dict:
    # 1. Define a function to load the dependency
    # e.g., look up additional data in own db
    return db.query("whatever")

@runner.stage(stage="<my_stage_name>")
def by_custom_data(
    # 2. Use that function as a dependency
    custom_data: Annotated[dict, Depends(my_custom_dependency)]
) -> str:
    # `custom_data` is automatically injected here.
    # ... do your thing
    # then, return name of task pathway.

Custom dependency functions can themselves rely on other dependencies, enabling more complex resource acquisition — see dep_video_iterator for an example of this.

---

Custom Agents Specification

This section defines the interface for Custom Agents. Use it to define agents via the library or when writing your own implementation.

Schema

type AgentPayload = {
  projectHash: string;
  dataHash: string;
  frame: number;
  objectHashes?: string[];
};

This schema aligns with the FrameData structure. The objectHashes field is optional — when present, it’s a list of strings.

Test Payload

When you register your Custom Agent in the platform’s Custom Agents section, you can test it with a test payload.

• If you modify the test payload, the platform verifies that your agent has access to the associated project and data.
• If you leave it unmodified, the platform instead sends a distinguished X-Encord-Agent header, which automatically triggers an appropriate response.

This lets you confirm that your agent is deployed correctly, that your session can see the agent (all requests to your agent originate from your browser session, not the Encord backend), and that it works on specific Projects.

Webhook Security

Encord provides webhook signing secrets to verify the authenticity of requests sent to your custom agent endpoints. Each custom agent endpoint receives a unique signing secret you can use to validate incoming webhook payloads.

To access your signing secret:

1. Navigate to your Workflow configuration in Encord.
2. Select your custom agent node.
3. Locate the Signing secret section.
4. Use the eye icon to reveal the secret, or the copy icon to copy it to your clipboard.

The signing secret is automatically generated when you configure your custom agent endpoint URL. Save your workflow after changing the URL to update the secret.

To verify webhook signatures:

Use the signing secret to verify that webhook requests to your custom agent endpoint originate from Encord. This helps protect your endpoint from unauthorized requests and ensures data integrity.

For implementation guidance, see the webhook security documentation.

Response

The platform displays your agent’s output using the AgentResponse type, making your agents more interactive and informative. For example, if the label state doesn’t meet the Custom Agent’s expectations, or if the agent’s function is to check the validity of current labels, the response type lets you communicate that information to the annotator.

Error Handling

Raise an EncordAgentException to handle errors appropriately. For example, if your agent expects polygons but receives a skeleton, use it to return an informative error to the Encord platform.

If an authorization issue occurs with the Encord platform (for example, a request tries to access a project the agent doesn’t have access to), the response body includes the authorization message in an AgentErrorResponse type:

type EditorAgentErrorResponse = {
  message?: string;
}

The platform displays this message so your agent can be used intuitively.