About the Iframely Processing Agent
If you see this agent accessing your URLs, it’s because one of our customers’ applications is using Iframely to perform automated processing — indexing, search, or AI-assisted retrieval — on their behalf. We act as a technical intermediary for that customer, not on our own; we don’t decide what to build with your content, and we don’t retain rights in it.
Iframely runs three separate agents, each scoped to what our customers can use it for:
- Preview Agent — used only for link previews and rich media embeds triggered by a human-shared URL. Does not allow automated processing or AI training under this identity.
- Processing Agent (this one) — handles automated processing requests on a customer’s behalf: Assist, Discover, Search, and AI-Input. Does not allow training.
- Training Agent — dedicated identity for AI-Train requests, where a publisher has permitted it. By default, it accesses only metadata and links; anything beyond that requires the publisher’s explicit permission.
If you’re looking for AI-training controls specifically, see the Training Agent doc — training is governed separately from everything on this page.
What we do
Iframely is a software-as-a-service, built for social and collaboration app developers, search and discovery products, and AI-assisted tools. This agent handles requests where a customer’s application needs to understand, index, or retrieve your content programmatically — for example, an AI assistant looking up a page someone just asked it to look at, or a search product indexing your site, or a recommendation engine suggesting your content to their users.
It handles the following of Iframely processing modes: Assist, Discover, Search, and AI-Input. See Processing modes for what each means and how our customers declare them.
User-Agent string
Iframely-Processing/1.3.1 (+https://iframely.com/docs/about-processing)Version and optional app name extension may and will change; the extension, when present, identifies the specific customer who triggered the request, as they’ve entered it with us. This is the identity referred to as the “Iframely Processing Agent” elsewhere in our documentation and policies.
What we access
Iframely can produce only a certain content data types — meta, media, structured entities, exceprts and fulltext. Iframely does not scrape web pages for arbitery content.
What’s Iframely makes available to a customer depends on the request’s mode, as declared by that customer.
Assist is for helping end-users and it gets the most access, including full body text unless signaled otherwise by your page. Search and Discover get your metadata, links, structured data, and a short excerpt for indexing purposes. AI-Input gets your metadata, links, and structured data, without body text.
Your own signals and policy — robots.txt, a Content-Signal directive, a license, or your Iframely dashboard — can extend or restrict any of this beyond the defaults. See Content access for the full breakdown.
Crawling behavior
Like the Preview Agent, the Processing Agent doesn’t go beyond a single URL — it is request-driven, on behalf of a specific customer request, one URL at a time.
Unlike the Preview Agent, it does check your robots.txt before fetching anything, following RFC 9309, since it acts on behalf or systems capable of autonomous, repeated access to your content rather than a single fetch of a URL someone just shared.
We minimize retrieval wherever structured or partial data is sufficient, and we don’t retain, aggregate, or repurpose content beyond what’s described.
Having a problem with the Processing Agent?
We’re happy to add you to (or remove you from) our ignorelist, work with you to improve how we parse your content, discuss alternatives to how we access your site, or just answer questions. Contact us at support@iframely.com.
Allow the Processing Agent on your network
If you have bot protection in place and want to permit training access, add the Training Agent to your allowlist separately from the Preview and Processing Agents. Use the appropriate user-agent string and see Allowlisting Iframely for IP lists, reverse DNS, Web Bot Authentication. It also describes transitive trust and gives the full request header reference.