1. Data API
  2. API format

Data API for Automated Processing

Data API is Iframely’s endpoint for customers involved in automated processing of URLs, including assistants, discovery and recommendation systems, search, AI input, and AI training.

Iframely’s Data API follows a publisher-first approach to data processing. Our Processing Agent and Training Agent are designed to be transparent to publishers and respect publisher signals. Beyond the default access levels, content provided to customers requires clear publisher signals or explicit dashboard permissions. Publisher signals can also restrict access below the default level.

The access object is a critical part of the Data API response. It describes the applicable processing triggers, publisher signals, and resulting access decisions. Customers can use it for traceability and audit logs, as well as to make their own decisions about how returned content may be used.

Data API follows a predefined structure; it is not a programmable scraper. It can fetch specific content pieces and does not go beyond them. Data API builds on the Preview APIs (Iframely API and oEmbed API). It adds three fixed content objects — entities, excerpt, and fulltext — where available and permitted.

Eligibility

Data API is subject to Iframely’s Automated Processing Terms.

To use Data API, customers need to declare their usage intent. We call it a Processing Mode. There are five modes you can use with Data API: assist, discover, search, ai-input, and ai-train.

Please see the full description of processing modes.

A mode can be selected for each API request individually through the mode= parameter, or set as a default in the account settings.

The mode is required to derive the access policy and apply it to available URL content. See the default access levels and how publisher signals affect them in Content access.

API request

  • url, api_key, and mode are required.
  • url must be URL-encoded.
  • mode must be one of assist, discover, search, ai-input, or ai-train. See Iframely processing modes for how to declare the right mode.

use is an additional optional intent parameter. It further describes how the content will be used and accepts immediate, reference, or full — see The use signal. Iframely passes this value to the publisher as part of the transitive trust headers. By default, it is empty.

mode identifies the processing scenario, while use further describes how the content will be used within that scenario.

For additional security, you can substitute api_key with the key parameter, the MD5 hash of your actual API key (see Allow origins).

Misrepresenting or obscuring your mode is a violation of our Automated Processing Terms.

API response

{
    "url": "https://en.wikipedia.org/wiki/The_Great_Wave_off_Kanagawa",
    "meta": {
        "title": "The Great Wave off Kanagawa",
        "description": "The Great Wave ...",
        "site": "Wikipedia",
        "...": "..."
    },

    "links": {
        "thumbnail": [
            {
                "href": "...",
                "type": "image/jpg",
                "rel": ["thumbnail", "og", "ssl"],
                "media": {
                    "width": 1280,
                    "height": 860
                },
                "content_length": 444875
            },
            {"..."}
        ],
        "logo": ["..."],
        "icon": ["..."]
    },

    "entities": {
        "article": {
            "@context": "https://schema.org",
            "@type": "Article",
            "name": "The Great Wave off Kanagawa",
            "url": "https://en.wikipedia.org/wiki/The_Great_Wa...",
            "...": "...",
            "image": "...",
            "headline": "..."
        },
        "navigation": {"...": "..."}
    },

    "excerpt": {
        "snippet": "Woodblock print by Hokusai ....",
        "headings": [
            {
                "level": 3,
                "text": "Depth and perspective"
            },
            {"..."}
        ],
        "hyperlinks": [
            {
                "href": "https://en.wikipedia.org/wiki/Katsushika_Hokusai",
                "text": "Katsushika Hokusai"
            },
            {"..."}
        ]
    },

    "fulltext": "...",

    "access": {
        "requested": {
            "modes": [
                "assist"
            ],
            "use": "reference"
        },
        "signals": {
            "license": {
                "url": "https://creativecommons.org/licenses/...",
                "name": "Creative Commons Attribution-ShareAlike...",
                "common_name": "CC BY-SA 4.0",
                "type": "CC"
            },
            "robots.txt": {
                "...": "..."
            },
            "directives": {
                "...": "..."
            }
        },
        "decision": {
            "meta": {
                "iframely_default": true,
                "supported_by": [
                    "license"
                ],
                "allowed": true
            },
            "links": {
                "...": "...",
                "allowed": true
            },
            "entities": {
                "...": "...",
                "allowed": true
            },
            "excerpt": {
                "...": "...",
                "allowed": true
            },
            "fulltext": {
                "...": "...",
                "allowed": true
            }
        }
    },

    "version": 1
}

Content objects

meta and links in Data API are the same preview objects as in the Iframely API. See meta and links for the complete field lists.

entities, excerpt, and fulltext are Data API content. entities contains structured data declared by the publisher. excerpt includes a snippet, headings, and hyperlinks from the main body. fulltext, if available, contains the main text of the page.

See Data API content for the specification of the content fields.

Content access

The API response includes an access object, a critical part of automated processing that describes the applicable default policy rules, publisher signals, and controls, along with the resulting access decision. Use it for audit and decision logs, and to assess how returned content may be used according to your own requirements.

The response can contain an empty or unavailable content type when the access decision does not allow it. Use access to understand why a content type was or was not returned.

See Content access for a full description of the access object.

Iframely remains a technical intermediary only. Customers remain responsible for complying with the terms of use of each individual publisher. Use the access object to make additional decisions about returned content.

See also