Superfeedr Documentation

Summary

What API to choose?
XMPP PubSub
PubSubHubbub API
- Implementation
- Notifications
Schema
- Status
- Entry

Introduction

If you have any question about this documentation or if you still need help with the Superfeedr, please join our Google Group, read our Frequently Asked Questions. We also strongly suggest the use of our dashboard (you must be logged in) to debug the real-time notification.

Code key

Outbound (requests) traffic have a grey background.

Inbound (responses + notifications) traffic have a green background.

What API to choose?

Superfeedr offers 2 different API : XMPP PubSub and HTTP PubSubHubbub. They have both purposes for which they've been created and, based on your goals using Superfeedr, you might want to select one or another. Here is a small chart that will help you choose the most appropriate in your case. (In this chart, the 'I' represent the endpoint that will receive the notifications).

XMPP PubSub

You can connect your jabber client (or component) to superfeedr by using the JID login-@superfeedr.com and your superfeedr password. You can also specify another JID in your settings.

The XMPP API is based on the XEP-0060 syntax. Additional data has been included within a superfeedr-specific namespace.

Adding a Feed

iq[@from] : should match the bare jid you entered in your settings.
iq[@id] : random string (enables you to match the superfeedr response with your query).
subscribe[@node] : use the url of the feed you want to monitor.
subscribe[@jid] : should be equal to the @[from] or, if you have chose an external component jid, should have the same domain. JID resources will be ignored.

<iq type="set" from="you@superfeedr.com" to="firehoser.superfeedr.com" id="sub1">
    <pubsub xmlns="http://jabber.org/protocol/pubsub">
        <subscribe node="http://domain.tld/feed1.xml" jid="you@superfeedr.com"/>    
        <subscribe node="http://domain.tld/feed2.xml" jid="you@superfeedr.com"/>
        <subscribe node="http://domain.tld/feed3.xml" jid="you@superfeedr.com"/>
    </pubsub>
</iq>

You can add up to 30 feeds in your subscription query, as long as they have all the same subscribe[@jid]. If you need to subscribe with multiple jids, we suggest that you send multiple subscription queries.

The server acknowledges the subscription(s) and sends the status information for each feed.

<iq type="result" to="you@superfeedr.com/home" from="firehoser.superfeedr.com" id="sub1">
    <pubsub xmlns="http://jabber.org/protocol/pubsub">
        <subscription jid="you@superfeedr.com" subscription="subscribed" node="http://domain.tld/feed1.xml">
            <status xmlns="http://superfeedr.com/xmpp-pubsub-ext">
                <http code="200">9718 bytes fetched in 1.462708s : 2 new entries.</http>
                <next_fetch>2009-05-10T11:19:38-07:00</next_fetch>
            </status>
        </subscription>
        <subscription jid="you@superfeedr.com" subscription="subscribed" node="http://domain.tld/feed2.xml">
            <status xmlns="http://superfeedr.com/xmpp-pubsub-ext">
                <http code="200">9718 bytes fetched in 1.462708s : 2 new entries.</http>
                <next_fetch>2009-05-10T11:19:38-07:00</next_fetch>
            </status>
        </subscription>
        <subscription jid="you@superfeedr.com" subscription="subscribed" node="http://domain.tld/feed3.xml">
            <status xmlns="http://superfeedr.com/xmpp-pubsub-ext">
                <http code="200">9718 bytes fetched in 1.462708s : 2 new entries.</http>
                <next_fetch>2009-05-10T11:19:38-07:00</next_fetch>
            </status>
        </subscription>
    </pubsub>
</iq>

In other cases, you will receive an iq with type="error", please check that your subscription query abide by the constraints explained above.

Removing a Feed

iq[@from] : should match the bare jid you entered in your settings.
iq[@id] : random string (enables you to match the superfeedr response with your query).
unsubscribe[@node] : use the url of the feed you want to stop monitoring.
unsubscribe[@jid] : use your 'bare' jid .

<iq type="set" from="you@superfeedr.com/home" to="firehoser.superfeedr.com" id="unsub1">
    <pubsub xmlns="http://jabber.org/protocol/pubsub">
        <unsubscribe node="http://domain.tld/feed1.xml" jid="you@superfeedr.com"/>
        <unsubscribe node="http://domain.tld/feed2.xml" jid="you@superfeedr.com"/>
        <unsubscribe node="http://domain.tld/feed3.xml" jid="you@superfeedr.com"/>
    </pubsub>
</iq>

You can remove up to 30 feeds in your unsubscription query.

The server acknowledges the unsubscription.

<iq type="result" from="firehoser.superfeedr.com" to="you@superfeedr.com/home" id="unsub1" />

Listing Feeds

iq[@from] : should match the bare jid you entered in your settings.
subscriptions[@page] : page number (subscriptions will be sent by batches of 30).
subscribe[@jid] : use your 'bare' jid .

<iq type="get" from="you@superfeedr.com/home" to="firehoser.superfeedr.com" id="subman1">
    <pubsub xmlns="http://jabber.org/protocol/pubsub" xmlns:superfeedr="http://superfeedr.com/xmpp-pubsub-ext">
        <subscriptions jid="you@superfeedr.com" superfeedr:page="3"/>
    </pubsub>
</iq>

The server sends the list of feed to which you're subscribed for the page requested, as well as the status information for each feed.

<iq type="result"  to="you@superfeedr.com/home" id="subman1" from="firehoser.superfeedr.com">
    <pubsub xmlns="http://jabber.org/protocol/pubsub"  xmlns:superfeedr="http://superfeedr.com/xmpp-pubsub-ext" >
        <subscriptions superfeedr:page="3">
            <subscription node="http://domain.tld/a/feed/atom.xml" subscription="subscribed" jid="you@superfeedr.com" >
                <status xmlns="http://superfeedr.com/xmpp-pubsub-ext">
                    <http code="200">9718 bytes fetched in 1.462708s : 2 new entries.</http>
                    <next_fetch>2009-05-10T11:19:38-07:00</next_fetch>
                </status>
            </subscription>
            <subscription node="http://domain2.tld/feed.rss" subscription="subscribed" jid="you@superfeedr.com" >
                <status xmlns="http://superfeedr.com/xmpp-pubsub-ext">
                    <http code="200">9718 bytes fetched in 1.462708s : 2 new entries.</http>
                    <next_fetch>2009-05-10T11:19:38-07:00</next_fetch>
                </status>
            </subscription>
        </subscriptions>
    </pubsub>
</iq>

Notifications

Once you're tracking a feed, your xmpp client must be connected at all times to receive the new entries published on this feed. We won't store any messages if your client is offline.

You can chose in your settings if you want to receive notifications each time we fetch and parse a feed, or just when there are new entries.

Please see the schema information for details on the status, as well as the feed and entry informations.

<message from="firehoser.superfeedr.com" to="you@superfeedr.com">
    <event xmlns="http://jabber.org/protocol/pubsub#event">
        <status feed="http://domain.tld/feed.xml" xmlns="http://superfeedr.com/xmpp-pubsub-ext">
            <http code="200">9718 bytes fetched in 1.462708s : 2 new entries.</http>
            <next_fetch>2009-05-10T11:19:38-07:00</next_fetch>
            <title>Lorem Ipsum</title>
        </status>
        <items node="http://domain.tld/feed.xml">
            <item chunks="1" chunk="1">
                <entry xmlns="http://www.w3.org/2005/Atom">
                    <title>Soliloquy</title>
                    <summary>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.</summary>
                    <link rel="alternate" type="text/html" href="http://superfeedr.com/entries/12345789"/>
                    <id>tag:domain.tld,2009:Soliloquy-32397</id>
                    <published>2010-04-05T11:04:21Z</published>
                </entry>
            </item>
            <item chunks="1" chunk="1">
                <entry xmlns="http://www.w3.org/2005/Atom">
                    <title>Finibus Bonorum et Malorum</title>
                    <summary>Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt.</summary>
                    <link rel="alternate" type="text/html" href="http://superfeedr.com/entries/12345788"/>
                    <id>tag:domain.tld,2009:Finibus-32398</id>
                    <published>2010-04-06T08:54:02Z</published>
                </entry>
            </item>
        </items>
    </event>
</message>

API Wrappers

XMPP can be scary. For those of you who don't want to mess with that, we have various wrappers (including Ruby, Perl, Python, Java and PHP) for the XMPP API. You can find them on our Github page. If you need one in another language, please contact us.

PubSubHubbub

Superfeedr is a PubSubHubbub hub; our hub is located at http://superfeedr.com/hubbub. We implement the Core Specification. The biggest advantage of using our PubSubHubbub API is that you can re-use your "subscriber" code with any other hub. You can also use any library that supports and implements pubsubhubub.

Implementation

Our hub supports HTTP Basic Authentication, you need to authenticate your HTTP requests. Here are few things to know about Superfeedr's hub :

We do not support hub.lease_seconds, this will be ignored
You can obviously use https:// and hub.secret for complete security. (Learn more)

Please read the Subscriber Best Practices and do not hesitate to use tools like curl or Hurl for debugging.

Subscriptions

The syntax for subscribe and unsubscribe is identical. You can also use the sync or async mode, based on the capabilities of your app. Please remember that verification will fail if it takes more than 3seconds.

Please refer to the Core Specification for Subscription. Here is a short step by step :

Send an POST request to http://superfeedr.com/hubbub, with HTTP Basic Authentication and the following params :
- hub.mode : subscribe or unsubscribe
- hub.verify : sync or async
- hub.callback : http://domain.tld/your/callback
- hub.topic : http//feed.you.want.to/subscribe/to
The hub will send a request to your callback url, to verify your intent : your callback must respond with 200 and must echo the hub.challenge provided by the hub.
If the hub returns 204, you're subscribed!, If it returns 202, the hub check the subscription later. If it returns anything else, something was wrong, you may want to check the body of hub's the response.

Notifications

One of the benefits of Superfeedr's hub is that you will benefit from our standard schema mapping. Please note however that our hub will do 'single' notifications and will not attempt retries in case of failure.

Schema

Whatever the original format (RSS, Atom, or any other namespace) is, the notification will use standard ATOM. We will match as much as we can into this format.

Status Schema

XMPP API Only. When you add a feed, list your feeds and when you receive notifications, Superfeedr will send you some status information for the concerned feed.

status[@feed] : contains the URL of the feed.
http[@code] : last HTTP status code, please see Status Code Definitions.
http : the content of that tag is a more explicit log message for your information.
next_fetch : the feed will be fetched at most before this time. A good practice is to keep track of that in your client and 'challenge' us if you see that a given feed wasn't fetched in time.
title : the feed title.

<status feed="http://domain.tld/feed.xml" xmlns="http://superfeedr.com/xmpp-pubsub-ext">
    <http code="200">9718 bytes fetched in 1.462708s : 2 new entries.</http>
    <next_fetch>2009-05-10T11:19:38-07:00</next_fetch>
</status>

Entry Schema

Notification entries will have the following form. It is standard ATOM. Here are the fields that you will see in the notification. Please note that an entry might not have all of them.

entry[@title] : The title of the entry.
entry[@id] : the Unique ID of the entry. If the original entry doesn't have an ID.
entry[@published] : The publication date (iso8601) of the entry.
entry[@content] : The content of the entry.
entry[@summary] (optional, unique) : The summary of the entry, only if the full content (entry[@content]) is not available.
category[@term] (optional, multiple) : a keyword related to the entry... (tag, category or topic)
link(at least one).
- link[@href] : the url related to the entry
- link[@rel] (optional) : the type of relation to that entry (alternate, reply... etc)
- link[@type] (optional) : MimeType of the link destination (tex/html by default)
- link[@title] (optional) : the link title
Author information(optional, multiple).
- name : the author's name (or nickname)
- email (optional) : the author's email address
- uri (optional) : the author's URI

<entry xmlns="http://www.w3.org/2005/Atom">
    <title>Soliloquy</title>
    <id>domain.tld-entries-12345</id>
    <published>2009-05-10T11:19:38-07:00</published>
    <summary>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.</summary>
    <content>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.</content>
    <category term="tag" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="category" scheme="http://www.sixapart.com/ns/types#tag" />
    <link href="http://domain.tld/entries/12345" rel="alternate" type="text/html" />
    <link href="http://domain.tld/entries/12345/comments.xml" rel="replies" type="application/atom+xml" /> 
    <author>
        <name>John Doe</name>
        <email>john@superfeedr.com</email>
    </author>
    <author>
        <name>Mike Smith</name>
        <uri>http://twitter.com/superfeedr</uri>
    </author>
</entry>