You know three things that would be cool?

• the ability to subscribe to the output of a Flickr API call in a feed aggregator
• the ability to get the results of Flickr API calls as…

Oh wait. That was a while ago. Wouldn’t it be great if you didn’t have to poll our API over and over just to see if photos were there, only to find out you waited too long since last time and now because of the results size limits you can’t get them all and you have to figure out how many you missed and then you have to make another call with the right offset to get those results but of course in between then and now the result set changed a bit so you aren’t sure if you really got them all and…

Wouldn’t it be great if Flickr had something that could just PuSH photos to you as they appeared, kind of like this?

Introducing the new (and experimental) flickr.push API methods. These allow you to subscribe to new uploads and updates from your contacts and favorites from your contacts. Let’s dive right in and see exactly how it all works:

The 20,000 ft overview is basically this:

1. You make an API call to Flickr asking to subscribe to one of several different photo feeds, providing a callback URL in the arguments.
2. A little verification dance ensues during which we make a request to your callback URL. If you respond appropriately we’re all good and from then on…
3. Live(-ish) updates are POSTed from Flickr to your callback URL in Atom 1.0 format.

Subscribing

The subscription system is based as closely as possible on Google’s Pubsubhubbub protocol, with a few wrinkles. One is that Flickr acts as the hub and the publisher all rolled in to one. We’re obviously not really “publishing” separate feeds of every single user’s contacts’ photos and faves to a central hub somewhere, we only create the feeds on demand when someone subscribes to them. So we couldn’t, for example, publish them all to a 3rd-party hub like Superfeedr. But the whole pubsubhubbub metaphor still works pretty well.

Another difference is that the subscription happens via an authenticated API call and not an HTTP POST; hopefully the reasons for this are obvious. We’ll get into them in detail a little bit later. But even though the mechanism for the subscription request is different we’ve tried to follow the protocol as closely as possible and keep the parameters the same. The Google PubSubHubbub Core 0.3 section on how the subscription flow works is a good place to start, and the rest of this post assumes you’ve read that and more or less understand what the interactions should be between hub and subscriber. Done? OK, here are the methods:

flickr.push.getTopics

This method just tells you what you can subscribe to. It returns something like this:

<rsp stat="ok">
  <topics>
    <topic name="contacts_photos" />
    <topic name="contacts_faves" />
  </topics>
</rsp>

yeah, yeah, you already get that part. You can currently subscribe to contacts’ photos (new uploads and updates) or contacts’ faves. So subscribe already!

flickr.push.subscribe

This method (which requires an authentication token with read permissions) takes almost exactly the same arguments as a “proper” PubSubHubbub subscribe HTTP request would. Wee differences:

topic – unlike the topic argument in the HTTP version (which is a URL), this is just one of the topic types returned by flickr.push.getTopics.

secret – currently not supported, so this parameter is omitted.

callback – this must be unique, i.e. you can’t use the same URL for more than one subscription.

Everything else works as you would expect – verification (either synchronous or asynchronous), the hub challenge string, subscription expiration/refreshing, unsubscribing (with flickr.push.unsubscribe) etc. Which brings us to:

flickr.push.getSubscriptions

This method also requires an authentication token with read permissions, and returns a list of subscriptions for the authenticated user, like so:

<rsp stat="ok">
  <subscriptions>
    <subscription topic="contacts_photos" callback="http://example.com/contacts_photos_endpoint?user=12345" pending="0" date_create="1309293755" lease_seconds="0" expiry="1309380155" verify_attempts="0" />
    <subscription topic="contacts_faves" callback="http://example.com/contacts_faves_endpoint?user=12345" pending="0" date_create="1309293785" lease_seconds="0" expiry="1309380185" verify_attempts="0" />
  </subscriptions>
</rsp>

Oh yeah, the docs:

flickr.push.subscribe
flickr.push.unsubscribe
flickr.push.getTopics
flickr.push.getSubscriptions

Feeds

The format of the feed that gets posted to your endpoint is currently limited to Atom 1.0, i.e. exactly what you’d get from something like

http://api.flickr.com/services/feeds/photos_public.gne?id=_YOUR_NSID_HERE_&format=atom

For the contacts_faves topic type it’s the same thing but with the addition of the atom:contributor element to indicate the user who faved the photo.

And that’s about it. Questions?

Privacy and Restrictions

The astute observer may notice that not all photos are being sent in the PuSH feeds. Since this is a new (and experimental) feature for Flickr, we’ve basically turned all of the privacy/safety restrictions on it up to 11, at least to start with. PuSH feeds currently only contain images that have public visibility and safe adultness level. In addition, users with their “Who can access your original image files” option set to anything other than “anyone” and users who are opted out of the API will not have their photos included in PuSH feeds.

While this may be a bit restrictive (for example since the API call is authenticated and the photos are coming from your contacts technically you should be allowed to see contacts only or friends/family photos for contacts that allow it),  we feel that since this is a new thing it’s better to start conservative and see how the feature is being used. It’s possible that we may relax some of these restrictions in the future, but for now a PuSH feed is essentially what a signed-out user could get just by grabbing the RSS feeds from various people’s photostreams.

You will also notice that for now we’ve limited the feature to pro account holders only.

So… What?

So what can you do with it? There’s the obvious: any web application which currently does some kind of polling of the Flickr API to get photos for its users can potentially be altered to receive the push feeds instead. More timely updates, cheaper/simpler for the application and as it turns out cheaper for Flickr, too – it’s often easier on our servers to push out events shortly after they happen and we’ve got them (often fresh in our cache) than it is to go and dig them up when they’re asked for some time later.

Some of the more interesting things that we hope these API methods will enable revolve around the more real-time nature of the events they expose. As an example of what’s possible in this space, Aaron Cope has created a little application he calls “Pua”. It’s a wonderfully simple way to surf Flickr without having to do much of anything; Pua takes you on a ride through your contacts’ photos and favorites, as they happen. Have a read about exactly what it is, why it’s called Pua and why he made it. If you ask nicely maybe Pua will give you an invite code.

Later

Hopefully there will be much more to come. Finer-grained controls on the subscriptions (safety levels, visibility levels, restricting to just new uploads or only certain types of updates, lightweight JSON feeds, etc.), new types of subscriptions (photos of your friends/family, photos from a particular location, photos having a particular tag, something to do with galleries…), and maybe some other stuff we haven’t thought of yet. Hey, wouldn’t it be cool if you didn’t need to run a web server on the other end to be the endpoint of the feeds?

Let us know what you’d like to see! What works, what doesn’t, what we got wrong and how to make it more useful to the people who want to Build Stuff (that’s you).

Fine Print

The Flickr PuSH feeds are part of the Flickr API, and thus fall under the API Terms of Service Agreement. This means all the usual things about respecting photo owners’ copyrights and all also the other good bits about API abuse. In other words, don’t try to subscribe to all of Flickr. Trust me, we’ll notice.