Tags in Space

A lot of you enjoyed our post (“Found in Space”) on the amazing astrometry.net project, and there have been some interesting followups.

A mysterious figure known only as “jim” paired up astronomy photos from Flickr with Google Sky. (You’re going to need the Google Earth plug-in for your browser — just follow the instructions on that page if you don’t have it.) In his technical writeup, “jim” explains how he used the Yahoo Query Language (YQL) to fetch the data. YQL is similar to the existing Flickr APIs, but it’s a query language like SQL rather than a set of REST-ish APIs. And both of those are really just ways to get data out of Flickr’s machine tag system, specifically the astro:* namespace. It’s turtles all the way down.

Who else is using astrotags? The British Royal Observatory in Greenwich is sponsoring a contest to determine the Astronomy Photographer of the Year and the whole thing is based on a Flickr group and extensive use of Flickr’s APIs. The integration is so seamless — galleries of photos and discussions are surfaced on their site as well as ours — you might as well consider Flickr to be their “backend” server. But they’ve also added much, such as great documentation about how to astrotag your photos as well as a concise explanation about how Astrometry.net identifies your photo, even among millions of known stars. (The sci-fi website io9 interviewed Fiona Romeo of the Royal Observatory about the contest; check it out.)

It’s dizzying how many services have been combined here — Astrometry.net grew out of research at the University of Toronto, web mashups use Google Sky for visualization in context, Yahoo infrastructure delivers and transforms data, the Royal Observatory at Greenwich provides leadership and expertise, and then little old Flickr acts as a data repository and social hub. And let’s not forget you, the Flickr community, and your inexhaustible creativity — which is the reason why all this can even come together.

All this was done with pretty light coordination and few people at Flickr were even aware what was going on until recently. I have no idea what the future is for APIs and a web of services loosely joined, but I hope we get to see more and more of this sort of thing.

API Responses as Feeds

You know three things that would be cool?

  • the ability to subscribe to the output of a Flickr API call in a feed aggregator
  • the ability to get the results of Flickr API calls as KML (or GeoRSS)
  • if all those devices that support RSS feeds (like photo frames!) also supported the Flickr API.

You see where I’m going with this don’t you?

You can already specify that you want the output format of a Flickr API call to be REST (POX), XML-RPC, SOAP (shudder, not sure that one still works), JSON, or serialized PHP. We always wanted to support formats like KML, or Atom but we were never quite sure how to represent the results of a call to flickr.photos.getInfo() or flickr.photos.licenses.getInfo() as a KML.

Last week we finally got around to pushing out our 80% solution — an experimental response format for API methods that use the standard photos response format that allows you to request API responses as as one of our many feed formats.

You can now get the output of flickr.photos.search(), or flickr.favorites.getList() as Atom, or GeoRSS, or KML, or whatever.

API Feed Types

The syntax is "&format=feed-{SOME_FEED_IDENTIFER}" where the feed identifiers follow the same convention you use when fetching…feeds.

  • feed-rss_100, API results will be returned as a RSS 1.0 feed
  • feed-rss_200, API results will be returned as a RSS 2.0 feed
  • feed-atom_10, API results will be returned as a Atom 1.0 feed
  • feed-georss, API results will be returned as a RSS 2.0 feed
    with corresponding GeoRSS and W3C Geo elements for geotagged
    photos
  • feed-geoatom, API results will be returned as a Atom 1.0 feed
    with corresponding GeoRSS and W3C Geo elements for geotagged
    photos
  • feed-geordf, API results will be returned as a RSS 1.0 feed
    with corresponding GeoRSS and W3C Geo elements for geotagged
    photos
  • feed-kml, API results will be returned as a KML 2.1 feed
  • feed-kml_nl, API results will be returned as a KML 2.1 network
    link feed

And remember, format is an API arg, and needs to be included in your API signature if you’re making a signed API call.

Namespaces, pagination, bits and bobs

You’ll find the feeds now include the venerable xmlns:flickr=”urn:flickr:” namespace. This is used to declare bits that don’t fit elsewhere like pagination.

Pagination information is passed as a single namespaced (‘urn:flickr:)
element under the feed’s root (or “channel” element if it has one). For
everything but RSS 1.0 based feeds it looks like this:

<flickr:pagination total="480" page="1" pages="96"
per_page="5" />

For RSS 1.0 we do a little RDF dance:

<flickr:flickr>
     <pagination>
             <total>480</total>
             <page>1</page>
                 <pages>32</pages>
             <per_page>15</per_page>
     </pagination>
</flickr:flickr>

Speaking of pagination, to start with we’ve enforced a maximum “per
page” limit of 15. If people have a reasonable use case we may
consider raising the limit but otherwise we need to account for the
extra data/size that feed formats add.

You’ll also find some extras like

<entry>
    ...
    <flickr:views>3</flickr:views>
    <flickr:original type="png" href="http://farm4.static.flickr.com/3074/2783931781_12f84e4079_o.png" width="640" height="480" />
</entry>

Error Handling

If an error occurs, the API will return a 400 HTTP status code.

Flickr error codes and message are returned as X-FlickrErrCode and X-FlickrErrMessage
HTTP headers. For example:

X-FlickrErrCode: 111
X-FlickrErrMessage: Format "feed-lolcat" not found

Caveats and Warnings

  • This is EXPERIMENTAL. It might change, it might go away, but we hope not. We also could potentially make it better based on all your awesome feedback.

  • This is not available for all methods. If you call photos.getInfo and
    ask for a feed response format all you will get is an error.

  • Just like any API call (or feed usage or really anything else) you are required to respect the copyright of the photographer.

  • These are still API calls. All the usual rules about usage apply.
    You are still bound by the Flickr API TOU and any other rules,
    capricious or not, we apply to API usage. Including rate limits. If
    you feed this in to an overly-aggressive aggregator we will make your
    API key cry. (In an entirely non-creepy way)

Putting it together: ego feeds

Turns out ‘kellan’ is a popular baby name these days, so whenever I go ego surfing Flickr I tend to see pictures of two year olds. This changes (for now) when I limit my searching to my friends photos. Using API feeds I can now build an ego feed of photos from my friends, like so:

flickr.photos.search:
   user_id => 51035734193@N01,
   contacts => all,
   text => kellan,
   sort => date-posted-desc,
   api_key => {API_KEY}
   auth_token => {AUTH_TOKEN}
   format => feed-atom_10

api.flickr.com/services/rest/?auth_token=xxxx&user_id=51035734193%40N01&
   contacts=all&format=feed-atom_10&sort=date-posted-desc&text=kellan
   &api_key=xxxx&method=flickr.photos.search&api_sig=xxxx

Putting it together: near home

Or a KML feed of the most interesting, safe, CC licensed photos, within 10 kilometers of a point (say your home), suitable for remixing:

flickr.photos.search:
   license => 1,2,4,5,7,
   sort => interestingness-desc,
   lat => 40.661699,
   lon => -73.98947,
   radius => 10,
   safe_search => 1,
   api_key => {API_KEY}
   format => feed-kml

api.flickr.com/services/rest/?auth_token=xxx&license=1%2C2%2C4%2C5%2C7&
  lat=40.661699&lon=-73.98947&radius=10&safe_search=1
  &format=feed-atom_10&api_key=xxxx&method=flickr.photos.search
  &api_sig=xxxxx

You get the idea.