Kellan in his blog post over here: Is a Firehose of Snowflakes a Nor’easter? correctly points that that I may have been slightly cagey about the awesomeness of what we actually put out yesterday, by saying …

“But because the documentation is quirky, I think people missed the significance. These are Flickr real time data APIs.”

My excuse is that’s my natural instinct to sneak new features past the lawyers, by dressing them up in a non-serious fashion :)

But in reality, this is just an excuse to post the following photo …

… see there I go again. Offsetting the important with light humor, go read Kellan’s post now for he is smart and knows what he’s talking about.

Now stop trying to get on explore & enjoy yourself

Photo by Las Tonterias points out

The New APIs

I’ll cut straight to the chase on this one, we’ve just launched two new API methods;

flickr.panda.getPhotos
flickr.panda.getList

The first call (flickr.panda.getPhotos) returns you a list of photos that the mystical Flickr Pandas are currently interested in, in the following format …

<photos interval="60000" lastupdate="1235765058272" total="120" panda="ling ling">
    <photo title="Shorebirds at Pillar Point" id="3313428913" secret="2cd3cb44cb"
        server="3609" farm="4" owner="72442527@N00" ownername="Pat Ulrich"/>
    <photo title="Battle of the sky" id="3313713993" secret="3f7f51500f"
        server="3382" farm="4" owner="10459691@N05" ownername="Sven Ericsson"/>
    <!-- and so on -->
</photos>

When calling this API method please ensure that your code uses the lastupdate and interval attributes to determine when to request new photos. lastupdate is a Unix timestamp indicating when the list of photos was generated and interval is the number of seconds to wait before polling the Flickr API again.

The second call (flickr.panda.getList) tells you which of the Mystical Flickr Pandas are around to request photos from.

Ling Ling and Hsing Hsing both return photos they are currently interested in, both have slightly different tastes in photos depending on their mood. The (currently) third Panda Wang Wang returns photos that have recently been geotagged, not quite realtime but close.

Important! No-one fully understands the whims and fancies of the Pandas at any moment in time, these APIs give you a direct dump of what they are looking at (filtered for safe and public photos only) at this very moment. We cannot promise that this will not include ladies (or men) in bikinis, or possibly even ladies not in bikinis. You should keep this in mind while planning for your target audience, I’ll touch on this a little bit more in the Implementation Hints section.

Philosophy

We’re always toying around with different ways to find and surface “interesting” photos. One method that people are generally familiar with is the Explore page, of which there’s always a lot of discussion. A good starting place for more Explore information is here with more general discussion in the Secrets Of Explore group.

But behind Explore is an amount of “Secret Sauce” a lot of which focuses on activity around a media object. The Explore page is generally a daily experience, although things often pop in and out during the day.

What we are doing with the Pandas is exposing a more real time experience, letting them each focus on various aspects of that activity, the first two Ling Ling and Hsing Hsing are fairly introspective and have different views of Flickr, but both focus on what they’ve noticed going on recently. The third Wang Wang is more a panda of the world and concentrates solely on geotagging. If you wanted to show photos being geotagged nearly as they happen, Wang Wang is the way to go.

From a numbers point of view, Explore shows 500 photos from the last 7 days … a Panda can get through around 150,000 photos per day.

A couple of caveats; Just because a Panda notices a photo, doesn’t mean that photo will make it into Explore, or even be destined to be considered by the Magic Donkey for Explore. Conversely, because a photo has made it into Explore doesn’t necessarily mean that a Panda will have noticed it. Also, because of the complexity and amount of stuff going on, no-one really knows exactly what Ling Ling and Hsing Hsing will find interesting, but that won’t stop us from tweaking the underlining code now and then.

Also, you may have noticed that this is one of the less serious APIs, it’s both experimental and there to hopefully encourage fun and play. Please respect the Pandas and don’t abuse them, help us protect this endangered species and keep them alive.

Implementation Hints

So, why are we doing all this messing around with “Pandas”?

Well, mainly because we want to give developers another stream of photos with which to play with. For example you could build your own Flickr Zeitgeist Badge or slideshow or tool to just show stuff that’s going on over at Flickr.

We built this …

… (many people wished we hadn’t) … the Rainbow Vomiting Panda of Awesomeness as an experiment (which used Ling Ling fwiw). Since then we’ve been tweaking the backend, and now finally released the API so you can all have a go.

It’s a stream of, on average, more interesting photos then you’d generally get from polling Everyone’s photos. The quality is pretty good, the best thing to do is watch The Panda for a while and figure out if a) you want to build something with a live stream of photos b) you can build something more better than a vomiting panda (which lets face it, it pretty hard to top!).

This is what I learnt while building it, it may help you too …

When you get a packet of photos back from a panda, its around 60 seconds worth of stuff that the Panda found interesting. The packet is sorted by what the Panda found most interesting in that timeframe to least. Because people can often find ladies in bikinis interesting (for some reason) there can sometimes be some activity around ladies in bikinis that the Pandas may pick-up on.

However, Pandas tend not to find ladies in bikinis that interesting (which may have something to do with why they’re in such peril) and when those photos appear they tend to happen in lower half of the packet of photos.

So in the case of the Vomiting Panda above, I just threw away the second half of each packet, rather than go through all of the photos before asking for the next packet. I found this to be a good thing to do anyway, even though there’s always a lot of amazing stuff going on on Flickr the tail end of some packet wasn’t always quite as amazing as I wanted. You may want to do things differently, but that’s just what I found while working with the APIs.

Send us stuff

Anyway, on that note, at some point I’ll try to do a round up of stuff people have built on these APIs. If you want to post what you build to the Flickr Hacks group and/or FlickrMail me I’ll start to build up a list and take it from there.

Photos by ohad* and psd.

The Jellyfish Nebula, by DJMcCrady,
with annotation by Astrometry.net

A robot intelligence has invaded Flickr. The “blind astrometry server” is a program which monitors the Astrometry group on Flickr, looking for new photos of the night sky. It then analyzes each photo, and from the unique star positions shown it figures out what part of the sky was photographed and what interesting planets, galaxies or nebulae are contained within. Not only does the photographer get a high-quality description of what’s in their photo, but the main Astrometry.net project gets a new image to add to its storehouse of knowledge.

Needless to say this is one of the coolest uses of Flickr groups and the API that we’ve ever seen. I recently discussed the project with team member Christopher Stumm, since he was the one who had the idea to hook it into Flickr.

With Astrometry.net, you’re distributing the work of cataloguing the sky to amateurs. Are we still in an age where the average person can make contributions to astronomical science?

Definitely. There’s a large number of excellent amateur setups out there, and they discover supernovae and minor planets regularly. Although science-grade data is generated, it can be difficult to use because it’s hard to find, and there’s typically no useful meta-data. We’re hoping to help with that problem.

The catalog we use to solve images was put together from surveys during the last 50 years – that’s a long time! We believe that if the information generated by the amateur astronomer community is harnessed we could build an open-source sky survey much faster.

On top of that, we would be able see what areas of the sky have interesting activity. Right now we’re using images from around the web to calculate the path comet Holmes took through the sky.

Your scale and rotation invariant hashing algorithm is fiendishly clever. Where does it come from?

It’s an adaptation of an old idea in computer vision — “geometric hashing” — to astronomical images. It was originally created by researchers who were trying to model associative memory; “that shape reminds me of something I’ve seen before”. The idea works great for astronomical pictures, because stars are easy to locate exactly. By adding a fast search method for similar-shaped arrangements of stars, and a check that eliminates coincidental matches, we’re able to match an image against the whole sky, usually in a matter of seconds.

Actually, sometimes we still do get false matches, but it turns out that that’s almost always because of some problem in either the image or our reference. These are places where we could use many amateur astronomer images to patch our reference catalog.

Have there been any surprises about the data you’ve received, or the response you’ve gotten? Any discoveries?

One thing that has surprised me has been the amount of positive feedback we’ve gotten. The project was covered on a few sites including kottke.org, Reddit, and O’Reilly’s Make magazine. After reading about it people had to test it out, so we saw pictures from people’s back yards, some people tried to fool the system by inputting hand-drawn images, and one person even passed in a screen shot from an iPhone application which shows you the night sky.

The submissions range from amazingly high quality to images where someone just took their camera and pointed it at the night sky with some trees and a house included too. I was surprised how robust our solver was to some of the obstructions we find in the images.

Overall I think it shows people are curious about the sky, but it can be pretty very overwhelming due to its sheer size. Hopefully we help make it a little more approachable by showing people what’s in their picture.

Why Flickr? How has the experience been?

I was a Flickr user, liked the service, and believed it offered much of what we were looking for at the time. Flickr had a huge user base which meant it was familiar and would be easy for users to upload images, submit them to our group, and offered basically unlimited scalability.

The image annotations offered by Flickr were really the icing on the cake and made the whole feature much more compelling, especially to people who aren’t astronomy buffs such as myself. On top of that the API made it easy to hook into. I can’t really complain about anything so far with the whole experience, it’s been a lot of fun.

Our very own waferbaby has been rocking it out with his series, “The Setup”, on hacker mise en place. (and by hacker we mean in every sense of the word, from _why the lucky stiff, to YACHT, to David Lanham)

And if you missed it, waferbaby had the Flickr dev team do our own little version of the “The Setup” last May, published for posterity at Trickr, or Humanising the Developers (Part 1) and (Part 2)

The Setup, check it out.

http://flickr.com/photos/tags/1234567890/

☆＼（*＾▽＾*）／☆