Category Archives: Uncategorized

Flipping Out

Posted on by

Flickr is somewhat unique in that it uses a code repository with no branches; everything is checked into head, and head is pushed to production several times a day. This works well for bug fixes that we want to go out immediately, but presents a problem when we’re working on a new feature that takes several months to complete. How do we solve that problem? With flags and flippers!

Feature Flags

Canadian Flag

Flags allow us to restrict features to certain environments, while still using the same code base on all servers. In our code, we’ll write something like this: 

if ($cfg.enable_unicorn_polo) {
    // do something new and amazing here.
}
else {
    // do the current boring stuff.
}

We can then set the value of $cfg.enable_unicorn_polo on each environment (false for production, true for dev, etc.). This has the additional benefit of making new feature launches extremely easy: Instead of having to copy a bunch of new code over to the production servers, we simply change a single false to true, which enables the code that has already been on the servers for months.

Feature Flippers

Knife-switch

Flags allows us to enable features on a per environment basis, but what if we wanted to be more granular and enable features on a per user basis? For that we use feature flippers. These allow us to turn on features that we are actively developing without being affected by the changes other developers are making. It also lets us turn individual features on and off for testing.

Here is what the Feature Flip page looks like:

Feature Flipper

Continuously Integrating

Feature flags and flippers mean we don’t have to do merges, and that all code (no matter how far it is from being released) is integrated as soon as it is committed. Deploys become smaller and more frequent; this leads to bugs that are easier to fix, since we can catch them earlier and the amount of changed code is minimized.

This style of development isn’t all rainbows and sunshine. We have to restrict it to the development team because occasionally things go horribly wrong; it’s easy to imagine code that’s in development going awry and corrupting all your data. Also, after launching a feature, we have to go back in the code base and remove the old version (maintaining separate versions of all features on Flickr would be a nightmare). But overall, we find it helps us develop new features faster and with fewer bugs.

On The Importance of Fun (And Some Holiday Snow)

Posted on by

Since we’re based in (mostly-) sunny San Francisco, Flickr’s code monkeys will be away for a few days eating excessive amounts of turkey and being appropriately thankful. Before heading off with friends and family, I thought I’d share a bit of “seasonal” JavaScript we have on the site, and a few notes about its inner workings.

Fun is Good

We build a lot of “serious” stuff at Flickr HQ, but we also recognize why it’s crucial that we save some time for the small things, the goofy and occasionally-irreverent parts of the site that remind people of a playful spirit. Outside of our daily work, many of us have our own personal geekery going on. It’s no coincidence that shiny, and even silly things people tinker with and build in their own time sometimes bleed over and become features or elements on Flickr; ideally, great ideas come from all sorts of places.

Let It Snow

Lights in the Snow (with ?snow=1), by mezzoblue

Last holiday season, we got around to making it snow on photo pages – just for the fun of it. If you happen to be on a photo.gne page and add ?snow=1, it might still even work.. We’ve also used a variant of the effect in the past for beta feature sign-up sequences, complete with cheesy MIDI renderings of “The Girl From Ipanema.” Again, this was “just because”; sometimes the web is most entertaining when it’s most unexpected. If you can occasionally make your users smile, giggle and laugh when using your site, chances are you’re doing something right – or, you’re running a comedy site that’s going downhill.

Wait, Snow is Hard

The snow effect was a JavaScript experiment made strictly as a test of DOM-2 event handlers, PNG and animation performance. Or, viewed in a more-cynical light, it was an evil plot to pollute websites with falling animated .GIFs and light CPUs on fire world web-wide.

To achieve a realistic effect, each snowflake has its own size, “weight” and x/y velocity properties which can be affected by “wind”. A slight parallax effect is achieved by the notion that larger flakes are closer to the screen and move faster, and so on.

It may not be surprising to note that it’s still expensive CPU-wise to animate many individual elements simultaneously. The effect is much smoother and has higher frame-rates these days when compared to early versions from 2003, but even modern browsers will readily eat up 50% CPU while running the effect. That said, the animation is keyed off of a single setInterval()-style loop with a low interval, so it will effectively try to run as fast as it can.

Revisiting Performance

“In theory there is no difference between theory and practice. But, in practice, there is.” — Yogi Berra

I had a theory that using text elements might be more efficient than image-based snow to animate, so I made the switch to using elements with the bullet entity • instead of PNGs with alpha transparency. No noticeable improvement was actually seen, despite my theories about drawing and moving images; HTTP requests were being saved, but the browser was still doing a lot of redraw or reflow work despite the elements using either absolute or fixed positioning. On my netbook-esque laptop with Firefox 3.5 under WinXP, animation would “freeze” when scrolling the window with position:fixed elements – presumably because my single interval-based JavaScript timer was blocked from running while scrolling was active.

In retrospect, what might be more efficient for overall CPU use is a number of absolutely-positioned “sheets” using a tiling background image with a pattern of snow. Each sheet of snow would move at the same speed and angle, but the number of unique elements being animated would be drastically reduced. JavaScript-based animation is not a science in any case, and having a large number of elements actively moving can significantly impact browser responsiveness. While this is not a common web use case for animation, it is interesting to note how the different browsers handle it.

Make Fun Stuff, and Learn!

Much of what I enjoy about front-end engineering is the challenge. I’ve learned a lot about browser performance and quirks just from prototypes and experiments, which can then be used in real production work.

While cross-browser layout and performance varies, it’s also rewarding to work on the occasional crazy idea or silly prototype and then refine it, features and quality-wise, to a point where it’s ready for a production site somewhere. JavaScript-based snow is at best a “perennial” feature, but the process of thinking about how to make something different and new work, and work well (theory) – and then researching, testing and hacking away to actually do it (practice) – is where the learning comes in.

Building authorized Flickr apps for the iPhone

Posted on by

only

You want to develop an iPhone app that interacts with Flickr content? This sounds pretty good. And the Flickr API provides you with an authorization workflow that is particularly adapted for this device. And Flickr members love their iPhones.

An authorization workflow you say, but why? Let me step back for a moment. First you may not need to authorize your application. This is needed only if your app needs to make authenticated API calls. That means if the users of your app access in a non anonymous way the Flickrverse: accessing non-public content, commenting, tagging, deleting, etc… In order for this to happen in a safe environment (for you and our users), a 3-way authorization needs to happen between Flickr, you and our mutual users, typical of social engineering interactions.

So how does this work?

Step 0

You need to setup your application.

  • Get an API key that uniquely identifies your application.
  • Configure your application to be web-based (yes, this can seem odd but this is the smoothest user experience on the iPhone) and specify the authorization callback URL (it will be used in Step 3.) I suggest not to use the http:// protocol reserved for the web but your own, like myapp://

There exists already a workflow explanation from developer stand point, but I’d like to add the specificities introduced by the use of a web-based authorization in an iPhone environment.

In the app, for each user you want to authorize:

Step 1

Create a login URL, specific to you application in the shape of http://flickr.com/services/auth/?api_key=%5Bapi_key%5D&perms=%5Bperms%5D&api_sig=%5Bapi_sig%5D and launch a web browser with this URL.

Step 2

Flickr will ask the user to sign-in into their account and present them with a page prompting them to authorize your application.

Step 3

If the user decides to authorize your application, Flickr will call the callback URL specified in Step 0. Here is the nice trick! This can actually launch back your application. All you have to do is to add a new entry for CFBundleURLTypes in your Info.plist:

 <key>CFBundleURLTypes</key>
 <array>
   <dict>
     <key>CFBundleURLName</key>
     <string>MyApp's URL</string>
     <key>CFBundleURLSchemes</key>
     <array>
       <string>myapp</string>
     </array>
   </dict>
 </array>

See Apple’s documentation on Implementing Custom URL Schemes for more details.

Step 4

Your application is launched back by Flickr (through the browser and the iPhone OS) with a frob as one of the argument of the URL. The app is effectively a Flickr auth handler. You can implement application:handleOpenURL: in a similar way as:

 - (BOOL)application:(UIApplication *)application handleOpenURL:(NSURL *)url
 {
        if (NSOrderedSame == [[url scheme] caseInsensitiveCompare:@”flickrApp”]) {
                // query has the form of "&frob=", the rest is the frob
                NSString *frob = [[url query] substringFromIndex:6];

                // Keep the frob for Step 5 and schedule Step 5

               return YES;
        } else {
               Return NO;
        }       
  }

Step 5

Your app makes an API call to convert this frob into a token. The frob is valid only for a certain time. The token will be used for the API calls that require authentication. This token is what uniquely identifies the use of your API key for a specific user on Flickr. You can save it using NSUserDefaults for next time the user uses the application without having to reauthorize the application. Even better to use KeyChain. Note that you should use checkToken to make sure the user has not de-authorized the application otherwise your authenticated call may fail for no apparent reasons.

I would like to take the opportunity of this blog post to recommend an excellent library to develop iPhone apps interacting with flickr: ObjectiveFlickr.

Have a good hack!

Jérôme Decq, from his home outside of Paris, singled handedly runs the Flickr Desktop Uploadr development, as well as hacking on making Flickr avaiable on a wide range of platforms, and photographing purple ducks.

In case you wanted to bake us a cake ….

Posted on by

Ian Sanchez has proposed a new geo data “test for freeness” in the spirit of the Debian project’s tests for free software.

A set of geodata, or a map, is libre only if somebody can give you a cake with that map on top, as a present. – Ian Sanchez

I mention this because the Flickr Shapefiles can be used unencumbered as a cake decoration. And we like cake. We like photos of cake as well. But we prefer cake.

Introducing The App Garden

Posted on by

Flickr has long had an extensive, well-documented API. Over the years, thousands of developers have taken advantage of it, coming up with some awesome apps. We love that.

We love it so much, we’ve revamped the /services/ section of Flickr, replacing it with The App Garden. What is it, you say? It’s a place for developers to promote their apps, right here on Flickr. We hope that it will make it easier for Flickr users to find the awesome apps that the Flickr API hackers have been building.

You will see that The App Garden already has some apps in it, and you might think “OOOH SHINY!!” You might also wonder how to get your app into the App Garden. I will show you!

Getting Started

We’ve tried to make things as simple and straight-forward as possible. You will find all of your API Keys under the Apps By Me page, which replaces the old API Key list. You will notice that they are all labeled as “Private” – we leave it up to you to decide when your app page is ready to be made public.

When you click on one of your apps, you will be taken to the owner view of your app page. This page is where you tell the world about your app – provide a description, link to a website, set screenshots, and add tags. When you’re ready, change the privacy setting to public. That will make your app visible to other users and allow it to show up in searches.

Managing Your Apps

Below the privacy settings, you will find the Admin section of the sidebar – your own little command center. You will find a link to a page with statistics for the app’s API Key (largely unchanged, though developers with higher user counts may notice a considerate speed up), as well as pages for disabling the key, editing the authentication flow for the key, and deleting the app altogether.

We love our API hackers and are happy to embrace them in a whole new way. We hope you like it.

More Info

App Garden FAQ
What is the App Garden?

Small Bridges (to Proximate Spaces)

Posted on by

photo by jordi ventura

A couple months ago Tom Taylor and Tom Armitage launched a web-based game based around geotagged Flickr photos called noticin.gs.

Noticings is a game about learning to look at the world around you.

Cities are wonderful places, and everybody finds different things in them. Some of us like to take pictures of interesting, unusual, or beautiful things we see, but many of us are moving so fast through the urban landscape we don’t take in the things around us.

Noticings is a game you play by going a bit slower, and having a look around you. It doesn’t require you change your behaviour significantly, or interrupt your routine: you just take photographs of things that you think are interesting, or things you see. You’ll get points for just noticing things, and you might get bonuses for interesting coincidences.

Maybe it’s just me (I don’t think so) but this is precisely the sort of thing we always hoped people would build on top of the Flickr API.

You “play” noticin.gs by uploading geotagged photos to Flickr and tagging them noticings. At the end of each day the noticin.gs robots query the Flickr API for new photos and assign points to each photo. Points are awarded for being the first noticing in a neighbourhood, because a player noticed something every day at lunchtime and so on; the scorings change and adapt with the game itself.


photo by Ben Terrett

Tom Taylor and I started talking about adding the “machine tags extras love” (remember, that is actually now a technical term on the Flickr engineering team) a while back. One idea was use the photo’s (noticin.gs) score as the key back in to their world but that seemed like an odd fit. Knowing the score for a photo doesn’t tell us how those points were awarded which is, really, the interesting part and what would we link to?

I’ll come back to the what-do-we-link-to part again later.

As it happens, every single noticing has its own web page and a unique ID that, conveniently, is the same as the photo that was noticed so we settled on noticings:id=PHOTO_ID as the tag that will be “expanded”. If it’s present we’ll ask the noticin.gs servers for the list of reasons that photo was awarded points and display the one with the highest score linking back to the noticin.gs page for that ID.

You can either add the special machine tag yourself or ask noticin.gs to do it for you automatically. To enable automagic machine tagging you’ll need to log in to noticin.gs and change the default settings. If you’re worried about creating yet another account for an yet another online service, don’t be: noticin.gs uses the Flickr Auth API itself to manage user accounts so “logging in” is as simple as authorizing noticin.gs to access your Flickr account (the way you would any other Flickr API application).

This is what it looks like once you’ve logged in:

Paul Mison has a lovely post about noticin.gs where he describes the game as “the biggest change to the way I post photos” to Flickr. That kind of thing makes all the bad days worth it.

Don’t forget: You can subscribe to an RSS feed of all the new photos machine tagged with noticings:id= and since the photos should all be geotagged already you can also create a network link for new photos and hop around from noticing to noticing in Google Earth.

Which is as good a segue as any to show a picture of a space ship. A “space ship.” I like this picture because it reminds me of machine tags.


photo by mattcottam

Which is as good a segue as any to talk about trains. But not just trains. Machine tags for trains. Actually, train stations.

We have a lot of pictures of train and subway stations. A casual search for the words subway OR metro alone yields 1.5 million results. If you add the word train to the list you get 5.7 million. That’s just searching for stuff in English.

Unfortunately, few transit systems have websites with pages dedicated to each station in their network (we’ll cut them some slack as they’re busy, you know, running the trains). A few do but none of them seem to have much in the way of a public API, even something as simple as a getInfo method.

So, a couple weekends ago I created Fake Subway APIs, which is a plain-vanilla XML-over-HTTP API service for returning information about train and subway stations. It doesn’t do much right now except return the name and URI for a station given its short code.

I did this because I wanted to make sure that the code we run to determine the “meaning” of a given machine tag always expects to be asking someone else for the answer. Even if a fake subway API is little more than a canned list of IDs and names it seemed important to go through the motions of treating machine tag extras as something external to Flickr.

My hope is that Fake Subways APIs will become irrelevant sooner rather than later, as individual services start building this stuff themselves. For now, though, it works and it means we can enable the “machine tags extras love” for four transit systems: BART in the San Francisco Bay Area, the metro (STM) in Montréal; Transport for London (aka the “Tube”) in, well, London; the National Rail Service in the UK.

The syntax is the same for all of them:

service name + ":station=" + station code

Like this:

  • For BART : bart:station=16th

    A complete list of BART station codes is available over here.

  • For the STM (aka the “metro”) : stm:station=m48

    A complete list of STM station codes is available over here.

  • For TFL (aka the “Tube”) : tfl:station=LON-N

    TFL machine tags are a bit of a bear compared to the others. Specifically, you need to indicate both the station code and the line code. This is a consequence of the way the TFL website is set up. A complete list of TFL station codes is available over here.

  • For the UK National Rail System : ukrail:station=HMN

    A complete list of National Rail station codes is available over here.

We chose those four because they were the ones which I knew to have a webpage for each station that could be linked to (or in the case of London Underground could be teased out because, let’s be honest, it’s the Tube) at the end of a machine tag.

Since all of this has started, hooks for the MBTA in Boston and the TTC in Toronto have been added to Fake Subway APIs so it seems reasonable to expect that we’ll add support for them here too.

Check it out, a train:

If your subway system isn’t listed please don’t take it personally. I work on this in the mornings over coffee and weekends when I’m sick and should be resting. The entire project is open source and I’d welcome contributions. Munich, for example…

One glaring omission is the New York City subway system, sometimes known as the Metropolitain Transportation Authority (MTA), because they don’t have proper webpages for the stations they operate. Fake Subway APIs provides a (fake) MTA API and even has fake/place-holder subway pages for each of the stations but where’s the fun in that?

One of the goals with the machine tags project has been to deliberately link outwards to the various sites, and services, rather than funnel everything through a single channel. A “small bridges” approach instead of an all roads lead to [INSERT BIG SITE HERE] model, so to speak.


photo by antimega

(Speaking of tubes…)

We’ve certainly had discussions around the idea of using Wikipedia as a sort of universal content resolution system, for things or people otherwise “missing” from the Interwebs. Tim Bray wrote a really good piece about this called “On Linking” a couple years ago. It’s not that we don’t love Wikipedia or support what they’re doing and they almost certainly have the most comprehensive list of trains stations anywhere on the Internet.

It’s just that we’d like to actively encourage as many people as possible to participate in what Tom Coates’ called the “Web of Data“, in a presentation in 2006, making their data available to to both humans and machines but also maintaining authorship of all that crunchy goodness. Tom’s slides are a bit opaque on their own and to date the best telling of the presentation has been Simon Willison and company’s collaborative note-taking, which I’ve liberally excerpted here:

Every new service that you create can potentially build on top of every other existing service.

Every service and piece of data that’s added to the web makes every other service potentially more powerful.

So the same things that keep the hippies happy keep the evil capitalists happy. They all have to play in the same ecosystem. If not, you end up in a backwater, disconnected from the cool stuff that’s happening. strength in sharing and participating.

So far, it’s an idea that worked pretty well for us if all the amazing stuff people have built on to top of the API is any measure.

If you look closely you’ll notice that I’ve had to link to an (Internet Archive) archived version of Simon’s site from 2006 since the notes are nowhere else to be found. There is still, obviously, lots of work left to be done no matter which road you prefer.

Also: While we’re talking about Wikipedia, Josh Clark’s Wikipedia Machine Tag Generator, which he built during the Yahoo! BBC Hack Day event in 2007, is just plain awesome.

So, where are we going with all of this? It’s a bit too soon to tell but one of the things I like about all of the recent machine tag work is that they start to expose geographies outside of the traditional grid of latitudes and longitudes. If that sounds a bit wooly and hand-wavey that’s because it is.

In concrete terms, one thing that’s pretty exciting is the ability to infer location for all those photos that aren’t geotagged yet but do have Upcoming, or foursquare, or OpenStreetMap machine tags or, yes, even train stations. All those services have their own APIs and aside from just pulling back coordinates you can use them to fill in simple, but important, details like whether a photo was taken indoors or outdoors.

And if we’re lucky they’ll start to show us the donut holes and the “place fields” (props to Matt Jones’ delicious links for that one) that we walk through every day but don’t recognize or don’t have names for yet.

photo by JLB

“That’s maybe a bit too dorky, even for us.”

Posted on by

Around the time we added support for Open Plaques machine tags Frankie Roberto, the project lead, asked: “What about supporting Open Street Map (OSM) way machine tags?”

My immediate response was something along the lines of “That’s maybe a bit too dorky, even for us.” Which meant that I kept thinking about it. And now we’re doing it.

If you’re not sure way what a “way” is, it’s best to start with OpenStreetMap’s own description of how their metadata is structured:

Our maps are made up of only a few simple elements, namely nodes, ways and relations. Each element may have an arbitrary number of properties (a.k.a. Tags) which are Key-Value pairs (e.g. highway=primary) …

A node is the basic element of the OSM scheme. Nodes consist of latitude and longitude (a single geospacial point) …

A way is an ordered interconnection of at least 2 and at most 2000 nodes that describe a linear feature such as a street, or similar. Should you reach the node limit simply split your way and group all ways in a relation if necessary. Nodes can be members of multiple ways.

Frankie’s interest is principally in marking up buildings in and around Manchester, where he lives. When he tags one of his photos with osm:way=30089216 we can fetch the metadata (the key-value pairs) for that way using the OSM API and see that it has the following properties:

<osm version="0.6" generator="OpenStreetMap server">
	<way id="30089216" visible="true" timestamp="2009-07-04T12:02:47Z" version="2" changeset="1728727" user="Frankie Roberto" uid="515">
		<nd ref="331415447"/>
		<nd ref="331415448"/>
		<nd ref="331415449"/>
		<nd ref="331415450"/>
		<nd ref="331415447"/>
		<tag k="architect" v="Woodhouse, Corbett & Dean"/>
		<tag k="building" v="yes"/>
		<tag k="created_by" v="Potlatch 0.10f"/>
		<tag k="name" v="St George's House"/>
		<tag k="old_name" v="YMCA"/>
		<tag k="start_date" v="1911"/>
	</way>
</osm>

That allows to us “expand” the original machine tag and display a short caption next to the photo, in this case: “St George’s House is a building in OpenStreetMap” with a link back to the web page for that way on the OSM site.

The technical terms for this process is “Adding the machine tags extra love“.

You may have noticed that there are a bunch of other key-value pairs in that example, like the name of the architect, that we don’t do anything with. Which attributes are we looking for, then? The short answer is: Not most of them. The complete list of map features in OSM is a bit daunting in scope and constantly changing. It would be nice to imagine that we could keep pace with the discussions and the churn but that’s just not going to happen. If nothing else, the translations alone would become unmanageable.

Instead we’re going to start small and see where it takes us. Here are the list of tagged features in a way or node definition that we pay attention to, and how they’ll be displayed:

  • k=name v={NAME}
    … is a feature in OpenStreetMap (If present, with another recognized tag we will display the name for the thing being described in place of the more generic “this is a…”)

  • k=building v=yes
    … is a building in OpenStreetMap

  • k=historic
    … is an historic site in OpenStreetMap

  • k=cycleway
    … is a bicycle path in OpenStreetMap

  • k=motorway (v=cycleway)
    … is a highway in OpenStreetMap (unless v is “cycleway” in which case it’s a bike path)

  • k=railway v=subway (or tram or monorail or light_rail)
    … is a subway (or tram or monorail or light_rail) line in OpenStreetMap

  • k=railway v=station
    … is a train station in OpenStreetMap; if the type of railway is also defined (above) then we’ll be specific about the type of station. I should mention that as of this writing we’re still waiting for the translations for “this is a train station” to come back because I, uh… anyway, real soon now.

  • k=waterway v=stream (or canal or river)
    … this is a stream (or canal or river) in OpenStreetMap

  • k=landuse v=farm (or forest)
    … this is a farm (or forest) in OpenStreetMap

  • k=natural v=forest (or beach)
    … this is a forest (or beach) in OpenStreetMap

Which means: We’ve almost certainly got at least some of it wrong. Anyone familiar with OSM features will probably be wondering why we haven’t included amentiy or shop tags since they contain a wealth of useful information. I hope we will, but it wasn’t clear how we should decide which features to support (more importantly, which to exclude) and the number of possible combinations were starting to get a bit out of hand and we have this little photo-sharing site to keep running.

This is the part where I casually mention that we’ve also added machine tags extra love for Four Square venues IDs. I’m just saying…

The features we’re starting with may seem a bit odd, with a heavy focus on natural land features (and train stations). Some of this is a by-product of the work we’ve been pursuing with the alpha shapes and “donut holes”, derived from geotagged photos, and some of it is just trying to shine the spotlight on places and environments that we take for granted.

Like I said, we’ve almost certainly got at least some of it wrong but hopefully we got part of it right and can correct the rest as we go. This one is definitely a bit more of an experiment than some of the others.

Finally, in the tangentially related department we finished wiring up the RSS/syndication feeds to work properly with wildcard machine tags. That means you can subscribe to a feed of all the (public) photos tagged with osm:way= or osm:node= or, if you’re like me, all the photos of places to eat in Dopplr with dopplr:eat=.

Enjoy!

Flickr is hiring

Posted on by

We’re looking for a few talented geeks to join our panda-wrangling, daily-deploying, shard-juggling squad, based in downtown San Francisco.

Flickr has openings for PHP and Front-end Engineers, an Engineering Manager, a MySQL DBA, Product Managers and a Designer.

papa: working on the Univac

If you think that you’ve got the chops, hop over to our jobs page for information on how to apply.

“Introducing astrotags”

Posted on by

The Royal Observatory Greenwich has posted an absolutely lovely video about “astrotags“, writing:

“Astrotags are a new way to label your astronomy photos with their celestial subject and its location. This short film, made by Jim Le Fevre and Mike Paterson for the Royal Observatory’s Astronomy Photographer of the Year exhibition, shows you how. So have a watch, then astrotag your pictures at the Astronomy Photographer of the Year group on Flickr. If everyone joins in we can make a beautiful and accurate map of the night sky… so pass the word on.”

We’ve written about astrotags before, in a couple of posts titled “Found in Space” and “Tags in Space“, and earlier this year Fiona Romeo, Head of Digital Media at the National Maritime Museum, spoke about the Observatory’s astrotagging project asking the question “what’s the space equivalent of geotagging”? at Webstock09.

Tangentially related, we’ve also updated the wildcard machine tag pages to display related tags based on the current namespace or predicate. For example, if you go to /photos/tags/astro:name= you’ll see these other related tags in the sidebar on the left:

Picture 10

Now we just need people to make some astrotagging galleries!

5 questions for Matt Biddulph

Posted on by

Matt Biddulph

Matt Biddulph takes lovely photos. And apparently when he isn’t taking photos he find times to be the founder and CTO of Dopplr. Which makes us happy, because we’re big Dopplr fans. If Flickr is the site where photos become social objects, Dopplr is that site for travel. Except with personal informatics, and positionally dependent rainbows. (we have a well documented weakness for rainbows)

Dopplr is consistently one of the most interesting projects tying together the loosely joined datastreams into a cohesive whole, and Matt’s meditations on how to do that are not to be missed.

Dopplr personal informatics coffeecup

1. What are you currently building that integrates with Flickr, or a past favorite that you think is cool, neat, popular and worth telling folks about? Or both.

Matt: I’m the CTO at Dopplr, and we use Flickr all over our site to integrate our users’ travel pictures and illustrate places in our “Social Atlas”. We’re particularly happy with Flickr’s recent integration of Dopplr places into their automatic machine tag handling

This year we’ve been radically upgrading our city information pages (e.g. http://dplr.it/san-francisco) and Flickr photos have been an important part of that. Designer Matt Jones came up with a combination of rich information visualisation and beautiful images, combining sparklines with photos of cities that we source from Flickr’s geotagged Creative-Commons licensed contributions.

The Flickr API is key to how these pages are made. Although ‘interestingness’ is a powerful sorting metric, we found that just asking for the most interesting picture of a city didn’t always give us the right aesthetic (just as Dan Catt’s pandas occasionally show a preference for bikinis). Instead, our developer Tom Insam built a workflow tool that shows site admins a list of the top pictures per place. It displays a preview of a sample data visualisation overlay, and limits the pictures to those licensed for derivative commercial works.

An admin can quickly work through our most popular cities, choosing pictures and a few design settings (such as whether to use black or white text for best contrast). Once a month, a script runs through our chosen images, regenerating this month’s data viz and removing any photos whose license has changed since our last check. This happens more often than we expected.

2. What are the best tricks or tips you’ve learned working with the Flickr API?

Matt: To make our city page Flickr integration possible, we had to correlate our city database with Flickr’s geotagging data. Our data is sourced from GeoNames and augmented by us and our users. Luckily Flickr provides a handy reverse-geocoder that maps a latitude/longitude point to an identifier. As a bonus, Flickr now uses the same “WOE IDs” as Yahoo’s Geo APIs, meaning that this correlation between our IDs and theirs can be used to lookup other rich geo data elsewhere on our site. We also started supporting WOE IDs in our own API in return, and we were very happy when Flickr started linking to our city pages from their own place pages.

Another tip for integrating Flickr API calls into web pages is to consider whether you can achieve a feature with purely client-side code. Rather than have our webserver tied up waiting for an HTTP request to return from Flickr, in certain places we serve up jQuery code to perform Flickr queries from the clientside, inserting the results into the page via ajax.

3. As a Flickr developer what would you like to see Flickr do more of and why?

Matt: I’m in complete agreement with Kellan — that realtime APIs are going to be big. For example, I want the ability to register an interest in something (“the latest CC-licensed photos of London”) and get the data pushed to me whenever new results are ready. This might be implemented using Web Hooks, like when github makes an HTTP POST for you whenever you push into a repository. It might be over XMPP, as used by Fire Eagle, when services register for publish-subscribe notifications of new location pings. The choice of technology is less important than the new possibilities (and design challenges) of the realtime web.

4. What excites you about Flickr and hacking? What do you think you’ll build next or would like someone else to build so you don’t have to?

Matt: The best thing about Flickr for hacking is the sheer amount of data that the community collects. The recent release of shapefiles derived from tags and geotagging is very exciting and has huge possibilities for geo mashups. I’m looking forward to firing up a Maps From Scratch instance and figuring out how to combine it with data from our service. There are some screenshots of my early experiments using Flickr’s clustr tool on our Social Atlas data at http://www.flickr.com/photos/mbiddulph/tags/clustr/

5. Besides your own, what Flickr projects and hacks do you use on a regular basis? Who should we interview next?

Matt: I nominate Dave Beckett, author of the C library and commandline tool Flickcurl because I use the commandline tool as a quick interface to Flickr data all the time. As a devotee of dynamic languages, I’m interested to hear about what it’s like to program the web in C.