Much Photos!

Introducing the New! Shiny! Photolist framework

photolist-sky
Blue skies. Mostly.

Here at Flickr, we have photos. Lots of photos. Like, billions and billions of photos. So, it’s pretty important for us to be able to show you more than one at once.

We have used what we call the “justified algorithm” to lay out photos for a while now, but as we move more and more pages onto our new-ish isomorphic node.js stack, we determined it was time to revisit the algorithm and create an updated implementation.

A few of us here in Frontend-landia got together to figure out all the things this new shiny should be able to do. With a lot of projects in full swing and on the near horizon, we came up with a pretty significant list, including but not limited to:
– Easy for developers to use
– Fit into any kind of container
– Support pagination (in both directions!) and infinite scroll
– Jank-free, butter-silky-baby-smooth scrolling
– Support layouts other than justified, like square thumbnails and grid layout with native aspect ratio

After some brainstorming, drawing of diagrams, and gummi bear consumption, we got to work building out the framework and the underlying algorithm.

Rejustification

photolist-sky
Drawing of diagrams

The basics of the justified algorithm aren’t too complex. The goal is for the layout module to accept a list of photo aspect ratios, and return a list of rectangles. A layout consists of a number of rows of items (photos), each with a target height and allowable height deviation above and below. This, along with the container width, gives us a minimum and maximum row aspect ratio.

Photolist: variable row height
Fig. 1: The justified algorithm: dimensions

We push each photo into a row; once the row is filled up, we move on to the next. It goes a little something like this:

  1. Iterate over each photo in the list to display
  2. Create a new row if there’s not currently an open row
  3. Attempt to add the photo to the current row at its native aspect ratio and at the target row height
  4. If the new row aspect ratio is less than the minimum row aspect ratio, continue adding photos until the aspect ratio is greater than the maximum aspect ratio
  5. Either keep or drop the last added photo, depending on which generates a row aspect ratio closer to the target row aspect ratio; adjust the row height as needed, and seal the row
  6. Repeat until all the photos have been laid out.

Photolist: row filling
Fig. 2: The justified algorithm: row filling

It’s Never That Easy…

The justified algorithm described above is the primary responsibility of the layout module. In practice, however, there are a number of other things the layout must handle to get good results for all use cases, and to communicate the results to other parts of the framework.

Diffs

One key feature of the layout module is how it organizes its results. To minimize the amount of processing required to update photos as the layout changes, the layout module returns pre-sorted diffs, each with a specific purpose:

  • new items, used to create new photos and put them in place
  • layout-changed items, used to resize/reposition existing photos
  • visibility-changed items, used to wake/sleep existing photos
  • widows and orphans (leading and trailing items) (read on!)

The container view can process only the parts of the layout response that are necessary, given the current state of the whole framework, to keep processing time down and keep performance up.

Widows and orphans


Annie the Musical,
Annie the Musical, by Eva Rinaldi

Some photolist pages on Flickr use infinite scrolling, and some display results one page at a time. Regardless of how a page shows its photos, it starts to feel messy when there is an incomplete row of photos hanging off the end of the page. If there is more content in the set, the last row should be full. However, since we fetch photos from the API in fixed batch sizes, things don’t always work out so nicely, leaving “leftovers” in the bottom row. Borrowing from typesetting terminology, we call these leftover photos orphans. (We can also paginate backwards; leftovers at the top are technically widows but we’ll just keep using the term orphans for simplicity.)

The layout notes these incomplete rows and hides them from the rest of the framework until the next page of content loads in. (This led to frequent and questionable metaphors about “orphan suppression” and “orphan rehydration.”) When orphans are to be hidden, the layout simply keeps them out of the diff. When the orphans are brought back in as the next page loads, the layout prepends them to the next diff. The container view is none the wiser.

This logic gets even more fun when you consider that it must perform in all of these use cases:

  • fixed page size (book-style) pagination
  • downward-scrolling infinite pagination
  • upward-scrolling infinite pagination (enter into an “infinite” content set somewhere other than the beginning); this requires right-to-left layout!

There’s also the case of the end of an “infinite” content set (scrolling down to the end or up to the beginning); in these cases, we still want the row to appear complete, and still must maintain the native aspect ratio of the photo. Therefore, we allow the row height to grow as much as it needs in this case only.

Bonus Round!

You might have noticed that Flickr is kind of a big site with, like, lots of photos. And we display photos in lots of different ways, with lots of different use cases. The photolist framework bends over backwards to support all of those, including:

  • forward and backward pagination
  • infinite scroll and fixed page-size pagination
  • specific aspect ratios (e.g. squares)
  • fixed number of rows
  • fast relayout (only a few milliseconds for thousands of photos)

Going into detail on each of those features is way beyond the scope of this blog post, but suffice it to say the framework is built to handle just about anything Flickr can throw at it. The one exception is the upcoming Camera Roll (coming soon to those of you who don’t yet have it!), which is Too Extreme for this framework, so we devised something special just for that page.

The whole enchilada


Mmm... enchiladas,
Mmm… enchiladas, by jeffreyww

The layout is at the heart of the photolist framework, but wait — there’s more! The main components of the framework are the layout (dissected above), the container view / controller, and the subviews (usually containing photos).

Photolist: the whole enchilada
Fig. 3: Relationship of view/controller, layout, and subviews, and changing subview states during downward scroll

The container view does a lot of fancy things, like:

  • loading in photos as you scroll down or paginate
  • triggering a relayout when the container size changes (i.e. when you resize your window)
  • matching up server-rendered HTML with clientside JavaScript objects (see isomorphic JavaScript, and an upcoming blog post about the Hermes stack at Flickr).

Its primary job, though, is to act as the conduit between the layout module and the individual subviews.

Every time a layout is processed or changes, it returns a “layout response” to the container view. The layout response contains a list of rectangles and wake/sleep flags (actually, a list of lists; see Diffs above); the container view relays that new information on to each individual subview to determine position and visibility. The container view doesn’t even need to know about the layout details — each subview adjusts itself to its layout data all on its own.

The subviews each have a decent amount of intelligence of their own, performing such tasks as:

  • choosing the most appropriate photo file size to fit the layout rectangle
  • adding/removing itself to/from the DOM as instructed by the layout to maintain good scroll performance
  • providing an annotation and interaction layer for titles, faves, comments, etc.

Coming soon to a webpage near you

The new photolist framework is certainly not a one-size-fits-all solution; it’s tailored for Flickr’s specific use cases. However, we tried to design and build it to be as broadly useful for Flickr as possible; as we continue to move parts of the site onto the new frontend stack and innovate new features, it’s critical to have solid components upon which we can build Flickr’s future. The layout algorithm is probably useful for many applications though, and we hope you gained some insight into how you might implement your own.

The photolist framework is already live in a number of places on the site, including the new Unified Search pages (currently in Beta), the Create / Wall art pages, the Group pool preview, and is coming soon to a number of other pages.

As always, if you’re interested in helping with that “and more” part, we’d love to have you! Stop by our jobs page and drop us a line.

33 Browser Stats You Just Might Believe

We care an awful lot about the kinds of browsers and computers visiting Flickr. As people update to the latest versions of their browsers, the capabilities we can build against improve, which lets us build cool new things. At the same time, if lots of people continue using older browsers then we have to do extra work to gracefully support them.

These days we not only have incredibly capable browsers, but thanks to the transparent and rapid update process of Chrome, Firefox, and soon Internet Explorer (hooray!), we can rely on new features rapidly showing up en masse. This is crazy great, but it doesn’t mean that we can stop paying attention to our usage statistics. In fact, as people spend more time on their phones, there’s as much of a need for a watchful eye as ever.

We’ve never really shared our internal numbers, but we thought it would be interesting to take a look at the browsers Flickr visitors used in 2014. We use these numbers constantly to inform our project planning. Since limitations in older browsers take time to support we have to be judicious in picking which battles to fight. As you’ll see below, these numbers can be quite dynamic with a popular browser dropping to nearly 0% market-share in just a year. Let’s dive in and see some specifics.

Fort Vancouver
Fort Vancouver by Kate Dickerson

Top level OSes and browsers

At the highest level we learn a lot by looking at our OS family data. Probably the most notable thing here is how much of our traffic is coming from mobile devices. Moreover, the rate of growth is eye-popping. And this is just our website – this data doesn’t include our iOS or Android clients at all. A quarter of our traffic is from mobile devices.

OSes in use on Flickr.com
2013 Q4 2014 Q4 Y/Y
Windows 56.55% 50.61% -5.94
Macintosh 21.49% 21.42% -0.07
iOS 11.09% 17.61% 6.52
Android 5.39% 7.82% 2.43
Other 5.48% 2.54% -2.94

Let’s slice things slightly differently and look at browser families. We greatly differ from internet-wide traffic in that IE isn’t the outright majority browser. In fact, it clocks in at only the #4 position. More than half of Flickr visitors use a Webkit/Webkit-heritage browser (Safari and Chrome, respectively). Chrome rapidly climbed into its leadership position over the last few years and it’s stabilized there. Safari is hugely buoyed by iOS’s incredible growth numbers, while IE has been punished by Windows’s Flickr market-share decline.

Browsers in use on Flickr.com
2013 Q4 2014 Q4 Y/Y
Chrome 35.71% 35.42% -0.29
Safari 24.11% 27.50% 3.39
Firefox 17.94% 18.29% 0.35
Internet Explorer 13.98% 10.31% -3.67
Other 8.26% 8.48% 0.22

Fine-grained details

We can go a step further and see many details in the individual versions of OSes and browsers out there. It’s one thing to say “Windows is down 6% over the year” but another to say “the growth rate for the latest version of Windows is 350% year over year.” When we look at the individual versions we can infer quite a bit of detail around update rates and changes in the landscape.

OS version details

A few highlights:

  • Windows 7 is on the decline, XP and Vista fell by roughly 50% each, and Windows 8 and 8.1 are surging ahead.
  • iOS 8.1 and Android 5.0 don’t appear in the list due to their late appearance in Q4. Our current monthly numbers have iOS 8.1 far outpacing every other iOS version.
  • OS X 10.10 has accelerated Mac user upgrades; since its launch 10.9 has shed over a percent per month, and the legacy versions have sharply accelerated their decline.
OS versions in use on Flickr.com
2013 Q4 2014 Q4 Y/Y
Windows NT 3.39% 0% -3.39
Windows XP 10.12% 4.49% -5.63
Windows Vista 3.56% 2.41% -1.15
Windows 7 36.29% 33.14% -3.15
Windows 8 2.01% 2.31% 0.30
Windows 8.1 1.06% 8.22% 7.16
Macintosh OS X 10.5* 0.65% 0.65
Macintosh OS X 10.6* 2.90% 2.90
Macintosh OS X 10.7* 1.91% 1.91
Macintosh OS X 10.8* 1.83% 1.83
Macintosh OS X 10.9* 8.26% 8.26
Macintosh OS X 10.10 0% 5.69% 5.69
iOS 4.3 0.19% 0% -0.19
iOS 5.0 0.12% 0% -0.12
iOS 5.1 0.59% 0% -0.59
iOS 6.0 0.42% 0% -0.42
iOS 6.1 2.02% 0.61% -1.41
iOS 7.0 7.36% 1.54% -5.82
iOS 7.1 0% 5.76% 5.76
iOS 8.0 0% 3.27% 3.27
Android 2.3 0.77% 0% -0.77
Android 4.0 0.82% 0% -0.82
Android 4.1 2.11% 1.22% -0.89
Android 4.2 0.84% 1.16% 0.32
Android 4.3 0.39% 0.56% 0.17
Android 4.4 0% 3.80% 3.80
Linux 4.37% 1.94% -2.43

* We didn’t start breaking out individual versions of OS X until Q1 2014. So unfortunately for this post we don’t have great info breaking down the versions of OS X, but we will in the future. OS X 10.10 did not exist in Q1 2014 so it’s counted as a natural 0% in our Q1 data.

Browser version details

These are the most dynamic numbers of the bunch. If there’s one thing they prove, it’s how incredibly effective the upgrade policies of Chrome and Firefox are. Where Safari and IE have years-old versions still hanging on (I’m looking at you Safari 5.1 and IE 8.0), virtually every Chrome and Firefox user is using a browser released within the last six weeks. That’s a hugel powerful thing. The IE team has suggested that Windows 10’s Project Spartan will adopt this policy, which is absolutely fantastic news. A few highlights:

  • Despite not being on a continuous upgrade cycle, Safari and IE were able to piggyback on successful OS launches to consolidate their users on their latest releases.
  • IE 8.0 is the only non-latest version of IE still holding on, thanks to its status as the latest version available for the still somewhat popular Windows XP.
OS versions in use on Flickr.com
2013 Q4 2014 Q4 Y/Y
Chrome 22.0.1229 1.67% 0% -1.67
Chrome 29.0.1547.76 1.39% 0% -1.39
Chrome 30.0.1599.101 8.94% 0% -8.94
Chrome 30.0.1599.69 3.74% 0% -3.74
Chrome 31.0.1650.57 6.08% 0% -6.08
Chrome 31.0.1650.63 6.91% 0% -6.91
Chrome 37.0.2062.124 0% 4.59% 4.59
Chrome 38.0.2125.104 0% 3.05% 3.05
Chrome 38.0.2125.111 0% 6.65% 6.65
Chrome 39.0.2171.71 0% 4.09% 4.09
Chrome 39.0.2171.95 0% 4.51% 4.51
Safari 5.0 1.96% 0% -1.96
Safari 5.1 5.60% 2.50% -3.10
Safari 6.0 6.21% 0.86% -5.35
Safari 7.0 7.29% 7.25% -0.04
Safari 7.1 0% 3.12% 3.12
Safari 8.0 0% 10.10% 10.10
Firefox 22.0 1.62% 0% -1.62
Firefox 24.0 5.50% 0% -5.50
Firefox 25.0 6.46% 0% -6.46
Firefox 26.0 1.90% 0% -1.90
Firefox 32.0 0% 4.92% 4.92
Firefox 33.0 0% 7.10% 7.10
Firefox 34.0 0% 3.52% 3.52
MSIE 8.0 3.69% 1.00% -2.69
MSIE 9.0 3.04% 1.22% -1.82
MSIE 10.0 5.94% 0% -5.94
MSIE 11.0 0% 6.69% 6.69
Generic WebKit 4.0* 3.18% 2.46% -0.72
Mozilla 5.0* 3.18% 4.80% 1.62
Opera 9.80 1.46% 0% -1.46

* These are catch-all versions of Mozilla-based and Webkit-based browsers that aren’t themselves Firefox, Safari, or Chrome.

A word on methodology

These numbers were anonymously collected using Yahoo’s in-house metrics libraries. The numbers here are aggregated over the course of three months each, making these numbers lagging indicators. This is why the latest releases, like Android 5.0 and iOS 8.1, are under-represented – they hadn’t yet enjoyed one full quarter when 2014 came to a close.

Further reading

There are a number of excellent sites out there watching similar browser statistics on a continuing basis. A few of them are:

  • Ars Technica – on a monthly basis they analyze raw data from Net Market Share with insightful commentary.
  • Net Market Share – while Ars does a bang-up job, it’s helpful to sift the data yourself to find the answers to your questions.
  • Peter-Paul Koch – No one shines a sharper light on the state of browsers than PPK, with just one example being his attention to disambiguating the various versions of Chromium out there (part two).

 
 

Flickr September 2014

Like this post? Have a love of online photography? Want to work with us? Flickr is hiring engineers, designers and product managers in our San Francisco office. Find out more at flickr.com/jobs.