About Phil Dokas

I do a lot of ᕕ( ᐛ )ᕗ at Flickr and this is unquestionably my favorite Wikipedia article.

33 Browser Stats You Just Might Believe

We care an awful lot about the kinds of browsers and computers visiting Flickr. As people update to the latest versions of their browsers, the capabilities we can build against improve, which lets us build cool new things. At the same time, if lots of people continue using older browsers then we have to do extra work to gracefully support them.

These days we not only have incredibly capable browsers, but thanks to the transparent and rapid update process of Chrome, Firefox, and soon Internet Explorer (hooray!), we can rely on new features rapidly showing up en masse. This is crazy great, but it doesn’t mean that we can stop paying attention to our usage statistics. In fact, as people spend more time on their phones, there’s as much of a need for a watchful eye as ever.

We’ve never really shared our internal numbers, but we thought it would be interesting to take a look at the browsers Flickr visitors used in 2014. We use these numbers constantly to inform our project planning. Since limitations in older browsers take time to support we have to be judicious in picking which battles to fight. As you’ll see below, these numbers can be quite dynamic with a popular browser dropping to nearly 0% market-share in just a year. Let’s dive in and see some specifics.

Fort Vancouver
Fort Vancouver by Kate Dickerson

Top level OSes and browsers

At the highest level we learn a lot by looking at our OS family data. Probably the most notable thing here is how much of our traffic is coming from mobile devices. Moreover, the rate of growth is eye-popping. And this is just our website – this data doesn’t include our iOS or Android clients at all. A quarter of our traffic is from mobile devices.

OSes in use on Flickr.com
2013 Q4 2014 Q4 Y/Y
Windows 56.55% 50.61% -5.94
Macintosh 21.49% 21.42% -0.07
iOS 11.09% 17.61% 6.52
Android 5.39% 7.82% 2.43
Other 5.48% 2.54% -2.94

Let’s slice things slightly differently and look at browser families. We greatly differ from internet-wide traffic in that IE isn’t the outright majority browser. In fact, it clocks in at only the #4 position. More than half of Flickr visitors use a Webkit/Webkit-heritage browser (Safari and Chrome, respectively). Chrome rapidly climbed into its leadership position over the last few years and it’s stabilized there. Safari is hugely buoyed by iOS’s incredible growth numbers, while IE has been punished by Windows’s Flickr market-share decline.

Browsers in use on Flickr.com
2013 Q4 2014 Q4 Y/Y
Chrome 35.71% 35.42% -0.29
Safari 24.11% 27.50% 3.39
Firefox 17.94% 18.29% 0.35
Internet Explorer 13.98% 10.31% -3.67
Other 8.26% 8.48% 0.22

Fine-grained details

We can go a step further and see many details in the individual versions of OSes and browsers out there. It’s one thing to say “Windows is down 6% over the year” but another to say “the growth rate for the latest version of Windows is 350% year over year.” When we look at the individual versions we can infer quite a bit of detail around update rates and changes in the landscape.

OS version details

A few highlights:

  • Windows 7 is on the decline, XP and Vista fell by roughly 50% each, and Windows 8 and 8.1 are surging ahead.
  • iOS 8.1 and Android 5.0 don’t appear in the list due to their late appearance in Q4. Our current monthly numbers have iOS 8.1 far outpacing every other iOS version.
  • OS X 10.10 has accelerated Mac user upgrades; since its launch 10.9 has shed over a percent per month, and the legacy versions have sharply accelerated their decline.
OS versions in use on Flickr.com
2013 Q4 2014 Q4 Y/Y
Windows NT 3.39% 0% -3.39
Windows XP 10.12% 4.49% -5.63
Windows Vista 3.56% 2.41% -1.15
Windows 7 36.29% 33.14% -3.15
Windows 8 2.01% 2.31% 0.30
Windows 8.1 1.06% 8.22% 7.16
Macintosh OS X 10.5* - 0.65% 0.65
Macintosh OS X 10.6* - 2.90% 2.90
Macintosh OS X 10.7* - 1.91% 1.91
Macintosh OS X 10.8* - 1.83% 1.83
Macintosh OS X 10.9* - 8.26% 8.26
Macintosh OS X 10.10 0% 5.69% 5.69
iOS 4.3 0.19% 0% -0.19
iOS 5.0 0.12% 0% -0.12
iOS 5.1 0.59% 0% -0.59
iOS 6.0 0.42% 0% -0.42
iOS 6.1 2.02% 0.61% -1.41
iOS 7.0 7.36% 1.54% -5.82
iOS 7.1 0% 5.76% 5.76
iOS 8.0 0% 3.27% 3.27
Android 2.3 0.77% 0% -0.77
Android 4.0 0.82% 0% -0.82
Android 4.1 2.11% 1.22% -0.89
Android 4.2 0.84% 1.16% 0.32
Android 4.3 0.39% 0.56% 0.17
Android 4.4 0% 3.80% 3.80
Linux 4.37% 1.94% -2.43

* We didn’t start breaking out individual versions of OS X until Q1 2014. So unfortunately for this post we don’t have great info breaking down the versions of OS X, but we will in the future. OS X 10.10 did not exist in Q1 2014 so it’s counted as a natural 0% in our Q1 data.

Browser version details

These are the most dynamic numbers of the bunch. If there’s one thing they prove, it’s how incredibly effective the upgrade policies of Chrome and Firefox are. Where Safari and IE have years-old versions still hanging on (I’m looking at you Safari 5.1 and IE 8.0), virtually every Chrome and Firefox user is using a browser released within the last six weeks. That’s a hugel powerful thing. The IE team has suggested that Windows 10’s Project Spartan will adopt this policy, which is absolutely fantastic news. A few highlights:

  • Despite not being on a continuous upgrade cycle, Safari and IE were able to piggyback on successful OS launches to consolidate their users on their latest releases.
  • IE 8.0 is the only non-latest version of IE still holding on, thanks to its status as the latest version available for the still somewhat popular Windows XP.
OS versions in use on Flickr.com
2013 Q4 2014 Q4 Y/Y
Chrome 22.0.1229 1.67% 0% -1.67
Chrome 29.0.1547.76 1.39% 0% -1.39
Chrome 30.0.1599.101 8.94% 0% -8.94
Chrome 30.0.1599.69 3.74% 0% -3.74
Chrome 31.0.1650.57 6.08% 0% -6.08
Chrome 31.0.1650.63 6.91% 0% -6.91
Chrome 37.0.2062.124 0% 4.59% 4.59
Chrome 38.0.2125.104 0% 3.05% 3.05
Chrome 38.0.2125.111 0% 6.65% 6.65
Chrome 39.0.2171.71 0% 4.09% 4.09
Chrome 39.0.2171.95 0% 4.51% 4.51
Safari 5.0 1.96% 0% -1.96
Safari 5.1 5.60% 2.50% -3.10
Safari 6.0 6.21% 0.86% -5.35
Safari 7.0 7.29% 7.25% -0.04
Safari 7.1 0% 3.12% 3.12
Safari 8.0 0% 10.10% 10.10
Firefox 22.0 1.62% 0% -1.62
Firefox 24.0 5.50% 0% -5.50
Firefox 25.0 6.46% 0% -6.46
Firefox 26.0 1.90% 0% -1.90
Firefox 32.0 0% 4.92% 4.92
Firefox 33.0 0% 7.10% 7.10
Firefox 34.0 0% 3.52% 3.52
MSIE 8.0 3.69% 1.00% -2.69
MSIE 9.0 3.04% 1.22% -1.82
MSIE 10.0 5.94% 0% -5.94
MSIE 11.0 0% 6.69% 6.69
Generic WebKit 4.0* 3.18% 2.46% -0.72
Mozilla 5.0* 3.18% 4.80% 1.62
Opera 9.80 1.46% 0% -1.46

* These are catch-all versions of Mozilla-based and Webkit-based browsers that aren’t themselves Firefox, Safari, or Chrome.

A word on methodology

These numbers were anonymously collected using Yahoo’s in-house metrics libraries. The numbers here are aggregated over the course of three months each, making these numbers lagging indicators. This is why the latest releases, like Android 5.0 and iOS 8.1, are under-represented – they hadn’t yet enjoyed one full quarter when 2014 came to a close.

Further reading

There are a number of excellent sites out there watching similar browser statistics on a continuing basis. A few of them are:

  • Ars Technica – on a monthly basis they analyze raw data from Net Market Share with insightful commentary.
  • Net Market Share – while Ars does a bang-up job, it’s helpful to sift the data yourself to find the answers to your questions.
  • Peter-Paul Koch – No one shines a sharper light on the state of browsers than PPK, with just one example being his attention to disambiguating the various versions of Chromium out there (part two).

 
 

Flickr September 2014

Like this post? Have a love of online photography? Want to work with us? Flickr is hiring engineers, designers and product managers in our San Francisco office. Find out more at flickr.com/jobs.

Avoiding Dragons: A Practical Guide to Drag ’n’ Drop

You, the enterprising programmer, know about parsing EXIF from photos in the browser and even how and why to power this parsing with web workers. “Bat,” you ask yourself, “how do I get those photos into the browser in the first place?”

The oldest and most low-tech solution is the venerable <input type="file" name="foo">. This plops the old standby file button on your page and POSTs the file’s contents to your server upon form submission.

To address many of this simple control’s limitations we debuted a Flash-based file uploader in 2008. This workhorse has been providing per-file upload statuses, batch file selection, and robust error handling for the last four years through Flash’s file system APIs.

These days we can thankfully do this work without plugins. Not only can we use XHR to POST files and provide all the other fancy info we’ve long needed Flash for, but now we can pair this with something much better than an <input>: drag and drop. This allows people drag files directly into a browser window from the iPhotos, Lightrooms, and Windows Explorers of the world.

Let’s take a look at how this works.

Foundations first

Workmen laying the cornerstone, construction of the McKim BuildingWorkmen laying the cornerstone, construction of the McKim Building by Boston Public Library

Let’s begin with our simple fallback, a – yes – <input type="file">.

HTML: <input type="file" multiple accept="image/*,video/*">
JS: Y.all('input[type=file]').on('change', handleBrowse);

Here we start with an <input> that accepts multiple files and knows it only accepts images and videos. Then, we bind an event handler to its change event. That handler can be very simple:

function handleBrowse(e) {
	// get the raw event from YUI
	var rawEvt = e._event;
	
	// pass the files handler into the loadFiles function
	if (rawEvt.target && rawEvt.target.files) {
		loadFiles(rawEvt.target.files);
	}
}

A simple matter of handing the event object’s file array off to our universal function that adds files to our upload queue. Let’s take a look at this file loader:

function loadFiles(files) {
	updateQueueLength(count);
	
	for (var i = 0; i < files.length; i++) {
		var file = files[i];
		
		if (File && file instanceof File) {
			enqueueFileAddition(file);
		}
	}
}

Looks clear – it’s just going over the file list and adding them to a queue. “But wait,” you wonder, “why all this queue nonsense? Why not just kick off an XHR for the file right now?” Indeed, we’ve stuck in a layer of abstraction here that seems unnecessary. And for now it is. But suppose our pretty synchronous world were soon to become a whole lot less synchronous – that could get real fun in a hurry. For now, we’ll put that idea aside and take a look at these two queue functions themselves:

function updateQueueLength(quantity) {
	state.files.total += quantity;
}

function enqueueFileAddition(file) {
	state.files.handles.push(file);
	
	// If all the files we expect have shown up, then flush the queue.
	if (state.files.handles.length === state.files.total) {
		for (var i = 0, len = state.files.total; i < len; i++) {
			addFile(state.files.handles[i]);
		}
		
		// reset the state of the world
		state.files.handles = [];
		state.files.total = 0;
	}
}

Pretty straightforward. One function for leaving a note of how many files we expect, one function to add files and see if we have all the files we expect. If so, pass along everything we have to addFile() which sends the file into our whirlwind of XHRs heading off to the great pandas in the sky.

Droppin’ dragons

Droppin’ dragonsDroppin’ dragons by Phil Dokas

While all of that is well and good, it was all for a ho-hum <input> element! Let’s hook a modern browser’s drag and drop events into this system:

document.addEventListener('drop', function(e) {
	if (e.dataTransfer && e.dataTransfer.files) {
		loadFiles(e.dataTransfer.files);
	}
});

The drag and drop API is a fairly complicated one, but it thankfully makes the task of reading files out of a drop event easy. Every drop will have a dataTransfer attribute and when there’s at least one file in the drag that member will itself have a files attribute.

In fact, when you’re only concerned about handling files dragged directly into the browser you could call it a day right here. The loadFiles() function we wrote earlier knows how to handle instances of the File class and that’s exactly what dataTransfer.files stores. Easy!

Put it up to eleven

While easy is a good thing, awesome is awesome. How could we make dragging files into a browser even better? Well, how about cutting down on the trouble of finding the folder with your photos somewhere on your desktop, opening it, and then dragging those files into the browser? What if we could just drag the folder in and call it a day?

goes to 11goes to 11 by Rick Kimpel

Try to drag a folder into the browser with the current state of our code; what happens? Our code tells the browser to treat all dropped file system objects as files. So what ultimately happens for folders is a very elaborate “nothing”. To fix this, we need to tell the browser how to handle directories. In our case, we want it to recursively walk every directory it sees and pick out the photos from each.

From here on out we’re going to be treading over tumultuous land, rife with rapidly changing specs and swiftly updating browsers. This becomes immediately apparent in how we begin to add support for directories. We need to update our drop event handler like this:

document.addEventListener('drop', function(e) {
	if (e.dataTransfer && e.dataTransfer.items) {
		loadFiles(e.dataTransfer.items);
	}
	else if (e.dataTransfer && e.dataTransfer.files) {
		loadFiles(e.dataTransfer.files);
	}
});

Items? Files? The difference is purely a matter of one being the newer interface where development happens and the other being the legacy interface. This is spelled out a bit in the spec, but the short of it is that the files member will be kept around for backwards compatibility while newer abilities will be built in the items namespace. Our code above prefers to use the items attribute if available, while falling back to files for compatibility. The real fun is what comes next.

You see, the items namespace deals with Items, not Files. Items can be thought of as pointers to things in the file system. Thankfully, that includes the directories we’re after. But unfortunately, this is the file system and the file system is slow. And JavaScript is single-threaded. These two facts together are a recipe for latency. The File System API tackles this problem with the same solution as Node.js: asynchronicity. Most of the functions in the API accept a callback that will be invoked when the disk gets around to providing the requested files. So we’ll have to update our code to do two new things: 1) translate items into files and 2) handle synchronous and asynchronous APIs.

So what do these changes look like? Let’s turn back to loadFiles() and teach it how to handle these new types of files. Taking a look at the spec for the Item class, there appears to be a getAsFile() function and that sounds perfect.

function loadFiles(files) {
	updateQueueLength(count);
	
	for (var i = 0; i < files.length; i++) {
		var file = files[i];
		
		if (typeof file.getAsFile === 'function') {
			enqueueFileAddition(file.getAsFile());
		}
		else if (File && file instanceof File) {
			enqueueFileAddition(file);
		}
	}
}

Easy – but, there’s a problem. The getAsFile() function is very literal. It assumes the Item points to a file. But directories aren’t files and that means this method won’t meet our needs. Fortunately, there is a solution and that’s through yet another data type, the Entry. An Entry is much like a File, but it can also represent directories. As mentioned in this WHATWG wiki document, there is a proposed method, getAsEntry(), in the Item interface that allows you to grab an Entry for its file system object. It’s browser prefixed for now, so let’s add that in as well.

function loadFiles(files) {
	updateQueueLength(count);
	
	for (var i = 0; i < files.length; i++) {
		var file = files[i];
		var entry;
		
		if (file.getAsEntry) {
			entry = file.getAsEntry();
		}
		else if (file.webkitGetAsEntry) {
			entry = file.webkitGetAsEntry();
		}
		else if (typeof file.getAsFile === 'function') {
			enqueueFileAddition(file.getAsFile());
		}
		else if (File && file instanceof File) {
			enqueueFileAddition(file);
		}
	}
}

So what we have now is a way of handling native files and a way of turning Items into Entries. Now we need to figure out if the Entry is a file or a directory and then handle that appropriately.

What we’ll do is queue up any File objects we run across and skip the loop ahead to the next object. But if we have an Item and successfully turn it into an Entry then we’ll try to resolve this down to a file or a directory.

function loadFiles(files) {
	updateQueueLength(count);
	
	for (var i = 0; i < files.length; i++) {
		var file = files[i];
		var entry, reader;
		
		if (file.getAsEntry) {
			entry = file.getAsEntry();
		}
		else if (file.webkitGetAsEntry) {
			entry = file.webkitGetAsEntry();
		}
		else if (typeof file.getAsFile === 'function') {
			enqueueFileAddition(file.getAsFile());
			continue;
		}
		else if (File && file instanceof File) {
			enqueueFileAddition(file);
			continue;
		}
		
		if (!entry) {
			updateQueueLength(-1);
		}
		else if (entry.isFile) {
			entry.file(function(file) {
				enqueueFileAddition(file);
			}, function(err) {
				console.warn(err);
			});
		}
		else if (entry.isDirectory) {
			reader = entry.createReader();
			
			reader.readEntries(function(entries) {
				loadFiles(entries);
				updateQueueLength(-1);
			}, function(err) {
				console.warn(err);
			});
		}
	}
}

The code is getting long, but we’re almost done. Let’s unpack this.

The first branch of our new Entry logic ensures that what was returned by webkitGetAsEntry()/getAsEntry() is something useful. When they error they return null and this will happen if an application provides data in the drop event that isn’t a file. To see this in action try dragging a few files in from Preview in Mac OS X – it’s odd behavior, but this adequately cleans it up.

Next we handle files. The Entry spec provides the brilliantly simple isFile and isDirectory attributes. These guarantee whether you have a FileEntry or a DirectoryEntry on your hands. These classes have useful – though as promised, asynchronous – methods and here we use FileEntry’s file() method and enqueue its returned file.

Finally, the unicorn we’re chasing – handling directories. This is a tad more complicated, but the idea is straightforward. We create a DirectoryReader which lets us read its contents through its readEntries() method which provides an array of Entries. And what do we do with these Entries? We recursively call our loadFiles() function with them! In this step we achieve recursively walking a branch of the file system and rooting out every available image. Finally, we decrement the count of expected files by 1 to indicate that this was a directory and it has now been suitably handled.

But there is one more thing.

In that final directory reading step we recursively called loadFiles() with an array of Entries. As of right now, this function only expects to handle Files and Items. Let’s patch up this oversight, add a final bit of error handling, and call it a day.

function loadFiles(files) {
	updateQueueLength(count);
	
	for (var i = 0; i < files.length; i++) {
		var file = files[i];
		var entry, reader;
		
		if (file.isFile || file.isDirectory) {
			entry = file;
		}
		else if (file.getAsEntry) {
			entry = file.getAsEntry();
		}
		else if (file.webkitGetAsEntry) {
			entry = file.webkitGetAsEntry();
		}
		else if (typeof file.getAsFile === 'function') {
			enqueueFileAddition(file.getAsFile());
			continue;
		}
		else if (File && file instanceof File) {
			enqueueFileAddition(file);
			continue;
		}
		else {
			updateQueueLength(-1);
			continue;
		}
		
		if (!entry) {
			updateQueueLength(-1);
		}
		else if (entry.isFile) {
			entry.file(function(file) {
				enqueueFileAddition(file);
			}, function(err) {
				console.warn(err);
			});
		}
		else if (entry.isDirectory) {
			reader = entry.createReader();
			
			reader.readEntries(function(entries) {
				loadFiles(entries);
				updateQueueLength(-1);
			}, function(err) {
				console.warn(err);
			});
		}
	}
}

All we need to do to handle an Entry is to rely on the fact that Entries have those oh-so-helpful isFile and isDirectory attributes. If we see those we know we have an Entry of one type or another and we know how to work with them, so just skip on down to the FileEntry and DirectoryEntry handling code.

And that, finally, is it. There are many specs with very new data types at play here, but through this turmoil we can achieve some very nice results never before possible in browsers.

Further reading

Flickr flamily floto

Like this post? Have a love of online photography? Want to work with us? Flickr is hiring engineers, designers and product managers in our San Francisco office. Find out more at flickr.com/jobs.