Speed tests: iDisk vs Dropbox

In this post I will show you the results of some upload/download speed tests that I have carried out in the past three days using iDisk and Dropbox. I was somehow surprised by the results that I got.

It is widely agreed that iDisk is an ill-designed service, probably one of the worst ever sold by Apple. The de-facto standard for file syncing on the cloud is Dropbox that offers a decent amount of space for free and that is available on most platforms.

Dropbox has also proven to be extremely reliable. This service has changed the way we carry around our files and its flexibility is limitless.

Inspired by this post on MacRumors and somehow surprised by the results, I’ve spent the past three days testing the upload/download speed of Dropbox and iDisk under different scenarios.

What I wanted to verify was whether iDisk performance is so bad compared to Dropbox. What I have learnt goes beyond that and the results that I want to share with you are interesting.

Scenarios

I have executed three types of scenario to compare the speed of iDisk and Dropbox.

Scenario 1

Repeated uploads/downloads of the same file through the web interface of iDisk and Dropbox from a public wireless network. The public library network I used for my tests became increasingly busy during the day and only at the end it got quieter again.

The purpose was to compare the upload/download speeds of the same file with different network loads.

Scenario 2

Repeated uploads/downloads of the same file through the web interface of iDisk and Dropbox from my home broadband with a constant network load.

The purpose was to measure the consistency of the uploads/download speeds.

Scenario 3

The purpose of this scenario was to assess the behavior of iDisk and Dropbox when uploading files of different type and size through the Finder.

Scenarios not considered

I have not run any tests in these two scenarios:

a. Time to upload the same file multiple times through the Finder because in this case Dropbox will not re-upload the file a second time.

When you upload a file to Dropbox, the underlying framework splits the file into chunks and calculate the hash (i.e. signature) for each of those chunks. At that point, only the chunks whose hash is not already stored on the cloud are uploaded. The result is a saving of the bandwidth used and more important an incredible upload speed, to the user eyes even greater than the network broadband specs.

In the case you upload a file, delete it and then upload exactly the same file again, the resulting second upload time is negligible because Dropbox simply restores the file with the same hash that was stored in the cloud before being deleted.

Interestingly, from my tests it appears that this technology is not being used when uploading files through the Dropbox web interface. That has allowed me to run the multiple upload tests of the same file.

The hashing process can be described with this simple diagram:

b. Download through the Finder.

No download tests have been carried out through the Finder due to the fact that Dropbox stores the files both on the filesystem and on the cloud making such tests irrelevant.

Configuration used

Scenario 1

  • HP 620 Core 2 Duo laptop, 2.2 GHz, 4GB RAM
  • Windows Vista SP2
  • Public network, unknown speed

Scenario 2

  • MacBook Pro 15”, 2.2 GHz, 4GB RAM
  • Mac OS 10.6.7
  • Home network, 8Mb download speed, upload 512kbps

Scenario 3

  • Same configuration as in Scenario 2
  • Uploads carried out through Finder
  • MobileMe iDisk Sync (from MobileMe preferences) set to Off

Test methodology

In both scenarios I used the stopwatch of my iPhone 4 to measure the time between:

  • For the web interface tests

    Measured the time between the Upload Now click event and the Upload Completed message.

  • For iDisk

    Paste Item event and the sound Finder plays when it completes a copy/paste operation on files.

  • For Dropbox

    Paste Item event and the moment the green checkmark appears on the file icon in Finder.

Disclaimer

I acknowledge that this testing methodology is far from being scientific.

My setup and test methods might have affected the results and other people might get different values even opposite from what I am sharing in this post.

The results posted on this blog are for personal use only and do not constitute a definitive proof on what service is best.

More tests, using different scenarios, different networks, and a systematic way to time the events are needed to calculate average times, variance values and to draw some more informative conclusions.

Experimental results

Scenario 1

The results of uploading/downloading the same 9.2 MB .pdf file multiple times through the web interface are as follows:

As you can see from the graph the amount of time taken to upload the same .pdf file increased during the day and only after Test 4 it dipped again. The reason of course is due to the network utilization. The public library where I was carrying out these tests, was very quiet in the morning and got busier and busier toward the middle of the day.

More important is the comparison between iDisk and Dropbox. In all five tests that I have run, iDisk has been faster.

The download times for the same 9.2 MB .pdf files are shown in this graph:

In 4 out 5 tests, Dropbox times have been lower than iDisk.

The surprise has been the upload times. This is a first indication that Dropbox is not the fastest of the two – at least not all the time.

It is worth noting that for some unknown reason the hashing mechanism that is at the origin of the incredible Dropbox performance is not being used when working through the web interface.

Scenario 2

I almost drew the same conclusions as in Scenario 1. Even in this case, the upload times were better for iDisk. This is something that will be confirmed in the next scenario but with some notable exceptions.

The download times are somehow reversed to Scenario 1:

I can’t say much about the download speeds. This last series of tests goes against the results I got in Scenario 1 and doesn’t really allow us to draw any conclusion.

The upload speeds on the other hand confirms the surprise I got in Scenario 1. There’s a huge gap between the times measured for iDisk uploads and the corresponding Dropbox ones.

The key to remember here is that I was uploading a single, fairly large file. More on this in the next scenario.

Scenario 3

As part of this scenario I have run six different tests by uploading different type of files and sizes in consecutive order:

  1. .zip file, 16.7 MB
  2. .mov file, 12.5 MB
  3. .mp3 file, 3.3 MB
  4. .pdf file, 1.2 MB
  5. A 909 kB folder containing 20 Microsoft Word files
  6. A 202 kB folder containing 12 text files

These are the results:

This is where we can start getting an idea of when one solution is better than the other. We also get a confirmation of the upload results obtained in Scenario 1 and Scenario 2.

  • For large single files, iDisk is consistently faster than Dropbox. This is shown in Test 1, Test 2 and Test 3. In Test 2 where I was uploading a QuickTime movie the advantage of iDisk is quite big
  • Where Dropbox excels is when you upload a bunch of smaller files like in Test 5 and Test 6. The gap in Test 5 is about five-fold

Conclusions

After running these tests, we can’t really say which service is faster, at least not in absolute terms. It would also be terribly wrong to try to name the best of the two, this was not the purpose of the tests. Each service in my opinion serves different purposes.

It’s worth mentioning a few points:

  • iDisk is faster when working with single large files
  • Dropbox excels when dealing with plenty of small files
  • I believe that the WebDAV protocol used by iDisk adds a substantial overhead on the data transmission. This overhead is negligible for large files but becomes a burden for small ones
  • Dropbox technology is way more sophisticated than iDisk and it has actually been since Dropbox was introduced. The fact that before uploading a file, the logic behind Dropbox prescreens the file by calculating the hash makes sure that in real world case scenario Dropbox is blazingly fast

I would be curious to know a little bit more about Dropbox technology and of course your results if you have the patience (or madness!) to run similar tests on your environment.

The conclusion that I am inclined to draw is that the data that I have obtained sort of prove what I suspected for quite some time.

Dropbox is perfect for real-time syncing across computers and gives its best when used as your on-the-go home folder, wherever you are in whatever platform you’re working. I personally use it daily to move files between three different computers, sync my 1Password keychain file etc.

iDisk is on the other hand better suited as an extra backup solution to be used beside the ubiquitous external hard-drive backup. If you combine it with an ftp software like Transmit you’re sure to have a very reliable network storage disk. I’ve been using iDisk in exactly this way for the past 4-5 years with excellent results.


References

mac.com vs me.com on iOS 4.3

I’m a long time MobileMe subscriber, or I should say a .mac subscriber. I love my mac.com email address and I was a bit crossed when Apple removed the possibility to use my mac.com email address in iOS 4.2 unless you previously had an account with a username@mac.com address on the iPhone.

When setting up a MobileMe account in iOS 4.2 the email address would default to username@me.com regardless of whether you typed in the login screen username@mac.com.

This was really annoying and I bet that many long time Apple users felt cheated by this. There was a workaround that came as a KB article from Apple. The workaround involved using iTunes to mirror on the iPhone the mac.com email account setup on mail.app on your Mac.

A big surprise happened when I installed iOS 4.3 on my iPhone 4 a few weeks ago. As usual whenever I install a new version of iOS, I also restore the iPhone as a new device.

Immediately after the restore, I added my MobileMe account to the iPhone and expected iOS to automatically change my email address to me.com even though I registered as mac.com.

To my surprise, the email address remained username@mac.com, no need to use the workaround to sync your email account with iTunes.

I believe that this was a bug on iOS 4.2. The proof is this Apple KB where the solution is to upgrade to iOS 4.3.

This ends the speculations that the domain mac.com is going to disappear and all longtime .mac users will have to switch to a less loved username@me.com email address.

Well done Apple, I’m so glad that I can keep on using my beloved mac.com email address.

The (lack) of usefulness of MobileMe System Status

I like MobileMe, I’ve been using it for nine years and despite the usual complaints about the speed of iDisk and some general reliability improvements that the web interface could undergo, I’ve been quite happy with it.

One of the things that really bugs me is the System Status History which contains entries like this one:

MobileMe Mail Maintenance – 03/17/2011 22:00 PDT – 03/18/2011 02:00 PDT
Due to scheduled maintenance, some MobileMe members may have been unable to access mail for up to 15 minutes. Normal service has been restored. We apologize for any inconvenience.

The keyword here is scheduled maintenance. In IT that is something that is decided days if not weeks/months ahead and it is usually preceded by an email notification or at least an alert on a portal.

Apple has the bad habit of posting these entries during or after the maintenance window.

The process usually goes like this:

  1. Damn, MobileMe is down again…
  2. Check Twitter for other users with the same problem
  3. Check the MobileMe System Status
  4. Huh,… scheduled maintenance. Why wasn’t I alerted?
  5. Go on with your life without MobileMe until the system is back up again

Has this ever bothered you as well or am I the only soul being frustrated by these things?