In this post I will show you the results of some upload/download speed tests that I have carried out in the past three days using iDisk and Dropbox. I was somehow surprised by the results that I got.
It is widely agreed that is an ill-designed service, probably one of the worst ever sold by Apple. The de-facto standard for file syncing on the cloud is that offers a decent amount of space for free and that is available on most platforms.
Dropbox has also proven to be extremely reliable. This service has changed the way we carry around our files and its flexibility is limitless.
Inspired by this post on MacRumors and somehow surprised by the results, I’ve spent the past three days testing the upload/download speed of Dropbox and iDisk under different scenarios.
What I wanted to verify was whether iDisk performance is so bad compared to Dropbox. What I have learnt goes beyond that and the results that I want to share with you are interesting.
Scenarios
I have executed three types of scenario to compare the speed of iDisk and Dropbox.
Scenario 1
Repeated uploads/downloads of the same file through the web interface of iDisk and Dropbox from a public wireless network. The public library network I used for my tests became increasingly busy during the day and only at the end it got quieter again.
The purpose was to compare the upload/download speeds of the same file with different network loads.
Scenario 2
Repeated uploads/downloads of the same file through the web interface of iDisk and Dropbox from my home broadband with a constant network load.
The purpose was to measure the consistency of the uploads/download speeds.
Scenario 3
The purpose of this scenario was to assess the behavior of iDisk and Dropbox when uploading files of different type and size through the Finder.
Scenarios not considered
I have not run any tests in these two scenarios:
a. Time to upload the same file multiple times through the Finder because in this case Dropbox will not re-upload the file a second time.
When you upload a file to Dropbox, the underlying framework splits the file into chunks and calculate the hash (i.e. signature) for each of those chunks. At that point, only the chunks whose hash is not already stored on the cloud are uploaded. The result is a saving of the bandwidth used and more important an incredible upload speed, to the user eyes even greater than the network broadband specs.
In the case you upload a file, delete it and then upload exactly the same file again, the resulting second upload time is negligible because Dropbox simply restores the file with the same hash that was stored in the cloud before being deleted.
Interestingly, from my tests it appears that this technology is not being used when uploading files through the Dropbox web interface. That has allowed me to run the multiple upload tests of the same file.
The hashing process can be described with this simple diagram:
b. Download through the Finder.
No download tests have been carried out through the Finder due to the fact that Dropbox stores the files both on the filesystem and on the cloud making such tests irrelevant.
Configuration used
Scenario 1
- HP 620 Core 2 Duo laptop, 2.2 GHz, 4GB RAM
- Windows Vista SP2
- Public network, unknown speed
Scenario 2
- MacBook Pro 15”, 2.2 GHz, 4GB RAM
- Mac OS 10.6.7
- Home network, 8Mb download speed, upload 512kbps
Scenario 3
- Same configuration as in Scenario 2
- Uploads carried out through Finder
- MobileMe iDisk Sync (from MobileMe preferences) set to Off
Test methodology
In both scenarios I used the stopwatch of my iPhone 4 to measure the time between:
-
For the web interface tests
Measured the time between the Upload Now click event and the Upload Completed message.
-
For iDisk
Paste Item event and the sound Finder plays when it completes a copy/paste operation on files.
-
For Dropbox
Paste Item event and the moment the green checkmark appears on the file icon in Finder.
Disclaimer
I acknowledge that this testing methodology is far from being scientific.
My setup and test methods might have affected the results and other people might get different values even opposite from what I am sharing in this post.
The results posted on this blog are for personal use only and do not constitute a definitive proof on what service is best.
More tests, using different scenarios, different networks, and a systematic way to time the events are needed to calculate average times, variance values and to draw some more informative conclusions.
Experimental results
Scenario 1
The results of uploading/downloading the same 9.2 MB .pdf file multiple times through the web interface are as follows:
As you can see from the graph the amount of time taken to upload the same .pdf file increased during the day and only after Test 4 it dipped again. The reason of course is due to the network utilization. The public library where I was carrying out these tests, was very quiet in the morning and got busier and busier toward the middle of the day.
More important is the comparison between iDisk and Dropbox. In all five tests that I have run, iDisk has been faster.
The download times for the same 9.2 MB .pdf files are shown in this graph:
In 4 out 5 tests, Dropbox times have been lower than iDisk.
The surprise has been the upload times. This is a first indication that Dropbox is not the fastest of the two – at least not all the time.
It is worth noting that for some unknown reason the hashing mechanism that is at the origin of the incredible Dropbox performance is not being used when working through the web interface.
Scenario 2
I almost drew the same conclusions as in Scenario 1. Even in this case, the upload times were better for iDisk. This is something that will be confirmed in the next scenario but with some notable exceptions.
The download times are somehow reversed to Scenario 1:
I can’t say much about the download speeds. This last series of tests goes against the results I got in Scenario 1 and doesn’t really allow us to draw any conclusion.
The upload speeds on the other hand confirms the surprise I got in Scenario 1. There’s a huge gap between the times measured for iDisk uploads and the corresponding Dropbox ones.
The key to remember here is that I was uploading a single, fairly large file. More on this in the next scenario.
Scenario 3
As part of this scenario I have run six different tests by uploading different type of files and sizes in consecutive order:
- .zip file, 16.7 MB
- .mov file, 12.5 MB
- .mp3 file, 3.3 MB
- .pdf file, 1.2 MB
- A 909 kB folder containing 20 Microsoft Word files
- A 202 kB folder containing 12 text files
These are the results:
This is where we can start getting an idea of when one solution is better than the other. We also get a confirmation of the upload results obtained in Scenario 1 and Scenario 2.
- For large single files, iDisk is consistently faster than Dropbox. This is shown in Test 1, Test 2 and Test 3. In Test 2 where I was uploading a QuickTime movie the advantage of iDisk is quite big
- Where Dropbox excels is when you upload a bunch of smaller files like in Test 5 and Test 6. The gap in Test 5 is about five-fold
Conclusions
After running these tests, we can’t really say which service is faster, at least not in absolute terms. It would also be terribly wrong to try to name the best of the two, this was not the purpose of the tests. Each service in my opinion serves different purposes.
It’s worth mentioning a few points:
- iDisk is faster when working with single large files
- Dropbox excels when dealing with plenty of small files
- I believe that the WebDAV protocol used by iDisk adds a substantial overhead on the data transmission. This overhead is negligible for large files but becomes a burden for small ones
- Dropbox technology is way more sophisticated than iDisk and it has actually been since Dropbox was introduced. The fact that before uploading a file, the logic behind Dropbox prescreens the file by calculating the hash makes sure that in real world case scenario Dropbox is blazingly fast
I would be curious to know a little bit more about Dropbox technology and of course your results if you have the patience (or madness!) to run similar tests on your environment.
The conclusion that I am inclined to draw is that the data that I have obtained sort of prove what I suspected for quite some time.
Dropbox is perfect for real-time syncing across computers and gives its best when used as your on-the-go home folder, wherever you are in whatever platform you’re working. I personally use it daily to move files between three different computers, sync my 1Password keychain file etc.
iDisk is on the other hand better suited as an extra backup solution to be used beside the ubiquitous external hard-drive backup. If you combine it with an ftp software like Transmit you’re sure to have a very reliable network storage disk. I’ve been using iDisk in exactly this way for the past 4-5 years with excellent results.
Thanks for the analysis. I've been trying to decide whether to use iDisk or Dropbox, and this is very helpful. I may wind up doing what you are doing as you describe in the last two paragraphs.
Thanks for your comment. I'm glad that you found my analysis useful. I'd be interested to hear if you get similar results.
Thank you so much for the analysis you made, it was really helpful, I am a mobile me user already, but your results helped me a lot. I am doing my thesis now, it is a master degree, and the topic is about Backup strategies for small business and personal usages … for devices like PDA, iPhone, etc. hence cloud services is vital.
The way you tested the service offered by Drop Box and iDisk was really bright and elegant.
Specially the way you showed the results.
Thank you again for this effort.
Thank you so much for the analysis you made, it was really helpful, I am a mobile me user already, but your results helped me a lot. I am doing my thesis now, it is a master degree, and the topic is about Backup strategies for small business and personal usages … for devices like PDA, iPhone, etc. hence cloud services is vital.
The way you tested the service offered by Drop Box and iDisk was really bright and elegant.
Specially the way you showed the results.
Thank you again for this effort.
Correct me if I am wrong but isn’t Idisk being decomissioned by Apple in June 2012. I was using Idisk with Transmit and it was working well. I have just moved everything to Dropbox because Apple introduced icloud and said they were closing down idisk
Thanks for your comment Craig.
You are right. I wrote the post 9 months ago and since then iDisk has become end of life.
Despite iDisk problems, it was nice to have such a feature so closely integrated with Mac OS X. Like many others I use Dropbox as well.