Technical groupsOpen sourceCareersResearchBlogContactConsulting Platform
Global consulting partnerEuropean product partnerGlobal Atlassian transformation leader
The Sneakernet: Towards A Much Faster Internet

10 April 2019 — by Matthias Meschede

After an unprecedented effort of scientists all over the world, the Event Horizon Telescope team has released the first image ever taken of a blackhole.

The image is generated by correlating petabytes of data, which are shipped from the telescopes to the data centers in hard disks via airplane. It turns out that this is way faster than using those old-school cables. Once more, innovation has been brought to us from our modern philosophers, this time staring into a lens pointed to the dark sky, last time staring into a lens pointed on tiny objects in a cave - the physicists.

As a new-school enterprise, Tweag immediately realized the potential of shipping hard disks and did a quick explorative study about future possibilities of this technology called the Sneakernet: Could we give a new life to the homing pigeon industry? How about transportation means that are optimized to carry incredible amounts of weight? How about transportation means that are designed to be fast as a bullet?

Consider a 4 terabytes (TB), 60 grams, 100 X 70 X 7 mm³ disk SSD disk. To get good transfer rates, we need to ship as many of these disks as fast as possible from one place to another. It’s all about shipping capacity and speed.

For instance, a human can carry about 30kg and walk around 5km/h. 30kg corresponds to 7500 of those 4TB hard disks. This means that a human can carry around 30,000 TB of data!

In case that this number doesn’t speak to you: it corresponds to 30,000,000 GB. Doesn’t speak to you either? Well, it’s 30 PB. Still not happy? Let’s go back to old-school technology then: 30 PB corresponds to the entire internet traffic of one month in the year 1999. Just think back and remember the endless ISDN and modem transfers of mp3’s and images, then think about what you can squeeze into a 4 TB hard disk and you’ll believe me.

The second factor that determines the transfer rate of a human carrier is speed. After all we have to bring our 30 PB of data from A to B and we want to do this in as little time as possible. The good old philosophical allegory time = distance / speed comes to help. Let’s say I walk 10 m to bring 30 PB of data to my colleagues office. Not considering the time that it will take her to prepare and clean her disk, this would take about 7 seconds, in line with Adam Ries (Germans know why).

The Sneakernet transfer rate equation is thus:

time_from_A_to_B = speed / distance
storage_capacity = load_capacity / disk_weight x disk_storage_space
rate = storage_capacity / time_from_A_to_B

A human is neither optimized for speed nor for load capacity (are we actually optimized for something?). Planes and ships fare much better in this sense. It turns out that a plane (Boeing 747 LCF with 200,000kg load) can carry the semi-satanic number of 3,333,333 hard disks that store 13 exabytes of data. That used to be the entire internet traffic of one month in the year 2009! Remember: Google, Youtube and all of these things came up and you need a plane to handle this kind of data. The plane is also fast enough to deliver this data internationally with reasonable delay. However there might be some package loss with certain providers.

Now brace yourself. There are ships called mammoth ultra large crude carriers (ULCCs) that can carry 550,000 Deadweight Tons (DWTs). They can deliver a shipload of 9,166,666,666 hard disks, containing 36,666 EB in total. A single shipload can fit the entire internet traffic of 2018 25 times! As you know, the internet barely exists for this long and traffic rates were much lower for most of its history. This brings us to the even bigger conclusion that a single shipload of SSD hard disks can store the entire traffic that has ever been exchanged in the history of the internet.

Wow, that’s impressive. Here is a little graph that shows how the Sneakernet compares to more traditional technology:

sneakernet

What about the time needed for loading and unloading data from the disks? And the food that the pigeons need to fly such large distances? What is the best sneaker brand for transferring data by hand?

We are sure that these questions can be answered and that the Sneakernet is the future. Besides the packet loss problem, that can become… problematic. And that so many hard disks don’t actually exist. Details.

About the authors
Matthias MeschedeMatthias initiates and coordinates projects at Tweag. He says he's a generalist "half scientist, half musician, and a third half of other things" but you'll have to ask him what that means exactly! He lives in Paris, and is regularly in Tweag's Paris office where you'll most likely find him in a discussion, writing long-form texts in vim or reading and occasionally writing code.
If you enjoyed this article, you might be interested in joining the Tweag team.
This article is licensed under a Creative Commons Attribution 4.0 International license.

Company

AboutOpen SourceCareersContact Us

Connect with us

© 2023 Modus Create, LLC

Privacy PolicySitemap