I have recently become interested in how we will keep up with the growing demand for wirelss data. Researchers at the University of Southern California are looking into algorithms for caching frequently-used data so a mobile can access it without going through a marco base station. The data could be cached on “helper nodes” that are femtocell base stations with low-speed backhaul links or on other mobile users’ devices configured to share data in a device-to-device (D2D) fashion.
Half of Internet traffic is streaming video. Streaming video is becoming more common on mobile devices. The industry expects mobile streaming video to grow faster than other traffic and become the dominant type traffic.
YouTube video popularity follows a Zipf distribution, in which a few viral videos account for most viewing. This distribution is informally expressed as the “Pareto principle” or the “80/20 rule”: In any system, usually 80% of the effects come from 20% of the causes. It’s not really a “rule” but it’s a distribution that crops up often, as in 20% of the members of a profession earning 80% of the money.
Storing a few popular videos is no problem because storage cost has decreased dramatically. The issue is in working out which nodes should store which videos. If there are several helper nodes in one area, it’s more efficient for each of them to store different sets of videos, increasing the number of videos that do not have to come from the main base station. This gets complicated when the helper nodes are mobile devices.
To maximize efficiency, the base station needs to keep track of frequently used files and the locations of helpers and use this information to direct specific helpers to caches specific files. Models indicate this could increase mobile network capacity by 10 to 100 times.
Pareto’s Head Attached to the Long Tail
The notion of saving a few popular videos reminds me of Chris Anderson’s book The Long Tail. Anderson argues that the Internet makes it possible to sell products that never would have had a large enough market to justify taking up shelf space in stores. This new marketplace to trade things of interest to a narrower audience takes away, Anderson says, from the large “blockbuster” items of broad interest.
Thinking about the question of delivering viral videos to phones, it occurs to me that the Internet not only allowed trading items of narrow interest; the Internet lends itself to “long-tail” items. Un-ACKed broadcast transmission (e.g. TV and radio) lends itself to the large head of the popularity distribution curve. We are now having to adapt it to "Pareto’s head".
This issue occured to me last week after watching Star Trek Into Darkness on its opening night last Wednesday. The movie was essentially a completely new reimagining of the 1982 film Star Trek: The Wrath of Khan. It borrowed some lines of dialogue from the 1982 film, but was mostly completely new. Why even bother bother starting with a 30-year-old film’s themes? Why not just create something new from scratch? I suspect the answer was in 1982 choices of media were limited to blockbusters. Many people received only a few television channels. Most people didn’t have access to a film projector and a library of old shows and movies to watch. So most everyone alive in 1982 at least heard of the The Wrath of Khan. One way to increase the chance of getting a large audience in today’s “long-tail” environment is to base a new film on an old classic. I am interested if this means that going forward there will be no new blockbusters that are not reimaginations of earlier stories and franchises. This is a question for someone in mass media.
Conclusion
Despite the ability of the Internet to narrowcast and the resulting newfound importance of the long tail, the data show the mobile media popularity distribution curve still has a very large head in front of the long tail. Adapting the mobile network to handle Pareto’s head will be one part of managing the rapidly growing demand for mobile data.