The Basics of Cloud Storage Latency Solutions

Storm Clouds Thumbnail

Recently, a company asked me to help solve what appeared to be a cloud storage latency problem with Amazon S3 for their clients outside of the U.S. Fortunately, there are many potential solutions available to solve this kind of simple latency issue that haunt a lot of cloud storage systems. In this case, determining which one was appropriate began with a review of Amazon S3’s cloud storage naming conventions.

Storm Clouds ThumbnailLets set the stage with a quick review.

Cloud storage is designed around the thought process of storing “objects” and not “files.” (It’s a slight change in metaphors — files to objects.) A file is usually associated with an actual storage location on a disk within a folder/directory hierarchy. It’s usually left to the user and/or application to decide where it’s going to live.

On the other hand, a cloud object — which is basically anything you want to store — never allows the user or application to decide where the object is going to be physically stored.  Instead, the cloud vendor gives the illusion of a folder/directory hierarchy to a given object (file) name. However, the object’s name is actually the combination of the “real” name the user/application gave it appended to the pseudo folder/directory/path that’s already been created.

Here’s an example, using the object/file name “readme.txt.”

Storage on your local machine:

Windows – c:tempmycoolappreadme.txt
*nix* – /home/mycoolapp/readme.txt

Storage on Amazon S3:


On Windows, you can see that the object has a file path consisting of the hard drive letter “c:” followed by the “temp” folder, which is in turn followed by the “mycoolapp” folder that contains the file (object) called “readme.txt.” Assuming a mount point or hard drive is involved, on a Unix/Linux/OS X system, the path is a little different though the same applies: “folder/folder/object(file).”

All that’s simple. Now let’s change the metaphor “file” to “object.” In the Amazon S3 example, think of the “uniquebucket” as your unique name space. It’s yours and yours alone. That means you have to create a unique bucket name.

It’s been said that S3’s worst design feature is the bucket-naming limitation, since it forces you to compete with everyone else for a unique name. For example, if you wanted your name to be “cloud,” too bad. “Cloud” already exists.

Next, you might think “mycoolapp” is a folder, but it’s really not. It’s actually part of the entire object name, which is known to Amazon S3 as “uniquebucket/mycoolapp/readme.txt.” So, what was once used as the file metaphor “readme.txt” is now using the object metaphor “uniquebucket/mycoolapp/readme.txt.”

In my next post, we’ll go over the latency issues that occur when mobile platforms communicate with cloud storage systems.

Post a Comment

Your email address will not be published.