Remember when life was simple, and you only had to pick either NAS or SAN? Now, a storage admin has many storage-format options to choose from, such as block, object, file, and NAS, and if you head over to our friends at AWS, they add a few more acronym options. Making the right choice when selecting what format will best serve your data and workflow needs can be confusing, but getting it right is critical to your organization’s overall success. A poor choice can lead to data and application services delays, inflated costs, lack of scalability, complex management frameworks, and a host of other issues. So, let’s dig into some of these formats and what workflows they are best suited for.
Block Storage
First stop is good ole block storage . It is when a raw volume of data storage is presented to a server, usually from a storage attached network (SAN) accessed via FC, SAS, or iSCSI. Each volume block can function as an individual hard drive; think of a USB drive you plug into your laptop, it is simply a mounted storage volume to be used by a file system or a database. Block storage is ones and zeros; there is no file system or metadata for tracking and visualizing data; the operating system must handle all the read/write of the blocks. The benefits of this option are throughput performance, low latency, and high IOPS. Typically, block is best suited in support of virtual infrastructure (hypervisors) and databases, given its tendency to be high performing. Though block is very versatile, its lack of metadata describing your data and ability to manage storage allocations on a file level, makes it less suited for workflows that are file centric.
File Storage
That is a nice segue to file-based storage. Unlike block storage, file-based storage (NAS, File System, Object) hide much of the complexity associated with block storage. The NAS simply appears as a drive letter on the network, and as the name suggests, does well at storing and managing files, whereas block storage lacks that higher-level organization of data and makes it more challenging. File-based storage is common in home directories and mainstream IT file shares due to its ability to easily share files across the network, as well as the ability to scale. File-based storage is also gaining popularity within more data-intensive workflows, such as seen in M&E, Oil & Gas, and research applications. This is mostly due to the need to access data as files rather than blocks or ones and zeros. It also helps that file-based storage has moved away from costly Fibre Channel to high-performing, lower-cost 100Gbe network. It also helps that scalability, shareability, and the overall TCO structure of file-based storage makes it attractive in file-centric workflows.
Object Storage
The new kid on-the-block though is giving file storage a run for its money and is very well suited for the new data-intensive IoT-connected world we now live in. Popularized by AWS S3 and Glacier, object storage is quickly becoming a de facto standard for storing massive amounts of unstructured data found in data science applications, web farms storing massive troves of music or photos, and AI/ML data repositories for workflows for genomic research to autonomous vehicles. So, what makes object so cool? First off, it is a simple concept where data is delivered as objects that have rich, searchable metadata and a unique ID number. Unlike file storage where you organize your files in directories and trees, objects are placed into massive scale, flat, highly durable, and available buckets or pools that can be on prem, in the cloud, or distributed across all of the above. That metadata in each object is a crucial key as it gives the applications and users the ability to quickly search and run analytics against very large unstructured data repositories. This is one reason its popular in machine learning and anything leveraging advanced analytics. Object storage also is inherently very secure – by using erasure coding, it provides very high levels of data durability, often exceeding 10-15 9’s. Meaning, once an object is created it is likely to be there forever. This makes object storage ideal for backup or archive where data preservation is a critical component. Finally, with object storage, your CFO will appreciate all this technical jargon given object storage is one of the lowest-cost storage solutions you can deploy aside from ever lovable magnetic media tape storage, of course.
Which Storage Format Meets Your Needs?
As you can see in the chart below and the paragraphs above, there are many ways to deploy storage and each has unique benefits for different use cases. Need performance for structured applications like databases? Consider block. Lot of users sharing files; NAS may be suitable. Building a data repository for cancer research or a future flying car? Object storage is likely the path to take. Not all storage is created equally, and it’s likely your data center will need a mix of solutions to meet the needs of your company. At Quantum, our technical storage specialists want to help you make informed decisions that best suit your organizations’ mission needs. We hope this blog provided some good insight and information, and in upcoming posts, we will tackle solving the data-management dilemma.
