Unstructured data is growing at up to 60% per year and will make up 80% of all data on the planet by 2025. For this reason, object storage solutions are entering a new era of mainstream adoption and use. Object storage allows you to easily store large amounts of unstructured data in a highly scalable and cost-effective way, making it a logical choice for backup and archiving, data analytics, and large, active unstructured data sets.
What Are Objects?
In its simplest form, object storage is a storage system where data elements are stored in repositories called objects. Objects can be of variable size, and are not only defined by their content, but also by their metadata and a unique identifier. Metadata can include simple things like creation date and time, but also unique characteristics about the data and searchable summary information. In contrast to accessing a file name and location within a file system, an object is accessed by a unique identifier created when the object is created and then used to retrieve the object later.
Built for Scale
A key element of a well-designed object storage system is its ability to scale gracefully, to grow from small terabyte-scale configurations up to massive scale, supporting billions of objects and 100s of petabytes in capacity. To prevent bottlenecks, object storage requires a ‘scale-out’ software architecture. This provides an expandable flat name space to address billions of objects and allows computing, networking, and storage resources to expand as the system’s capacity grows to maintain performance.
Always Available Access
Objects stores are accessed over a standard Ethernet network in one of two ways, through API calls or HTTP/HTTPS requests, most commonly through software in the form of web-based applications, packaged and open-source software solutions, and custom-developed in-house applications and scripts. Today, most object stores are accessed via AWS S3, a de facto standard command set. Most notably, the S3 protocol runs securely over Ethernet, making it easy to integrate object storage into your computing environment and access your data from varying distances, across private networks, WANs, and the Internet.
Durability, Security, and Availability
Another key attribute of object storage is the emphasis in their design of data durability, security, and availability. With so much data being stored, it’s important that the system and data are always accessible, and data is securely stored and confidently retained over a long period of time, for years and sometimes, decades.
Object storage systems use erasure coding, a data protection technique superior to classic RAID and storage replication technologies. Erasure coding breaks an object up into many piece parts, called shards, then calculates and maintains multiple copies of parity information for these shards; the data shards and parity shards are then distributed across the underlying hardware infrastructure. This ensures continuity of data and data access in the case of a data loss, component failure, or a site disaster.
Not All Object Storage Systems Are the Same
While object storage systems have a lot of commonalities, it is also important that not all object storage systems are the same. For many systems, the concept of scale only extends to the terabyte range and reliability and availability capabilities don’t meet the needs of massive enterprise deployments.
Performance at scale. Whereas scalability is a key attribute promoted by many vendors, implementations can suffer as the system scales, causing painful, performance-crushing data rebalancing across system resources. This forces customers to manage a fleet of object storage systems despite their original goal of a simple, scalable solution.
Recommendation: Seek real-world references of customers that are running single name space instances that scale to at least a petabyte in scale, and even higher if your capacity requirements extend into 10’s or 100’s of petabytes.
Efficiency. A well-designed system should be manageable at any scale, from terabytes to exabytes, by a single administrator on a part-time basis.
Recommendation: Go beyond the demo to truly understand the system’s design and management toolset available. Bypass the downloadable UI-centric demoware to consider a ‘proof-of-concept’ deployment to validate the system design and applicability to your use case.
Proactive data protection. With the amount of the data you are storing, its security, availability, and protection against failure, is paramount.
Recommendation: Look for advanced proactive data integrity verification to guarantee data consistency over the long term and always-on, cloud-based monitoring to support your efforts.
An everlasting archive. Most object storage systems naturally grow over time and evolve to support additional use cases. The object storage solution you deploy is a key strategic asset for both mining and preserving your organization’s data assets for the long term.
Recommendation: Consider your object storage vendor as a strategic partner working with you to protect and extract value from your digital assets. Value vendor longevity and understand how well systems preserve your data and maintain data accessibility when transitioning to new generations of software and hardware.
About Quantum ActiveScale
Quantum ActiveScale is an enterprise-class object storage solution designed from the ground up for data durability, security, and availability of massive unstructured data sets and use cases ranging from terabytes to exabytes in scale. With S3 compatibility and an optimized two-layer storage architecture, ActiveScale simply and cost-effectively scales performance and capacity without bound in support of analytical workloads, active archiving, long term retention, and cold data storage. Learn more at: www.quantum.com/object-storage