Vast Data Alternative With Quobyte — A Comparison

Quobyte
5 min readMay 21, 2024

--

Vast data uses a traditional two tiered storage architecture that’s over 25 years old: hardware redundant NFS Gateway nodes (VAST Cbox) with a fast write cache and a SAN backend (VAST Dbox), with the difference that NVMe-over-Fabric replaced traditional fiber channel.

Performance Bottleneck: NFS Gateways

NFS Gateways are a severe performance bottleneck since NFS doesn’t have any load balancing capabilities. Once a client connects to a gateway, it communicates with that single endpoint only.

Performance Bottleneck: Write Cache in CBoxes

VAST’s file system design is based around fast flash write caches in the NFS gateways (CBoxes). Once these fill up, which often happens with high throughput parallel workloads, you’re down to the performance of the backend storage, which is significantly slower.

Performance Bottleneck: NFS Gateway Cache Consistency

The NFS gateways have to coordinate cached metadata (i.e., provide cache consistency). This can result in significant performance penalties when concurrently modifying the same directory or file from two or more gateways, which includes many AI/ML, big data, and HPC applications.

Performance Bottlenecks: Reads From the Secondary Tier DBoxes (SAN Backend)

The two tiered architecture means that reads require a lookup in the NFS gateway (including potential cache consistency communication between the NFS gateways), plus another hop to the slower backend store (SAN). Since the fast flash in the NFS gateways is only a small cache, you can’t control which data to keep in the fast tier.

Availability Limitation: CBoxes Hardware Redundancy in One Chassis

VAST’s NFS gateways (CBoxes) are custom appliances based on the 30-year-old “dual controller” architecture. Redundancy is implemented in hardware and has to reside in the same chassis. This severely limits availability.

Quobyte: A Modern, Resilient, Low-Latency Architecture Without Bottlenecks

Quobyte’s architecture is based on scalable resiliency: a modern shared nothing distributed architecture on commodity servers with no performance bottlenecks. It is built on the same concepts that enable Google and the other hyperscalers to provide unprecedented scale and reliability with minimal manpower.

Quobyte Architecture Diagram

As a parallel file system, Quobyte provides clients with direct access to the data in a single hop, without any bottlenecks like NFS gateways. All media — fast NVMe, QLC flash, and HDD — can be accessed directly for reads and writes. Data placement is handled by the policy engine, which gives you full control over the full data lifecycle. And all of this on cost-effective commodity servers from your favorite vendors, or on the public clouds, or hybrid.

A Detailed Comparison of VAST Data vs. Quobyte

Commodity Hardware Instead of VAST Storage Appliances

Quobyte runs on standard servers from any vendor, which means you can buy hardware from a highly competitive market without any lock-in. In addition, you can standardize the server models in your data center to further optimize cost and streamline operations. Admins and data center operators already know how to install and manage servers; there is no training, no learning curve, and no added complexity.

In contrast, with the appliance model you are stuck with VAST specific hardware and their pricing. The custom hardware will add complexity as “yet another thing” in your data center that your admins and team have to learn how to manage.

Finally, Quobyte is software that runs on servers in your data center, a colo (including “at the edge”) and on cloud VMs or bare metal machines. You can re-create the same storage environment anywhere, including hybrid set-ups. Appliances always limit you in terms of where you can run and how fast you can migrate.

Transparent and Fair Pricing vs. VAST Appliance Pricing

Quobyte offers transparent and fair pricing that gives you predictable cost for renewals and extensions. We don’t lure you in with low initial costs only to surprise you with massive price hikes for additional capacity and renewals. Since Quobyte runs on commodity servers and drives from any vendor, you can benefit from a range of competitors in the server market.

You can expand your cluster with newer hardware at any time, keeping the existing hardware: This protects your investment and makes it easy to grow your cluster quickly, on demand, and in flexible increments, from a single drive to 100s of servers at once.

Fast NVMe and Cost-Effective HDD

Many applications, including artificial intelligence and deep learning, need both media types to combine high performance for training and low cost for storing large data sets, intermediate results, and models. Instead of the cumbersome external tiering of flash-only systems like VAST data. External tiering requires you to buy and manage a second storage system and users get frustrated with having to wait for data to be paged in.

Quobyte offers automated, policy-controlled, and transparent tiering between multiple media types like fast NVMe, QLC flash storage, and HDD. With direct access to all tiers, Quobyte offers high-performance reads from any media type without having to add the latency of another “hop” to data on the QLC backend. In addition, you can keep your data on cost-optimized read-only TLC flash for low latency reads on 10s of petabytes.

Linear performance scaling

With Quobyte, client machines can directly communicate with every storage server without any NFS gateway bottlenecks. This has the huge advantage that the performance of your Quobyte cluster increases linearly with every server you add — instantly more throughput, IOPS, capacity, and consistent low latency. You can start with 4 servers and go to 100s in one cluster — double your servers, double your performance.

In comparison, NFS based systems have scalability limits due to the NFS gateway consistency coordination and the fact that NFS clients can only talk to a single gateway.

Other Features

  • Unified storage
    Similar to Vast data, Quobyte offers a single namespace for file and object. However, Quobyte also offers unified ACLs across Linux (POSIX, NFS), Windows, and S3. Compared to Vast Data’s separate ACLs for each platform, you only have to manage one set of ACLs in Quobyte. This reduces admin overhead and removes potential security risks.
  • Storage Security
    Quobyte uses its own secure protocol for client-to-server communication and can provide features that VAST’s NFS lacks, e.g., End-to-end data encryption, X.509 certificates, end-to-end checksums, and selective TLS.
  • Analytics
    With Quobyte’s native driver comes the ability to monitor client side application IO. Quobyte shows you the top activity in your cluster by application, Slurm/PBS job, client and volume.

Similar to VAST data’s catalog, Quobyte provides the File Query Engine to run highly efficient metadata searches. However, the engine in Quobyte runs directly on the file system metadata, doesn’t require any extra copies or resources, and always runs on the latest version of your file system.

How To Choose A VAST Data Competitor

Choosing the right storage system is a challenge. While all shared file system storage looks the same on the surface, the details like architecture, protocols, and operational simplicity make a huge difference. You can read more about the benefits of a software-storage solution here, the drawbacks of NFS, and why you should consolidate your NAS with Quobyte.

Originally posted on Quobyte’s blog on May 07, 2024.

--

--

Quobyte

Quobyte empowers customers by providing real software storage so that they can keep up with the ever-increasing amounts of data in today’s data-driven world.