Quobyte Core Services
These services run on worker nodes with local drives (nvme, HDD…) and implement the actual Quobyte file system. We recommend running these services on a pool of dedicated worker nodes to avoid the negative impact of other workloads on the file system performance. As an alternative, you can run the entire Quobyte cluster on bare metal Linux and simply connect it to your Kubernetes cluster.
Quobyte API
The Quobyte CSI Plugin communicates with your Quobyte cluster via the API service e.g. to provision volumes or updated quotas. You’ll also use this service for managing your cluster.
Quobyte Client Handles IO
The Quobyte client runs on all worker nodes in your Kubernetes cluster. The client runs as DaemonSet on all worker nodes in your Kubernetes cluster. Quobyte clients automatically mount new volumes and make them available to the host via a bi-directional volume mapping. The Quobyte client receives the file system operations from the applications on the worker nodes and translates them to RPC calls to the various Quobyte core services.
StorageClasses Connect Quobyte with Kubernetes
You can create as many StorageClasses as you need and connect them to Quobyte by selecting csi.quobyte.com as the provisioner. For each StorageClass you can specify labels to tell the Quobyte Policy Engine how you’d like the data for these volumes stored. Learn more in this tutorial.
Quobyte CSI Plugin for Kubernetes Persistent Volume Claims
The Quobyte CSI Plugin connects your Kubernetes cluster with your Quobyte cluster by dynamically provisioning persistent volumes. The plugin listens for PersistentVolumeClaims (PVC) that have a StorageClass that is configured to use Quobyte. Once a new persistent volume claim is received, the Quobyte CSI plugin communicates with the Quobyte API to provision the new persistent volume on Quobyte. For PVCs that have a size, the CSI plugin also creates a Quobyte volume quota.
Once the new volume has been created, all Quobyte clients across the cluster automatically mount it. The CSI plugin tells Kubernetes where the volume is mounted on the local worker nodes, e.g. /home/quobyte/mounts/pvc-volume-123. Kubernetes then maps this path into the application’s container at the requested location, e.g. in /data.
Quobyte Helm Chart for Easy Installs
The Quobyte Helm Chart makes it easy to install or update all three components (Core Services, Clients, and CSI Plugin) on a Kubernetes cluster. You can also use it to install just the CSI plugin and client if your Quobyte cluster runs outside of Kubernetes.
Secure Persistent Volumes with Quobyte
User IO Mapping with Access Keys
One of the big security gaps with containers is the arbitrary uid/gids that applications in containers can use to issue IO operations. On shared file systems this renders any access control useless as applications can impersonate any user on the system.
Quobyte uses Access Keys — the same tokens your users can use for object access — to map all IO from a container to the user that owns the application. When deploying an applications users have to provide their credentials (access key plus secret key). These credentials are passed on to the Quobyte client, which maps all IO from the container onto the user matching the Access Key. Hence, providing proper access control on shared volumes, including ReadWriteMany.
Encrypted Persistent Volumes
Volumes can be encrypted end-to-end with AES-XTS. This feature can be enabled on a per-volume basis using the Quobyte Policy Engine. The data is encrypted and decrypted by the Quobyte client, i.e. data is encrypted at rest and in transit.
Communication over TLS
In addition to data encryption, the communication between Quobyte clients and services can be protected with TLS. Administrators can configure which networks require TLS and which networks are trusted for plaintext communication.
Originally posted on Quobyte’s blog on October 19, 2022.
