GPFS
| Company type | Software |
|---|---|
| Industry | Computer software |
| Founded | 1998 |
| Founder | IBM |
| Headquarters | Armonk , United States |
Area served | Worldwide |
| Products | Clustered file system software |
| Owner | IBM |
| Website | https://www.ibm.com/products/spectrum-scale |
GPFS (originally known as IBM Spectrum Scale), developed by IBM, is a robust clustered file system designed for high-performance environments. It accommodates deployment in shared-disk, shared-nothing distributed parallel configurations, or hybrid arrangements. GPFS is utilized by numerous top-tier commercial organizations and supercomputing systems listed on the Top 500 List.
A prominent example of GPFS deployment is the Summit at Oak Ridge National Laboratory, which led the November 2019 Top 500 rankings. Summit achieves a performance of 200 Petaflops by leveraging over 9,000 POWER9 processors alongside 27,000 NVIDIA Volta GPUs, with its storage infrastructure managed through the Alpine filesystem.
GPFS facilitates simultaneous high-throughput file access across various nodes within a cluster. It supports operating systems such as AIX, Linux, and Microsoft Windows Server, and can operate in mixed environments combining AIX, Linux, and Windows across platforms like x86, Power, or IBM Z architectures.
History
The origins of GPFS trace back to the Tiger Shark file system, an IBM research initiative launched in 1993 at the Almaden Research Center. Initially aimed at high-throughput multimedia workloads, Tiger Shark’s design proved highly effective for scientific computing applications.
Another significant antecedent was IBM’s Vesta filesystem, developed between 1992 and 1995 at the Thomas J. Watson Research Center. Vesta pioneered file partitioning techniques to optimize parallel applications running on high-performance multicomputers equipped with parallel I/O subsystems. This partitioning enabled concurrent access to multiple disjoint segments within a single file, enhancing scalability by abstracting data distribution across I/O nodes.
Commercialized as PIOFS around 1994, Vesta/PIOFS was eventually replaced by GPFS around 1998. The key innovation introduced by GPFS was substituting Vesta/PIOFS’s specialized interface with a standard Unix API, thereby concealing parallel I/O complexity from end users while preserving performance levels. Additionally, GPFS shared components with IBM Multi-Media Server and IBM Video Charger products, which is reflected in many GPFS utilities bearing the mm prefix (denoting multi-media).
In 2010, IBM demonstrated GPFS with Shared Nothing Cluster functionality (GPFS-SNC), officially launched as GPFS version 3.5 in December 2012 and currently known as FPO (File Placement Optimizer).
Architecture
GPFS is a clustered file system that segments files into blocks smaller than 1 megabyte and distributes these blocks across multiple nodes within a cluster.
Data is stored on conventional block storage devices enhanced by an internal RAID layer that virtualizes volumes to provide redundancy and enable parallel data access, functioning similarly to RAID block storage. Furthermore, GPFS can replicate data at the file level across different volumes.
Notable architectural characteristics include:
- Metadata distribution encompassing the directory hierarchy without reliance on a central "directory controller" or "index server."
- Optimized indexing mechanisms designed for handling extremely large directories.
- Distributed locking mechanisms that uphold comprehensive POSIX filesystem semantics, including exclusive file locks.
- Partition awareness: network partitions divide the filesystem into distinct groups detected via heartbeat protocols; the largest partition remains operational to allow graceful degradation.
- Online maintenance capabilities such as disk addition and data rebalancing performed without interrupting service, ensuring continuous availability of both the filesystem and associated supercomputing clusters.
Additional features encompass high availability, compatibility with heterogeneous clusters, disaster recovery solutions, security measures, support for DMAPI, integration with HSM, and facilitation of ILM.
Comparison with Hadoop Distributed File System (HDFS)
The Hadoop ecosystem includes HDFS, designed to store vast amounts of data on standard hardware without relying on RAID or SAN configurations.
- Both systems divide files into blocks that are distributed among multiple nodes.
- GPFS provides comprehensive POSIX-compliant filesystem features.
- While GPFS spreads directory indexes and metadata across the cluster, HDFS centralizes these functions on Primary and Secondary Namenodes, which demand significant RAM resources.
- GPFS utilizes smaller block sizes, whereas HDFS opts for larger blocks (typically 64 MB or more) to minimize the storage burden on Namenodes. Handling numerous small files in HDFS can rapidly exhaust filesystem indexing capacity, constraining scalability.
Information Lifecycle Management
Disks are organized into storage pools based on criteria such as performance level, physical location, or reliability—for instance, distinguishing high-speed Fibre Channel drives from cost-effective SATA disks—enabling tiered storage strategies within a single filesystem.
A fileset represents a subtree within the namespace, effectively dividing the filesystem into administratively manageable segments. These filesets act as control points for quotas and policies that govern data placement and migration. Depending on user-defined rules, data within a fileset may reside across multiple storage pools.
There are two main categories of policies: file placement and file management. File placement policies determine where newly created files are stored, considering factors like filename patterns, user identity, or associated fileset. File management policies handle actions such as relocating, replicating, or deleting files based on attributes like last access time or file size, all without modifying the directory hierarchy.
The policy engine is highly scalable and capable of running simultaneously on numerous nodes, enabling it to enforce management rules over billions of files within a matter of hours.