GFS2
| Industry | Software |
|---|---|
| Founded | 1993 |
| Founder | Bob Young, Marc Ewing |
| Headquarters | Raleigh , United States |
| Products | Global File System 2 (GFS2), Red Hat Enterprise Linux, Fedora, CentOS |
| Owner | Red Hat, Inc. |
| Website | www |
In the realm of computing, the Global File System 2 (GFS2) represents a shared-disk file system created by Red Hat specifically for Linux cluster environments. It allows every node within the cluster to simultaneously access the identical shared block storage directly, setting it apart from distributed file systems that distribute data among various nodes. Additionally, GFS2 can operate as a local filesystem on standalone machines.
GFS2 does not follow a client-server architecture; instead, all participating nodes function as equals. Implementing GFS2 necessitates hardware capable of shared storage access alongside a lock manager to regulate concurrent storage usage. This lock manager is designed modularly, enabling GFS2 to utilize the distributed lock manager (DLM) in clustered configurations, while employing a "nolock" manager for standalone scenarios. Previous iterations of GFS incorporated GULM, a server-based lock manager featuring failover capabilities.
Both GFS and GFS2 are distributed as free software under the terms of the GNU General Public License.
History
The origins of GFS trace back to 1995 when University of Minnesota professor Matthew O'Keefe and his students initiated its development. Initially targeting Silicon Graphics' IRIX operating system, it was later ported to Linux (version 2.4) in 1998, leveraging Linux’s open-source nature. Between 1999 and 2000, Sistina Software managed it as an open-source project before transitioning it to proprietary status in 2001.
Following this, developers branched OpenGFS from the last publicly available version of GFS, improving compatibility with OpenDLM. Both projects eventually ceased after Red Hat acquired Sistina in December 2003 and subsequently released GFS along with cluster infrastructure components under the GPL license by mid-2004.
Red Hat invested in ongoing development focused on resolving bugs and enhancing stability, culminating in the creation of GFS2. This version, bundled with its distributed lock manager, was integrated into the Linux kernel starting with version 2.6.19. Red Hat Enterprise Linux (RHEL) 5.2 introduced GFS2 as a kernel module for testing purposes, with full support established by RHEL 5.3.
GFS2 is embedded within Fedora, Red Hat Enterprise Linux, and CentOS distributions. Commercially supported versions of GFS2 are offered by Red Hat on RHEL platforms. Starting from RHEL 8.3, GFS2 extends support to cloud computing environments utilizing shared storage devices.
Significant version milestones include:
- v1.0 (1996) exclusive to SGI IRIX
- v3.0 ported to Linux
- v4 introduced journaling capabilities
- v5 added redundant lock management
- v6.1 (2005) incorporated distributed lock management
- Linux kernel 2.6.19 merged GFS2 and DLM
- RHEL 5.3 marked the first fully supported GFS2 release
Hardware
The design of both GFS and GFS2 primarily targets storage area network (SAN) infrastructures. Although they can operate as single-node filesystems, their advanced features depend on SAN deployment, which may be achieved through technologies such as iSCSI, Fibre Channel, ATA over Ethernet (AoE), or any Linux-compatible block device shared among nodes, including DRBD.
Communication for the distributed lock manager (DLM) relies on an IP-based network—most commonly Ethernet. While some SAN configurations may integrate storage and DLM traffic on the same network, it is generally advisable to keep these networks separate.
A critical requirement for GFS is a fencing mechanism that isolates malfunctioning nodes to maintain cluster stability. Typical fencing solutions involve power control devices or remote management interfaces such as DRAC, IPMI, or ILO. Virtualized and hypervisor-level fencing alternatives are also applicable. Fencing safeguards recovery processes by preventing failed nodes from interfering and can facilitate automatic rebooting once recovery concludes.
Distinctions from a Local Filesystem
While GFS and GFS2 largely replicate the behavior of local filesystems, they exhibit differences driven by cluster-specific requirements and interface constraints:
- The flock() system call cannot be interrupted by signals.
- When using the fcntl() F_GETLK command, the returned process IDs may correspond to processes running on any node within the cluster, rendering signal delivery ineffective.
- The lock_dlm cluster locking module does not support leases, although leases are available on local filesystems.
- Both dnotify and inotify function solely within a single node and are generally discouraged in clustered environments; upcoming enhancements may provide improved inotify cluster support.
- Only GFS2 offers support for the splice system call.
A vital cluster-oriented capability is the cache management system implemented through glocks (pronounced "Gee-locks"), which operate across the entire cluster. Each inode is associated with two glocks: the iopen glock, which monitors processes that have the inode open, and the inode glock, which governs cache states. These states—UN (unlocked), SH (shared read lock), DF (deferred read lock), and EX (exclusive)—correspond directly to DLM lock modes.
In EX mode, inodes can cache both dirty data and metadata; SH mode allows caching only clean data; DF mode permits caching metadata exclusively and is utilized for direct I/O operations; UN mode prohibits any caching. Exclusive locks serialize modifications to prevent conflicts, implying that operations such as file creation or deletion within the same directory, or concurrent writes to a single file, are most efficiently managed by one node. Although multi-node access is possible, it incurs additional overhead due to cache flushing.
Performance bottlenecks are common in email server setups where many clients access mail spools simultaneously. Dividing mail spools into distinct directories assigned per cluster node can greatly enhance throughput.
Journaling
Both GFS and GFS2 implement journaling capabilities; GFS2 supports journaling modes akin to those found in ext3. In data=writeback mode, only metadata is journaled—this is the sole journaling mode available in GFS. However, journaling can be enabled on individual zero-length data files with limitations such as no support for mmap or sendfile system calls. GFS2 introduces an "inherit-journal" directory attribute that causes all files and subdirectories within to inherit journaling settings. This feature offers an alternative to the data=journal mount option supported by ext3, which neither GFS nor GFS2 implement.
GFS2 also supports data=ordered mode, ensuring that dirty data is flushed before the journal commit completes. This approach prevents uninitialized blocks from appearing after node failures and corresponds to the default behavior of ext3. As of 2010, GFS2 does not provide data=journal mode but employs a unified on-disk format for both regular and journaled files. This design relaxes restrictions on modifying journaling attributes when closing files, aligning with ext3 practices.
For efficiency, each cluster node maintains its own journal—implemented as disk extents in GFS and as regular files in GFS2. The maximum number of nodes that can simultaneously mount the filesystem depends on how many journals are available.
Improvements of GFS2 Over GFS
GFS2 offers a range of advancements compared to its predecessor GFS, including:
- A dedicated metadata filesystem serving as a separate root (refer below)
- Introduction of GFS2-specific trace points starting with kernel version 2.6.32
- An XFS-style quota interface available since kernel 2.6.33
- Implementation of cached ACLs beginning with kernel 2.6.33
- Support for "discard" commands pertinent to thin provisioning and SCSI TRIM operations
- Default activation of I/O barriers when supported (configurable from kernel 2.6.33 onward)
- FIEMAP ioctl functionality for retrieving inode disk layout information
- Integration of Splice (system call) support
- mmap and splice compatibility for journaled files facilitated by a unified on-disk format
- A significantly reduced number of tunable parameters simplifying configuration
- Ordered write mode consistent with ext3 behavior (whereas GFS only supports writeback mode)
Compatibility and the GFS2 Meta Filesystem
GFS2 was engineered to facilitate straightforward upgrades from GFS by preserving much of the original on-disk architecture, including the big-endian byte ordering. Notable distinctions encompass:
- The creation of a "meta filesystem" dedicated to accessing system-related files
- A consistent on-disk format applied to both journaled and standard files
- Journals implemented as regular system files rather than specialized extents
- The introduction of additional "" system files
- Minor modifications to the inode structure and indirect block arrangement
The journaling mechanisms in GFS and GFS2 are not compatible. Transitioning between versions necessitates using the offline utility to revise metadata, which leverages unused blocks within GFS journals to establish the required files. The majority of data remains unaffected throughout this update.
Serving as an alternative root inside the primary filesystem, the GFS2 meta filesystem contains internal system files for GFS2 operations. This subsystem functions seamlessly, with mounting and unmounting processes automatically handled by GFS2 management tools.
See also
- Comparison of file systems
- GPFS, ZFS, VxFS
- Lustre
- GlusterFS
- List of file systems
- OCFS2
- QFS
- SAN file system
- Fencing
- Open-Sharedroot
- Ceph (software)