ZFS
| Company type | Software |
|---|---|
| Industry | Computer software, Data storage |
| Founded | 2001 |
| Founder | Jeff Bonwick, Bill Moore, Matthew Ahrens |
| Owner | Oracle Corporation (proprietary version) |
| Parent | Sun Microsystems (original developer) |
ZFS—initially known as the Zettabyte File System—is a file system integrated with volume management, created by Sun Microsystems and launched within the Solaris platform in 2001. Between 2005 and 2010, ZFS was distributed as open source under the OpenSolaris project until Oracle Corporation acquired Sun Microsystems and transitioned ZFS to a proprietary model. During this timeframe, ports of ZFS were developed for Linux, Mac OS X (under the name MacZFS), and FreeBSD.
Following the discontinuation of OpenSolaris, the illumos project forked it in 2010, preserving ZFS's open source development. The OpenZFS initiative was founded in 2013 to coordinate ongoing open source contributions to ZFS. This organization manages the central codebase, while various companies handle platform-specific integrations and quality assurance. OpenZFS has become widely embraced across Unix-like operating systems, delivering enterprise-grade storage capabilities and scalable data management solutions.
Overview
Data storage management encompasses two primary functions: managing the physical volume management of block devices—such as hard drives and SD cards—which are structured into logical units like VDEVs (Virtual Devices in ZFS), and overseeing data organization on these units through file systems or alternative storage methods.
For instance, a RAID configuration combining two hard disks with an SSD cache is administered via Intel's Rapid Storage Technology (RST), which is embedded in desktop chipsets and firmware. To the Windows operating system, this setup appears as a single NTFS-formatted volume, without visibility into underlying operations like caching or RAID reconstruction. This illustrates the distinction between device-level management and file-level management on the volume.
ZFS distinguishes itself by unifying these functions, operating simultaneously as a volume manager and a file system. It maintains comprehensive knowledge of physical disk status, logical layouts, and every stored file. Designed to safeguard against data corruption from hardware faults, operating system failures, or bit rot, ZFS continuously verifies, repairs, and optimizes all storage processes—capabilities that exceed those of traditional controllers paired with separate file systems.
Additionally, ZFS provides sophisticated dataset- and pool-level snapshots and replication, including snapshot-based cloning. According to documentation from FreeBSD, these features rank among its "most powerful," outperforming snapshot functionalities found in other file systems. It supports extremely high snapshot counts without degrading performance, enabling frequent snapshots that protect against data loss due to user error or malicious activity during system modifications. Snapshots can be live-rolled back or used for historical data views on expansive file systems, minimizing dependence on conventional backup and restore methods. Furthermore, snapshots can be cloned into new independent file systems. At the pool level, ZFS can create snapshots known as "checkpoints," allowing rollback of significant operations affecting entire pools or datasets.
2004–2010: Sun Microsystems Era
In 1987, AT&T joined forces with Sun Microsystems to unify prominent Unix variants—Berkeley Software Distribution (BSD), UNIX System V, and Xenix—culminating in Unix System V Release 4 (SVR4). This platform evolved into Solaris, succeeding SunOS 4.
ZFS development commenced at Sun under engineers Jeff Bonwick, Bill Moore, and Matthew Ahrens beginning in 2001, with a public announcement made in 2004. By late 2005, its source code had been incorporated into Solaris development branches and was included in Solaris 10’s major update released in June 2006. As a significant innovation, ZFS enhanced scalability and data management for enterprise storage infrastructures.
Initially proprietary software, Solaris gradually embraced open source when Sun released most of its code under the CDDL license in 2005 as part of the OpenSolaris initiative. Under this open-source framework, ZFS was adapted for Linux distributions, Mac OS X (MacZFS), and FreeBSD platforms. The file system is engineered to handle extraordinarily large data volumes—up to 256 quadrillion zettabytes.
In September 2007, NetApp initiated legal action against Sun alleging patent infringement related to ZFS’s Write Anywhere File Layout technology; Sun responded with counterclaims. The dispute concluded with an undisclosed settlement reached in 2010.
2010–present: Oracle Development and OpenZFS Evolution
Starting in 2005, various adaptations of ZFS were introduced. Following Oracle's acquisition of Sun Microsystems in 2010, Oracle transitioned ZFS to a proprietary model, while open-source progress persisted independently under the OpenZFS initiative beginning in 2013.
Overview
ZFS offers several features crucial for enterprise data management, including:
- Engineered for scalable and durable data storage with zero tolerance for data loss and extensive customization options.
- Implements hierarchical checksums for all data and metadata, storing them alongside parent blocks to guarantee data integrity and facilitate automatic corruption repair.
- Allows configuration of multiple data or metadata replicas to improve fault recovery.
- Supports automatic rollback of recent modifications upon detection of errors or inconsistencies.
- Employs automated self-healing mechanisms—often transparent to users—through error correction codes, redundant copies, write-intent logs (SLOG/ZIL), RAID/RAID-Z parity schemes, and mirrored datasets.
- Provides native compatibility with traditional RAID configurations and advanced RAID-Z layouts that enhance striping efficiency while minimizing rebuild efforts.
- Integrates tiered storage and cache management that utilize file system awareness, delivering performance improvements beyond conventional volume-based approaches.
- Includes built-in snapshotting, backup, and replication capabilities, though external tools are necessary for management orchestration.
- Features native compression alongside resource-intensive deduplication to maximize storage utilization.
- Enables efficient RAID rebuilds by focusing exclusively on missing or corrupted blocks, thereby reducing recovery durations.
- Maintains independence from specific RAID hardware; ZFS pools remain accessible across diverse hardware platforms or operating systems without risking data loss.
- Utilizes sophisticated caching techniques achieving high cache hit ratios (commonly exceeding 80%), including dedicated fast devices (SLOG/ZIL) that transform synchronous writes into asynchronous operations to boost write throughput.
- Offers extensive tunability to optimize performance across various workload profiles.
- Suitable for deployment within high-availability clusters and computing infrastructures, although not initially designed explicitly for such environments.
Data Integrity
A defining characteristic that sets ZFS apart from other file systems is its rigorous emphasis on preserving data integrity. It safeguards user information against silent data corruption caused by factors such as data degradation, power interruptions, disk firmware defects, phantom writes, erroneous I/O operations, DMA parity faults, driver malfunctions, and accidental overwrites.
Research dating back to 1999 revealed that prevalent filesystems of the era—including UFS, Ext, XFS, JFS, and NTFS—along with hardware RAID solutions lacked adequate mechanisms to prevent corruption. Early analyses suggest ZFS delivers superior protection while outperforming UFS in speed, establishing it as a contemporary alternative.
ZFS ensures data integrity through a checksum mechanism based on either a Fletcher-based algorithm or SHA-256 hashing applied throughout the filesystem hierarchy. Every block of data is checksummed, with the checksum stored not within the block but in its parent pointer. This verification process extends upward to the root node, forming a structure known as a Merkle tree. Unlike most filesystems that embed checksums with their data blocks, ZFS’s approach places each block’s checksum in its parent reference, enabling comprehensive validation of the entire storage pool.
When accessing any block—be it data or metadata—ZFS verifies the associated checksum. If validation succeeds, the data are delivered to the requester; if discrepancies are detected, ZFS attempts self-repair by retrieving intact copies from redundant storage layers such as internal mirroring. Additional redundancy can be achieved by configuring or , which store multiple copies at the expense of usable capacity. Essential pool management metadata are also duplicated by default to enhance reliability.
In cases where damaged blocks exist but recoverable copies are available or can be rebuilt using checksums and parity information, ZFS automatically repairs corrupted data and reestablishes redundancy. If no viable recovery options exist, the pool transitions into a faulted state that halts further operation until corrective actions are taken.
By default, ZFS does not verify the consistency of cached memory contents (ARC), relying on enterprise-grade hardware equipped with error-correcting RAM. Nevertheless, in-memory data verification can be activated through debug settings when necessary.
RAID
To ensure data integrity, ZFS relies on maintaining multiple copies of data or parity details, generally spread across several disks. This functionality is commonly achieved through a RAID controller or software RAID embedded within the file system itself.
Preference for avoiding hardware RAID controllers
While ZFS can function with hardware RAID setups, it achieves superior performance and enhanced data protection when granted direct access to raw storage devices. ZFS requires precise disk status feedback to validate data writes and utilizes advanced algorithms to manage caching, flushing, and disk operations efficiently.
The use of hardware, firmware, or other "soft" RAID controllers that modify the ZFS-to-disk I/O pathway can negatively impact both performance and data reliability. Controllers that cache data or consolidate drives into a single logical unit obscure the detailed disk-level visibility ZFS depends on, potentially causing degraded performance, slower recovery from failures, or even data loss. For example, hardware RAID cards may obstruct ZFS’s ability to detect disk faults, array degradation, or corruption, and hinder its capacity to optimally distribute and repair stored data. Furthermore, many RAID controllers embed proprietary metadata on drives that complicate software RAID access. In case of controller malfunction, recovering data might necessitate identical replacement hardware, which could be unavailable.
Contrary to other systems where RAID cards offload processing to boost speed and dependability, such controllers typically diminish both aspects when used with ZFS.
If connecting disks through a RAID or similar controller is unavoidable, it is recommended to reduce controller intervention by employing a straightforward HBA, a basic fanout card, or configuring the device in JBOD mode (disabling RAID features and caching). Nonetheless, even with JBOD enabled, some RAID cards may still interfere due to caching mechanisms or drive timeout policies—often requiring drives equipped with Time-Limited Error Recovery (TLER)/CCTL/ERC capabilities to prevent drive dropouts. Not all controllers are appropriate even when their RAID functions are turned off.
ZFS's solution: RAID-Z and mirroring
ZFS implements "software" RAID through RAID-Z (a parity-based scheme akin to RAID 5) and disk mirroring (similar to RAID 1), providing considerable flexibility without needing dedicated hardware.
RAID-Z utilizes variable stripe widths where each block forms its own RAID stripe, allowing full-stripe writes. Coupled with ZFS’s copy-on-write architecture, this design eliminates the write hole error and offers better performance than conventional RAID 5 by avoiding read-modify-write cycles.
Because stripe sizes fluctuate dynamically, reconstructing RAID-Z arrays depends on filesystem metadata that reveals the RAID layout—information accessible only through ZFS’s unified logical and physical data structures. This close integration also enables verification of every block using a 256-bit checksum, delivering data integrity far beyond standard RAID implementations.
RAID-Z detects and repairs silent data corruption through "self-healing": upon encountering read errors, ZFS employs parity information to locate and fix faulty disks before supplying data back. Both RAID-Z and mirroring operate without requiring NVRAM or write caches while maintaining reliable high-speed storage on commodity drives.
ZFS offers five different RAID-Z configurations: striping (no redundancy), RAID-Z1 (tolerance for one disk failure), RAID-Z2 (two disks), RAID-Z3 (three disks), along with mirroring (which can withstand failure of all but one disk).
The development of RAID-Z3 was driven by the advent of multi-terabyte drives in the early 2000s; extended rebuild times increased vulnerability to data loss due to stressed disks. By incorporating triple parity protection, RAID-Z3 substantially improves redundancy and lowers the risk of losing data during lengthy rebuild processes.
Resilvering and Scrubbing (Array Synchronization and Data Integrity Verification)
Unlike the traditional Unix/Linux utility fsck used for file system repairs, ZFS does not incorporate a direct counterpart. Instead, it employs an integrated scrub process that continuously inspects and rectifies silent data corruption on active, mounted file systems without requiring downtime. Important differences between the two include:
- fsck mandates an offline file system for operation, whereas scrub functions on live systems.
- fsck generally verifies only metadata integrity, while scrub validates both metadata and actual data content.
- fsck may fail to recover data with corrupted checksums; in contrast, ZFS stores checksums separately from the data blocks, improving repair success rates.
- ZFS keeps multiple copies of metadata and data distributed across disks and mirrors, enabling scrub to detect and correct widespread corruption more effectively.
- Scrubbing involves a comprehensive scan of all data and metadata, which can take hours on large RAID configurations compared to fsck’s quicker metadata-only checks.
- While fsck attempts to repair issues with potential partial data loss, scrub depends on redundancy and will mark the pool as faulty if recovery is unattainable.
Sun/Oracle advises performing scrubs monthly for enterprise-grade disks and weekly for commodity drives to uphold data integrity and system stability.
Capacity
ZFS is designed as a 128-bit file system, allowing it to manage approximately 1.84 × 1019 times more data than 64-bit systems such as Btrfs. Its maximum capacity thresholds are set at levels far beyond practical use; for instance, completely filling a single zpool would require an infeasible 3×1024 terabytes of storage.
Notable theoretical limits consist of:
- Maximum single file size: 16 exbibytes (264 bytes)
- Maximum directory entries: 248
- Maximum attribute size: 16 exbibytes
- Maximum attributes per file: 256 (restricted to 248 by directory file count)
- Maximum zpool size: 256 quadrillion zebibytes (2128 bytes)
- Maximum devices per zpool: 264
- Maximum file systems per zpool: 264
- Maximum zpools per system: 264
Encryption
Within Oracle Solaris, ZFS encryption is embedded directly into the I/O pipeline, enabling blocks to be sequentially compressed, encrypted, checksummed, and deduplicated during write operations. Encryption settings are configured at the dataset level and can be modified without interrupting system availability. Wrapping keys are inherited by descendant datasets, while each dataset generates unique data encryption keys shared exclusively with its children. Additionally, there is a facility to rotate encryption keys without re-encrypting existing stored data.
By 2019, this encryption capability was fully incorporated into OpenZFS version 0.8.0 for Debian and Ubuntu Linux platforms.
Some users have encountered issues with native ZFS encryption, particularly during send/receive procedures, although the root causes have yet to be definitively identified.
Read/Write Efficiency
ZFS dynamically distributes data storage across all vdevs and devices within a pool to optimize performance. Its write allocation strategy adapts when new disks join the pool. Data writes are balanced based on available free space in each vdev so that less-utilized vdevs receive a higher share of writes, preventing scenarios where some vdevs fill up prematurely while others remain underused. This load balancing improves read speeds by enabling parallel access across multiple disks. Proper management of pools and vdevs is crucial to avoid inefficiencies caused by uneven storage consumption.
Over time, free space within ZFS pools may become fragmented; however, no native defragmentation tool exists. Observations suggest that significant fragmentation combined with heavy disk activity can negatively impact overall performance.
Storage Components, Spares, and Allocation Limits
ZFS storage pools can utilize hot spare drives that automatically take over when a disk begins to fail. In configurations using mirrored devices, disks can be organized by their physical enclosure to ensure continued functionality even if an entire chassis fails.
Pools may comprise a mix of different device types, enabling the combination of diverse storage media to increase overall capacity. This adaptability facilitates scalable growth and efficient distribution of storage space among file system entities such as datasets or ZVOLs.
Every file system instance within a zpool has access to the full capacity of all vdevs. System administrators have the ability to impose quotas to restrict the amount of space each instance can consume and set reservations to guarantee a minimum allocation of storage.
Caching Layers: ARC, L2ARC, Transaction Groups, ZIL, SLOG, Special VDEV
ZFS implements several caching tiers to enhance read and write throughput while balancing cost considerations. Data that is frequently requested is stored in RAM caches, whereas less frequently accessed information is moved to slower devices like solid-state drives (SSDs). Data that is rarely accessed remains on traditional hard disk drives. ZFS dynamically shifts data among these layers based on access patterns.
There are four main cache types: two dedicated to reads and two for writes, each comprising a RAM-based cache and a fast-storage (typically SSD) cache.
| Location | Read Cache | Write Cache | |
|---|---|---|---|
| Primary cache | RAM | The ARC (Adaptive Replacement Cache) operates here and is active whenever sufficient memory is available. By caching frequently used data in RAM, ARC minimizes disk reads. When RAM is limited, ARC's effectiveness declines, leading to increased disk access and reduced system performance. | Write operations are managed through transaction groups, which accumulate changes over intervals of 5–30 seconds before flushing them to disk. This batching improves disk efficiency but introduces a slight risk of data loss during sudden power or hardware failures. ZFS addresses this risk with write journaling supported by the secondary write cache layers (SLOG/ZIL). If write input exceeds processing speed, the system temporarily halts accepting new data until disks catch up. |
| Secondary cache and Intent log | High-speed storage devices such as SSDs, which are hot-swappable in recent ZFS releases | The optional L2ARC ("Level 2 Adaptive Replacement Cache") stores additional read data on SSDs, greatly enhancing read speeds and accelerating Data deduplication when the deduplication table fits entirely within it. Initial population of L2ARC from an empty state can require several hours. Losing L2ARC reduces read performance but does not result in data loss. | The SLOG (Separate Log device) holds the ZIL (ZFS Intent Log). This dedicated device speeds up synchronous write acknowledgments by quickly recording writes to persistent storage, protecting against data loss during power outages or write errors by ensuring writes are safely committed before being applied to the main pool. Without a SLOG device, the ZIL resides on the primary pool itself, which slows down confirmation of synchronous writes. The SLOG device is seldom read except during recovery after failures. For workloads with heavy synchronous writes—common in ESXi, NFS, and database systems—the SLOG significantly enhances both performance and reliability.
Because losing the log device can result in recent writes being lost, it is advisable to mirror the log device for redundancy. Earlier iterations of ZFS risked total pool failure if the log device failed; modern versions have reduced this vulnerability. |
ZFS also incorporates additional caching mechanisms such as per-VDEV caches and divisions within ARC that balance user data with system metadata, providing granular control over performance tuning.
Special VDEV Category
Starting with OpenZFS version 0.8, it is possible to configure a Special VDEV category designed to preferentially house filesystem metadata, optionally including the Data Deduplication Table (DDT) and small filesystem blocks. This allows metadata to reside on high-speed solid-state drives, while standard file data continues to be stored on traditional spinning disks. Such an arrangement enhances the speed of metadata-heavy tasks like filesystem traversal, scrubbing, and resilvering, without requiring a complete SSD-based storage solution.
Copy-on-write transactional framework
ZFS utilizes a copy-on-write transactional object architecture where each block pointer carries a 256-bit checksum or hash (with options such as Fletcher-2, Fletcher-4, or SHA-256) that is validated during reads. Rather than overwriting existing data blocks directly, any modifications result in new blocks being written, accompanied by updated metadata blocks. For improved efficiency, these changes are batched into transaction groups and leverage the ZIL (ZFS Intent Log) write cache for synchronous write operations. The organization of blocks and their checksums follows a hierarchical tree structure.
Snapshots and clones
The copy-on-write mechanism allows ZFS to preserve previous data blocks when new writes occur, enabling the swift creation of consistent snapshots that represent the entire filesystem state at a specific moment. Snapshots efficiently conserve space by sharing unchanged blocks and are inherently read-only; however, they are not substitutes for full backups. Restoration can be performed at the level of entire snapshots or individual files.
Writable snapshots, known as "clones," establish separate filesystems that share common data blocks. As modifications happen within clones, new blocks are allocated while unaltered blocks remain shared among them—utilizing copy-on-write principles to maintain storage efficiency.
Snapshot transmission and reception
ZFS allows file systems to be migrated between pools or sent to remote systems using the send command, which generates a stream representation of the filesystem’s state. This stream can correspond to either a complete snapshot or an incremental delta between snapshots, with size proportional to the amount of changed data. This functionality supports efficient offsite backup synchronization and facilitates high availability through pool mirroring.
Dynamic striping
By employing dynamic striping across all devices in a pool, ZFS maximizes throughput by automatically increasing stripe width as additional devices are added. This approach ensures an even distribution of write operations across all disks within the zpool.
Variable block sizes
ZFS supports variable block sizes with a default of 128 KB and allows tuning of maximum block size to suit different workload requirements. When compression is enabled, blocks are stored on disk in compressed form at smaller sizes, which reduces storage consumption and enhances I/O performance while incurring additional CPU processing overhead.
Simplified filesystem creation
The process of creating or expanding a ZFS filesystem inside a storage pool is straightforward and comparable in complexity to creating a directory, rather than involving traditional volume management tasks.
Adaptive endianness
To ensure cross-platform compatibility, ZFS pools and filesystems store metadata using an endian-adaptive format. Metadata blocks are written in the native byte order of the system and undergo byte-swapping upon reading if necessary. The actual stored data remains unchanged, preserving consistency at the application level across different hardware architectures.
Deduplication
Data deduplication was incorporated into the ZFS codebase in October 2009, with OpenSolaris ZFS development builds becoming accessible starting December 2009 (build 128).
Implementing deduplication effectively demands a large amount of RAM—commonly ranging from 1 to 5 GB per terabyte of storage capacity. The memory requirement is influenced by the count of unique data blocks and the storage consumed both on disk and in memory, which can be evaluated using utilities such as zpool and zdb. When memory or cache availability is insufficient, virtual memory thrashing may occur, significantly impairing system performance or leading to memory shortages. Additionally, deduplication imposes a high CPU load during write operations, further affecting overall system responsiveness.
Certain storage providers deploy customized versions of ZFS to attain elevated data compression ratios. In 2012, prominent examples included GreenBytes and Tegile. Oracle acquired GreenBytes in May 2014, primarily for its ZFS deduplication and replication technologies.
Because deduplication requires substantial system resources and can negatively influence performance, it is typically advised only when system capacity and data profiles are well-suited to benefit from this space optimization approach.
Additional capabilities
- Defined I/O priority utilizing deadline scheduling.
- Advertised globally optimal sorting and aggregation of I/O operations.
- Support for multiple independent prefetch streams with automatic detection of length and stride.
- Parallelized directory operations executed in constant time.
- Comprehensive end-to-end checksumming through a "Data Integrity Field," facilitating detection and recovery from data corruption using redundancy; supports three hashing algorithms: fletcher (optimized for speed), SHA256 (standard), and Skein (salted).
- Native transparent filesystem compression compatible with LZJB, gzip, LZ4, and Zstd algorithms.
- Advanced data scrubbing and resilvering (resynchronization) mechanisms.
- Distribution of load and space utilization evenly across disks within the pool.
- Ditto blocks feature enabling configurable replication levels per filesystem, creating additional copies for user data and metadata primarily to guard against corrupted sectors by distributing replicas across multiple devices.
- The ZFS architecture (copy-on-write combined with superblocks) guarantees safety when using disks with write caches, provided write barriers are respected, delivering both reliability and enhanced performance.
- On Solaris platforms, adding entire disks to a ZFS pool automatically activates their write caches; managing partial disks does not enable this feature. In contrast, FreeBSD’s GEOM framework manages disk flushes for partitions, circumventing this limitation.
- Enforcement of quotas at the user, group, project, and dataset levels.
- Filesystem-level encryption introduced with Solaris 11 Express and OpenZFS 0.8; on other platforms, encrypted disks can be utilized similarly (for example, FreeBSD’s GELI).
- Ability to import pools in read-only mode.
- Capability for data recovery through rolling back complete transactions during zpool import.
- Snapshots can be generated manually or scheduled automatically; they present previous data states as complete read-only filesystems or as historical file versions when accessed via CIFS/SMB/Samba (“Previous versions” on Windows) or AFP (“Apple Time Machine”).
- Disks may be assigned as 'spares' to facilitate automatic fault management by activating spare drives and transparently resilvering data upon failure events.
Limitations
- Up through Solaris 10 Update 11 and Solaris 11.2, reducing the count of top-level vdevs within a pool was not feasible except for hot spares, cache, and log devices, nor was it possible to decrease pool capacity. This capability was under development and eventually introduced in Solaris 11.4 (August 2018) and OpenZFS 0.8 (May 2019), allowing online shrinking by removing non-redundant top-level vdevs.
- As of 2008, it was not possible to add an additional disk as a column to an existing RAID Z, RAID Z2, or RAID Z3 vdev; instead, users could create a new RAID Z vdev and incorporate it into the zpool.
- Conventional nested RAID setups such as RAID 51 cannot be configured natively in ZFS without external tools. Vdevs must consist solely of raw disks or files rather than other vdevs, restricting complex nesting options. Nevertheless, because ZFS stripes data across multiple vdevs, configurations equivalent to RAID 50 or RAID 60 are frequently implemented.
- Changing the number of devices within a top-level vdev typically requires taking data offline, destroying the pool, and re-creating it with the desired device count. One exception is the ability to add redundancy to an existing mirror at any time. Moreover, if all top-level vdevs are mirrors with adequate redundancy, the zpool split command can remove one vdev from each top-level vdev, generating a second pool containing identical data.
Data recovery
ZFS does not rely on utilities like fsck since it is engineered to self-heal. When storage pools are properly designed with sufficient redundancy, conventional repair tools become unnecessary. However, in cases of pool corruption caused by hardware failures, flawed design, or accidental damage, traditional ZFS lacked sophisticated mechanisms for partial data recovery from damaged pools.
Recent advancements in ZFS have greatly enhanced its recovery functions:
- Failure or removal of caching devices no longer results in pool loss; at worst, only the most recent transactions stored in the ZIL might be lost while preserving overall data integrity.
- For pools that cannot be mounted, modern versions of ZFS attempt to restore the latest consistent state by rolling back recent changes. Thanks to copy-on-write architecture, older versions of data and metadata often remain accessible, allowing recovery up to the point where overwritten blocks limit restoration.
- Diagnostic utilities such as zdb (ZFS debugger) and dtrace assist in troubleshooting mount issues and support manual repairs by bypassing health checks to mount damaged pools.
- , OpenZFS has introduced advanced recovery features including:
- Rewritten code with comprehensive diagnostics that simplify identifying and repairing corruption;
- Options to trust or ignore stored pool configurations, permitting mounts despite missing or faulty top-level vdevs and enabling rollback beyond problematic configuration changes. This facilitates copying recoverable files for safety and potentially reconstructing data from redundant copies;
- Solutions for disks accidentally removed from one pool and added to another, restoring lost metadata and making the pool readable again.
OpenZFS and ZFS
Following Oracle Corporation’s acquisition of Sun in 2010, public development of ZFS and OpenSolaris ceased. Some contributors branched off the final OpenSolaris release into the Illumos project. Because of ZFS’s benefits, it has been ported across various platforms with differing feature sets. To coordinate efforts and avoid fragmentation, OpenZFS was founded in 2013.
Matt Ahrens, one of ZFS’s architects, observed that by 2019 more than half of the original OpenSolaris ZFS code had been replaced within OpenZFS through community contributions, resulting in “Oracle ZFS” and “OpenZFS” becoming incompatible both politically and technically.
Commercial and Open Source Offerings
- In 2008, Sun Microsystems introduced storage appliances in the 7000 series, built on the ZFS file system.
- Oracle launched the ZS3 line of ZFS-based filers in 2013, which secured leading positions in the SPC-2 performance benchmark.
- Also in 2013, iXsystems debuted NAS solutions employing ZFS technology—FreeNAS (currently TrueNAS CORE) targeting small office/home office users, alongside TrueNAS for enterprise clientele.
- By 2014, Netgear had released ReadyDATA, a collection of enterprise-grade NAS devices utilizing ZFS.
- In 2015, rsync.net unveiled a cloud storage service allowing users to create zpools and transfer data through zfs send/receive commands.
- Starting in 2020, iXsystems began development on TrueNAS SCALE, a hyperconverged software solution based on ZFS designed to serve both SOHO and enterprise environments.
Oracle Corporation, Closed Source Transition, and Forking (Post-2010)
After acquiring Sun Microsystems in January 2010, Oracle Corporation promptly discontinued the OpenSolaris distribution along with its open source development framework. By August of the same year, Oracle halted public updates to Solaris OS and Networking source code, effectively reverting Solaris 11 to a proprietary closed source platform.
In reaction to these changes, the illumos initiative was launched on August 3, 2010, by key Solaris engineers committed to maintaining open source Solaris development and completing the open sourcing of components not previously released by Sun. Established as the Illumos Foundation—a 501(c)6 trade association based in California—the project initially refrained from becoming a fork or standalone distribution. Nevertheless, following Oracle's termination of OpenSolaris, illumos forked the final Solaris ON release to evolve into an independent operating system. The open source version of ZFS remained a core component within illumos.
Due to ZFS’s extensive adoption across multiple platforms, the OpenZFS umbrella project was formed in 2013. This collective effort coordinates collaborative development on the core ZFS codebase while permitting maintainers to oversee platform-specific adaptations.
Version History
| Legacy release |
| Template:Free |
| ZFS Filesystem Version | Release Date | Key Enhancements |
|---|---|---|
| 1 | OpenSolaris Nevada build 36 | Initial introduction |
| 2 | OpenSolaris Nevada b69 | Directory entries enhanced with object type information (file, directory, etc.) |
| 3 | OpenSolaris Nevada b77 | Added SMB sharing support, case-insensitivity features, system attributes, and integrated antivirus compatibility |
| 4 | OpenSolaris Nevada b114 | Implemented userquota, groupquota, userused, and groupused properties |
| Template:Free | OpenSolaris Nevada b137 | Introduced system attributes; symlinks recognized as separate object types |
| ZFS Pool Version | Release Date | Major Features |
|---|---|---|
| 1 | OpenSolaris Nevada b36 | Initial release |
| 2 | OpenSolaris Nevada b38 | Added Ditto Blocks for enhanced redundancy |
| 3 | OpenSolaris Nevada b42 | Introduced hot spares, double-parity RAID-Z (raidz2), and improved RAID-Z accounting mechanisms |
| 4 | OpenSolaris Nevada b62 | Enabled zpool history tracking capabilities |
| 5 | OpenSolaris Nevada b62 | Dataset gzip compression support added |
| 6 | OpenSolaris Nevada b62 | Introduced "bootfs" property for pools |
| 7 | OpenSolaris Nevada b68 | Support for separate Intent Log (ZIL) devices implemented |
| 8 | OpenSolaris Nevada b69 | Delegation of administrative privileges to non-root users enabled |
| 9 | OpenSolaris Nevada b77 | Added CIFS server functionality and dataset quota management |
| 10 | OpenSolaris Nevada b77 | Support introduced for cache devices within storage pools |
| 11 | OpenSolaris Nevada b94 | Improvements made to zpool scrub and resilver operations performance |
| 12 | OpenSolaris Nevada b96 | Snapshot properties were added |
| 13 | OpenSolaris Nevada b98 | New properties including usedbysnapshots, usedbychildren, usedbyrefreservation, and usedbydataset introduced |
Operating system compatibility
The following table outlines various operating systems, distributions, and extensions that incorporate ZFS support, along with their respective zpool versions and Solaris build numbers when relevant:
| Operating System | Zpool Version | Sun/Oracle Build Number | Notes |
|---|---|---|---|
| Oracle Solaris 11.4 | 49 | 11.4.51 (11.4 SRU 51) | |
| Oracle Solaris 11.3 | 37 | 0.5.11-0.175.3.1.0.5.0 | |
| Oracle Solaris 10 1/13 (Update 11) | 32 | ||
| Oracle Solaris 11.2 | 35 | 0.5.11-0.175.2.0.0.42.0 | |
| Oracle Solaris 11 November 2011 Release | 34 | b175 | |
| Oracle Solaris Express November 2010 Release | 31 | b151a | licensed solely for evaluation purposes |
| OpenSolaris June 2009 Release | 14 | b111b | |
| OpenSolaris (final development version) | 22 | b134 | |
| OpenIndiana | 5000 | b147 | illumos-based distro; build code 'b151a' name conflict noted |
| Nexenta Core version 3.0.1 | 26 | b134+ | GNU userland environment |
| NexentaStor Community Edition 3.0.1 | 26 | b134+ | supports up to 18 TB; includes web-based management interface |
| NexentaStor Community Edition 3.1.0 | 28 | b134+ | GNU userland environment |
| NexentaStor Community Edition 4.0 | 5000 | b134+ | supports up to 18 TB; features web administration interface |
| NexentaStor Enterprise Edition | 28 | b134+ | commercial release with web admin tools |
| Debian GNU/kFreeBSD "Squeeze" (No longer supported) | 14 | Requires installation of the "zfsutils" package | |
| Debian GNU/kFreeBSD "Wheezy-9" (No longer supported) | 28 | Requires installation of the "zfsutils" package | |
| FreeBSD | 5000 | ||
| zfs-fuse version 0.7.2 | 23 | known performance limitations; project discontinued | |
| ZFS on Linux version 0.6.5.8 | 5000 | ||
| KQ Infotech's ZFS on Linux | 28 | project defunct; source merged into LLNL-supported ZFS on Linux | |
| BeleniX version 0.8b1 | 14 | b111 | lightweight live-CD based on OpenSolaris |
| Schillix version 0.7.2 | 28 | b147 | lightweight live-CD derived from OpenSolaris |
| StormOS "hail" release | built upon Nexenta Core and Debian; succeeded by Dyson OS | ||
| Jaris distribution | Japanese Solaris-based variant derived from OpenSolaris | ||
| MilaX version 0.5 | 20 | b128a | compact live-CD built on OpenSolaris foundations |
| FreeNAS editions: | |||
| 15 | |||
| 28 | based on FreeBSD version 8.3 | ||
| 5000 | based on FreeBSD versions from 9.1 onward | ||
| 5000 | based on FreeBSD releases 11.4 and 12.2 respectively | ||
| 22 | b134 | KDE desktop environment included | |
| 22 | b130 | embedded network-attached storage system | |
| 28 | b151a | embedded NAS platform | |
| 28/5000 | illumos/Solaris foundation | supports OpenIndiana, OmniOS, Solaris with Linux management utilities | |
| 28/5000 | illumos-OmniOS branch | community-driven minimal stable/LTS storage server | |
| 28/5000 | illumos build b151+ | minimal live distribution designed for cloud and KVM hypervisor environments | |
| 5000 |
supported via MacZFS; replaced by OpenZFS on OS X | ||
| 28 |
supported through ZEVO; superseded by OpenZFS on OS X | ||
| 22 | |||
| 6 | |||
| 5000 |
native integration since2014 using ZFS on Linux | ||
| 5000 |
native support provided through installable binary modules | ||
| 5000 |
See also
- Comparison of file systems
- List of file systems
- Versioning file system—Catalog of file systems with versioning capabilities
Bibliography
External links
- Fork Yeah! The Rise and Development of Illumos—Presentation detailing the history of Solaris, Sun Microsystems' move to open source, the creation of ZFS, and its subsequent fork following Oracle's takeover.
- The best cloud File System was created before the cloud existed (archived December 15, 2018)
- Comparison of SVM mirroring and ZFS mirroring
- EON ZFS Storage (NAS) distribution
- End-to-end Data Integrity for File Systems: A ZFS Case Study
- ZFS – The Zettabyte File System (archived February 28, 2013)
- ZFS and RAID-Z: The Über-FS?
- ZFS: The Last Word In File Systems, authored by Jeff Bonwick and Bill Moore (archived August 29, 2017)
- Visualizing the ZFS intent log (ZIL), April 2013 article by Aaron Toponce
- Features of illumos including OpenZFS
- Earlier wiki resource with additional links: Getting Started with ZFS, September 15, 2014 (archived December 30, 2018), part of the illumos documentation