DFSORT
| Company type | Software |
|---|---|
| Industry | Mainframe computing |
| Founded | 1968 |
| Products | Sort/Merge utilities for IBM mainframes |
The Sort/Merge utility is a mainframe application created to organize records within a file, combine already sorted files, or duplicate specific records. These tools employ conventional sorting algorithms enhanced by proprietary improvements.
In the early days of mainframes, limited main memory capacity was common, while large datasets were frequently processed. As a result, initial Sort/Merge utilities focused on optimizing sorting operations on secondary storage, such as tape or disk. The 1968 OS/360 Sort/Merge introduced five "sequence distribution techniques" designed to accommodate available devices.
Historically, the name SORT has denoted the preferred sorting program installed at a site, encompassing IBM's Sort/Merge as well as third-party solutions like SYNCSORT and CASORT. IBM’s DFSORT is often recognized by its program identifier ICEMAN (component ICE); the original OS/360 Sort/Merge bore the code IERRCO00, component IER, also referred to simply as SORT.
Virtual storage systems
Prior to the System/370, every IBM mainframe operating system included built-in sort/merge utilities. With the advent of virtual storage platforms such as DOS/VS and OS/VS, IBM separated much software into chargeable products, offering sort/merge solutions like 5734-SM1 (OS Sort/Merge) and later 5740-SM1 (OS/VS Sort/Merge), which was subsequently renamed Data Facility Sort (DFSORT).
In 1990, IBM incorporated the BLOCKSET merge algorithm into DFSORT as an enhancement over the OS/360 Sort/Merge. Originally developed by an IBM Systems Engineer in 1963, BLOCKSET was rediscovered in archival materials and implemented decades afterward.
Usage
Sort/Merge ranks among the most extensively utilized applications within mainframe environments, accounting for approximately twenty percent of overall processing capacity.
Contemporary Sort/Merge utilities offer functionalities beyond basic sorting and merging, including file duplication, record filtering or exclusion, summarization, elimination of duplicates, data reformatting, appending information, and report creation. These tools support numerous "user exits," which are load modules or object decks dynamically linked during runtime. Working storage datasets (SORTWK01 through SORTWKnn) may be allocated on disk or tape; however, the BLOCKSET algorithm mandates disk-based working storage. Generally, increasing the number of working storage datasets improves processing efficiency.
Competition
Given its critical role, multiple vendors provide their own sort/merge packages compatible with IBM mainframes and operating systems such as z/OS, z/VM, and z/VSE. These alternatives largely maintain compatibility with IBM’s SORT while often incorporating additional capabilities. Leading sort/merge products include:
- DFSORT from IBM
- SyncSort by Syncsort, Inc.
- CA-Sort by CA Technologies
(Some providers also offer variants for platforms including Unix, Linux, and Windows.)
Migration
Sort/Merge functionality remains essential in mainframe settings. Transitioning to environments like Unix, Linux, or Windows necessitates a sort/merge tool; products such as MFSORT from Micro Focus and AHLSORT replicate DFSORT features outside of mainframe systems.
IBM OS/360 SORT
Prior to the development of virtual storage operating systems, the volume of input data sets exceeded the capacity of main memory for complete sorting. To address this, SORT implemented a replacement selection technique aimed at reducing memory consumption.[1] The utility focused on sequence distribution methods, either default or user-defined, to enhance the efficiency of secondary storage "sort work" (SORTWK) files. These strategies effectively managed the distribution of partially ordered record sequences.
OS/360 SORT provided five distinct distribution methods:[1]
- Magnetic tape-based:
- Balanced (BALN) – demanded 12,000 bytes of main memory and utilized 2x+1 tape drives for intermediate storage, where x represents the number of input tape volumes, capped at 15 reels.
- Polyphase (POLY) – required 12,000 bytes of memory and three intermediate tape devices; applicable to a single input reel only.
- Oscillating (OSCL) – needed 21,000 bytes and up to max(x+2,4) intermediate tapes, with x being the count of input volumes, up to 15.
- Direct access storage:
- Balanced (BALN) – consumed 13,000 bytes and between three to six disk work areas; sorting capacity was influenced by both main and auxiliary storage availability.
- Crisscross (CRCX) – incompatible with IBM 2311 or IBM 2301 devices; required 24,000 bytes and six to seventeen auxiliary work areas; overall capacity was dependent on accessible storage.
IBM OS/VS SORT
The tape-based distribution methods from OS/360 SORT were preserved in OS/VS SORT under the term "conventional techniques." However, disk sorting strategies were updated and replaced by four new methods:
- FLR-Blockset designed for fixed-length records
- VLR-Blockset tailored for variable-length records
- Peerage optimized for fixed-length records
- Vale suitable for both fixed and variable-length records