Jump to content

Btrieve

From Alfawiki
Btrieve Technologies, Inc.
Company typePrivate
IndustrySoftware, Database management
PredecessorSoftCraft
FoundedFebruary 1982; 44 years ago (1982-02) in Austin, Texas, United States
FounderDoug Woodward, Nancy Woodward
FateRebranded as Pervasive Software
Key people
Nancy Woodward (Chairman), Doug Woodward (CTO), Ron Harris (CEO)
ProductsBtrieve database software, Scalable SQL

Btrieve is a transactional database software designed as a navigational database utilizing the Indexed Sequential Access Method (ISAM) to enable rapid data access. It has been made available in versions compatible with DOS, Linux, legacy Microsoft Windows, 32-bit IBM OS/2, and Novell NetWare platforms.

Initially developed as a record management system by SoftCraft, Btrieve was created by Doug and Nancy Woodward, with partial financing from Doug's brother, Loyd Woodward. Doug Woodward gained full ownership after receiving half the shares as a wedding gift. Following its commercial success, Novell purchased Btrieve in 1987 to incorporate it into their NetWare operating system, while continuing to support DOS environments. Integrated into every NetWare release from versions 2.x through 4.x, Btrieve established a strong presence as an embedded database solution targeting mid-market users. In 1994, Novell divested Btrieve and its associated technology back to Doug and Nancy Woodward along with Ron Harris, forming Btrieve Technologies, Inc. (BTI).

With the introduction of version 6.15, Btrieve was restructured into a modular design functioning as one of two front-end interfaces for the standardized MicroKernel Database Engine (MKDE). The Btrieve front-end maintained compatibility with the traditional Btrieve API, while the alternative front-end, Scalable SQL, offered a relational database system featuring its proprietary SQL dialect. After launching Btrieve 6.15 alongside ScalableSQL version 4, the company rebranded itself as Pervasive Software prior to going public. Subsequently, both Btrieve and ScalableSQL were unified under Pervasive.SQL or PSQL, which was later renamed Actian Zen. While Btrieve continued to be supported for a short period thereafter, ScalableSQL was phased out, with customers encouraged to transition to Pervasive.SQL that supported both SQL and Btrieve-based applications.

Architecture

Btrieve does not classify as a relational database management system (RDBMS). Originally described as a record manager or navigational database, it manages operations such as record insertion, retrieval, modification, and deletion by leveraging ISAM indexing and storage techniques. A fundamental component of its architecture is the MicroKernel Database Engine, which enables modular backend integration within Pervasive.SQL. This design accommodates both the navigational Btrieve engine and the relational Scalable SQL engine.

Contemporary versions of Btrieve support two types of transactions: system transactions, which consist of batches of non-transactional operations or user-defined transactions, and user transactions that manipulate actual data records. System transactions are primarily used to facilitate batch processing and enhance ease of data recovery.

The structure of a Btrieve file is composed entirely of pages that are transferred between memory and storage during input/output processes. Prior to version 6.0, files included data pages, index pages, and a file control record, with indexes pointing directly to physical pages on disk. Starting with version 6.0, logical pages are mapped to physical pages through page allocation tables. The file control record contains essential metadata such as the total number of pages within the file. To safeguard against data corruption during updates, earlier releases utilized a technique called pre-image paging, whereas later versions adopted shadow paging. This change introduced significant modifications to the file format and resulted in compatibility challenges between version 6 and preceding editions.

History

Throughout its existence, Btrieve has been owned and enhanced by four distinct companies: SoftCraft, Novell, Btrieve Technologies, Inc. (which later became Pervasive Software), and Actian Corporation. It continues to enjoy a dedicated developer community, with Pervasive Software founding the "Btrieve Society" to provide ongoing support for programmers.

In the DOS environment, versions of Btrieve up to 5 operated as a terminate-and-stay-resident (TSR) API database engine, supporting multi-user database access through record locking mechanisms. The network edition functioned in a comparable manner.

Initial DOS releases were priced at approximately US$1,000; however, the TSR executable was allowed to be freely distributed alongside applications.

SoftCraft era

Btrieve was introduced in February 1982 by SoftCraft, a company based in Austin, Texas, founded by Doug and Nancy Woodward—Doug serving as vice president of development and Nancy as president. The product line progressed with several iterations: the 2.x series in 1983; version 3.0 following MS-DOS 2.0’s support for file handles; the client/server-enabled 3.1 C/S in 1985; and version 4.0 in 1986, with 4.1 adding features such as extended key types and supplementary indexes.

Although widely adopted by more than 5,000 developers—particularly within financial sectors—Btrieve functioned as an API database engine rather than a complete DBMS like dBase II, which was less expensive and offered more comprehensive features. In response, SoftCraft launched Xtrieve in 1984, a menu-driven interface that utilized a new .DDF data dictionary to enforce relational database principles.

Novell acquisition

Novell expanded its NetWare product line by acquiring SoftCraft in 1987. Following the acquisition, Nancy Woodward took on the role of vice president and general manager of Novell’s Austin division, while Doug Woodward became vice president of Advanced Database Technologies. Shortly thereafter, Btrieve 5.0 was released as a native NetWare application incorporating features such as auto-increment keys, the BROUTER network server, separate data-only and key-only files, and optional data compression. The 1990 release of version 5.1 improved transaction management with enhanced logging and roll-forward capabilities and extended APIs across DOS, OS/2, and Windows platforms. Version 6.0 arrived in June 1992 but suffered from limited marketing efforts and compatibility issues with earlier versions due to architectural changes like shadow-paging, which constrained its market expansion.

Before being acquired, SoftCraft had developed XQL, an SQL interpreter designed to meet industry standards beyond Xtrieve’s scope. This technology evolved into NetWare SQL, launched in 1989 as a basic implementation of the IBM-standard SQL interpreter.

Btrieve Technologies, Inc.

By 1994, Novell had largely ceased pursuing the vision of NetWare as a comprehensive operating system and began divesting many of its acquisitions. Due to a two-year delay in releasing version 6, Btrieve received minimal promotional support during this period. After prolonged negotiations spanning two years, Novell announced on January 26, 1994, that ownership of Btrieve would be transferred to Btrieve Technologies, Inc. (BTI). The transaction was finalized on April 29, 1994, with Nancy Woodward appointed as Chairman, Doug Woodward named Chief Technical Officer, and Ron Harris—formerly associated with Texas Instruments and a founding executive at Citrix Systems, Inc.—assuming the role of CEO.

The product underwent a complete rewrite, culminating in the release of version 6.15 for DOS, Windows, and OS/2 on July 1, 1994. Concurrently, Novell SQL was rebranded as Scalable SQL to reflect the change in ownership. In 1995, version 6.15 was further expanded to support Novell NetWare, Windows NT Server, and Windows NT/95, establishing Btrieve as a cross-platform database solution incorporating the Micro Kernel Database Engine (MKDE) architecture.

Pervasive Software

In 1996, BTI underwent a rebranding to become Pervasive Software, simultaneously renaming its flagship product to Pervasive.SQL. The company went public in September 1997, aiming to strengthen its foothold in the relational database sector and reposition itself as an SQL provider, while still advancing the development of Btrieve. The release of version 6.30 preserved the MKDE architecture. That same year, ScalableSQL 4.0 and Btrieve 7.0 were also launched.

By 2000, Novell encountered backlash after substituting the complete Pervasive.SQL package with a restricted 90-day trial version in NetWare 5.1 and subsequent editions. The most recent significant release, Pervasive PSQL Summit v11, was introduced in September 2010.

Actian Corporation

Actian Corporation acquired Pervasive Software in 2013. Subsequently, in February 2016, Actian released Btrieve 12, continuing the product’s development under its new management.

Btrieve for DOS

SoftCraft created a DOS client-based variant of Btrieve, described as a "Btrieve engine operating on a specific workstation." This engine accessed files directly through operating system calls, enabling record modifications either locally or across a network. It allowed up to five users simultaneous database access. All record manipulation was performed on the workstation running the engine. Later iterations introduced two operational modes: SEFS (Single-Engine File Sharing) and MEFS (Multi-Engine File Sharing).

Btrieve for NetWare

The NetWare version of Btrieve shared similarities with the DOS edition but incorporated features tailored to NetWare environments. It operated a server process named BSERVER on the file-sharing server to manage data input/output alongside the network file system. Initially implemented as a NetWare Value-Added Process (VAP) in NetWare 2.x (BSERVER.VAP), it was later redeveloped as a NetWare Loadable Module (NLM) for versions 3.x and beyond.

The BSERVER process managed record access and forwarded requested data to other servers through the BROUTER process. Client machines utilized requesters available for DOS, OS/2, Windows, and UnixWare platforms to initiate database I/O operations via BREQUEST.EXE, which communicated requests to BSERVER and relayed responses back to applications.

The BROUTER facilitated routing requests to database replicas on different servers by coordinating communication among multiple server processes using File Server Tables that detailed server names, addresses, and routing data. It also handled access control mechanisms such as locking through SPX protocol communications and the BSPXCOM module.

Btrieve for DOS employed SEFS and MEFS sharing modes and supported both exclusive and concurrent transactions over network connections.

Btrieve for Windows

Before transitioning to the MKDE codebase rewrite, Btrieve for Windows featured SEFS and MEFS file sharing capabilities alongside shadow-paging, supporting both exclusive and concurrent locking mechanisms. It distinguished between version 6.x and 6.1 files: version 6.x enabled partial record operations without locking entire records, accommodated records exceeding 64KB in size, implemented VATs and ACSs, introduced new data types, supported percentage operations (manipulating records based on their physical file location), and managed duplicate keys. Additionally, version 6.x allowed dynamic addition or removal of any index type (previous versions only permitted dropping supplemental indexes). Version 6.1 brought enhancements such as concurrent and system transactions, optional key renumbering, case-insensitive ACS tables, and improved locking functionality.

Btrieve for Windows could function either as a client connecting to databases via SEFS or MEFS modes or by directly accessing the Btrieve server itself.

Client-side Btrieve

In the client-side Btrieve setup, database files were stored either locally or on network drives mapped through DOS NET USE. When applications initiated operations, they called WBTRCALL.DLL, which loaded the client engine after checking the BTI.INI configuration file. This process triggered the loading of the local interface (WBTRLOCL.DLL), which could then load the Btrieve engine (WBTR32.EXE) into memory to handle database requests. The engine relied on Win32 system libraries to perform file operations on the database files.

Client-side Btrieve connecting to server-based Btrieve

The client version was also capable of accessing server-based Btrieve through a DOS requester that required DOS Protected Mode Interface (DPMI) to utilize extended memory in Protected Mode.

Applications invoked WBTRCALL.DLL, which referenced BTI.INI to determine whether data access was local or remote. For remote connections, it employed Windows DPMI to execute a DOS requester (BREQUEST.EXE) that established a network link with the server. The server then processed the requests and sent completion notifications back to the requester.

Btrieve for Windows NT/Windows 95

Launched in 1995 alongside editions for Netware and Windows NT Server, Btrieve for Windows NT/95 advanced to version 6.15 and incorporated the MKDE. It continued supporting SEFS and MEFS file sharing modes, shadow-paging, as well as exclusive and shared locking mechanisms. This release added support for null key values, permitting records without key data to be stored without indexing, thereby minimizing unnecessary searches. Additionally, it introduced system and user transactions (refer to System and user transactions). Version 6.15 also featured variable-tail allocation tables, which were included in the Windows NT/95 build.

Btrieve for Windows NT/95 was offered in two configurations: standalone workstation and client/server.

Standalone workstation

Under the standalone workstation configuration, all record processing took place locally. The MKDE (W32MKDE.EXE) accessed database files directly through Windows file system APIs, utilizing lock files to coordinate concurrent access.

Applications interfaced with either the Btrieve API or Microkernel Interface (WBTRV32.DLL), which forwarded requests to the MKDE that accessed files on local or network drives.

This design could encounter problems during network interruptions; if a connection failed during an update, related data fields might become inconsistent, potentially leading to database corruption despite safeguards such as pre-image paging.

Client/Server

In the client/server arrangement, record processing occurred on a Windows file server accessed via mapped network drives (established using NET USE). Access rights were governed by user authentication or mapping credentials.

On Windows 95 systems, the MKDE interface (WBTRV32.DLL) determined whether to operate in workstation or server mode based on configuration files. On Windows NT, it checked the registry when both server (NTMKDE.EXE) and workstation (W32MKDE.EXE) processes ran simultaneously. In workstation mode, files were accessed directly; in server mode, communication occurred through client DLLs (W32BTICM.DLL or NTBTICM.DLL) paired with corresponding server DLLs on the mapped drive. The server DLL then interacted with the server MKDE (NTMKDE.EXE) to perform record updates and confirm completion.

This client/server framework enhanced stability by allowing the server MKDE to better detect and recover from network failures compared to the standalone workstation model.

Configuration

A configuration tool was supplied alongside Btrieve to modify MKDE parameters, encompassing:

  • File parameters: overseeing files, handles, record locks, indexes, and log files. This involved setting limits on open files, logical handles, record locks per client, index balancing, and file format configurations. It managed operation logging for selected files and enabled choosing file sharing methods (MEFS or SEFS). The system transaction hold limit defined the maximum number of transactions during write operations on shared files.
  • Memory allocation: defining buffer sizes assigned by the Microkernel.
  • Client/System transactions: options controlling transaction support and logging behavior.
  • System resources/directories: settings for supported clients and threads along with system file path locations.
  • Trace operations: advanced tracing capabilities primarily intended for debugging Btrieve functions.

Pervasive.SQL 7

Launched in March 1998, Pervasive.SQL 7 combined Scalable SQL 4 with Btrieve 7.0 and was compatible with Windows 95, Windows NT 3.51 & 4, NetWare, and DOS. To resolve upgrade compatibility challenges, the company introduced a component-based framework named SmartComponents. This approach embedded component identifiers within filenames and utilized dynamic binding of "glue files" (DLLs loaded only when required) through an "Abstract OS Services DLL," which located the most current suitable component version by interpreting encoded filenames.

The earlier Btrieve 6.x log format was replaced by a centralized PVSW.LOG featuring a standardized, enhanced structure along with improved error reporting.

While the MKDE persisted in Pervasive.SQL 7, it was adapted to align with the new dynamic component design. Applications interact with a services manager that scans configured directories for encoded filenames matching this pattern: <syntaxhighlight lang="bnf"> 

<filename> ::= <platform-code> "BIF" <major-functional-level> <minor-functional-level>
<platform-code> ::= "W1" | "W2" | "W3" | "W9" | "WT" | "NW" | "O3"
<major-functional-level> ::= <number>
<minor-functional-level> ::= <number> <number>
<number> ::= "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"

</syntaxhighlight>

Platform codes embedded in filenames
Code Platform
W1 Windows 3.1x, including Windows for Workgroups (Win16)
W2 Extended Windows (32-bit Watcom Extender)
W3 Windows 95, Windows NT (Win32)
W9 Windows 95
WT Windows NT
NW NetWare versions 3.x and 4.x
O3 OS/2 (32-bit)

The dynamically loaded DLL ("glue module") connects with the MKDE, which decides whether to function in workstation or server mode—either routing requests to the database server or directly accessing database files in workstation mode.

Pervasive.SQL 2000/2000i

Pervasive.SQL 2000 and its iteration 2000i retained the architectural foundation of Pervasive.SQL 7; the latter introduced the i*Net server. Both versions continued employing the same component framework supporting either Btrieve or Scalable SQL engines alongside the MKDE. This release broadened platform compatibility to include Red Hat Linux, Caldera OpenLinux, SUSE, and Solaris. Integration with Terminal Services was enhanced; however, only a single database engine instance could operate per terminal server, limiting concurrent session support.

Pervasive.SQL V8

Introduced in December 2002, Pervasive.SQL V8 advanced the performance of Btrieve and SQL applications by incorporating several new technologies:

  • Client-side caching enhances read operations by locally storing portions of the database.
  • Turbo Write Acceleration (TWA) minimizes disk interactions by batching disk write operations.
  • Transaction Logging improves performance while providing moderate protection against failures.

The mid-cycle Security Feature Pack (version 8.5) brought database-level security controls, eliminating dependence on operating system file permissions. This enabled administrators to manage access to Btrieve data without granting file system rights. Additionally, client/server configurations no longer required network shares or mapped drives, allowing secure Btrieve data connections through URI strings.

Pervasive PSQL v9

PSQL version 9 introduced Java-based graphical user interfaces built on Eclipse for both Windows and Linux platforms, improving user experience. It delivered notable enhancements in SQL performance and syntax across ADO.Net, JDBC, ODBC, and OLE DB drivers. The maximum size for Btrieve files expanded from 64GB in the 8.x series to 128GB with the 9.0 file format and further to 256GB in the 9.5 format.

This release also reintroduced the DDF Builder tool, enabling users to define metadata for existing Btrieve files and access data via SQL interfaces. Although Full Text Search (FTS) functionality was added during this cycle, it was later removed from subsequent offerings.

All MKDE variants preserved backward read compatibility with earlier Btrieve file formats without modifying them unless an upgrade was explicitly requested. Files formatted as 5.x or older required rebuilding to version 6.x or later to support write operations from PSQL 9.0 or newer engines.

Pervasive PSQL v10

Launched in September 2007, Pervasive PSQL v10 marked the first edition compatible with 64-bit operating systems, updating the Btrieve API and Distributed Tuning Interface (DTI) accordingly. Despite this, components such as Pervasive PSQL Workgroup and SDKs remained 32-bit. This version targeted compatibility with Windows Vista and the forthcoming Windows Server 2008; Server, Workgroup, and Client editions supported Vista, with the Server edition certified for Windows Server 2008.

Version 10 introduced Xtreme input/output (XIO), a 32-bit database accelerator that extended cached memory usage beyond the traditional 2GB limit on 32-bit Windows systems. XIO enhanced I/O throughput through techniques like update compression and optimized write processes.

Digital license enforcement, termed Product Authorization, was first implemented in v10 for trial downloads and online sales. By Service Pack 3 (November 2009), this system expanded to include products distributed via resellers and ISV partners. SP3 also added compatibility with Windows 7.

Backward compatibility was maintained using the 9.5 file format standard; however, maximum data file size increased to 256GB with page sizes up to 16,384 bytes. Support was discontinued for legacy platforms including NetWare, Windows NT, Windows 98, Windows ME, DOS 6.22, and 16-bit applications in v10—though Pervasive continued offering support for NetWare through version 9.

In 2010, version 11 was released to harness multithreading capabilities for accelerated database processing.

Pervasive PSQL v11

Launched in September 2010, Pervasive PSQL version 11 introduced a redesigned database engine aimed at boosting performance and scalability on multi-core processors. It enhances parallel thread execution, enabling concurrent task processing across multiple cores. Key improvements include refined low-level synchronization that allows several users to access cached pages simultaneously, while non-user operations such as checkpoints run independently on separate cores. Additionally, multi-core processing improves simultaneous file access by multiple users.

Support for multi-core architectures is integrated across all v11 editions, including 32- and 64-bit Windows and Linux Server versions as well as the 32-bit Workgroup edition. Internal testing demonstrated performance increases of up to 300% compared to version 10 on an 8-core Microsoft Server 2008 Enterprise (64-bit).

Windows versions incorporated IPv6 support while maintaining IPv4 compatibility for both Btrieve and DTI access protocols.

The 64-bit server editions come equipped with fully 64-bit relational/SQL engines, navigational/Btrieve engines, and ODBC drivers bundled within both server and client installations.

The software development kit was enhanced with the introduction of the Pervasive PSQL ADO.NET Data Provider 3.5, compatible with .NET Framework versions through 4.0 and including support for Entity Framework 1.0. Moreover, PDAC access methods received updates to support Embarcadero's RAD Studio versions 2009 and 2010.

Product Authorization capabilities were extended to OEM partners via a web-based license management portal. Offline activation became possible for all users through the addition of telephone authorization.

While retaining the file format from version 9.5 to ensure backward compatibility, Pervasive PSQL v11 discontinued support for Windows 2000 systems.

Pervasive PSQL Ecosystem

Pervasive provides a suite of supplementary products designed to augment the core PSQL database management system. The latest releases—AuditMaster version 7, Backup Agent version 3, and DataExchange version 4—were made available in December 2010.

  • Pervasive AuditMaster offers real-time monitoring of all database activities (both Btrieve and SQL), facilitating the tracking of sensitive data modifications and enabling alerts or automated responses.
  • Pervasive Backup Agent operates in continuous mode, permitting backup software to perform reliable copies of live databases without interrupting service.
  • Pervasive DataExchange supports synchronization and replication across multiple PSQL instances to guarantee critical data remains accessible.

Btrieve 12

Released in February 2016 by Actian, Btrieve 12 supports Windows Vista through Windows 10 as well as Windows Server editions from 2008 and 2012. It preserves file format and API compatibility with Btrieve version 6.15, ensuring seamless read/write operations on legacy files originating from earlier DOS and 16-bit applications.

See also

Sources

External links

Retrieved from "https://www.alfawiki.org/index.php?title=Btrieve&oldid=25515"