DTrace
| Company type | Software framework |
|---|---|
| Industry | Computer software |
| Founded | November 2003 |
| Founder | Bryan Cantrill, Mike Shapiro, Adam Leventhal |
| Successor | illumos (fork incorporating DTrace) |
Area served | Unix-like operating systems, Windows Server 2025 |
Key people | Bryan Cantrill, Mike Shapiro, Adam Leventhal |
| Products | Dynamic tracing system for kernel and application diagnostics |
| Owner | Originally Sun Microsystems; now Oracle Corporation and open-source communities |
| Parent | Sun Microsystems (original), Oracle Corporation (current) |
| Website | dtrace |
DTrace is an advanced dynamic tracing system initially created by Sun Microsystems to facilitate real-time diagnosis of kernel and application problems within live production settings. Originally developed for Solaris, it was made available under the free Common Development and Distribution License (CDDL) through OpenSolaris and its derivative illumos. Since then, DTrace has been adapted for numerous Unix-like platforms. Notably, Windows Server 2025 will incorporate DTrace as part of its native system utilities.
This framework delivers an extensive analysis of system performance metrics, covering areas such as CPU, memory, filesystem, and network activity generated by running processes. It also enables granular examination, including the inspection of function call parameters and tracking file access on a per-process basis.
After Oracle's 2010 acquisition of Sun Microsystems and the subsequent termination of OpenSolaris development, former Solaris developers initiated the illumos project—an open-source fork that integrates DTrace technology.
In 2011, Oracle announced plans to port DTrace to Linux, with official repositories hosted on GitHub since 2019. Prior to this, unofficial Linux adaptations existed without altering the original licensing. By 2017, Oracle had re-licensed the DTrace kernel components under the GPLv2+ license and user-space utilities under both GPLv2 and the UPL. Microsoft contributed by porting DTrace from FreeBSD to Windows in 2018.
The OpenDTrace initiative commenced in 2016 on github, preserving the original CDDL license while incorporating additional code under the BSD 2-Clause license. Its mission is to develop a portable, OS-independent DTrace implementation compatible with macOS, FreeBSD, OpenBSD, NetBSD, Linux, and embedded platforms.
Description
Sun Microsystems engineered DTrace to offer deep operational visibility that empowers users to optimize and debug both applications and operating systems efficiently.
Users create tracing programs—scripts—using the D programming language (distinct from other languages named "D"). This language draws inspiration from C, incorporating specialized functions and variables tailored for tracing purposes. The structure of D scripts resembles that of AWK scripts, consisting of one or more probes (instrumentation points), each associated with specific actions. These probes act similarly to pointcuts found in aspect-oriented programming. When probe conditions are satisfied—such as when a file is opened, a process initiates, or a particular line of code executes—the corresponding action triggers. Probes can inspect runtime information by accessing the call stack and contextual variables, then log data or adjust variables to establish event correlations.
DTrace emphasizes production safety by minimizing the probe effect during monitoring and ensuring disabled probes incur no performance penalties. This capability is vital given the availability of tens of thousands of probes. Additionally, users can dynamically introduce new probes as needed.
Command line examples
DTrace scripts may be executed directly from the command line by specifying probes alongside their associated actions. Sample commands include:
<syntaxhighlight lang="bash">
- Trace new processes along with their arguments
dtrace -n 'proc:::exec-success { trace(curpsinfo->pr_psargs); }'
- Monitor files opened by processes
dtrace -n 'syscall::open*:entry { printf("%s %s",execname,copyinstr(arg0)); }'
- Count syscalls grouped by program name
dtrace -n 'syscall:::entry { @num[execname] = count(); }'
- Count syscalls grouped by syscall name
dtrace -n 'syscall:::entry { @num[probefunc] = count(); }'
- Count syscalls grouped by process ID and program name
dtrace -n 'syscall:::entry { @num[pid,execname] = count(); }'
- Report disk I/O size per process
dtrace -n 'io:::start { printf("%d %s %d",pid,execname,args[0]->b_bcount); }'
- Aggregate pages paged in per process
dtrace -n 'vminfo:::pgpgin { @pg[execname] = sum(arg0); }' </syntaxhighlight>
Although some scripts extend to hundreds of lines, most advanced troubleshooting requires only a few dozen lines. The DTraceToolkit offers over 200 open-source scripts authored by Brendan Gregg (the author of the definitive DTrace book), providing comprehensive documentation and practical examples for both technical specialists and business professionals.
Supported platforms
Initially launched in November 2003, DTrace became an official part of Sun's Solaris 10 in January 2005. It was the inaugural component of the OpenSolaris initiative to be released as open-source under the Common Development and Distribution License (CDDL).
DTrace plays a fundamental role in illumos and its derivative distributions and is included by default in FreeBSD and NetBSD.
Apple incorporated DTrace support starting with Mac OS X 10.5 "Leopard," which featured a graphical interface known as Instruments. The system includes over 40 scripts from the DTraceToolkit, providing utilities for analyzing disk I/O and process execution. Mac OS X implements a flag (P_LNOATTACH) that can limit tracing on specific processes, initially restricting some probe capabilities; this limitation was resolved by OS X 10.5.3. Nonetheless, since the release of El Capitan, System Integrity Protection has imposed default restrictions on DTrace’s ability to access protected binaries.
The Linux version of DTrace has been available since 2008, with active development hosted on GitHub. It supports core standard providers such as fbt, syscall, and profile, along with a unique "instr" provider. However, several Solaris-specific providers remain unsupported as of 2013. The Linux implementation functions as a loadable kernel module, thereby avoiding direct kernel source alterations and circumventing licensing conflicts between CDDL and GPL; however, loading this module marks the kernel as tainted.
In 2007, QNX Software Systems began integrating DTrace into the QNX operating system.
Oracle Corporation introduced beta-stage DTrace support for Oracle Linux in 2011 as a technology preview within the GPLv2-licensed Unbreakable Enterprise Kernel release 2. Full general availability was announced in December 2012.
On March 11, 2019, Microsoft made a version of DTrace available for Windows 10 insider builds and subsequently incorporated it as a native diagnostic tool in Windows Server 2025.
Language and application providers
When paired with a compatible language provider, DTrace can extract detailed code context such as function names, source files, and line numbers. It can also interact with dynamic memory management and garbage collection features when supported by the language. Languages supported include assembly language, C, C++, Java, Erlang, JavaScript, Perl, PHP, Python, Ruby, shell script, and Tcl.
Application providers allow DTrace to monitor application-level operations by tracing through system calls into kernel space. Prominent applications equipped with DTrace providers include MySQL, PostgreSQL, Oracle Database, Oracle Grid Engine, and Firefox.
Authors and awards
DTrace was developed by Bryan Cantrill, Mike Shapiro, and Adam Leventhal.
The creators earned accolades in 2005 from InfoWorld and Technology Review for their pioneering work. In 2006, DTrace secured the highest honor at The Wall Street Journal's Technology Innovation Awards. Additionally, the authors received the Software Tools User Group (STUG) award from USENIX in 2008.
See also
- eBPF – A Linux kernel tracing mechanism providing capabilities akin to DTrace, available since kernel version 4.9
- ftrace – A Linux kernel tracing framework designed for monitoring scheduling events, interrupts, memory-mapped I/O, and CPU power management
- ktrace – A utility in BSD Unix and macOS for tracing interactions between the kernel and programs
- ltrace – A Linux debugging tool that displays calls from user applications to shared libraries
- strace – A Linux command-line utility that tracks system calls and signals utilized by processes
- SystemTap – A scripting language and toolset for dynamic instrumentation of Linux systems
- LTTng
- IBM ProbeVue
References
External links
- DTrace Tools – Brendan Gregg’s collection of DTrace examples (2004)
- DTraceToolkit – A repository of DTrace scripts
- DTrace book scripts – GitHub repository containing scripts from the DTrace book
- FreeBSD DTrace page – Includes tutorials and concise command examples
- DTrace book
- DTrace guide – Documentation for Illumos DTrace
- Dynamic Tracing with DTrace & SystemTap – Complimentary book featuring practical examples and exercises
- DTrace Hands On Lab – Guided course for mastering DTrace step-by-step
- DLight Tutorial – Interactive graphical tool for C/C++ developers based on DTrace technology; included with Oracle Solaris Studio prior to version 12.4
- Exploring Leopard with DTrace – Tool for debugging and system exploration
- Google Tech Talk on DTrace by Bryan Cantrill
- Hidden in Plain Sight, Sun Microsystems, authored by Bryan Cantrill
- Official Oracle Linux port of DTrace: