Kepler scientific workflow system

Kepler

Company type	Open-source software project
Industry	Scientific workflow management, Bioinformatics software
Founded	2002
Headquarters	Santa Barbara, California , United States
Products	Kepler scientific workflow platform
Website	kepler-project.org

Kepler is an open-source platform designed for creating, running, reusing, evolving, archiving, and distributing scientific workflows. It provides capabilities such as process and data monitoring, tracking provenance, and rapid data transfer. Scientific workflows are represented as directed graphs where computational units are nodes and the connections between them depict data movement.

Within Kepler, these nodes are referred to as 'Actors' and the connections as 'channels'. The system includes a graphical user interface that facilitates workflow construction on desktop environments, alongside a runtime engine capable of executing workflows either through the GUI or command-line interface. It supports distributed execution across clusters or grid infrastructures. Kepler is tailored for scientific modeling and analysis by structuring computational operations into workflows that simulate data progression through sequential processing steps to fulfill scientific goals.

Scientific workflow

A scientific workflow integrates both data and processing steps into an organized, configurable sequence designed to automate computational approaches addressing scientific challenges. Scientific workflow systems typically incorporate graphical interfaces that streamline the integration of various technologies, thereby boosting researchers’ efficiency.

Access to scientific data

Kepler enables direct retrieval of scientific datasets stored in popular repositories. For example, it connects to resources housed in the Knowledge Network for Biocomplexity (KNB) Metacat server, which uses Ecological Metadata Language for metadata description. Additional supported data sources include those accessible via DiGIR, OPeNDAP, GridFTP, JDBC, SRB, among others.

Models of Computation

What sets Kepler apart from other bioinformatics workflow management systems is its clear separation between the workflow’s structural design and its computational semantics, permitting multiple computation models to be applied to a single workflow graph. It incorporates several models inherited from the Ptolemy system, such as Synchronous Data Flow (SDF), Continuous Time (CT), Process Network (PN), and Dynamic Data Flow (DDF).

Hierarchical workflows

The platform supports hierarchical workflows that allow complex processes to be constructed from smaller, reusable components. This modular design empowers users to store and reuse components across different workflows efficiently.

Workflow semantics

Kepler enables semantic annotation of workflow elements by employing ontology-based terms. These annotations facilitate advanced functionalities including improved search capabilities, automated validation processes, and enhanced editing support within workflows.

Sharing workflows

Workflows and their components created in Kepler can be shared by exporting them as Kepler Archive (KAR) files—an extension of the Java JAR file format. KAR files may be distributed via email, hosted on websites, or uploaded to the Kepler Component Repository—a centralized resource accessible through web portals and web service APIs. Users can search for components and integrate them directly within Kepler’s workflow composition interface.

Provenance

Provenance plays a crucial role in scientific workflows by allowing users to trace the origins of results, reproduce experiments, and verify data processing steps. Capturing provenance involves logging details about data origins, transformations applied, components used, and parameter configurations, which collectively support reproducibility and validation of outcomes. Nonetheless, current tools provide limited querying capabilities for provenance in complex workflows beyond straightforward models like DAGs.

Kepler history

Initiated in 2002 by scientists involved with the Science Environment for Ecological Knowledge (SEEK) and Scientific Data Management (SDM) initiatives, the Kepler Project was established at the National Center for Ecological Analysis and Synthesis (NCEAS) at UCSB along with the San Diego Supercomputer Center at UC San Diego. Originating from Ptolemy II—a system developed at UC Berkeley for modeling concurrent and real-time processes—Kepler transformed into a workflow management platform that supports scientific investigations through modular units known as actors. Its development has been a collaborative effort across multiple disciplines, including ecology, molecular biology, physics, chemistry, oceanography, hydrology, library science, and computer science.

See also

• Apache Taverna
• Discovery Net
• VisTrails
• LONI Pipeline
• Bioinformatics workflow management systems
• DataONE Investigator Toolkit

External links

• Kepler Project website
• Kepler Component Repository
• Ptolemy II project website
• Knowledge Network for Biocomplexity (KNB) Data archive
• List of software tools related to workflows on the DataONE website