Apache Mahout
| Company type | Open source software project |
|---|---|
| Industry | Machine learning, data mining, big data |
| Founded | 2009-04-07 |
| Products | Scalable machine learning algorithms and libraries |
| Parent | Apache Software Foundation |
| Website | mahout |
Apache Mahout is an initiative under the Apache Software Foundation that provides free, scalable machine learning algorithms with a strong emphasis on linear algebra. Initially built on the Apache Hadoop framework, Mahout has transitioned to prioritize Apache Spark for distributed computation. It supplies Java and Scala libraries designed for mathematical computations and primitive data structures, facilitating scalable analytics and data processing. The project is continuously advancing by integrating various algorithms tailored to meet enterprise-level machine learning requirements.
Samsara
Mahout-Samsara introduces a Scala-based domain-specific language (DSL) that allows developers to articulate algorithms succinctly using syntax reminiscent of R, thereby improving code readability and developer efficiency.
<syntaxhighlight lang="scala"> val G = B %*% B.t - C - C.t + (ksi dot ksi) * (s_q cross s_q) </syntaxhighlight>
Backend agnostic
The design of Mahout separates its domain-specific language from the execution layer, enabling compatibility with multiple computational backends. Although current development predominantly targets the Apache Spark engine, the platform remains flexible for integration with other engines; historical implementations have included and Apache Flink, with illustrative examples accessible within the source code repository.
GPU/CPU accelerators
To overcome performance limitations inherent in JVM computations, "native solvers" have been introduced. These solvers relocate BLAS operations from the JVM heap to off-heap or GPU memory, utilizing multiple CPUs or GPUs through the ViennaCL library. ViennaCL is a highly optimized C++ library implementing BLAS functionality via OpenMP and OpenCL. As of version 14.1, the OpenMP implementation is considered stable, whereas the OpenCL variant is still in an experimental phase.
Recommenders
Mahout offers implementations of recommendation algorithms such as Alternating Least Squares, Co-Occurrence, and Correlated Co-Occurrence—the latter being a distinctive recommender that extends co-occurrence analysis across several data dimensions.
Transition from MapReduce to Apache Spark
In its early stages, Mahout’s primary algorithms for clustering, classification, and batch collaborative filtering were executed on Apache Hadoop using MapReduce. Nonetheless, support was also available for single-node or non-Hadoop cluster environments; for instance, the 'Taste' collaborative-filtering module originated as an independent project not reliant on Hadoop.
From release 0.10.0 onwards, Mahout adopted a backend-independent programming model named "Samsara," featuring an algebraic optimizer decoupled from specific backends and a Scala DSL that harmonizes both in-memory and distributed algebraic operations. Supported execution platforms include Apache Spark, , and Apache Flink. The deprecation of MapReduce algorithm support began in 2014.
Release history
| Version | Release Date | Notes |
|---|---|---|
| 0.1 | 2009-04-07 | |
| 0.2 | 2009-11-18 | |
| 0.3 | 2010-03-17 | |
| 0.4 | 2010-10-31 | |
| 0.5 | 2011-05-27 | |
| 0.6 | 2012-02-06 | |
| 0.7 | 2012-05-16 | |
| 0.8 | 2013-07-25 | |
| 0.9 | 2014-02-01 | |
| 0.10.0 | 2015-04-11 | Introduced Samsara DSL |
| 0.10.1 | 2015-05-31 | |
| 0.10.2 | 2015-08-06 | |
| 0.11.0 | 2015-08-07 | |
| 0.11.1 | 2015-11-06 | |
| 0.11.2 | 2016-03-11 | |
| 0.12.0 | 2016-04-11 | Added Apache Flink engine |
| 0.12.1 | 2016-05-19 | |
| 0.12.2 | 2016-06-13 | |
| 0.13.0 | 2017-04-17 | |
| 0.14.0 | 2019-03-07 | Source only release (no binaries) |
| 14.1 | 2020-10-07 |
Developers
The development of Apache Mahout is overseen by a community guided by the Project Management Committee (PMC). The current members of the PMC are Andrew Musselman, Andrew Palumbo, Drew Farris, Isabel Drost-Fromm, Jake Mannix, Pat Ferrel, Paritosh Ranjan, Trevor Grant, Robin Anil, Sebastian Schelter, and Stevo Slavić.