Apache Impala
| Company type | Open source software |
|---|---|
| Industry | Software, Big data |
| Founded | 2012 |
| Headquarters | N/A |
| Products | Massively parallel processing SQL query engine |
| Website | impala |
Apache Impala is an open source massively parallel processing (MPP) SQL query engine designed to operate on data stored within a computer cluster that runs Apache Hadoop. Its development drew inspiration from Google's F1 system and it was introduced in 2012.
Description
Operating atop Apache Hadoop, Apache Impala facilitates scalable and low-latency SQL querying on data housed in HDFS and Apache HBase without requiring any data relocation or conversion. It seamlessly integrates with Hadoop’s native file formats, metadata management, security protocols, and resource scheduling frameworks utilized by MapReduce, Apache Hive, and other components of the Hadoop ecosystem.
Impala is aimed at analysts and data scientists who need to perform SQL-based analytics or use business intelligence platforms on Hadoop datasets. This approach supports both interactive queries and large-scale batch processing within a unified environment, leveraging shared data and metadata to avoid reliance on specialized infrastructures or proprietary data formats.
Prominent capabilities include:
- Compatibility with storage systems such as HDFS, S3, Microsoft Azure Blob Storage, Apache HBase, and Apache Kudu
- Support for various Hadoop file formats including text, LZO, SequenceFile, Avro, RCFile, Parquet, and ORC
- Integration with Hadoop security mechanisms like Kerberos authentication and Ldap
- Detailed role-based access control through Apache Ranger
- Utilization of metadata structures, SQL syntax, and the ODBC driver originally developed for Apache Hive
In 2013, the introduction of the columnar storage format Apache Parquet enhanced Impala’s query efficiency. That same year, Amazon Web Services incorporated support for Impala within Elastic MapReduce. By 2014, MapR included Impala within its distribution. In 2015, Cloudera unveiled the columnar storage solution Apache Kudu alongside Impala, proposing both projects for donation to the Apache Software Foundation. Impala achieved the status of an Apache Top-Level Project in November 2017.
See also
- Apache Drill — an open source initiative influenced by Dremel
- Dremel — Google’s analogous technology
- Trino — open source SQL engine developed by creators of Presto
- Presto — Facebook-originated open source SQL engine supported by Teradata
External links
- Apache Impala official website
- Impala GitHub source repository