Yahoo Sherpa
| Company type | Product |
|---|---|
| Industry | Cloud storage, NoSQL database |
| Founder | Yahoo! |
| Products | Hosted distributed key-value data store with geographic replication |
| Owner | Yahoo! |
Sherpa is a cloud storage solution created by Yahoo!. It functions as a hosted, distributed key-value data store with geographic replication. This NoSQL platform is engineered to satisfy the demands for scalability, availability, and low latency required by Yahoo!'s web services. Sherpa delivers features such as elastic scalability, multi-tenancy support, a worldwide presence for reduced local latency, asynchronous data replication, RESTful APIs, consistency controls at the record level, high availability, data compression, secondary indexing, and replication on a per-record basis.
Architecture
Sherpa is designed as a multi-tenant environment where applications save data within tables segmented into smaller units known as tablets. These tablets are partitioned either by key hashing or range and reside on nodes called storage units. A routing component oversees the association between application tablets and storage units, directing client requests appropriately. Client operations include get, set, delete, and scan actions using unique primary keys.
Data model
Within Sherpa’s key-value architecture, data is stored as JSON blobs inside tables that enforce unique primary keys without requiring fixed schemas. It supports scanning entire tables with filter predicates and provides various table types—such as distributed hash tables, distributed ordered tables, mastered tables, and merging tables—chosen based on the application's access patterns and query needs.
Scalability
Sherpa achieves scalability by dividing each user-defined table into multiple tablets that serve as work and tenancy units. Each tablet holds a specific range of records, allowing the system to efficiently manage very large quantities of tables, tablets, and records.
Elasticity
The platform supports horizontal scaling by incorporating additional machines without causing application downtime. Elasticity operations encompass assigning, reassigning, and splitting data partitions.
Fault-tolerance
To ensure resilience against failures, data is automatically replicated across several nodes and data centers. Failures of individual nodes remain invisible to applications. Sherpa employs a dependable transactional message bus that guarantees at-least-once delivery of replication messages.
Tunable consistency
Sherpa provides adjustable consistency settings ranging from per-record timeline consistency with serialized writes directed to a master copy to eventual consistency modes.
Selective record replication
Replication scope can be controlled at both the record and table levels.
Backup
The backup capability allows storing multiple historical versions of entire tables in offline storage for later retrieval of previous individual record states.
Secondary indexes
Sherpa supports asynchronous secondary indexes to enable querying based on non-primary key attributes.