Jump to content

Ceph (software)

From Alfawiki
Ceph
Company typeOpen-source software
IndustryComputer storage
Founded2004
FounderSage Weil
Area served
Worldwide
ProductsObject storage, block storage, file storage
Websiteceph.io

Ceph (pronounced ) is an open-source, software-defined storage system that provides object, block, and file storage capabilities based on a distributed cluster architecture. It operates without a single point of failure and can scale up to exabyte capacities. Starting from version 12 (Luminous), Ceph directly manages both HDDs and SSDs through its BlueStore backend and includes support for a POSIX-compliant filesystem.

Utilizing commodity hardware and Ethernet IP networks, Ceph achieves fault-tolerant data replication without the need for specialized equipment. It ensures high availability and robust data integrity by employing replication, erasure coding, snapshots, and cloning techniques. Engineered to be self-managing and self-healing, Ceph reduces the need for manual administration while lowering operational expenses.

Prominent enterprise users such as CERN, OVH, and DigitalOcean showcase Ceph’s ability to deliver scalable and dependable storage solutions in extensive production settings.

Design

Ceph operates with five distinct daemon types:[1]

  • Cluster monitors () oversee node status (active or failed), manage cluster configuration, determine data distribution, and maintain overall cluster state.
  • OSDs () handle large-scale data storage using the BlueStore backend,[2] which replaced the older Filestore backend starting with version 12.x.
  • Metadata servers () control access to inodes and directories within CephFS by managing metadata operations.
  • HTTP gateways () offer object storage interfaces compatible with Amazon S3 and OpenStack Swift APIs.
  • Managers () are responsible for cluster monitoring, upkeep tasks, and integration with external monitoring tools such as Prometheus and Zabbix.Cite error: The opening <ref> tag is malformed or has a bad name

These daemons are fully decentralized and may be deployed on separate servers or combined within a converged architecture. Clients communicate directly with the appropriate cluster components.[3]

To improve throughput, Ceph distributes data across numerous devices and nodes in a manner akin to RAID. It supports dynamic load balancing by replicating heavily accessed services across multiple nodes.[4]

Since 2017, BlueStore has been the default production backend,[5] offering superior latency performance and configuration flexibility compared to the filesystem-based Filestore, which incurs additional processing overhead. Filestore is scheduled for deprecation with the Reef release anticipated in mid-2023. For Filestore OSDs, XFS was the preferred filesystem; although Btrfs was available, it exhibited less stability, whereas ext4 was discouraged due to limited metadata handling.Cite error: The opening <ref> tag is malformed or has a bad name BlueStore continues to utilize XFS for a minor metadata partition.Cite error: The opening <ref> tag is malformed or has a bad name

Object storage S3

Ceph provides distributed object storage via the RADOS Gateway (), which presents an interface compatible with both Amazon S3 and OpenStack Swift APIs.

The Ceph RGW scales effectively and is often employed in high-density storage environments supporting large-scale applications such as Big Data lakes, backup systems, archives, IoT, media streaming platforms, video capture solutions, as well as deployment images for virtual machines and containers.

Its software libraries enable client applications to interact directly with the reliable autonomic distributed object store (RADOS). Core libraries facilitate access to Ceph's RADOS Block Device (RBD), RADOS Gateway, and Ceph File System, providing unified management alongside replication and data protection capabilities.

The "librados" libraries support multiple programming languages including C, C++, Java, PHP, and Python. The RADOS Gateway exposes a RESTful interface fully compatible with native Amazon S3 and OpenStack Swift protocols.

Block storage

Ceph offers thin-provisioned block devices that automatically distribute and replicate data throughout the cluster. Its RADOS Block Device (RBD) integrates seamlessly with Kernel-based Virtual Machines (KVM) and supports configurations using either HDDs, SSDs, or a combination of both.

Designed to handle workloads demanding high throughput and IOPS—such as databases, virtual machines, analytics, AI, and machine learning—Ceph RBD increasingly leverages SSDs equipped with NVMe interfaces to boost performance.

Built atop the core RADOS infrastructure, RBD benefits from advanced capabilities including clones and snapshots. Distributing volumes across the cluster enhances performance for sizable block device images through striping.

Access to distributed, highly available block storage from Microsoft Windows and VMware vSphere clients is facilitated by "Ceph-iSCSI" gateways via the iSCSI protocol. Ceph RBD images are exposed as Logical Units (LUs) through load-balanced, fault-tolerant iSCSI targets.

Because configuration data resides within the Ceph RADOS object store, ceph-iscsi gateways operate statelessly, allowing for flexible scaling or replacement. This architecture underpins a truly distributed, resilient, and self-healing enterprise storage solution built on commodity hardware using open source technologies.

Ceph block devices can be virtualized to provide storage services for virtual machines across multiple platforms such as OpenShift, OpenStack, Kubernetes, OpenNebula, Ganeti, Apache CloudStack, and Proxmox Virtual Environment.


File storage

CephFS, Ceph’s file system, is constructed on the same RADOS base that supports its object storage and block device functionalities. The CephFS metadata server (MDS) translates directories and filenames into objects within the RADOS cluster. This metadata server cluster scales dynamically and redistributes metadata to balance load evenly across nodes, ensuring optimal performance and avoiding bottlenecks.

Clients access the POSIX-compliant file system primarily through a Linux kernel client; an older FUSE-based client remains available. Servers operate as conventional Unix daemons.

Ceph’s file storage solutions are frequently employed for use cases such as log aggregation, messaging systems, and general-purpose file storage.


Dashboard

Since 2018, Ceph has provided a Dashboard web interface aimed at streamlining cluster administration. Developed collaboratively by the Ceph community under LGPL-3 licensing, it utilizes ceph-mgr along with technologies like Python, Angular, and Grafana. The Dashboard’s main page received a refresh in early 2023.

Earlier dashboard projects—including Calamari (2013–2018), OpenAttic (2013–2019), VSM (2014–2016), Inkscope (2015–2016), and Ceph-Dash (2015–2017)—have been phased out.


Crimson

Initiated in 2019, the Crimson project reengineers the OSD data path to minimize latency and reduce CPU consumption. It addresses the disparity between rapidly advancing storage media such as NVMe and 3D XPoint and comparatively slower CPU speeds. Crimson aims to serve as a backward-compatible drop-in replacement for existing OSD implementations.

While maintaining compatibility with BlueStore through AlienStore, Crimson is concurrently developing a native ObjectStore named SeaStore alongside CyanStore for experimental purposes. SeaStore replaces BlueStore's RocksDB transaction layer to enhance parallelism and scalability.

History

Ceph originated as a project developed by Sage Weil during his doctoral dissertation at the University of California, Santa Cruz (UCSC). His work was supervised by Professor Scott A. Brandt and supported by the Advanced Simulation and Computing Program (ASC), which included contributions from national laboratories like Los Alamos National Laboratory (LANL), Sandia National Laboratories (SNL), and Lawrence Livermore National Laboratory (LLNL). Weil initially wrote Ceph's code in 2004 while interning at LLNL over the summer, concentrating on scalable management of filesystem metadata, which later became known as Ceph's Metadata Server (MDS). In 2005, prompted by Scott Brandt, Weil produced a fully operational prototype of the Ceph file system. Ceph was publicly introduced through Weil’s presentations at USENIX OSDI 2006 and the Supercomputing Conference that same year.

Following his graduation in 2007, Weil dedicated himself to Ceph's development full-time, joined shortly thereafter by Yehuda Sadeh Weinraub and Gregory Farnum. On March 19, 2010, Linus Torvalds incorporated the Ceph client into version 2.6.34 of the Linux kernel, which was officially released in May 2010. In 2012, Weil established Inktank Storage to offer professional services and support specifically for Ceph.

In April 2014, Red Hat acquired Inktank, consolidating Ceph development efforts to produce an enterprise-level production release with comprehensive support and ongoing maintenance.

The Ceph Community Advisory Board was formed in October 2015 to steer the development of open source software-defined storage solutions. This board included representatives from organizations such as Red Hat, Intel, Canonical, CERN, Cisco, Fujitsu, SanDisk, and SUSE.

In November 2018, the Linux Foundation established the Ceph Foundation as a successor to the advisory board. Founding members comprised Amihan, Canonical, China Mobile, DigitalOcean, Intel, OVH, ProphetStor Data Services, Red Hat, SoftIron, SUSE, Western Digital, XSKY Data Technology, and ZTE.

By March 2021, SUSE ceased active sales of its Ceph-based Enterprise Storage product to concentrate on Rancher's Longhorn solution for storage. This strategic shift redirected SUSE’s storage initiatives toward key customers while discontinuing its Enterprise Storage offerings.

Release history

Release history
Name Release First release End of
life 		Milestones
Argonaut Template:Version July 3, 2012 Initial major stable release
Bobtail Template:Version January 1, 2013
Cuttlefish Template:Version May 7, 2013 Stabilization of ceph-deploy tool
Dumpling Template:Version August 14, 2013 May 2015 Introduced namespace, region features and monitoring REST API
Emperor Template:Version November 9, 2013 May 2014 Enabled multi-datacenter replication for RGW
Firefly Template:Version May 7, 2014 April 2016 Added erasure coding, cache tiering, primary affinity; experimental key/value OSD backend and standalone RGW
Giant Template:Version October 29, 2014 April 2015
Hammer Template:Version April 7, 2015 August 2017
Infernalis Template:Version November 6, 2015 April 2016
Jewel Template:Version April 21, 2016 June 1, 2018 Stable CephFS introduced; experimental BlueStore backend; daemons no longer run with root privileges
Kraken Template:Version January 20, 2017 August 1, 2017 Stable BlueStore implementation; erasure coding support for RBD pools
Luminous Template:Version August 29, 2017 March 1, 2020 Introduced pg-upmap balancer
Mimic Template:Version June 1, 2018 July 22, 2020 Stable snapshots and Beast; launch of official GUI Dashboard
Nautilus Template:Version March 19, 2019 June 1, 2021 Added asynchronous replication and automatic retry for write failures due to defect remapping
Octopus Template:Version March 23, 2020 June 1, 2022
Pacific Template:Version March 31, 2021 June 1, 2023
Template:Version April 19, 2022 June 1, 2024 Automatic configuration of min_alloc_size for new media
Template:Version August 3, 2023 August 1, 2025
Template:Version September 26, 2024 September 19, 2026
Template:Version TBA


Supported Platforms

While Ceph is primarily engineered for Linux, it has also been partially adapted for Windows, offering production-level compatibility with Windows Server 2016, Windows Server 2019, and Windows Server 2022. Certain functionalities might be restricted due to the absence of a UNIX socket implementation. Development and testing can also be conducted on Windows 10 and Windows 11. Both Ceph RBD and CephFS are functional on Windows systems, although OSD support is not available. Moreover, Ceph has a version implemented for FreeBSD.

MicroCeph

Introduced by Canonical in 2022, MicroCeph is a streamlined Ceph deployment solution designed to accommodate users without extensive expertise, utilizing the snap packaging format. This deployment is platform-agnostic, scalable, isolated from the host OS environment, and requires minimal configuration and upkeep. MicroCeph supports all primary Ceph storage protocols—block, file, and object—and can be set up with comprehensive disk encryption, making it particularly suitable for efficient edge storage deployments.

Name Origin

The term "Ceph" originates from "cephalopod", a class of mollusks that includes creatures like squids and octopuses, reflecting the system’s highly parallel architecture. Additionally, the name subtly references "Sammy," the banana slug mascot of the UCSC, highlighting a connection between cephalopods and banana slugs as members of the mollusk family.

See also

Further reading

External links

  1. ↑ Cite error: Invalid <ref> tag; no text was provided for refs named ibm-developerworks
  2. ↑ Cite error: Invalid <ref> tag; no text was provided for refs named bluestore
  3. ↑ Cite error: Invalid <ref> tag; no text was provided for refs named lwn
  4. ↑ Cite error: Invalid <ref> tag; no text was provided for refs named lcse
  5. ↑ Cite error: Invalid <ref> tag; no text was provided for refs named luminous
Retrieved from "https://www.alfawiki.org/index.php?title=Ceph_(software)&oldid=27318"