Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardware

Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardware

• 1.
  Gary Howard, SeniorSolution Architect, Data Centers 1
• 2.
  DCG Storage Group2 Driven by:  Mobile  Social Media  Internet of Things  Big Data and Cloud Source: IDC – The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things - April 2014 StorageCostStructureNeedsaFundamentalShift 1 2 3 5 7 12 19 30 48 77 125 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 Storage Capacity in Terabytes 62%CAGR 2% IT Budget CAGR As data volumes continue to grow, IT needs storage solutions that can reduce costs while still meeting other storage requirements
• 3.
  3 STORAGE TRANSFORMING KEY DRIVERS CLOUD CLOUD SERVICES 1 STORAGEMEDIA TRANSITIONS 3 3D XPOINT™ TECHNOLOGY NEXT GENERATION ARCHITECTURES 2 CLOUD CLOUD SERVICES NEXT GENERATION ARCHITECTURES STORAGE MEDIA TRANSITIONS 3D XPOINT™ TECHNOLOGY
• 4.
  DCG Storage Group4 WHYSTORAGEMODERNIZATIONISESSENTIAL Internet of Things Media Transition Scale Out Scale Up Storage Modernization: Seamless data access anywhere, at anytime, on any device at the required performance AGILE, Automated & Secure Infrastructures and Business Models Cloud Enterprise 01 1010101 101010101010 010101010101 101010101010 Orchestration
• 5.
  DCG Storage Group MovefromScale-uptoScale-OutArchitectures 5 Supportsstorage capacity growth cost effectively Standard Ethernet network Data distributed across multiple nodes or clusters Flexible design to support multiple workloads Separate, dedicated networks Data stored in proprietary storage hardware Optimized to run only a specific workload Compute nodes working together external network Scalesperformance&capacity Scale-OutScale-up Single System internal network SCALESBYADDINGDISKSFORCAPACITY
• 6.
  DCG Storage Group OpenSourceScale-OutCephwithoptimalcommercialsupport 6 Clients CephStorage Cluster Server 1 Ceph OS CPU Memory Disks Network Server 2 Ceph OS CPU Memory Disks Network Server 3 Ceph OS CPU Memory Disks Network Server 4 Ceph OS CPU Memory Disks Network Object Storage Block Storage File Storage One platform that provides Object Storage, Block Storage, S3 interface and File
• 7.
  7 Intel’sroleinstorage AdvancetheIndustry OpenSource&Standards BuildanOpenEcosystem Intel®StorageBuilders Endusersolutions Cloud,Enterprise IntelTechnologyLeadership Storage Optimized Platforms Intel®Xeon® E5-2600 v4 Platform Intel® Xeon® Processor D-1500 Platform Intel® Converged Network Adapters 10/40GbE Intel® SSDs for DC & Cloud Storage Optimized Software Intel® Intelligent Storage Acceleration Library Storage Performance Development Kit Intel® Cache Acceleration Software SSD & Non-Volatile Memory Interfaces: SATA , NVMe PCIe, Form Factors: 2.5”, M.2, U.2, PCIe AIC New Technologies: 3D NAND, Intel® Optane™ Ceph community contributions on workload profiling, latency analysis and performance optimizations 90+ partners Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. Storage solutions architectures Intel solution architects have deep expertise on Ceph for low cost and high performance usage helpingcustomerstoenableamodernstorageinfrastructure
• 8.
  How well can Cephperform? Which of my workloads can it handle? Which server hardware is required for it to perform well?
• 9.
  Optimization Criteria Properties Example Uses IOPS-optimized Lowest cost per IOPS  Highest IOPS  Meets minimum fault domain recommend- ation (single server is less than or equal to 10% of the cluster)  Typically block storage  3x replication on hard disk drive (HDD) or 2x replication on Intel® SSD DC Series  MySQL on OpenStack clouds Throughput- optimized  Lowest cost per given unit of throughput  Highest throughput  Highest throughput per BTU  Highest throughput per watt  Meets minimum fault domain recommend- ation (single server is less than or equal to 10% of the cluster)  Block or object storage  3x replication  Active performance storage for video, audio, and images  Streaming media Capacity- optimized  Lowest cost per TB  Lowest BTU per TB  Lowest watt per TB  Meets minimum fault domain recommend- ation (single server is less than or equal to 15% of the cluster)  Typically object storage  Erasure coding common for maximizing usable capacity  Object archive  Video, audio, and image object archive repositories
• 10.
  Open Stack StarterS M L 64TB 256TB + 1PB + 2PB + IOPS Optimized • Ceph block (RBD) • Intel® P3700s w/ co-located write journals OR • Intel® S3610 w/ P3700 write journals in a 4:1 ratio • Multiple OSDs per flash drive • 10 Intel® Xeon® E5 cores per P3700; 4 per S3610 • 2x or 3x replication (with backup) Throughput Optimized • Ceph block or object (RBD or RGW) • HDDs w/ P3700 or S3710 write journals • 1 Xeon core per 2 HDDs (e.g. with 24 HDDs, 12 core Intel® Xeon® E5-2650 v4) • Single OSD per HDD • 10GbE  40GbE with >12 HDD per chassis • 3x replication Cost-Capacity Optimized • Ceph object (RGW) • HDD drives with no SSD journal • 1 Xeon core per 2 HDD • Single OSD per HDD • Erasure-coded
• 11.
  11 Recent Intel testingshows it’s possible to reach 1.4M IOPS Cephfordatabaseworkloads Using a MySQL database workload, we measured 400K OLTP QPS using a mix of 70/30% Select/Update
• 12.
  DCG Storage Group Cephperformancerange 12 Perceivedperformance range Actual performance range (measured)
• 13.
  DCG Storage Group Availableresources 13 https://www.redhat.com/en/resources/ red-hat-ceph-storage-servers-intel®- processors-and-ssds https://www.percona.com/resources /videos/mysql-cloud-head-head- performance-lab https://rh2016.smarteventscloud. com/connect/sessionDetail.ww?S ESSION_ID=88853&tclass=popup Watchfor it posted to www.intel.com by Friday!
• 14.
  SummaryandNextSteps  Ceph canhelp deliver on the promise of the cloud, using next generation storage architectures  Flash technology enables new capabilities in small footprints  Ceph and MySQL provide a compelling case for converged storage that can support latency sensitive analytics workloads  Next steps:  Access the available resources to learn more about how Ceph can be deployed for your workloads  Consider a Ceph pilot with Red Hat and Intel Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Any difference in system hardware or software design or configuration may affect actual performance. See configuration slides in backup for details on software configuration and test benchmark parameters. 14
• 16.
  DCG Storage Group Storagemodernization 16 ½latency 2x throughput 50% CapEX reduction Storage Intel® Differentiated Storage ServicesApps Intel® CAS “Web” “Analytics’” .. . 1IDF 2015 Class SDS002 http://intelstudios.edgesuite.net//idf/2015/sf/aep/SSDS002/SSDS002.html IDF2015 Technology Demo: https://www.youtube.com/watch?v=vtIIbxO4Zlk Results of Yahoo* internal benchmark testing - Ruiping Sun, Principal Architect, Yahoo*. Hardware/Software Config: 8 OSD Nodes, each: HP ProLiant DL180 G6 ySPEC 39.5, 2x Xeon X5650 2.67GHz (HT enabled, total 12 cores, 24 threads), Intel 5520 IOH-36D B3 (Tylersburg), 48GB 1333MHz DDR3 (12x4GB PC3-10600 Samsung DDR3-1333 ECC Registered CL9 2Rx4), 10*8TB 7200 RPM SATA HDDs, 1*1.6TB Intel P3600 SSD (10GB journal per OSD, 1.5TB cache) (CAS config only), 2*HP NC362i/Intel 82576 Gigabit NICs, 2*Intel 82599EB 10GbE NICs, RHEL 6.5 w/kernel 3.10.0-123.4.4.el7
• 17.
  Pushing Buying BehaviorsAway from Norm Cost efficiency Speed of provisioning Vendor lock-in Storage scaling Admin Governance IOPS per GB better faster less lock-in elastic pool more expertise more control broad options better faster more lock-in silos* less expertise less control narrow options public cloud storage NAS/SAN storage appliances software defined storage commodity servers
• 18.
  Illustrating 10x Rangeof Ceph Performance 1 DVD movie per second with 3-node cluster (standard 2U Ceph servers, hybrid) E-Book MP3 Song Audio CD DVD Movie HD Movie Blue-ray Movie UHD Movie 1 Blue-ray movie per second with 3-node cluster (standard 2U Ceph servers, all-flash)
• 19.
  Range of CephUse-Case Examples Workload IO Profiles Workload Examples Searchable Examples Workload IO Characteristics Hardware Characteristics IOPS MySQL MariaDB PostgreSQL Medallia High IOPS/GB Smaller random IO Read/write mix Sled-based chassis Intel® DC P3700 10 Xeon® cores per P3700 10GbE Balanced/Throughpu t Digital media distro Server virtualization (OpenStack Cinder) Bloomberg Target, Walmart yahoo!, Comcast High MBps/TB Larger sequential IO Read/write mix Standard -> dense chassis HDD -> Intel® DC P3700 Balanced core : drive ratio 10GbE -> 40GbE Capacity-Archive Digital media archive Object archive Big Data archive yahoo! CERN Low cost/GB Sequential IO Write mostly Dense -> ultra-dense chassis HDD Low core : drive ratio 10GbE