CloudStack Architecture Future

Sheng Liang, CTO Cloud Platforms, Citrix Systems
July 21, 2012
Users


        SS        CP                            Cloud
VR                                  EC2
                                                Stack     API Layer
        VM        VM

                                                        Access control
     System VM
     Framework                      Orchestration                          Plug-in
                                       Engine                            Framework

Event Framework
                                               Resource Management
                                                                                             Plug-ins
                                          Message Bus                                        (for storage,
             vSphere    XenServer           KVM            SRX       NetScaler     Other     networking,
             Resource   Resource          Resource       Resource    Resource    Resources   hypervisor,
                                                                                             etc.)
Future: CloudStack storage
                                                                  CloudStack
Primary Storage                                                   Mgmt Server

• Can be configured zone-wide
• Supports true EBS implementations

                                                 AWS-style      AWS-style          AWS-style
                                                 Availability   Availability       Availability
                                                   Zone           Zone               Zone

Secondary Storage
• No longer needed zone-wide NFS server                           Object Storage
• Region-wide object store for templates, ISOs
  and snapshots
• Templates and ISOs cached in Primary Storage
• Apache Foundation project
• Hadoop Core includes:
 ᵒ HDFS - Distributed file system derived from Google File System
 ᵒ Map/Reduce – Distributed compute engine
• Written in Java
• Runs on Linux, Mac OS/X, Windows, and Solaris
• Optimized for commodity hardware
Now: management server clusters, zones, and regions

                     Region 1                                       Region 2
      Data            Data            Data           Data            Data            Data
     Center          Center          Center         Center          Center          Center

     Primary                         Standby        Primary                         Standby
      Mgmt                            Mgmt           Mgmt                            Mgmt
      Server                          Server         Server             DR           Server
     Cluster
                         DR          Cluster        Cluster                         Cluster




    Availability    Availability    Availability   Availability    Availability    Availability
       Zone            Zone            Zone           Zone            Zone            Zone




                   Object Storage                                 Object Storage
Future: management server cluster per zone

                     Region 1                                       Region 2
      Data            Data            Data           Data            Data            Data
     Center          Center          Center         Center          Center          Center


      Mgmt            Mgmt           Mgmt            Mgmt            Mgmt           Mgmt
      Server          Server         Server          Server          Server         Server
      Cluster         Cluster        Cluster         Cluster         Cluster        Cluster




    Availability    Availability    Availability   Availability    Availability    Availability
       Zone            Zone            Zone           Zone            Zone            Zone




                   Object Storage                                 Object Storage
• Implement rigorous state machines for all operations
• Customizable workflow engine
• Enhance plug-in framework
 ᵒ Storage plug-in
• Stateless, recreatable, and restartable
• CloudStack manages auto-scaling of System VMs
• System VMs may be dedicated (e.g., virtual router) or shared (SSVM
  and CPVM)
• Potentially expose System VM framework to 3rd-party VMs?
 ᵒ NetScaler VPX
 ᵒ User VM
S3 API

                                  NFS, CIFS, HDFS,
                                  3rd-Party Object
             S3 API VM              Store API, …
S3 API VM                                            Object Store
…           …                 …           (e.g., NetApp, Hadoop, Cloudian)
             S3 API VM


CloudStack Compute Nodes
• Clean up access control code
• API servers run as a separate service
 ᵒ Running in System VM
• Built on message bus
• Treats management server DB as the desired state
• Some downsides:
 ᵒ Prevents use of native virt mgmt tools
 ᵒ Bugs could cause disastrous results (like stopping/destroying guest VMs)
• An alternative model: CloudStack functions as a virt admin user,
  enabling super admin to use native virt mgmt tools
Future model

  Userh        CloudStack
               Mgmt Server
                              vCenter    ESXi Cluster

                 vCenter
                  Client
 Admin                       XenServer
                              Cluster
               XenCenter


                             Hyper-V
                Systems
                             Cluster
                 Center
• Register existing virt clusters
• CloudStack must assume any of the underlying networks, storage,
  hosts, and VMs can change state any time
• Expose additional virt mgmt capabilities as cloud API
Manage existing server virt clusters

                   CloudStack Region



    AWS-style       AWS-style          AWS-style       Existing      Existing
    Availability    Availability       Availability   Server Virt   Server Virt
      Zone            Zone               Zone          Clusters      Clusters




                      Object Storage
Work better. Live better.

CloudStack Architecture Future

  • 1.
    CloudStack Architecture Future ShengLiang, CTO Cloud Platforms, Citrix Systems July 21, 2012
  • 2.
    Users SS CP Cloud VR EC2 Stack API Layer VM VM Access control System VM Framework Orchestration Plug-in Engine Framework Event Framework Resource Management Plug-ins Message Bus (for storage, vSphere XenServer KVM SRX NetScaler Other networking, Resource Resource Resource Resource Resource Resources hypervisor, etc.)
  • 3.
    Future: CloudStack storage CloudStack Primary Storage Mgmt Server • Can be configured zone-wide • Supports true EBS implementations AWS-style AWS-style AWS-style Availability Availability Availability Zone Zone Zone Secondary Storage • No longer needed zone-wide NFS server Object Storage • Region-wide object store for templates, ISOs and snapshots • Templates and ISOs cached in Primary Storage
  • 4.
    • Apache Foundationproject • Hadoop Core includes: ᵒ HDFS - Distributed file system derived from Google File System ᵒ Map/Reduce – Distributed compute engine • Written in Java • Runs on Linux, Mac OS/X, Windows, and Solaris • Optimized for commodity hardware
  • 5.
    Now: management serverclusters, zones, and regions Region 1 Region 2 Data Data Data Data Data Data Center Center Center Center Center Center Primary Standby Primary Standby Mgmt Mgmt Mgmt Mgmt Server Server Server DR Server Cluster DR Cluster Cluster Cluster Availability Availability Availability Availability Availability Availability Zone Zone Zone Zone Zone Zone Object Storage Object Storage
  • 6.
    Future: management servercluster per zone Region 1 Region 2 Data Data Data Data Data Data Center Center Center Center Center Center Mgmt Mgmt Mgmt Mgmt Mgmt Mgmt Server Server Server Server Server Server Cluster Cluster Cluster Cluster Cluster Cluster Availability Availability Availability Availability Availability Availability Zone Zone Zone Zone Zone Zone Object Storage Object Storage
  • 7.
    • Implement rigorousstate machines for all operations • Customizable workflow engine • Enhance plug-in framework ᵒ Storage plug-in
  • 8.
    • Stateless, recreatable,and restartable • CloudStack manages auto-scaling of System VMs • System VMs may be dedicated (e.g., virtual router) or shared (SSVM and CPVM) • Potentially expose System VM framework to 3rd-party VMs? ᵒ NetScaler VPX ᵒ User VM
  • 9.
    S3 API NFS, CIFS, HDFS, 3rd-Party Object S3 API VM Store API, … S3 API VM Object Store … … … (e.g., NetApp, Hadoop, Cloudian) S3 API VM CloudStack Compute Nodes
  • 10.
    • Clean upaccess control code • API servers run as a separate service ᵒ Running in System VM
  • 11.
    • Built onmessage bus • Treats management server DB as the desired state • Some downsides: ᵒ Prevents use of native virt mgmt tools ᵒ Bugs could cause disastrous results (like stopping/destroying guest VMs) • An alternative model: CloudStack functions as a virt admin user, enabling super admin to use native virt mgmt tools
  • 12.
    Future model Userh CloudStack Mgmt Server vCenter ESXi Cluster vCenter Client Admin XenServer Cluster XenCenter Hyper-V Systems Cluster Center
  • 13.
    • Register existingvirt clusters • CloudStack must assume any of the underlying networks, storage, hosts, and VMs can change state any time • Expose additional virt mgmt capabilities as cloud API
  • 14.
    Manage existing servervirt clusters CloudStack Region AWS-style AWS-style AWS-style Existing Existing Availability Availability Availability Server Virt Server Virt Zone Zone Zone Clusters Clusters Object Storage
  • 15.