Designing CloudStack Clouds 
Geoff Higginbottom 
CTO ShapeBlue 
geoff.higginbottom@shapeblue.com 
Twitter: @CloudStackGuru
 Cloud Architect & ShapeBlue CTO 
@shapeblue #ccceu14 
 Specialise in…. 
 Designing & Building Clouds based on Apache CloudStack / Citrix 
CloudPlatform 
 Developing CloudStack training 
 Blogging and sharing CloudStack knowledge 
 Involved with CloudStack before donation to Apache 
 Designed Clouds for Cloudera, SunGard, Ascenty, BskyB, Trader Media, 
M5 Hosting, Team Cymru, Interoute, University of Pennsylvania and 
many many more… 
 CloudStack Committer 
About Me
About ShapeBlue 
“ShapeBlue are expert builders of public & 
private clouds. They are the leading global 
Apache CloudStack integrator & consultancy” 
@shapeblue #ccceu14
WHY? 
@shapeblue #ccceu14
Type of Cloud 
Public 
Enterprise 
Test & Dev 
@shapeblue #ccceu14
Requirements 
Integration 
Security 
Type 
Scale 
Workloads 
Monitoring 
SLAs 
Preferred Technologies 
User Experience 
Templates 
Portal 
Logging 
Managed Services 
Bursting 
Existing Skills 
Storage Time Scales 
@shapeblue #ccceu14
Design Team 
Billing 
CloudStack ‘Expert’ 
Storage 
@shapeblue #ccceu14 
Marketing 
QA / Testing 
Networking 
Compute 
Product Development 
Support 
Decision Makers
Training 
Design Team need to 
understand CloudStack, 
so train them first 
@shapeblue #ccceu14
Design Cycle 
@shapeblue #ccceu14 
Design is a reiterative 
process, with some 
design decisions 
impacting on others
Scale 
@shapeblue #ccceu14
Workloads 
 Enterprise Workloads 
should be a known quantity 
 Public Cloud Workloads are 
unknown 
@shapeblue #ccceu14
Capacity Calculations 
@shapeblue #ccceu14 
 VM Profiles 
 Memory 
 CPU 
 Storage Capacity 
 Storage IOPS 
 Hypervisor Overhead 
 Storage Performance 
 Network Performance
Hypervisor Choice 
 Hypervisor Selection and Decisions in CloudStack by Tim Mackey 
 http://open.citrix.com/cloud-computing-vids/video/latest/hypervisor-selection- 
and-decisions-in-cloudstack-by-tim-mackey.html 
@shapeblue #ccceu14 
 Existing Skills 
 Feature Comparison 
 Zone Type, Snapshots, VXLAN, IPv6, SDN, VPC, PVLAN, Storage 
 Licensing Costs 
 Supportability 
 Traditional Server vs Blades 
 More than one – Hypervisor Agnostic
 Heavily influenced by Requirements, 
@shapeblue #ccceu14 
Scale, and Zone modes 
 Is often the driver for other 
technology choices 
 Avoid single points of failure 
 Keep it simple 
Networking
Zone Networking Modes 
 ‘One size sits fits all’ may not be the best approach 
 Each Zone can be a different Network Type 
@shapeblue #ccceu14 
 Basic 
 Basic + Security Groups 
 Basic + Security Groups + EIP / ELB 
 Advanced 
 Advanced + Security Groups
Hypervisor Networking 
@shapeblue #ccceu14 
 How many NICs 
 10GB / 1GB 
 Bonding / Multipath 
 Converged 
 Traffic Allocations 
 Management 
 Guest 
 Public 
 Storage 
 High Bandwidth Services
Storage 
 Primary Storage 
 Local 
 Lack of HA 
 Shared 
 NFS 
 iSCSI 
 Fibre Channel 
 Performance is critical, 
IOPS are king 
@shapeblue #ccceu14
Storage 
 Secondary Storage 
 NFS 
 S3 
 Swift 
@shapeblue #ccceu14
@shapeblue #ccceu14 
 Resource Allocation 
 All Public 
 Some Public, Some Dedicated 
 All Dedicated 
 Reseller Model 
 Account/Domain Relationship 
 1-to-1 
 1-to-many 
Domains and Accounts
@shapeblue #ccceu14 
 Allocate resources to VMs 
 CPU 
 RAM 
 Storage Performance 
 Tagging 
 Cost associated with them 
 Public or Private (linked to Domains) 
 Keep them realistic 
Service Offerings
Templates & ISOs 
@shapeblue #ccceu14 
 Pre-Defined VM images 
 Base OS, or fully installed Apps 
 Licensing (RHEL, Windows) 
 Self Build via ISOs 
 Allow user generated Public? 
 Allow user upload / download? 
 Lifecycle Management
Testing 
 Define test and acceptance criteria 
 Develop test plans (manual UI and scripted API) 
 Run tests to confirm initial build is good 
 Use tests for testing future upgrades and expansions 
@shapeblue #ccceu14
Add on Services 
Managed Services 
@shapeblue #ccceu14 
Billing 
Object Storage 
VM Monitoring 
Backup 
Anti Virus
Management Farm 
CloudStack Management 
@shapeblue #ccceu14 
SQL DB 
LDAP 
DNS 
Load Balancers 
Portal 
Billing 
Monitoring 
3rd Party Services 
Automation 
Admin 
vCenter
DC2 Software Management Farm 
Primary Storage Network (iSCSI or NFS) Secondary Storage Network (NFS) 
Example Logical Network Diagram Dual Zone 
Ver Date Description Issuer/ Reviewer 
@shapeblue #ccceu14 
DC1 Software Management Farm 
CS Management Servers 
CS MySql Servers 
NetScaler VPXs 
CM Automation Server 
Deployment Server 
Amysta App Server 
Amysta MySQL Server 
Management Network 
Management Hosts 
Compute Hosts 
Guest Networks 
(Multiple VLANs) 
Public Networks 
(Multiple VLANs) 
Secondary Storage Network (NFS) Primary Storage Network (iSCSI or NFS) 
Secondary Primary 
DC1 Compute 
Management VMs 
Storage Network 
CS Management Servers 
CS MySql Servers 
NetScaler VPXs 
CM Automation Server 
Deployment Server 
Amysta App Server 
Amysta MySQL Server 
Core 
Network 
Management Hosts 
Management VMs 
Storage Network 
Guest Networks Compute Hosts 
(Multiple VLANs) 
Public Networks 
(Multiple VLANs) 
Primary Secondary 
Users Portal Access 
WWW WWW 
WWW 
Tennant 1 Tennant 2 Tennant 3 
Tennant 1 Tennant 2 Tennant 3 
Private Cloud Tennant VMs on Compute in either DC1 or DC2 
Each Network is isolated via VLANs or SDN technologies. 
Public Cloud Tennant 
VMs on Compute in 
either DC1 or DC2 
Each Tennant has an Isolated Network 
protected by Virtual Router/Firewall. 
Each Network is isolated via VLANs or 
SDN technologies. 
Virtual Routers Public Networks are 
connected directly to the Internet 
enabling users to have full control of 
Firewall & Load Balancing features. 
DC2 Compute 
Public Network(s) 
Virtual Routers 
Guest Networks 
Guest VMs 
Management Network 
Client access to Portal is 
Global Load Balanced by 
Citrix NetScaler VPXs 
running on Management 
Farm using One-Arm 
Configurations 
Firewall functionality 
should be provided by 
existing Firewalls running 
in HA Pair 
Direct Access to Virtual Routers 
1.0 15/08/13 1st releas e G Higginbottom / G Sirett 
Network 
Diagram
Just the one Cloud? 
@shapeblue #ccceu14 
 Production 
 Very Strict Configuration Management 
 Pre-Production 
 Same design as Production 
 Smaller, but with all key components 
 Strict Configuration Management 
 Testing 
 Probably gets rebuilt every few months
“Build it and they will come” 
is the motto of a fool. 
“Build it, take it to them, ask them to buy 
and serve them well” 
is the motto of a successful person. 
© Larry Winget 
@shapeblue #ccceu14
“Design for tomorrow, 
build for today” 
© Geoff Higginbottom 2013 
@shapeblue #ccceu14
Designing CloudStack Clouds 
Geoff Higginbottom 
CTO ShapeBlue 
geoff.higginbottom@shapeblue.com 
Twitter: @CloudStackGuru

Designing CloudStack Clouds

  • 1.
    Designing CloudStack Clouds Geoff Higginbottom CTO ShapeBlue geoff.higginbottom@shapeblue.com Twitter: @CloudStackGuru
  • 2.
     Cloud Architect& ShapeBlue CTO @shapeblue #ccceu14  Specialise in….  Designing & Building Clouds based on Apache CloudStack / Citrix CloudPlatform  Developing CloudStack training  Blogging and sharing CloudStack knowledge  Involved with CloudStack before donation to Apache  Designed Clouds for Cloudera, SunGard, Ascenty, BskyB, Trader Media, M5 Hosting, Team Cymru, Interoute, University of Pennsylvania and many many more…  CloudStack Committer About Me
  • 3.
    About ShapeBlue “ShapeBlueare expert builders of public & private clouds. They are the leading global Apache CloudStack integrator & consultancy” @shapeblue #ccceu14
  • 5.
  • 6.
    Type of Cloud Public Enterprise Test & Dev @shapeblue #ccceu14
  • 7.
    Requirements Integration Security Type Scale Workloads Monitoring SLAs Preferred Technologies User Experience Templates Portal Logging Managed Services Bursting Existing Skills Storage Time Scales @shapeblue #ccceu14
  • 8.
    Design Team Billing CloudStack ‘Expert’ Storage @shapeblue #ccceu14 Marketing QA / Testing Networking Compute Product Development Support Decision Makers
  • 9.
    Training Design Teamneed to understand CloudStack, so train them first @shapeblue #ccceu14
  • 10.
    Design Cycle @shapeblue#ccceu14 Design is a reiterative process, with some design decisions impacting on others
  • 11.
  • 12.
    Workloads  EnterpriseWorkloads should be a known quantity  Public Cloud Workloads are unknown @shapeblue #ccceu14
  • 13.
    Capacity Calculations @shapeblue#ccceu14  VM Profiles  Memory  CPU  Storage Capacity  Storage IOPS  Hypervisor Overhead  Storage Performance  Network Performance
  • 14.
    Hypervisor Choice Hypervisor Selection and Decisions in CloudStack by Tim Mackey  http://open.citrix.com/cloud-computing-vids/video/latest/hypervisor-selection- and-decisions-in-cloudstack-by-tim-mackey.html @shapeblue #ccceu14  Existing Skills  Feature Comparison  Zone Type, Snapshots, VXLAN, IPv6, SDN, VPC, PVLAN, Storage  Licensing Costs  Supportability  Traditional Server vs Blades  More than one – Hypervisor Agnostic
  • 15.
     Heavily influencedby Requirements, @shapeblue #ccceu14 Scale, and Zone modes  Is often the driver for other technology choices  Avoid single points of failure  Keep it simple Networking
  • 16.
    Zone Networking Modes  ‘One size sits fits all’ may not be the best approach  Each Zone can be a different Network Type @shapeblue #ccceu14  Basic  Basic + Security Groups  Basic + Security Groups + EIP / ELB  Advanced  Advanced + Security Groups
  • 17.
    Hypervisor Networking @shapeblue#ccceu14  How many NICs  10GB / 1GB  Bonding / Multipath  Converged  Traffic Allocations  Management  Guest  Public  Storage  High Bandwidth Services
  • 18.
    Storage  PrimaryStorage  Local  Lack of HA  Shared  NFS  iSCSI  Fibre Channel  Performance is critical, IOPS are king @shapeblue #ccceu14
  • 19.
    Storage  SecondaryStorage  NFS  S3  Swift @shapeblue #ccceu14
  • 20.
    @shapeblue #ccceu14 Resource Allocation  All Public  Some Public, Some Dedicated  All Dedicated  Reseller Model  Account/Domain Relationship  1-to-1  1-to-many Domains and Accounts
  • 21.
    @shapeblue #ccceu14 Allocate resources to VMs  CPU  RAM  Storage Performance  Tagging  Cost associated with them  Public or Private (linked to Domains)  Keep them realistic Service Offerings
  • 22.
    Templates & ISOs @shapeblue #ccceu14  Pre-Defined VM images  Base OS, or fully installed Apps  Licensing (RHEL, Windows)  Self Build via ISOs  Allow user generated Public?  Allow user upload / download?  Lifecycle Management
  • 23.
    Testing  Definetest and acceptance criteria  Develop test plans (manual UI and scripted API)  Run tests to confirm initial build is good  Use tests for testing future upgrades and expansions @shapeblue #ccceu14
  • 24.
    Add on Services Managed Services @shapeblue #ccceu14 Billing Object Storage VM Monitoring Backup Anti Virus
  • 25.
    Management Farm CloudStackManagement @shapeblue #ccceu14 SQL DB LDAP DNS Load Balancers Portal Billing Monitoring 3rd Party Services Automation Admin vCenter
  • 26.
    DC2 Software ManagementFarm Primary Storage Network (iSCSI or NFS) Secondary Storage Network (NFS) Example Logical Network Diagram Dual Zone Ver Date Description Issuer/ Reviewer @shapeblue #ccceu14 DC1 Software Management Farm CS Management Servers CS MySql Servers NetScaler VPXs CM Automation Server Deployment Server Amysta App Server Amysta MySQL Server Management Network Management Hosts Compute Hosts Guest Networks (Multiple VLANs) Public Networks (Multiple VLANs) Secondary Storage Network (NFS) Primary Storage Network (iSCSI or NFS) Secondary Primary DC1 Compute Management VMs Storage Network CS Management Servers CS MySql Servers NetScaler VPXs CM Automation Server Deployment Server Amysta App Server Amysta MySQL Server Core Network Management Hosts Management VMs Storage Network Guest Networks Compute Hosts (Multiple VLANs) Public Networks (Multiple VLANs) Primary Secondary Users Portal Access WWW WWW WWW Tennant 1 Tennant 2 Tennant 3 Tennant 1 Tennant 2 Tennant 3 Private Cloud Tennant VMs on Compute in either DC1 or DC2 Each Network is isolated via VLANs or SDN technologies. Public Cloud Tennant VMs on Compute in either DC1 or DC2 Each Tennant has an Isolated Network protected by Virtual Router/Firewall. Each Network is isolated via VLANs or SDN technologies. Virtual Routers Public Networks are connected directly to the Internet enabling users to have full control of Firewall & Load Balancing features. DC2 Compute Public Network(s) Virtual Routers Guest Networks Guest VMs Management Network Client access to Portal is Global Load Balanced by Citrix NetScaler VPXs running on Management Farm using One-Arm Configurations Firewall functionality should be provided by existing Firewalls running in HA Pair Direct Access to Virtual Routers 1.0 15/08/13 1st releas e G Higginbottom / G Sirett Network Diagram
  • 27.
    Just the oneCloud? @shapeblue #ccceu14  Production  Very Strict Configuration Management  Pre-Production  Same design as Production  Smaller, but with all key components  Strict Configuration Management  Testing  Probably gets rebuilt every few months
  • 28.
    “Build it andthey will come” is the motto of a fool. “Build it, take it to them, ask them to buy and serve them well” is the motto of a successful person. © Larry Winget @shapeblue #ccceu14
  • 29.
    “Design for tomorrow, build for today” © Geoff Higginbottom 2013 @shapeblue #ccceu14
  • 30.
    Designing CloudStack Clouds Geoff Higginbottom CTO ShapeBlue geoff.higginbottom@shapeblue.com Twitter: @CloudStackGuru

Editor's Notes

  • #6 Seems like a good idea? Boss wants one? Solid Business Case?
  • #11 It’s not a linear path, some choices affect others
  • #13 Understanding workloads are important – they drive the design Enterprise = Known Workloads Public = Unknown Workloads