Download to read offline
Uploaded bypanagenda
DNUG46 - Build your own private Cloud environment
The document discusses building a private cloud environment, emphasizing the use of Kubernetes for automating deployment, scaling, and management of containers. It outlines key topics such as provisioning, backup strategies, log management, and security, while also recommending tools and best practices for creating a flexible and scalable infrastructure. The author, Nico Meisenzahl, also touches on the importance of managing workloads, persistent storage, and continuous integration/deployment pipelines.
More Related Content
Similar to DNUG46 - Build your own private Cloud environment
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
DNUG46 - Build your own private Cloud environment
- 1. Build your ownprivate Cloud environment Nico Meisenzahl, panagenda nico.meisenzahl@panagenda.com @nmeisenzahl
- 2. Make Your DataWork For You Build your own private Cloud environment Nico Meisenzahl DNUG 46, Essen
- 3. @panagenda Consultant. Blogger, speaker IBMCloud Champion & Docker Community Leader Loves K8s, containers & automation. His desk is a ping pong table. Nico Meisenzahl @nmeisenzahl https://meisenzahl.org nico@meisenzahl.org https://panagenda.com/modernization nico.meisenzahl@panagenda.com
- 4. What is Cloud? •someone else’s computer • you do not need to care about everything • can abstract – Hardware → Infrastructure-as-a-Service – underlying OS/machine → Platform-as-a-Service – everything except your business logic → Function-as-a-Service 4
- 5. What is aprivate on-premises Cloud? • for your colleagues – someone else’s computer – a platform which abstracts various things – examples: • deploy, run and scale containers • just care about the code and let the system manage everything else • for you – your computer 😉 – a complex environment with multiple requirements • flexibel, scalable and fast • secure and state-of-the-art – various topics like compute, storage, high-availability, security, …. 5
- 6. Some words onContainers • A container consists of one or more processes that are isolated from the rest of the system • All files necessary for execution are provided by a separate image 6
- 7. Some words onKubernetes • open-source system for automating deployment, scaling, and management of containerized applications using a declarative approach • greek for captain • abstracts underlying machines and OS • state-of-the-art for modern application deployment • used to manage your workload – Think of “vSphere for your containers” 7
- 8. Things you needto take care of • Kubernetes environment – provisioning, scalability, high-availability – update/backup strategy • Storage • Monitoring, Metrics • Log management • Security – Authentication, Authorisation, RBAC – Workload security • Surroundings – Registry, CI/CD, Vulnerability scanning, Operators, etc. – Service mesh, Functions/Serverless 8
- 9. Kubernetes provisioning • Kubernetescan be complex • “Kubernetes the hard way” is great for learning but not for production • Tools which can support you – Rancher Kubernetes Engine (RKE) – KubeOne by Loodse – IBM Cloud Private – kubeadm – https://kubernetes.io/docs/setup/pick-right-solution • design and size your cluster based on your needs – 1,3,5,7 master nodes, worker nodes, etcd nodee? • external dependencies: Storage, Load-Balancer, vSphere... 9
- 10. Kubernetes update strategy •one minor releases (1.x) every 3 months • the last 3 minor releases are maintained/supported – EOL after 9 months! • define your update strategy based on your needs – “in-place update” by adding/removing nodes – “side-by-side” migration 10
- 11. Kubernetes Nodes • Nodesaren’t persistent! – scaling up/down nodes based on your workload (autoscaling!) – updates (create new ones and delete old ones) • therefore use – a immutable approach instead of inplace upgrades – automation and Infrastructure-as-Code approaches • rethink your OS and Container Runtime – do you need a full-blown OS? – is Docker Engine needed? 11
- 12. Kubernetes backup strategy •Kubernetes API is stateful, everything else is stateless – stores its data in a etcd database • Applications deployed in the Cluster store their data externally • Restore etcd – recreate your scripted infrastructure – restore etcd snapshot • Rebuild everything with IaC – recreate your scripted infrastructure – run your pipelines to deploy your applications 12
- 13. Persistent Storage • statefulapplications need volumes to store their data • worker nodes need to be stateless – scaling services on a multi-node environment – up/down scaling nodes • auto-provisioning is key – no manual tasks when deploying/scaling services – Persistent Volume Claims with Provisioner • NFS, Ceph, GlusterFS, Minio, ... 13
- 14. Log management • acentral log management is needed to effectively debug applications – applications are scaled on multiple nodes – complex applications consist of many microservices • collect logs using – a logging agent on node-level • stdout/stderr are collected by Container runtime – a sidecar container approach • logging container collects logs and forwards them • Elasticsearch, Logstash/Fluentd, Kibana as logging stack 14
- 15. Monitoring & Metrics •the chosen system needs to “understand” Kubernetes • Prometheus is the common solution for monitoring and metrics collection for services as well as Kubernetes itself – collects and stores metrics – altering using Alertmanager – Dashboards with Grafana • Prometheus pulls metrics using HTTP – from URL – using Prometheus Exporter library or sidecar – dynamic Container monitoring using Kubernetes API (kube-state-metrics agent) 15
- 16. Authentication, Authorisation • externalservice to provide/manage users • Authentication – X509 Client certificates – OpenID – Bearer Tokens – static password file – Service Account (managed by K8s) – ... • Authorisation – used to define access level for different users and namespaces – Role Based Access Control (RBAC) is recommended – resources access is defined with verbs (get, create, delete, …) and identities (user, group, ...) 16
- 17. Cluster workload security •you will run different applications in one cluster! • use Pod Security Policies (PSP) to define what a pod is allowed to do – permit root, allowed user/group IDs – allowed volume types – allowed Linux capabilities, namespaces – ... • only use secure Images (vulnerability scanning) • permit usage of untrusted Container Registries 17
- 18. Ecosystem • Container Registry/ Chart repository – to store internal images – include security topics like vulnerability scanning • Kubernetes Operators – can be used to deploy and manage custom ressources by extending the API – example: Resource called “mongodb” which deploys a production ready cluster • CI/CD – build continuous pipelines to package/build/deploy your applications – use templating to be able to deploy to various environments (e.g. Helm) 18
- 19. Some more thoughts... • move logic from your code into your platform by using a service mesh – timeout/retry management, load-balancing, ... – end-to-end encryption • allow your Devs to only care about their code – until now, we only talked about Platform-as-a-Service – built Function-as-a-Service on top of K8s 19
- 20. Questions? • Slides →https://www.slideshare.net/nmeisenzahl 20
- 21. Headquarters, Austria: panagenda GmbH(Ltd.) Schreyvogelgasse 3/10 AT 1010 Vienna Phone: +43 1 89 012 89 Fax: +43 1 89 012 89-15 E-Mail: info@panagenda.com Headquarters, Germany: panagenda GmbH (Ltd.) Lahnstraße 17 DE 64646 Heppenheim Phone: +49 6252 67 939-00 Fax: +49 6252 67 939-16 E-Mail: info@panagenda.com USA: panagenda Inc. 60 State Street, Suite 700 MA 02109 Boston Phone: +1 617 855 5961 Fax: +1 617 488 2292 E-Mail: info@panagenda.com Germany: panagenda Consulting GmbH (Ltd.) Donnersbergstrasse 1 DE 64646 Heppenheim Phone: +49 6252 67 939-86 Fax: +49 6252 67 939-16 E-Mail: info@panagenda.com The Netherlands: Trust Factory B.V. 11th Floor, Koningin Julianaplein 10 NL 2595 AA The Hague Phone: +31 70 80 801 96 E-Mail: info@trust-factory.com © 2007-2015 panagenda Make Your Data Work For You
- 22. DNUG e.V. Pappelallee 78/79 10437Berlin Telefon: +49 30 20898805 0 Telefax: +49 30 20898805 1 E-Mail: info@dnug.de Web: http://www.dnug.de