Download as PDF, PPTX
![Example of Puppet codepackage { 'nginx':
ensure => present,
} ->
file { '/etc/nginx/nginx.conf':
ensure => file,
owner => 'root',
group => 'root',
mode => '0644',
content => template('mymodule/nginx.conf.erb'),
notify => Service['nginx'],
} ->
file { '/etc/nginx/conf.d/global.conf':
ensure => file,
owner => 'root',
group => 'root',
mode => '0644',
content => template('mymodule/nginx_global.conf.erb'),
notify => Service['nginx'],
}
service { 'nginx':
ensure => running,
enable => true,
}](https://image.slidesharecdn.com/dockerandpuppet-151125220146-lva1-app6891/75/Docker-and-Puppet-for-Continuous-Integration-18-2048.jpg)
![Dockerfile - example
FROM ubuntu:latest
MAINTAINER Giacomo Vacca <gv@rtcsoft.net>
RUN apt-get update
# Install nginx
RUN apt-get -y -q install nginx
# Prepare target dir for website (Must match global.conf)
RUN mkdir -p /var/www/html
# Configure nginx
COPY nginx/nginx.conf /etc/nginx/nginx.conf
COPY nginx/global.conf /etc/nginx/conf.d/global.conf
EXPOSE 8080
CMD [ "/usr/sbin/nginx" ]
https://github.com/giavac/docker-experiments/tree/master/simple_nginx](https://image.slidesharecdn.com/dockerandpuppet-151125220146-lva1-app6891/75/Docker-and-Puppet-for-Continuous-Integration-23-2048.jpg)
![Example of Puppet codepackage { 'nginx':
ensure => present,
} ->
file { '/etc/nginx/nginx.conf':
ensure => file,
owner => 'root',
group => 'root',
mode => '0644',
content => template('mymodule/nginx.conf.erb'),
notify => Service['nginx'],
} ->
file { '/etc/nginx/conf.d/global.conf':
ensure => file,
owner => 'root',
group => 'root',
mode => '0644',
content => template('mymodule/nginx_global.conf.erb'),
notify => Service['nginx'],
}
service { 'nginx':
ensure => running,
enable => true,
}](https://crownmelresort.com/image.slidesharecdn.com/dockerandpuppet-151125220146-lva1-app6891/75/Docker-and-Puppet-for-Continuous-Integration-18-2048.jpg)
![Dockerfile - example
FROM ubuntu:latest
MAINTAINER Giacomo Vacca <gv@rtcsoft.net>
RUN apt-get update
# Install nginx
RUN apt-get -y -q install nginx
# Prepare target dir for website (Must match global.conf)
RUN mkdir -p /var/www/html
# Configure nginx
COPY nginx/nginx.conf /etc/nginx/nginx.conf
COPY nginx/global.conf /etc/nginx/conf.d/global.conf
EXPOSE 8080
CMD [ "/usr/sbin/nginx" ]
https://github.com/giavac/docker-experiments/tree/master/simple_nginx](https://crownmelresort.com/image.slidesharecdn.com/dockerandpuppet-151125220146-lva1-app6891/75/Docker-and-Puppet-for-Continuous-Integration-23-2048.jpg)
Uploaded byGiacomo Vacca
Docker and Puppet for Continuous Integration
The document discusses the integration of Docker and Puppet in continuous integration processes for software development. It highlights the benefits of using Docker containers for building, testing, and deploying applications while maintaining clean environments, and emphasizes the necessity of configuration management tools like Puppet to automate server configurations. The presentation also covers various tools and workflows associated with Jenkins, Docker orchestration, and interaction opportunities between these technologies.
More Related Content
![Vagrant + Docker provider [+Puppet]](https://cdn.slidesharecdn.com/ss_thumbnails/npoggivagrant-docker-140725100147-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Docker and Puppet for Continuous Integration
![[workshop] The Revolutionary WebRTC](https://cdn.slidesharecdn.com/ss_thumbnails/therevolutionarywebrtc-161216083417-thumbnail.jpg?width=640&height=640&fit=bounds)
Docker and Puppet for Continuous Integration
- 1. Docker and Puppetfor Continuous Integration Giacomo Vacca, Founder at RTCSoft, gv@rtcsoft.net
- 2. About me • 15years “in the trenches cubicles” • Developer of RTC (VoIP, IM, WebRTC) solutions • Founder of RTCSoft in 2015 @giavac https://github.com/giavac gv@rtcsoft.net
- 3. Docker usage scenarios •Run services • Packaging/Deployment mechanism • Fast prototyping • Continuous Integration/ Delivery
- 4. Continuous Integration 1/2 •A team of developers work on the same project • Each developer commits several times per day • Each commit is potentially verified (Build, Deploy, Test) • The project is always kept in a stable/releasable state
- 5. Continuous Integration 2/2 •Developers work on personal/bug/feature (short-lived) branches • These branches are merged into the “release” branch for the release Image source: www.atlassian.com
- 6. CI components Image source:myself (This is why it sucks)
- 7. Server-side apps forLinux • Apps are built, packaged, deployed and tested • Unit testing during the builds • Component/Integration/Load testing after test deployments • Artefacts are uploaded into file servers or deb/yum repos
- 8. Old school: “Buildand run” Build the app on the runtime machine • PROBLEMS: • Repeat the build on each host • Unnecessary pollution of the runtime host • Maintenance nightmare: the easiest upgrade is problematic • Potential exposure of source code to external attacks
- 9. Using a “Buildmachine” Dedicate a machine to the various builds • PROs: • Keep the runtime environment reasonably clean • Source code is protected inside the private network • CONs: • You’re reducing a multi-dimensional problem into a bi-dimensional one • You’ll pay the price one day!
- 10. • Widely adopted,Open Source CI tool • Core + many plugins • Integrates well with git and Docker • Can manage multiple “slaves” https://jenkins-ci.org/
- 11. Jenkins and Masterworker • “Cron jobs manager with a GUI” • What’s a Jenkins job? • The simplest case: Jenkins runs the jobs inside its own host • “Master” worker, no slaves involved • Still you need to manage Jenkins’ configuration…
- 12. Building machineS Assign aVM per building scenario (app/OS) • A step in the right direction • CONs: • Hard to maintain (see Configuration Management later) • VM resources allocated even when the builds are not running • Explosion of number of VM types to use (OS/versions matrix) • VMs are “heavy”
- 13. Jenkins and Slaves •Configure a number of “slave workers” • Assign a slave to a job • But you need to maintain all those slaves…
- 14. Configuration Management You needsomething better than that.
- 15. Configuration Management intro •Define programmatically the configuration of a machine • Many tools available: Puppet, Ansible, Chef, Salt, etc. • Make configuration predictable AND fix divergence automatically • Track changes during time (it’s all text) • Define a formal language across developers, sysadmin, Ops
- 16. • Puppet definesthe final state of a host (What, not How) • Has its own declarative syntax • Written in Ruby • Designed for Master-Slave interaction, can be used Standalone • Idempotent https://puppetlabs.com/
- 17. Configuration Management You really,really need it. Even if you like 3-ways diffs and hate the term “idempotence”.
- 18. Example of Puppetcodepackage { 'nginx': ensure => present, } -> file { '/etc/nginx/nginx.conf': ensure => file, owner => 'root', group => 'root', mode => '0644', content => template('mymodule/nginx.conf.erb'), notify => Service['nginx'], } -> file { '/etc/nginx/conf.d/global.conf': ensure => file, owner => 'root', group => 'root', mode => '0644', content => template('mymodule/nginx_global.conf.erb'), notify => Service['nginx'], } service { 'nginx': ensure => running, enable => true, }
- 19. Back to Jenkinsand multiple Slaves • We saw the approach with multiple slaves, with VMs as slave • Functionally valid • Cumbersome to maintain • What if we could have “on demand”, lightweight workers? e.g. Docker containers?
- 20. • A newform of virtualisation (w/ real HW performance) • A delivery mechanism • Many “images” freely available • “Here’s the ingredients and recipe” VS “Here’s the cake” https://www.docker.com/
- 21. Virtual Machines vsDocker Source: https://www.docker.com/what-docker
- 22. Docker workflow • Buildan image (Dockerfile + base images) • docker build -t gvacca/nginx . • Store image in a registry (Public, e.g. Dockerhub or private) • docker push gvacca/nginx • Pull image on a target host • docker pull gvacca/nginx • Run container on a target host • docker run -it gvacca/nginx
- 23. Dockerfile - example FROMubuntu:latest MAINTAINER Giacomo Vacca <gv@rtcsoft.net> RUN apt-get update # Install nginx RUN apt-get -y -q install nginx # Prepare target dir for website (Must match global.conf) RUN mkdir -p /var/www/html # Configure nginx COPY nginx/nginx.conf /etc/nginx/nginx.conf COPY nginx/global.conf /etc/nginx/conf.d/global.conf EXPOSE 8080 CMD [ "/usr/sbin/nginx" ] https://github.com/giavac/docker-experiments/tree/master/simple_nginx
- 24. Build a Dockerimage gvacca@dockerubuntu:~simple_nginx$ sudo docker build -t gvacca/simple-nginx . Sending build context to Docker daemon 7.168 kB Step 1 : FROM ubuntu:14.04 ---> 1d073211c498 … Step 2 : MAINTAINER Giacomo Vacca <gv@rtcsoft.net> ---> Running in 10a8334becfd ---> 559310abf4fb Removing intermediate container 10a8334becfd Step 3 : RUN apt-get update ---> Running in 873b09debab0 … Step 8 : EXPOSE 8080 ---> Running in a27f73413c02 ---> f5ef8527f2a3 Removing intermediate container a27f73413c02 Step 9 : CMD /usr/sbin/nginx ---> Running in d99de19cc782 ---> df76d20cd00f Removing intermediate container d99de19cc782 Successfully built df76d20cd00f
- 25. Run the container! gvacca@dockerubuntu:simple_nginx$sudo docker run -d --name nginx -p 8080:8080 -v $PWD/website:/var/www/html/website gvacca/simple-nginx add8d371f82f615ebbd121cea804511dcdafac893b14325366744797424fa44c gvacca@dockerubuntu:simple_nginx$ sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES add8d371f82f gvacca/simple-nginx "/usr/sbin/nginx" 11 seconds ago Up 9 seconds 0.0.0.0:8080->8080/tcp nginx gvacca@dockerubuntu:simple_nginx$ ps aux |grep docker root 792 0.0 4.7 693136 23740 ? Ssl Nov04 11:20 /usr/bin/docker daemon root 1977 0.0 2.6 152692 13172 ? Sl 11:26 0:00 docker-proxy - proto tcp -host-ip 0.0.0.0 -host-port 8080 -container-ip 172.17.0.2 -container-port 8080
- 26. Intermediate topics • Volumes •Network (was ‘link’ before 1.9) • Debugging (‘inspect’) • ARG (after 1.9) • Docker on OSX/Windows (Docker Machine) • Interacting with Kernel modules
- 27. Docker orchestration • DockerComposer • Docker Swarm • Google’s Kubernetes • Apache Mesos • Puppet & others
- 28. Let’s start exploringinteraction opportunities between the tools seen so far…
- 29. Puppet to manageJenkins • https://forge.puppetlabs.com/rtyler/jenkins • Manage Jenkins as a service • Install plugins automatically, etc • Or build your own module! • Install .war • Configure Jenkins (config.xml) • Configure jobs ($JENKINS_PATH/jobs/JOB/config.xml)
- 30. Docker as Slavefor Jenkins • Associate specific Docker images to specific Jenkins build jobs • Keep builds clean & reproducible (“Non-event releases”) • Run slave containers on demand - stop them when not needed. • Jenkins Plugin: https://wiki.jenkins-ci.org/display/JENKINS/Docker +Plugin • Other Cloud methods, e.g. Mesos
- 31. Jenkins to buildDocker images • A Jenkins job: • Checks out a Dockerfile • Builds the image • Upload the new image in a repo • May re-use images for new builds (careful about integrity)
- 32. Puppet to manageDocker containers • Module for docker: https://forge.puppetlabs.com/garethr/docker • Installs Docker and required dependencies • Launches the Docker daemon • Sets DNS, users and other configuration items • Pull images from selected repos • Run containers (image + command)
- 33. Run Jenkins insideDocker • Manage Jenkins with a dedicated image + volume with data • From the official Docker registry: https://hub.docker.com/_/jenkins/ docker run -d -p 8080:8080 -p 50000:50000 -v ~:/var/jenkins_home jenkins • TCP 8080: Web UI • TCP 50000: interface to connect slaves
- 34. Docker inside Docker Butyou need the privileged mode
- 35. Conclusion • There aremany opportunities to automate Continuous Integration • CI for server-side apps poses specific challenges • You can’t ignore Configuration Management • Docker represents a huge opportunity • You need to find your own use case and solution
- 36. APPENDIX - Puppetvs Docker • Some stop using Puppet for Configuration Management once they move their apps inside containers • A Dockerfile defines “how” to build an image • A Dockerfile must be distribution-specific (yum or apt?) • Puppet is OS-agnostic: the differences are hidden inside modules
- 37.
- 38. Recommended Books • “TheDocker book”, J. Turnbull, http://www.amazon.co.uk/Docker- Book-Containerization-new-virtualization-ebook/dp/B00LRROTI4 • “Continuous Delivery”, J. Humble, http://www.amazon.com/ Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/ 0321601912 • “Pro Puppet”, J. Turnbull, http://www.amazon.co.uk/Pro-Puppet- Second-Professional-Apress/dp/1430260408
- 39.