Download as PDF, PPTX
More Related Content
Similar to Put your Java apps to sleep? Find out how - John Matthew Holt (Waratek)
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
Put your Java apps to sleep? Find out how - John Matthew Holt (Waratek)
- 1. Put your Java Apps to Sleep? Find out How… John Ma(hew Holt | JAX London 2013 ! © Copyright Waratek 2013
- 2. John Matthew Holt! • CTO and co-‐Founder of Waratek • Been doing JVM engineering for 10 years • Invented “Java VirtualizaCon” to transform how JVMs operate in virtualized and cloud-‐ compuCng environments • Lead the design of the first Type-‐1 VMM for a JVM © Copyright Waratek 2013
- 3. What will wecover?! • First some background… – Why virtualize INSIDE the JVM? – Think “VMware for Java” • PuSng you Java Apps to Sleep… – Why are Java apps Cred? (or restated: why should they be slept?) – Let your apps nap: “wake-‐on-‐LAN for JVCs” © Copyright Waratek 2013
- 4. First some background…! • Why virtualize INSIDE the JVM? © Copyright Waratek 2013
- 5. Because…! • Because JVMs make bad bedfellows with HW hypervisors (HV) • A JVM thwarts HW hypervisor features like: – The GC in a JVM thwarts memory-‐overcommit in a HW HV – The GC’s heap memory, and the JIT’s permgen/ codecache, thwarts ‘transparent page sharing’ (memory de-‐dup) in a HW HV – Dedicated GC and JIT for EVERY Java applicaCon wastes lots and lots of CPU Cme and memory capacity doing the same thing over, and over, and over, and… © Copyright Waratek 2013
- 6. Today’s JVMs areOLD! • Were designed ~20 years ago, and haven’t changed design since then • JVM design PREDATES the major 2 trends of last decade: virtualizaCon and cloud-‐compuCng • Only 2 innovaCons in JVMs in 20 years: – Beder JIT compilers – Concurrent GCs – ….all blindly focused on single-‐tenant performance at the expense of mulC-‐tenant efficiency © Copyright Waratek 2013
- 7. Server Java hasproblems! • JVMs are grossly inefficient users of CPU and memory – A JVM assumes it can use all available CPU and memory capacity for itself and its single applicaCon, eg: – 1 JBoss can consume up to 500MB RSS with no apps – 1 Tomcat can consume up to 250MB RSS with no webapps – Running mulCple JVMs on a server rapidly exhausts physically memory while CPU uClisaCon averages 5% • >100 different JVM versions makes compliance/support a NIGHTMARE • Security vulnerabiliCes and compromises are EVERYWHERE inside Java © Copyright Waratek 2013
- 8. VMWare does notvirtualize Java well! • Realworld stats using VMware: – >80% of Server JVMs have <=2GB of heap memory • Yet actual applicaCon heap uClisaCon averages <20% – A 2GB JVM heap needs ~3GB of OS RSS memory, which in turn needs ~4GB of VMWare tenant memory • …reducing actual applicaCon memory uClisaCon to <10% – CPU uClisaCon running Java on Vmware averages <7% • Yet physical memory is exhausted because of running mulCple JVMs © Copyright Waratek 2013
- 9. Java is acompliance nightmare! • Realworld stats: – 93% of organisaCons are running versions of Java at least 5 years old – The average organisaCon has 51 disCnct versions of Java installed – <1% of installed Java is upgraded to the latest version • Recurring enterprise conversaCon: – QuesCon: “what version of Java do you run?” – Answer: “every version except the latest version!” © Copyright Waratek 2013
- 10. Java is asecurity nightmare! • Realworld stats: – In last 12 months alone, >250 CVE alerts issued by US-‐ CERT/NIST for Java – The most prevalent version of Java has 96 “perfect 10” vulnerabiliCes – “There are virtually no modern versions of Java without any known severe vulnerabiliCes” (source: Bit9) • CriCcal Security Patch Updates are released so oven that developers and admins can’t keep up © Copyright Waratek 2013
- 11. Virtualization is theanswer…! • …but HIGHER up the Server Stack • VirtualizaCon is creeping up the Server Stack already – x86 HW got virtualized in 1999 (VMware et al) – x86 OS got virtualized in 2006 (OpenVZ, LXC, et al) – …now it’s Java’s turn in 2013 • Virtualizing Java means virtualizing at the TOP of the Server Stack, whereas VMware virtualizes at the BOTTOM © Copyright Waratek 2013
- 12. Cost-per-tentant to zero! • The higher up virtualizaCon goes up the Server Stack, the cheaper a “tenant” becomes • Virtualizing INSIDE the JVM gets close to “zero-‐ cost tenants”: – Helloworld in <1MB of tenant memory – Examples.war (Tomcat demo webapps) in <5MB of tenant memory – …Java becomes CHEAPER than python/ruby/perl when the JVM itself gets virtualized! © Copyright Waratek 2013
- 13. Virtualization == Security! • Virtualizing the JVM makes Server Java secure and compliant – Virtualized Java Apps run as “tenants” in a shared JVM – Tenants run in Java Virtual Container (JVC) “jails” – Tenants are isolated and “locked-‐down” – Tenants can run any Java version • Doesn’t mader what “Java version” a tenant uses, it gets the latest JVM • Means a “Java version” becomes just a tenant API for the convenience of developers • Admins control the REAL version at the JVM level so that tenants (developers) can stop caring about “versions” altogether © Copyright Waratek 2013
- 14. First some background…! • Think “VMWare for Java” © Copyright Waratek 2013
- 15. A hypervisor INSIDEa JVM! • Waratek has wriden a type-‐1 VMM that runs INSIDE the Oracle HotSpot and OpenJDK JVMs – >300,000 lines of Java code, <5K lines of C-‐code – Can easily run on any OS/CPU pair (only Linux/x86 supported so far) – Does EVERYTHING VMware does, but INSIDE the JVM • • • • • • Virtual FS Virtual networks/IPs/NICs CPU pinning and quotas Memory overcommit, elasCc memory, memory quotas I/O QoS, rate-‐limiCng Etc… – …all with ZERO code change to Java applicaCons © Copyright Waratek 2013
- 16. A real VMMinside the JVM! © Copyright Waratek 2013
- 17. VMWare for Java! JavaCode JVC Java App java.* APIs SecurityManager Waratek Java Hypervisor Process C/C++ Code JNI Libs • Waratek virtualizes Java exactly like VMware virtualizes HW – Waratek inserts a “Java Hypervisor” between the JVM and the Java App + java.* APIs + SecurityManager – The Java app PLUS java.* APIs and SecurityManager run inside a Java Virtual Container (JVC) “jail” – For a Java App to go outside of its JVC “jail” it has to “trap” to the Java Hypervisor Waratek JVM © Copyright Waratek 2013
- 18. VMWare for Java! JavaCode JVC Java App java.* APIs SecurityManager Waratek Java Hypervisor Process C/C++ Code JNI Libs Waratek JVM • Every incoming and outgoing access by a JVC is intercepted and checked by the Java Hypervisor, e.g: – – – – – – – – – – – – – – Class loading intercepted and checked JNI invocaCon intercepted Parameter and return value checks File access (delete/copy/move) checks Network operaCon checks IO rate-‐limiCng and flow-‐control CPU Cme control Memory consumpCon control Buffer overflow checks Null pointer checks Type safety checks OS access checks JNI Libraries isolated … © Copyright Waratek 2013
- 19. Slots into existingVM tools! • You can manage the Java hypervisor exactly like you manage exisCng HW hypervisors: – – – – SSH CLI RedHat LibVirt/KVM integraCon (beta) VMWare vCAC integraCon shortly OpenStack integraCon shortly • Plus other management interfaces – – – – – HTTP REST API Local and remote JMX APIs /proc/PID/* pseudo-‐filesystem for JVM and each JVC Vmware CloudFoundry integraCon (beta) RedHat OpenShiv integraCon soon © Copyright Waratek 2013
- 20. A HelloWorld JVCin LibVirt XML! © Copyright Waratek 2013
- 21. JVCs make Javasecure! Java Code JVC Java App java.* APIs SecurityManager Waratek Java Hypervisor Process C/C++ Code JNI Libs Waratek JVM • If a Java App inside a JVC compromises either/both of java.* APIs or SecurityManager, then only that JVC is effected and NOT the Java Hypervisor or JVM or other JVCs • JNI Libraries are isolatable in their own OS process • Thus a compromised JVC cannot: – – – – – – – Insert malicious code outside its JVC Delete/move/copy files outside its VROOT Trigger buffer overflows Do null-‐pointer segmentaCon faults Perform unsafe heap memory access Fork OS processes etc… © Copyright Waratek 2013
- 22. “Come in, we’reopen”! • The Java Hypervisor has an open API for unlimited customizaCon and extensibility • Anyone can write their own PURE-‐JAVA plugins to the Java hypervisor to customize ANYTHING about JVCs, eg: – Make a DropBox.com account the virtual filesystem for a JVC – Create a virtual LAN between distributed JVCs anywhere in the world – Clone live network I/O of an acCve JVC to a test JVC for side-‐by-‐side applicaCon tesCng © Copyright Waratek 2013
- 23. Example hypervisor plugin! • Waratek wrote ElasCcat as one example of a Java Hypervisor plugin • ElasCcat is open source, is <1000 lines of Java code, and loads into the Java hypervisor at startup • ElasCcat virtualizes Apache Tomcat so it becomes the world’s first mulC-‐tenant JavaEE container: – Tomcat runs in one JVC and each webapp runs in it’s own dedicated JVC – If one webapp crashes it won’t crash any other JVC – …all this achieved with ZERO code change to Tomcat or webapps, and without them knowing they’ve been virtualized! © Copyright Waratek 2013
- 24.
- 25. Putting Java tojSleep…! • Why are Java apps Cred? (or: why should they be slept?) © Copyright Waratek 2013
- 26. Java Apps aretired!! • Realworld stats: – >50% of PaaS apps are “dead bone idle” at any given Cme – The average Dev/Test environment runs at <10% sustained uClizaCon – Lots and lots of enterprise Java apps are only used periodically, e.g. end-‐of-‐week, end-‐of-‐month… • But these apps have SLAs and can’t be shut-‐off! © Copyright Waratek 2013
- 27. Fact from thefield:! • So frustrated was one Fortune500 admin that he shut-‐off a bunch of Java apps that had been idle for 2 weeks (no TCP traffic) and waited to see what would happen… the results were: – …a small number of app-‐owners noCced immediately (1-‐2 days) – …the largest number of app-‐owners noCced within 1-‐2 weeks – …and a small number of app-‐owners NEVER noCced at all! © Copyright Waratek 2013
- 28. Wouldn’t it begreat if…! • …you could “compress” an idle app and re-‐ awaken it instantly WITHOUT THE APP KNOWING? • …and without the END-‐POINTS knowing? • …and without ever RESTARTING the app later? © Copyright Waratek 2013
- 29. Actually it exists…almost!! • The technology for this nearly already exists: – It’s called “wake-‐on-‐LAN” (WoL) • …but there’s a problem: WoL only works at the HARDWARE level, so is of NO USE in mulCtenant virtualized cloud-‐compuCng environments • Think about it: – As long as ONE tenant on a server is acCve then the server HAS to stay alive and WoL CANNOT be used! © Copyright Waratek 2013
- 30. Putting Java tojSleep…! • Let your apps nap: wake-‐on-‐LAN for JVCs © Copyright Waratek 2013
- 31. “Wake-on-LAN for JVCs”! • What’s needed is a NEW kind of WoL that “naps” single idle tenants on a mulCtenant server… • …and reclaims the memory/CPU/IO quotas that were provisioned for the now-‐napped app, to be used to run other new apps • This is what Waratek jSleep does for Java apps! © Copyright Waratek 2013
- 32. jSleep in aSlide! • Waratek jSleep works inside the Java Hypervisor to do four things: – 1) monitor the acCvity of individual apps (JVCs) – 2) put idle JVCs to sleep aver defined inacCvity thresholds and reclaim their CPU/mem/IO quotas – 3) monitor the I/O connecCons (TCP/UDP Sockets, ServerSockets, Files, etc) of a sleeping JVC – 4) and awaken a sleeping JVC when I/O acCvity is detected, reinstaCng reclaimed quotas • …it’s wake-‐on-‐LAN for JVCs! © Copyright Waratek 2013
- 33. jSleep is VeryPowerful! • Lots of “knobs and dials” make it possible to define sophisCcated sleep/awake profiles that respond to: – CPU acCvity – Network/IO acCvity – IO Cmeouts – Wall-‐clock Cme – …or just about anything else • THERE’S ALSO a built-‐in network profile that allows for immediate configuraCon via the JIRSH CLI © Copyright Waratek 2013
- 34. Simple jSleep Example! • Set a jSleep policy based on network traffic of defined sockets/ports, eg: – jirsh # sleepinfo 1 –-set network=120000,include_local=:8080,10.1.0.8:4444 – Above command: • Suspends JVC 1 aver the network interface is idle 120s • The include_local param is a series of host:port combinaCons that specify what traffic of remote hosts and/or local ports is allowed to wake the JVC, so in above: – Traffic from any remote host to local port 8080 will wake the JVC – Traffic from remote host 10.1.0.8 to local port 4444 will wake the JVC © Copyright Waratek 2013
- 35. jSleep API in60 Seconds! • 2 packages, 8 classes: – cloudvm.jsleep.event.* • • • • BytesReceivedEvent CpuEvent NetworkIdleEvent TimeoutEvent – cloudvm.jsleep.profile.* • • • • JSleepProfile JSleepProfile.MemoryType JSleepProfileState NetworkProfile © Copyright Waratek 2013
- 36. jSleep API in60 Seconds! • Main class: cloudvm.jsleep.profile.JSleepProfile – Every JVC can have one of these – Defines the two CORE methods: • checkSuspend( VirtualContainerMXBean ) • checkResume( VirtualContainerMXBean ) – Is an abstract class so anyone can write their own custom sleep/resume profile – Loaded and configured using JMX (local or remote) © Copyright Waratek 2013
- 37. Custom jSleep APIExample! • Here’s a custom jSleep profile… public boolean checkSuspend(VirtualContainerMXBean paramVirtualContainerMXBean) { Date currentDate = new Date(); if(currentDate.aver(sleepDate) && currentDate.before(resumeDate)) { return true; } else { return false; } } public boolean checkResume(VirtualContainerMXBean paramVirtualContainerMXBean) { Date currentDate = new Date(); if(currentDate.before(sleepDate) || currentDate.aver(resumeDate)) { return true; } else { return false; } } © Copyright Waratek 2013
- 38. What happens when…! • …a JVC sleeps?: – The threads of the JVC are suspended on the OS kernel (but NOT destroyed) – The JVC’s dead objects are asynchronously reclaimed by the GC – If the JSleepProfile was configured with MemoryType.VARIABLE, the JVCs unused memory quota will be reclaimed for use by other JVCs – All open file descriptors will be kept open – …all this in <1 second © Copyright Waratek 2013
- 39. What happens when…! • …a JVC wakes up?: – The threads of the JVC are resumed on the OS kernel – If the JSleepProfile was configured with MemoryType.VARIABLE, the JVM will reinstate the prior memory quota of the JVC – Any file descriptors with received but unread data will now be “readable” by the applicaCon – …all this in <1 second © Copyright Waratek 2013
- 40. Why you shouldcare…! • Dev/Test, PaaS, and middleware environments are characterized by LOTS of “applicaCon idleness” • Idle apps today are treated NO DIFFERENTLY than acCve, busy apps • Years ago H/W wake-‐on-‐LAN was an effecCve technology for powering down idle servers… but it no longer works in mulCtenant cloud infrastructure • jSleep is “wake-‐on-‐LAN for JVCs” and can increase the number of Java apps-‐per-‐server by up to 10x – …this means a reducCon in server sprawl and OpEx up to 90% © Copyright Waratek 2013
- 41. Summary! • Recap and links… © Copyright Waratek 2013
- 42. Biggest change toJava in yrs! • A hypervisor inside the JVM transforms Java with: – Ultra-‐liteweight JVCs for near zero-‐cost tenants – JVC “jails” for Java security and isolaCon – Mixed-‐version JVCs for version consolidaCon and legacy compliance – jSleep (“Wake-‐On-‐LAN for JVCs”) for near-‐zero footprint of idle tenants © Copyright Waratek 2013
- 43. Links! • General informaCon about the Waratek JVM – hdp://www.waratek.com • jSleep, JVI (Java VirtualizaCon Interface), and JMX APIs – hdp://javadoc.waratek.com • ElasCcat source code – hdps://github.com/waratek/elasCcat-‐driver © Copyright Waratek 2013
- 44.