Download as PDF, PPTX
More Related Content
Similar to JEEConf 2019 | Let’s build a Java backend designed for a high load
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
JEEConf 2019 | Let’s build a Java backend designed for a high load
- 1. Let’s build aJava backend designed for a high load Alex Moskvin CTO@Plexteq
- 2. About myself • CTO@PlexteqOÜ • Ph.D in information technology area • Interests • Software architecture • High loaded systems • Everything under the hood • AI/ML + BigData • Knowledge sharing ;) • Follow me • https://twitter.com/amoskvin • https://www.facebook.com/moskvin.aleksey 2
- 3.
- 4.
- 5. High load What clients do see: - Low responsiveness of the service -Sporadic errors - Interrupted connections 5
- 6. High load What sysops/devopsdoes see: - Large iowait -High CPU contention - High RAM usage - High number of open files (processes, sockets, threads) - High contention on target data storage - High GC activity 6
- 7.
- 8.
- 9. Problem :: systemlayout 9
- 10. Problem :: keyaspects 1. Interaction with remote/external services 2. Processing data from remote/external services 3. Storing data 4. Handling interaction with clients 10
- 11. Problem :: keystack 11
- 12. Problem :: keystack 12
- 13. Distributed services The 8 fallacies of distributed computing 1. The network is reliable 2.Latency is zero 3. Bandwidth is infinite 4. The network is secure 5. Topology doesn’t change 6. There is one administrator 7. Transport cost is zero 8. The network is homogeneous 13
- 14.
- 15. Distributed services ::example 1 15
- 16. Distributed services ::watch timeouts Issue #1: Code relies on default timeouts Connect Read 16
- 17. Distributed services ::watch timeouts Issue #1: Code relies on default timeouts Connect Read By default timeouts are INFINITE 17
- 18. Distributed services ::watch timeouts Issue #1: Solution 18
- 19.
- 20. Distributed services ::handle errors! Issue #2: No error handling Solution 1. Handle them all 2. Check response code • i.e. 429 is a requirement to stop sending requests 20
- 21. Distributed services ::fail fast Issue #3: Code relies on high availability of external service Solutions: 1. Fail-fast 21
- 22. Distributed services ::circuit breaker Fail-fast with circuit breaker 22
- 23. Distributed services ::circuit breaker Stability pattern used when calling remote functions 23
- 24. Distributed services ::Circuit breaker CLOSED OPEN HALF OPEN If state = OPEN && grace period passed -> retry If request succeeded -> CLOSED Otherwise -> OPEN Service is OK Service is FAILING 24
- 25. Distributed services ::Circuit breaker Fail-fast with circuit breaker 25
- 26. Interaction with externalservices 100 threads concurrently accessing remote failing resource Default (infinite connect timeout) Circuit breaker + 2s timeout 26
- 27. Processing, offloading andstoring data 27
- 28.
- 29.
- 30. Processing data ::deserialization Issue #1: Deserialization is expensive with large payloads 30
- 31. Processing data ::deserialization Issue #1: Deserialization is expensive with large payloads Consequences: 1. Heap saturation -> potential OOM 2. GC pauses -> high latencies 3. Deserialization is sequential -> thread is not doing anything useful until deserialization completes 31
- 32. Processing data ::deserialization Any ideas? 32
- 33. Processing data ::streaming deserialization Issue #1: Deserialization is expensive with large payloads Solutions: 1. Stream based processing 33
- 34. Processing data ::streaming deserialization Your favorite deserializer more likely supports streaming processing already! 34
- 35. Processing data ::streaming deserialization XML • SAX (Streaming API for XML) JSON • Gson • Jackson 35
- 36. Processing data ::streaming deserialization 36
- 37.
- 38.
- 39.
- 40.
- 41. Storing data Our recommendation: 1. Domain model and entity relationships 2.Scaling strategy for your data 3. Approach for achieving optimal read/write performance 41
- 42. Storing data In-heap ACID compliantBASE compliant 42
- 43. Storing data In-heap ACID compliantBASE compliant Time series, monitoring dataCustomer billing dataMonitored device online status (deviceID:status) Our problem: 43
- 44.
- 45.
- 46.
- 47.
- 48. Storing data • No GC involved •Low heap usage (10x time smaller) • Map is shared between multiple JVM processes • Map could be replicated across multiple nodes (commercial feature) 48
- 49.
- 50. Storing data Relational storage. Issues: 1. ACID is expensive 2.Operations are blocking 3. Connection pool is limited 50
- 51. Storing data Relational storage. Solution: 1. Avoid pessimistic locking 2.Transaction isolation > Read Committed is a no-go 3. Scale up! 4. Connection pool implementation matters (HikariCP) 5. Connection pool sizes on app side and on DB size must correlate 6. Use indices wisely 7. Know your execution plans! 51
- 52.
- 53.
- 54.
- 55.
- 56.
- 57.
- 58.
- 59.
- 60. Handling client requests https://docs.spring.io/spring-framework/docs/current/javadoc- api/org/springframework/scheduling/annotation/EnableAsync.html By default, Spring will be searching for an associated thread pool definition: either a unique TaskExecutorbean in the context, or an Executor bean named "taskExecutor" otherwise. If neither of the two is resolvable, a SimpleAsyncTaskExecutor will be used to process async method invocations. https://docs.spring.io/spring-framework/docs/current/javadoc- api/org/springframework/core/task/SimpleAsyncTaskExecutor.html NOTE: This implementation does not reuse threads! Consider a thread-pooling TaskExecutorimplementation instead, in particular for executing a large number of short-lived tasks. 60
- 61.
- 62.
- 63. Handling client requests Pros: 1.System is more predictable • Target resource (i.e. MongoDB) load is managed and constrained with a thread pool (a kind of DOS protection) • Thread allocation is managed and constrained 63
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.