Download as PDF, PPTX
Uploaded byYan Cui
How to choose the right messaging service for your workload
The document provides guidance on selecting the appropriate AWS messaging service based on workload requirements, focusing on event-driven architecture and the various service options like Kinesis, EventBridge, SNS, and SQS. It highlights key considerations such as scaling constraints, cost efficiency, error handling, and observability in messaging systems. Additionally, it outlines the differences between task and event-driven messaging and the importance of understanding trade-offs in service capabilities.
More Related Content
![[Azure Governance] Lesson 4 : Azure Policy](https://cdn.slidesharecdn.com/ss_thumbnails/azuregovernance-lesson4-azurepolicy-190519183014-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to How to choose the right messaging service for your workload
![Choosing the right messaging service for your serverless app [with lumigo]](https://cdn.slidesharecdn.com/ss_thumbnails/choosingtherightmessagingserviceforyourserverlessappwithlumigo-201217193955-thumbnail.jpg?width=640&height=640&fit=bounds)
How to choose the right messaging service for your workload
- 1. How to choosethe right messaging service for your workload
- 3.
- 4.
- 5.
- 6.
- 8. AWS services arethe new data structures. Understanding their trade-offs is the new big O notation.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21. picking the rightAWS service is a valuable skill
- 22.
- 23.
- 25. What are thescaling constraint for the messaging service?
- 26. Google “{service name}quotas”
- 30. How does Lambda’s concurrencyscale with throughput?
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38. more concurrency isnot always better…
- 39. if you want… maximum throughput precisecontrol over throughput SNS EventBridge Kinesis Provisioned
- 40. if you want… maximum throughput precisecontrol over throughput SNS EventBridge Kinesis Provisioned
- 41.
- 42. Reserved Concurrency aretaken out of the available regional Lambda concurrency
- 43. managing Reserved Concurrencyfor many functions is dif fi cult and error prone, easy to create more problems than it solves
- 44.
- 45. always factor scaleinto the equation
- 46. $10.836 1 msg/s fora month, 1KB per msg 1 x 60s x 60m x 24hr x 30days @ $0.014 per mil + 24hrs x 30days @ $0.015 per shard per hr $2.592 SNS SQS EventBridge Kinesis 1 x 60s x 60m x 24hr x 30days @ $1.00 per mil 1 x 60s x 60m x 24hr x 30days @ $0.40 per mil 1 x 60s x 60m x 24hr x 30days @ $0.50 per mil $1.037 $1.296
- 47. Kinesis on-demand modepricing
- 48. $10.836 1 msg/s fora month, 1KB per msg 1 x 60s x 60m x 24hr x 30days @ $0.014 per mil + 24hrs x 30days @ $0.015 per shard per hr $2.592 SNS SQS EventBridge Kinesis Provisioned 1 x 60s x 60m x 24hr x 30days @ $1.00 per mil 1 x 60s x 60m x 24hr x 30days @ $0.40 per mil 1 x 60s x 60m x 24hr x 30days @ $0.50 per mil $1.037 $1.296 Kinesis On-Demand 1kb x 60s x 60m x 24hr x 30days @ $0.08 per GB ingested + 24hrs x 30days @ $0.04 per stream per hr $28.998
- 49. $47.088 1,000 msg/s fora month, 1KB per msg 1000 x 60s x 60m x 24hr x 30days @ $0.014 per mil + 24hrs x 30days @ $0.015 per shard per hr $2592.00 SNS SQS EventBridge Kinesis Provisioned 1000 x 60s x 60m x 24hr x 30days @ $1.00 per mil 1000 x 60s x 60m x 24hr x 30days @ $0.40 per mil 1000 x 60s x 60m x 24hr x 30days @ $0.50 per mil $1036.80 $1296.00 Kinesis On-Demand 1000kb x 60s x 60m x 24hr x 30days @ $0.08 per GB ingested + 24hrs x 30days @ $0.04 per stream per hr $226.55
- 50. services that chargeby uptime are order(s) of magnitude cheaper when running at scale
- 51. National broadcaster ofFinland
- 52. National broadcaster ofFinland 500+ millions events per day, peaks at 600K+ messages/min
- 53. National broadcaster ofFinland 500+ millions events per day, peaks at 600K+ messages/min Anahit Pogosova
- 54. National broadcaster ofFinland 500+ millions events per day, peaks at 600K+ messages/min Anahit Pogosova decisions driven by cost ef fi ciency
- 55. National broadcaster ofFinland 500+ millions events per day, peaks at 600K+ messages/min Anahit Pogosova
- 56. National broadcaster ofFinland 500+ millions events per day, peaks at 600K+ messages/min Anahit Pogosova fallback in case Kinesis is down
- 57.
- 59.
- 60. DLQs only capturethe failed event
- 61.
- 64.
- 68. prefer Lambda destinationover DLQs (you can use both together, but there’s no clear reason for doing so)
- 71.
- 72.
- 73.
- 74. A measure ofhow well the internal state of an application can be inferred from its external outputs
- 75.
- 78. X-Ray doesn’t tracethrough many popular messaging services.
- 79. X-Ray doesn’t tracethrough many popular messaging services. Need separate solution (e.g. correlation IDs) for Lambda logs.
- 81. all the indirectinvocations are accounted for
- 83. all the Lambdalogs from the transaction in chronological order
- 86.
- 87. No manual instrument. Bettersupport for messaging services. Supports containers and Lambda.
- 88. “which messaging serviceshould I use?”
- 90.
- 91. Task Event “Something happened” “Godo a thing”
- 92. Task Event “Something happened” “Godo a thing” Intended for a target receiver.
- 93. Task Event “Something happened” “Godo a thing” Intended for a target receiver. Often expects an answer.
- 94. Task Event “Something happened” “Godo a thing” Intended for a target receiver. Publishers are obvious to subscribers. Often expects an answer.
- 100.
- 101. EventBridge SNS FIFO DoStuff!
- 102. EventBridge SNS FIFO DoStuff! order is lost
- 103.
- 104. EventBridge SNS FIFO SQSFIFO Do Stuff (in order)! Other subscribers (not my problem!)
- 106.
- 107. EventBridge Kinesis Lambda Other subscribers (notmy problem!) send domain events
- 108.
- 109.
- 110.
- 112. https://theburningmonk.com/hire-me Advise Training Delivery “Fundamentally, Yanhas improved our team by increasing our ability to derive value from AWS and Lambda in particular.” Nick Blair Tech Lead
- 113.