Downloaded 74 times
Uploaded bySplunk
Clear the Mist from your Clouds with Splunk
The document discusses Splunk's approach to cloud and hybrid cloud environments, emphasizing the importance of security, compliance, and operational control in cloud migrations. It outlines customer requirements for cloud migration, such as building unique services and maintaining cost visibility, and introduces Splunk's strategies for effective monitoring and managing hybrid infrastructures. Key considerations include maintaining security governance, optimizing resource usage, and utilizing collaboration with cloud platforms like AWS, Azure, and GCP for enhanced insights and data management.
In this document
Powered by AI
Introduction to Splunk as a key player in cloud migration, presented by Yuval Tenenbaum in 2019.
Challenges in migrating to cloud and hybrid cloud environments, including security, compliance, networking, and financial planning.
Various environments in cloud computing, focusing on customer experience and what businesses aim to achieve during migration.
Splunk's approach for managing hybrid cloud infrastructures, monitoring, and optimizing resources including security and cost.
Methods for collecting data in cloud environments, with various patterns for AWS, Azure, and Google Cloud.
Different deployment architectures for Splunk instances, focusing on cost, efficiency, and search capabilities.
The importance of hybrid monitoring and examples of integrating Splunk with AWS services for security purposes.
Wrap-up of the session with an emphasis on participant feedback.
More Related Content
Similar to Clear the Mist from your Clouds with Splunk
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
Clear the Mist from your Clouds with Splunk
- 1. © 2019 SPLUNKINC.© 2019 SPLUNK INC. Clear the Mist from your Clouds with Splunk SplunkLive London - June 2019 Yuval Tenenbaum Director – SE Architects EMEA
- 2. © 2017 SPLUNKINC. Migration To Cloud & Hybrid Cloud Insights is Top Of Mind
- 3. © 2019 SPLUNKINC. ► Enables Least privileged model at the highest operational control ► Mitigates Risk – lower the ‘blast radius’ of impactful events ► Achieve Agility- deploy & run environments programmatically at scale ► Cost optimisation- clear ‘line of sight’ into the cost of running workloads Hybrid Cloud – Think Differently Legacy Model Least privileged
- 4. © 2019 SPLUNKINC. ► Split Investment may slow down your cloud adoption – Spreading your resources across multiple clouds means that you may not get critical mass or a fast ROI ► Portability - How many of us will actually move workloads around? ► Cloud Broker concept – Putting a “bloatware” between you and your cloud api’s instead of working natively with these cloud API’s Is it Really All Good Stuff? I used to be indecisive now I’m definitely going multi- cloud
- 5. © 2019 SPLUNKINC. Cloud - Same Challenges-Different Environments ► Security • Are we firewalled correctly? • Do we use all necessary security features? ► Compliance • Are we following all published standards? ► Networking • Placed servers on the correct network? ► Financial • Stayed within budget? ► Capacity Planning • Used resources optimally? And all of that in a decentralized Model…
- 6. © 2019 SPLUNKINC. Customer experience??? SAAS Hybrid Everything - What happens when we stack them? ON PREMISES Legacy systems (Mainframe…) Facilities Dev/PreProd Storage Backup Archive DR Security VMs Containers Micro services AWS (Application 1)Access / Security Database StorageDev Compute Containers App engine GCP (Big Data project 1) Dataflow AWS (Archive) Azure (Application 1) VMs Database VM sets Traffic mger
- 7. © 2017 SPLUNKINC. So How Can Splunk Clear up this Cloudy Mist? Know your Clouds…..
- 8. © 2019 SPLUNKINC. ► Splunk has working relationships with AWS, Azure, and GCP ► We have customers successfully running Splunk Enterprise BYOL within AWS, Azure, and GCP ► We have proven strategies to get data in from AWS, Azure, and GCP Cloud Vendor Relationships
- 9. © 2017 SPLUNKINC. Splunk’s Approach to Hybrid Cloud One Consolidated Solution Manage Hybrid Infrastructure Cost, Capacity and Resource Management Cloud Migration Splunk takes the place of the multitude of monitoring tools because sometimes one is better than many. Deploy Splunk in Hybrid setup (on-prem, saas, byol) and deal with Hybrid infrastructure complex monitoring Understand how your resources are performing – and how many are being used – then optimize utilization and billing. Get visibility at all stages of the migration process (landing zones)– whether before, during or long after.
- 10. © 2017 SPLUNKINC. In the Beginning…… Cloud Migration
- 11. © 2019 SPLUNKINC. What Customers Want To Achieve When Migrating to the Cloud ► Build - Differentiate yourself by building unique and valuable services ► Move Fast - From initial idea to a service which can be monetized ► Stay Secure - Make sure that what we build is secure and compliant ▶ Manage Cost – Control what you spend and gain visibility into future cost
- 12. © 2019 SPLUNKINC. Path To Successful Cloud Migration Measure the baseline user experience and performance, as well as define acceptable post-migration levels. Security assessment – build a well architected and compliant landing zones Performance metrics should be closely monitored & compared to the baseline. Throughout the migration, end-to-end monitoring can help SecOps teams stay ahead of any potential risks. Continuous monitoring should be used to measure acceptable metrics and success. Leverage a platform that shows insights into cost, shared services, monitoring, Security & compliance BEFORE DURING AFTER
- 13. © 2019 SPLUNKINC. Challenges With Building & Maintaining Landing Zones ▶ Define & maintain an Account structure ▶ Define your network architecture and monitor it continuously ▶ Define & maintain a security governance and compliance baseline Migrate Land Operate & Optimize
- 14. © 2019 SPLUNKINC. Additional Considerations ▶ Define & maintain centralized logging ▶ Define & maintain Cost Allocation
- 15. © 2019 SPLUNKINC. How Can Splunk Help (1)? ▶ Tell you who is accessing your accounts, from where and what are they doing?
- 16. © 2019 SPLUNKINC. How Can Splunk Help (2)? ▶ Tell you if anyone is breaking your security policies? • Is encryption used everywhere • Has the root account has MFA enabled • Suspicious AWS S3 Activities • IAM Password policies are kept as you defined in your security baseline?
- 17. © 2019 SPLUNKINC. How Can Splunk Help (3)? ▶ Help you understand your network topology and gain visibility into who is trying to access it ▶ Help you gain visibility into performance & right sizing of your key workloads ▶ Help you understand historic and future cost
- 18. © 2019 SPLUNKINC. AWS Analytic Stories - ES Content Updates
- 19. © 2019 SPLUNKINC. Migration Dashboards
- 20. © 2017 SPLUNKINC. So How Do We Collect Cloud Data to do this Hybrid Monitoring?
- 21. © 2017 SPLUNKINC. Getting Data In Cloud Patterns
- 22. © 2017 SPLUNKINC. General Getting Data In Routes Pull or Push, Add-Ons or Serverless Poll/Request API Data Data Cloud Serverless Code Add-On HEC “Push”
- 23. © 2017 SPLUNKINC. GDI : AWS
- 24. © 2019 SPLUNKINC. It May Look a Bit Complicated
- 25. © 2019 SPLUNKINC. ► AWS Config can be pulled with a Splunk Heavy Forwarder with the SQS Based S3. Anything via CloudWatch Logs or CW events, can be pushed with Kinesis Firehose to Splunk AWS Pull vs. Push Config Events SNS Topic Notification SQS Subscription Notification Pulls Event from S3 Bucket Splunk Pull SQS Notification HEC PushPull CloudWatch Logs
- 26. © 2019 SPLUNKINC. AWS Source Matrix There are many options to GDI in AWS but Splunk can help Data Type Recommended Input Type Billing Billing CloudWatch CloudWatch CloudFront Access Logs SQS based S3 Config SQS based S3 Config Rules Config Rules Description Description ELB Access Logs SQS based S3 Inspector Inspector CloudTrail SQS Based S3 S3 access logs SQS Based S3 VPC Flow Logs (CW Logs) Kinesis With SQS Based S3 you can scale out data collection by configuring multiple inputs to ingest logs from the same S3 bucket without creating duplicate data. Kinesis Firehose is recommended for CloudWatch Logs data collection
- 27. © 2017 SPLUNKINC. GDI : Azure & O365
- 28. © 2019 SPLUNKINC. 3 Log Types in Azure 1) Control/Management, 2) Data Plane, 3) Processed Events Control: System Configuration and Management Data Plane: Provisioned Service and Diagnostic Data Processed Events: Alerts & Recommendations
- 29. © 2019 SPLUNKINC. { REST } Storage Event Hub
- 30. © 2019 SPLUNKINC. ► Splunk can pull data from Azure using a Heavy Forwarder and collect data from either the MS Blob or a REST API using the modular input. Azure can push data using the Event Hub to Azure Functions which can be sent to Splunk’s HEC. Azure Pull vs. Push MSBlob HEC PushPull Splunk Indexers Activity Monitor Event Hub Azure Function Event Hub
- 31. © 2019 SPLUNKINC. Azure Add-on Landscape
- 32. © 2019 SPLUNKINC. Getting O365 Data In Azure Active Directory Application OAUTH2 REST Splunk Add-on for Microsoft O365 Office 365
- 33. © 2017 SPLUNKINC. GDI : Google Cloud
- 34. © 2019 SPLUNKINC. Getting GCP Data In REST Splunk Add-on for Google Cloud Platform Billing PubSub Monitoring StackDriver
- 35. © 2019 SPLUNKINC. ► Initial: • Most customers will generate around 1-10GB when they are setting up their Public Cloud deployments and enabling services. • As they mature - 10-50GB. ► More instances and deployed apps in Cloud, 50-200GB. ► Most customers are 100-200GB / day of Public Cloud data. ► All-in Cloud Companies : 500GB-1TB range. ► Less common >1TB ► O365 - ~400 to 500 KB per user per day (50K users = 25 GB/day) ► Best way to analyze the amount of data is to spin-off a test environment and look at the numbers. How Much Data?
- 36. © 2017 SPLUNKINC. Collection Deployment Architectures
- 37. © 2019 SPLUNKINC. ► Central Splunk Instance • One Instance to manage – lower “Instance/Storage” costs • Data egress cost considerations (data transfers from each cloud) • Local or Distributed Heavy Forwarders ► Splunk Instance per Cloud, 1 “Master” view • One Instance in each Cloud – potential higher “Instance/Storage” cost • Management of Splunk in each Cloud • “Master” Search Head needed for Hybrid Search – latency impact • Lower egress cost ► Hybrid • Mix of both options balancing out Costs/Hybrid Search Deployment Architecture 3 Patterns
- 38. © 2019 SPLUNKINC. Option 1 Public/Private Cloud / Splunk Cloud Single Splunk InstanceHeavy Forwarder (Add-On) Heavy Forwarder (Add-On) Heavy Forwarder (Add-On) Note Options for Serverless/HEC input direct to Central Instance Cloud Data
- 39. © 2019 SPLUNKINC. Option 2 Public/Private Cloud Distributed Hybrid SearchSplunk Indexer(s) Splunk Indexer(s) Splunk Indexer(s) Search Head Search Results
- 40. © 2019 SPLUNKINC. Option 3 Distributed Search Splunk Indexer(s) & Master Search Splunk Indexer(s) Heavy Forwarder (Add-On) Cloud Data Search Results
- 41. © 2019 SPLUNKINC.© 2017 SPLUNK INC. © 2017 SPLUNK INC. OUR MISSION ….Including Cloud data!
- 42. © 2019 SPLUNKINC. Hybrid Monitoring Collect & store machine data generated by on-premises IT sources and public cloud sources simultaneously, and can correlate across both to monitor, alert, analyse, troubleshoot and investigate.
- 43. © 2017 SPLUNKINC. Pulling it all together: Example Cloud Innovation, Integration and Use Case AWS Security Hub + Splunk Phantom Bi-Directional Integration
- 44. © 2019 SPLUNKINC. AWS Security Hub - Findings
- 45. © 2019 SPLUNKINC. Phantom - EC2 Instance- Investigate & Notify
- 46. © 2019 SPLUNKINC. Geo Location & IP Reputation
- 47. © 2019 SPLUNKINC. Prompting The Analyst- Quarantine Instance
- 48. © 2019 SPLUNKINC. Phantom- Isolate ES2 Instance Playbook
- 49. © 2019 SPLUNKINC.
- 50. © 2019 SPLUNKINC. Back To AWS Security Hub
- 51. © 2019 SPLUNKINC.© 2019 SPLUNK INC. Don't forget to rate this session in the .conf18 mobile app Thank You.