Download as PDF, PPTX
Uploaded byCathrine Wilhelmsen
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics (Global Azure Norway 2021)
This document introduces data integration in Azure Synapse Analytics, emphasizing the importance of data in business intelligence and analytics projects. It details the functionalities of Azure Synapse Analytics, including data ingestion, transformation, and integration capabilities without needing to write code. The session also highlights the collaborative nature of the platform for data engineers, analysts, scientists, and consumers.
More Related Content
![[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...](https://cdn.slidesharecdn.com/ss_thumbnails/draganberic-lakehousearchitecturewithdeltalakeanddatabricks-221130080712-6e817e95-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics (Global Azure Norway 2021)
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics (Global Azure Norway 2021)
- 1. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Pipelines & Data Flows: Introduction to Data Integration in Azure Synapse Analytics Cathrine Wilhelmsen Global Azure Norway · April 16th, 2021
- 2. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Session Abstract Do you regularly need to get data for your projects? Yep! 🙋♀️ Data is at the core of every Business Intelligence, Data Science, and Machine Learning project. You need data to understand what has happened in the past, to predict what may happen in the future, to discover patterns and anomalies, and to gain the insight necessary for making faster and better decisions. But before you can do any of those things, you need to ingest, store, transform, integrate, and prepare your data. Guess what? You can do all of those things in Azure Synapse Analytics – without having to write any code! In this session, we will cover the fundamentals of data integration in Azure Synapse Analytics. First, we will discuss when Azure Synapse Analytics is the right tool of choice. Then, we will go through what Pipelines and Data Flows are, and when to use them. Finally, we will see how easy it is to ingest and transform data both on-premises and in the cloud.
- 3. © 2021 CathrineWilhelmsen (hi@cathrinew.net) @cathrinew cathrinew.net
- 4. © 2020 CathrineWilhelmsen (hi@cathrinew.net) Data Warehousing Business Intelligence Artificial Intelligence Big Data and Analytics Machine Learning Data Science
- 5. © 2020 CathrineWilhelmsen (hi@cathrinew.net) Data Warehousing Business Intelligence Artificial Intelligence Big Data and Analytics Machine Learning Data Science
- 6. © 2020 CathrineWilhelmsen (hi@cathrinew.net) What? When? Why?
- 7. © 2020 CathrineWilhelmsen (hi@cathrinew.net) Ingest Store Transform Integrate Prepare
- 8. © 2020 CathrineWilhelmsen (hi@cathrinew.net)
- 9. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Azure Synapse Analytics What? Pipelines & Data Flows How? Data Integration When? …the next 45 minutes…
- 10.
- 11. © 2021 CathrineWilhelmsen (hi@cathrinew.net) What is Azure Synapse Analytics? All-in-one platform for analytical projects: • Data Lake (All Data) • Data Warehouse (Relational Data) • Data Analytics (SQL, Spark) • Data Integration (Ingest, Transform)
- 12. © 2021 CathrineWilhelmsen (hi@cathrinew.net) What is Azure Synapse Analytics? Deeply integrated with other services: • Azure Purview • Azure Machine Learning • Azure Cosmos DB • Power BI
- 13. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Who can use Azure Synapse Analytics? Built for collaboration between: • Data Engineers • Data Analysts • Data Scientists • Data Consumers
- 14. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Integration Data
- 15. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Data Integration in Azure Synapse Analytics Code-First Scripts, Notebooks Designer-First Pipelines, Data Flows
- 16. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Data Integration in Azure Synapse Analytics Copy Data Transform Data
- 17.
- 18. © 2021 CathrineWilhelmsen (hi@cathrinew.net) What are Pipelines?
- 19. © 2021 CathrineWilhelmsen (hi@cathrinew.net) What are Activities?
- 20. © 2021 CathrineWilhelmsen (hi@cathrinew.net) What are Datasets?
- 21. © 2021 CathrineWilhelmsen (hi@cathrinew.net) What are Linked Services?
- 22. © 2021 CathrineWilhelmsen (hi@cathrinew.net) What are Data Flows?
- 23. © 2021 CathrineWilhelmsen (hi@cathrinew.net) What are Triggers?
- 24.
- 25. © 2021 CathrineWilhelmsen (hi@cathrinew.net)
- 26. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Code-First Scripts, Notebooks Designer-First Pipelines, Data Flows
- 27.
- 28. © 2021 CathrineWilhelmsen (hi@cathrinew.net) Azure Synapse or Azure Data Factory?
- 29. © 2021 CathrineWilhelmsen (hi@cathrinew.net) @cathrinew cathrinew.net hi@cathrinew.net cathrinew.net/adf