From the course: Hands-On Introduction: Data Engineering

Unlock the full course today

Join today to access over 24,900 courses taught by industry experts.

A real-world data engineering task

A real-world data engineering task

From the course: Hands-On Introduction: Data Engineering

Start my 1-month free trial Buy for my team

A real-world data engineering task

“

- [Instructor] As a data engineer, you are the Swiss Army knife of the modern enterprise. You have the ability to simplify complex data-centric requirements into discreet and manageable tasks. In addition to technical knowledge, you also have a deep understanding of the business context of your organization. Your work often involves taking vague requests and turning them into repeatable workflows. For example, analysts might ask for pipelines to perform healthcare analytics or understand population movement. Each of these tasks may differ in size, depth, or urgency, but they all share one common element. They're able to be distilled into the ETL model. To see this in practice, let's look at a real world data engineering example. One day a business manager asks you, as a data engineer, the following. "Every day, I download a file from the internet containing the list of top level domains. I ask my team to give me a…

Contents