From the course: End-to-End Real-World Data Engineering Project with Databricks

Unlock this course with a free trial

Join today to access over 24,900 courses taught by industry experts.

Create a Databricks notebook to load a CSV file to the customer delta table

Create a Databricks notebook to load a CSV file to the customer delta table

From the course: End-to-End Real-World Data Engineering Project with Databricks

Start my 1-month free trial Buy for my team

Create a Databricks notebook to load a CSV file to the customer delta table

“

- [Instructor] To load our customer data, we can continue on this notebook itself or we can create a new notebook as well. To make the things more structured, let's create a new notebook. So for that, let's click on Notebook, and say Bronze_layer_Customer_load. Now in this notebook, we're going to load our customer data. Now you remember where we have uploaded our customer data? We went to Catalog, Browse DBFS, we went to this customer_data, and here is our customer.csv file. Now right click on this and copy the path, because from this location, we're going to load the data. Now go back to our notebook again. Now to load the data, let's just say the file path is equal to the path which we have copied. Now to create the DataFrame, say df = spark.read.csv. Because we are reading the file in the CSV format, let's give the file path. And we want to do header=True as well, because the first line in our CSV file is of a header. And I also do inferSchema=True, because this will automatically…

Contents