From the course: Cloud Hadoop: Scaling Apache Spark
Join today to access over 24,900 courses taught by industry experts.
Spark session objects - Apache Spark Tutorial
From the course: Cloud Hadoop: Scaling Apache Spark
Spark session objects
- [Instructor] As we start programming Apache Spark, we will be working with a SparkSession. And the way that we reference that depends on the version of Spark we're working with. Now, if you're working with Spark 1.x, you will see SQLContext, capitalized the way it is here, or sc. If you're working with Spark 2.0, you'll see the word "spark" with a small s. Now, the reason I'm calling this out is because I've seen a lot of inconsistency in samples. I've also seen that the word "Spark" in 2.0 doesn't fully implement all the possible methods that I've worked with in SQLContext, or sc. So frankly, sometimes you see 'em mixed in the code, and it's a little bit confusing, so I wanted to call it out here.
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.