%md-sandbox

<div style="text-align: center; line-height: 0; padding-top: 9px;">

  <img src="https://databricks.com/wp-content/uploads/2018/03/db-academy-rgb-1200px.png" alt="Databricks Learning" style="width: 1200px">

</div>

%md-sandbox

<h2 style="color:red">Instructor Note</h2>

#### The purpose of this notebook:

* Introduce the various Spark entry points, namely `SparkSession`.

* Introduce students to the API documentation.

* Introduce students to the `DataFrameReader` class.

* The payoffs for this notebook include...

  * How to read in data from CSV

  * Understanding the difference between using **inferSchema** and specifying a schema.

* Regarding `printRecordsPerPartition(..)`, it 

  * converts the specified `DataFrame` to an RDD

  * counts the number of records in each partition

  * prints the results to the console.

%md

# Reading Data - CSV Files

**Technical Accomplishments:**

- Start working with the API documentation

- Introduce the class `SparkSession` and other entry points

- Introduce the class `DataFrameReader`

- Read data from:

  * CSV without a Schema.

  * CSV with a Schema.

%md

## ![Spark Logo Tiny](https://files.training.databricks.com/images/105/logo_spark_tiny.png) Classroom-Setup<br>

For each lesson to execute correctly, please make sure to run the **`Classroom-Setup`** cell at the start of each lesson (see the next cell) and the **`Classroom-Cleanup`** cell at the end of each lesson.

%md

##![Spark Logo Tiny](https://files.training.databricks.com/images/105/logo_spark_tiny.png) Entry Points

Our entry point for Spark 2.0 applications is the class `SparkSession`.

An instance of this object is already instantiated for us which can be easily demonstrated by running the next cell:

%md-sandbox

<h2 style="color:red">Instructor Note</h2>

* Have the students run the cell.

* Point out the data type shown in the REPL's output.

Scala: `org.apache.spark.sql.SparkSession@50789153`

Python: `pyspark.sql.session.SparkSession object at 0x7fd4cfece690`

%md

It's worth noting that in Spark 2.0 `SparkSession` is a replacement for the other entry points:

* `SparkContext`, available in our notebook as **sc**.

* `SQLContext`, or more specifically it's subclass `HiveContext`, available in our notebook as **sqlContext**.