Read CSV and JSON file format in spark 2.0

Read CSV with spark 2.0 STEP 1. Open the spark-shell and fire the following command. scala> spark.read.format(“csv”).option(“header”,”true”).load(“F:/Hadoop Youtube/customer.csv”) STEP 2. Display the result with show command scala> .show +—–+——+———–+——-+———-+—-+——+ |empno| ename|designation|manager| hire_date|... Read more »

Querying on CSV file using Spark SQL

Apache Spark SQL is a Spark module to simplify working with structured data using DataFrame and DataSet abstractions in Python, Java, and Scala. These abstractions are the distributed collection of data organized... Read more »