Create a Scala project on Spark using Scala IDE

Scala is one of the most exciting languages for programming Big Data. It is a multi-paradigm language that fully supports functional, object-oriented, imperative and concurrent programming. It is a strong language, it means a convenient form of self-documenting code. 

Apache Spark is written in Scala, and any library that purports to work on distributed run times should at the very least be able to interface with Spark. 

When you want to create a project vs Scala, you don't like write program vs Spark-Shell and console (windows) or terminal (Unix). So you need configure Eclipse vs Spark. Before configuring, you have installed Spark on your machine. 

Requirements:

Installed successfully Spark on your machine ( Windows, Unix).

Scala IDE 2.10 (it's better for Spark current version).

Configuration:

Open command prompt ( or terminal) and go to Spark boot directory 

cd c:\spark-1.6.0

type this command sbt eclipse




c:\spark-1.6.0\sbt eclipse




The configuration will take some minutes




On screen displays something like this picture

Add jar file to Scala project

1. Create a Scala project

2. Right click on project, choose Build Path\ Add External Archives 




3. Import jar file

• An dialog box opens and go to jar file directory: C:\spark-1.6.0\assembly\target\scala-2.10
• Select jar file: spark-assembly-1.6.0-hadoop2.2.*.jar

Ok, You completely created Scala project on Spark. And now coding!