- Computers & Software»
- Computer Science & Programming
Create a Scala project on Spark using Scala IDE
Scala is one of the most exciting languages for programming Big Data. It is a multi-paradigm language that fully supports functional, object-oriented, imperative and concurrent programming. It is a strong language, it means a convenient form of self-documenting code.
Apache Spark is written in Scala, and any library that purports to work on distributed run times should at the very least be able to interface with Spark.
When you want to create a project vs Scala, you don't like write program vs Spark-Shell and console (windows) or terminal (Unix). So you need configure Eclipse vs Spark. Before configuring, you have installed Spark on your machine.
You installed successfully Spark on your machine ( Windows, Unix).
Scala IDE 2.10 (it's better for Spark current version).
Open command prompt ( or terminal) and go to Spark boot directory
type this command sbt eclipse
The configuration will take some minutes
On screen displays something like this picture
Add jar file to Scala project
1. Create a Scala project
2. Right click on project, choose Build Path\ Add External Archives
3. Import jar file
An dialog box opens and go to jar file directory: C:\spark-1.6.0\assembly\target\scala-2.10
Select jar file: spark-assembly-1.6.0-hadoop2.2.*.jar
Ok, You completely created Scala project on Spark. And now coding!