ArtsAutosBooksBusinessEducationEntertainmentFamilyFashionFoodGamesGenderHealthHolidaysHomeHubPagesPersonal FinancePetsPoliticsReligionSportsTechnologyTravel
  • »
  • Technology»
  • Computers & Software»
  • Computer Science & Programming

Create a Scala project on Spark using Scala IDE

Updated on March 30, 2016

Scala is one of the most exciting languages for programming Big Data. It is a multi-paradigm language that fully supports functional, object-oriented, imperative and concurrent programming. It is a strong language, it means a convenient form of self-documenting code.
Apache Spark is written in Scala, and any library that purports to work on distributed run times should at the very least be able to interface with Spark.
When you want to create a project vs Scala, you don't like write program vs Spark-Shell and console (windows) or terminal (Unix). So you need configure Eclipse vs Spark. Before configuring, you have installed Spark on your machine.

You installed successfully Spark on your machine ( Windows, Unix).
Scala IDE 2.10 (it's better for Spark current version).

Open command prompt ( or terminal) and go to Spark boot directory

cd c:\spark-1.6.0

type this command sbt eclipse

c:\spark-1.6.0\sbt eclipse

The configuration will take some minutes

On screen displays something like this picture

Add jar file to Scala project

1. Create a Scala project

2. Right click on project, choose Build Path\ Add External Archives

3. Import jar file
An dialog box opens and go to jar file directory: C:\spark-1.6.0\assembly\target\scala-2.10
Select jar file: spark-assembly-1.6.0-hadoop2.2.*.jar

Ok, You completely created Scala project on Spark. And now coding!


    0 of 8192 characters used
    Post Comment

    No comments yet.