Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Partial Spark 1.3.0 update #65

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open

Partial Spark 1.3.0 update #65

wants to merge 31 commits into from

Conversation

gone-phishing
Copy link
Member

Though the code compiles for spark 1.3.0 with hadoop 2.5.2, a lot of dependency issues needs to be fixed. We can have a separate branch for these updates

@gone-phishing
Copy link
Member Author

Pushing a few more commits that resolves multiple scala version warnings.
There are still overlapping jars warnings present, and I've tried excluding a few of them by adding exclusions in the pom file, but then they don't have any effect on compilation and they can simply be ignored.

gone-phishing and others added 24 commits April 15, 2015 23:22
@gone-phishing
Fixing imports from dbpedia core jar
45fa37a
@gone-phishing
Adding scala version required
a4043b8
@gone-phishing
Fixed FunSuite for scalatest issue (FunSuite needs to be a trait and …
784bbb4 
…so we use FunSuiteLike)
@gone-phishing
Using FunSuite as a trait
f702ce1 
FunSuiteLike is the name of the trait
@gone-phishing
FunSuiteLike trait is used
311cbd9
@gone-phishing
Debugging the DistDownloadConfig.scala file for reading params
d482ca7
@gone-phishing
Merge remote-tracking branch 'origin/spark_1.3.0-update' into spark_1…
408ecff 
….3.0-update
@gone-phishing
Commenting the distconfig variable
480757c
@gone-phishing
Commenting the distconfig variable
1f642e4
@gone-phishing
Recompiling spark with scala 2.11.4
3f9ff7c
@gone-phishing
Changing a lot of config properties
0d4935e
@gone-phishing
Update README.md
3493129
@gone-phishing
Modifying the pom file
276e58d
@gone-phishing
Modifying the pom file
962320e
@gone-phishing
Merge remote-tracking branch 'origin/spark_1.3.0-update' into spark_1…
f7b989e 
….3.0-update
@gone-phishing
Changing a lot of config properties
3702a50
@gone-phishing
Removing data files
ebd5c52
@gone-phishing
Editing config files for download
5b39ec6
@gone-phishing
Adding KryoSerializer for extractors
f4a0881
@gone-phishing
Working extraction for li languages
e0192bd
@gone-phishing
Working extraction for li languages
56ca1ec
@gone-phishing
Updating pom files for latest extraction framework build
cb7bbb1
@gone-phishing
Jsch authentication fixed using private key
215cc3e
@gone-phishing
Adding the data folder in the repo using a dummy file
a265679
@intracer
Copy link

any updates on this?

@gone-phishing
Copy link
Member Author

parallel downloads using akka clustering and extraction of information using apache spark is well supported at this stage. Are there any issues ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@gone-phishing @intracer