Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Working example/generic usecase #46

Open
MaximusPrimus opened this issue May 13, 2019 · 4 comments
Open

Working example/generic usecase #46

MaximusPrimus opened this issue May 13, 2019 · 4 comments
Assignees
Labels

Comments

@MaximusPrimus
Copy link

This seems to be great. Can you provide a few lines of code of a working example for this so it becomes easier to use for first-timers.

Thanks for the hep !

@joshdevins
Copy link
Collaborator

What exactly are you trying to do? There are "apps" that have example code in them or you can use those drivers directly on your data. E.g. https://github.com/soundcloud/spark-pagerank/blob/master/src/main/scala/com.soundcloud.spark.pagerank/PageRankApp.scala#L68-L91

Do you want to build a graph or just run PageRank on an existing graph? Check out the drivers.

@joshdevins
Copy link
Collaborator

See: https://github.com/soundcloud/spark-pagerank#usage

Feel free to do a PR to improve the documentation if you think there is something that can be clearer. I prefer not putting code in the README as it can drift from the implementation since it's not under test.

@MaximusPrimus
Copy link
Author

Thank you for your reply. I went through the usage file and had doubts, created this question. I have a TSV file with (src, dst, weights). Downloaded the .jar file. My aim is to form a graph and run the weighted page rank. I followed the documentation and went through some code but this gives me an error :-

GraphBuilderApp.run(Array("--input=/temp/followers1.txt", "--output=/tmp"), spark)
error : org.kohsuke.args4j.CmdLineException: "--input /temp/followers1.txt" is not a valid option
at org.kohsuke.args4j.CmdLineParser.parseArgument(CmdLineParser.java:419)
at com.soundcloud.spark.pagerank.GraphBuilderApp$.run(GraphBuilderApp.scala:30)
... 49 elided

Just wanted to see an example which could help me understand how to run the apps on my data.

Many thanks !

@joshdevins
Copy link
Collaborator

You need to submit a Spark job with the GraphBuilderApp as the driver/application. See: https://spark.apache.org/docs/latest/submitting-applications.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

2 participants