Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

training embedding in Entity-linking with GPU #4

Open
CharlieUnderwood opened this issue Feb 19, 2020 · 1 comment
Open

training embedding in Entity-linking with GPU #4

CharlieUnderwood opened this issue Feb 19, 2020 · 1 comment

Comments

@CharlieUnderwood
Copy link

CharlieUnderwood commented Feb 19, 2020 •
edited
Loading

I have a Nvida GPU on windows with jdk version 11.0.6 and cuda==9.0. I have successfully trained embedding on cpu with a dependency addition into build.gradle file:

compile group: 'org.nd4j',name: 'nd4j-native-platform',version:'0.9.1'

I am aware that if I want to use GPU instead of cpu as computing resource, I have to replace
compile group: 'org.nd4j',name: 'nd4j-native-platform',version:'0.9.1'
to
compile group: 'org.nd4j', name: 'nd4j-cuda-9.0-platform', version: '1.0.0-beta3'.

However, the following error occurred:

PS E:\pgx_samples_master\entity_linking> ./gradlew createEmbeddings --args="-g graph_p -i data/graph_data/graph_small.js
on"
Configuration on demand is an incubating feature.

Task :createEmbeddings
18:16:18.803 INFO : Starting embeddings creation...
18:16:18.850 INFO : Setting up embeddings creation...
18:16:18.850 INFO : Input graph: data/graph_data/graph_small.json
18:16:18.850 INFO : Creating Pgx session...
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/E:/pgx_samples_master/libs/pgx-19.3/shared-lib/third-party/guice-4.2.2.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
18:16:20.397 INFO : Session created
18:16:20.397 INFO : Opening input graph: data/graph_data/graph_small.json...
18:16:22.365 INFO : Loaded input graph: graph_small_v, loading time: 1 sec
18:16:22.396 INFO : Creating undirected graph...
18:16:22.802 INFO : Undirected graph created, exec. time: 0 sec
Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at oracle.pgx.api.PgxFuture.get(PgxFuture.java:99)
at oracle.pgx.api.beta.mllib.DeepWalkModelBuilder.build(DeepWalkModelBuilder.java:284)
at create_embeddings.CreateEmbeddingsMain$.createModel(CreateEmbeddingsMain.scala:50)
at create_embeddings.CreateEmbeddingsMain$.main(CreateEmbeddingsMain.scala:93)
at create_embeddings.CreateEmbeddingsMain.main(CreateEmbeddingsMain.scala)
Caused by: java.lang.ExceptionInInitializerError
at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.(NativeOpExecutioner.java:53)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at java.base/java.lang.Class.newInstance(Class.java:584)
at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:6185)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:6087)
at org.nd4j.linalg.factory.Nd4j.(Nd4j.java:201)
at oracle.pgx.mllib.backend.dl4j.Dl4jBackend.initialize(Dl4jBackend.java:25)
at oracle.pgx.mllib.dl4j.deepwalk.DeepWalkModelDl4j.(DeepWalkModelDl4j.java:59)
at oracle.pgx.mllib.dl4j.deepwalk.DeepWalkModelDl4jProvider.getModel(DeepWalkModelDl4jProvider.java:15)
at oracle.pgx.engine.mllib.DeepWalkServerModel.(DeepWalkServerModel.java:73)
at oracle.pgx.engine.instance.InstanceManager.createDeepWalkModel(InstanceManager.java:235)
at oracle.pgx.engine.CoreMlLibImpl.lambda$createDeepWalkModel$35(CoreMlLibImpl.java:301)
at oracle.pgx.engine.exec.FunctionRequest.doCall(FunctionRequest.java:28)
at oracle.pgx.engine.exec.Task.call(Task.java:326)
at oracle.pgx.engine.exec.Task.run(Task.java:246)
at oracle.pgx.engine.exec.SameThreadExecutor.execute(SameThreadExecutor.java:32)
at oracle.pgx.engine.exec.SameThreadExecutor.managedExecute(SameThreadExecutor.java:76)
at oracle.pgx.engine.exec.Task.submitTo(Task.java:230)
at oracle.pgx.engine.exec.Task.trySubmit(Task.java:210)
at oracle.pgx.engine.exec.ExecutionManager.tryDispatch(ExecutionManager.java:63)
at oracle.pgx.engine.Server.visit(Server.java:367)
at oracle.pgx.engine.Server.run(Server.java:284)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: ND4J is probably missing dependencies. For more information, please refer to: http://nd4j.org/getstarted.html
at org.nd4j.nativeblas.NativeOpsHolder.(NativeOpsHolder.java:51)
at org.nd4j.nativeblas.NativeOpsHolder.(NativeOpsHolder.java:19)
... 28 more
Caused by: java.lang.UnsatisfiedLinkError: no jnind4jcpu in java.library.path: [C:\Program Files\Java\jdk-11.0.6\bin, C:\WINDOWS\Sun\Java\bin, C:\WINDOWS\system32, C:\WINDOWS, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\libnvvp, C:\Program Files\Docker\Docker\Resources\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\libnvvp, C:\Windows\system32, C:\Windows, C:\Windows\System32\Wbem, C:\Windows\System32\WindowsPowerShell\v1.0, C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common, C:\Program Files (x86)\scala\bin, C:\Program Files\dotnet, H:\go\bin, C:\Program Files (x86)\Microsoft VS Code\bin, C:\WINDOWS\system32, C:\WINDOWS, C:\WINDOWS\System32\Wbem, C:\WINDOWS\System32\WindowsPowerShell\v1.0, C:\WINDOWS\System32\OpenSSH, C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR, C:\Program Files\Java\jdk-11.0.6\bin, C:\spark-2.4.0-bin-hadoop2.7\bin, D:\MATLAB\runtime\win64, D:\MATLAB\bin, H:\Git\cmd, C:\Program Files\gradle-6.1.1\bin, C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.5.0, D:\anaconda, D:\mysql\server\MySQL Shell 8.0\bin, C:\Users\QLR.DESKTOP-BS1O4DP\AppData\Local\Microsoft\WindowsApps, ., D:\GoLand 2019.1.1\bin, ., .]
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2660)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827)
at java.base/java.lang.System.loadLibrary(System.java:1870)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:976)
at org.bytedeco.javacpp.Loader.load(Loader.java:777)
at org.bytedeco.javacpp.Loader.load(Loader.java:684)
at org.nd4j.nativeblas.Nd4jCpu.(Nd4jCpu.java:10)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.bytedeco.javacpp.Loader.load(Loader.java:739)
at org.bytedeco.javacpp.Loader.load(Loader.java:684)
at org.nd4j.nativeblas.Nd4jCpu$NativeOps.(Nd4jCpu.java:63)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at org.nd4j.nativeblas.NativeOpsHolder.(NativeOpsHolder.java:29)
... 29 more
Caused by: java.lang.UnsatisfiedLinkError: no nd4jcpu in java.library.path: [C:\Program Files\Java\jdk-11.0.6\bin, C:\WINDOWS\Sun\Java\bin, C:\WINDOWS\system32, C:\WINDOWS, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\libnvvp, C:\Program Files\Docker\Docker\Resources\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\libnvvp, C:\Windows\system32, C:\Windows, C:\Windows\System32\Wbem, C:\Windows\System32\WindowsPowerShell\v1.0, C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common, C:\Program Files (x86)\scala\bin, C:\Program Files\dotnet, H:\go\bin, C:\Program Files (x86)\Microsoft VS Code\bin, C:\WINDOWS\system32, C:\WINDOWS, C:\WINDOWS\System32\Wbem, C:\WINDOWS\System32\WindowsPowerShell\v1.0, C:\WINDOWS\System32\OpenSSH, C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR, C:\Program Files\Java\jdk-11.0.6\bin, C:\spark-2.4.0-bin-hadoop2.7\bin, D:\MATLAB\runtime\win64, D:\MATLAB\bin, H:\Git\cmd, C:\Program Files\gradle-6.1.1\bin, C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.5.0, D:\anaconda, D:\mysql\server\MySQL Shell 8.0\bin, C:\Users\QLR.DESKTOP-BS1O4DP\AppData\Local\Microsoft\WindowsApps, ., D:\GoLand 2019.1.1\bin, ., .]
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2660)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827)
at java.base/java.lang.System.loadLibrary(System.java:1870)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:976)
at org.bytedeco.javacpp.Loader.load(Loader.java:765)

I have cuda 9.0 toolkit in my windows PATH, could you help to look at this error?
@martijndwars
Copy link
Contributor

CC @rpatra.

martijndwars pushed a commit that referenced this issue Aug 27, 2020
@orausch
Merge pull request #4 in PGX/pgx-samples from feature/paper-classific…
961dd2a 
…ation to master

* commit 'ec86daa8e47382432535e1b444948d6a0876d33f':
  Add Cora example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@martijndwars @CharlieUnderwood