Pre-Computed Recommendations not working

[varunraj1493]

Hi ,
i am using neo4j 3.4.9 version and graphAware 3.4.9.52.11 version. Followed same steps mentioned as in read me with graphaware-server-community-all-3.4.9.52.jar file copied to plugins folder. I getting this error while trying to start neo4j

please Help me , am badly tuck..

2019-03-18 09:08:40.484+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@f1a45f8' was successfully initialized, but failed to start. Please see the attached cause exception "NULL". Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@f1a45f8' was successfully initialized, but failed to start. Please see the attached cause exception "NULL".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@f1a45f8' was successfully initialized, but failed to start. Please see the attached cause exception "NULL".
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:68)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:220)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:111)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:79)
at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:32)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase@f1a45f8' was successfully initialized, but failed to start. Please see the attached cause exception "NULL".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:466)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:212)
... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory, /home/varun/Downloads/neo4j-community-3.4.7/data/databases/candidate_graph
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:212)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:125)
at org.neo4j.server.CommunityNeoServer.lambda$static$0(CommunityNeoServer.java:58)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:88)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'com.graphaware.runtime.bootstrap.RuntimeKernelExtension@b016b4e' was successfully initialized, but failed to start. Please see the attached cause exception "NULL".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:466)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.extension.KernelExtensions.start(KernelExtensions.java:84)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:208)
... 9 more
Caused by: java.lang.NoSuchFieldError: NULL
at org.springframework.expression.TypedValue.(TypedValue.java:31)
at org.springframework.expression.spel.ast.StringLiteral.(StringLiteral.java:34)
at org.springframework.expression.spel.standard.InternalSpelExpressionParser.maybeEatLiteral(InternalSpelExpressionParser.java:616)
at org.springframework.expression.spel.standard.InternalSpelExpressionParser.eatStartNode(InternalSpelExpressionParser.java:434)
at org.springframework.expression.spel.standard.InternalSpelExpressionParser.eatPrimaryExpression(InternalSpelExpressionParser.java:286)
at org.springframework.expression.spel.standard.InternalSpelExpressionParser.eatUnaryExpression(InternalSpelExpressionParser.java:279)
at org.springframework.expression.spel.standard.InternalSpelExpressionParser.eatPowerExpression(InternalSpelExpressionParser.java:255)
at org.springframework.expression.spel.standard.InternalSpelExpressionParser.eatProductExpression(InternalSpelExpressionParser.java:236)
at org.springframework.expression.spel.standard.InternalSpelExpressionParser.eatSumExpression(InternalSpelExpressionParser.java:219)

[ikwattro]

Can you list all the plugins you have installed and their versions.

[varunraj1493]

@ikwattro that issue is resolved, but i am stuck with the precomputations. Now my cluster is getting started but still pre-computation is not happening. These are the logs

2019-04-04 13:14:28.331+0000 INFO Bootstrapping module with order 1, ID reco, using com.graphaware.reco.neo4j.module.RecommendationModuleBootstrapper
2019-04-04 13:14:28.348+0000 INFO Constructing new recommendation module with ID: reco
2019-04-04 13:14:28.348+0000 INFO Trying to instantiate class com.phenom.graphaware.SimilarJobsProcessor
2019-04-04 13:14:28.975+0000 INFO Attempting to instantiate as a singleton...
2019-04-04 13:14:28.976+0000 INFO Attempting to instantiate using public no-arg constructor...
2019-04-04 13:14:28.985+0000 INFO Success.
2019-04-04 13:14:29.102+0000 INFO Node Inclusion Policy set to com.graphaware.common.policy.inclusion.composite.CompositeNodeInclusionPolicy@e41b3ea8
2019-04-04 13:14:29.103+0000 INFO Max recommendations set to 20
2019-04-04 13:14:29.107+0000 INFO Registering module reco with GraphAware Runtime.
2019-04-04 13:14:29.109+0000 INFO GraphAware Runtime bootstrapped, starting the Runtime...
2019-04-04 13:14:29.273+0000 INFO Starting GraphAware...
2019-04-04 13:14:29.275+0000 INFO Loading module metadata...
2019-04-04 13:14:29.277+0000 INFO Loading metadata for module reco
2019-04-04 13:14:29.468+0000 INFO Module reco seems to have been registered for the first time.
2019-04-04 13:14:29.471+0000 INFO Creating fresh metadata for module reco.
2019-04-04 13:14:29.799+0000 WARN Did not find a suitable node in 1000 attempts. Are you sure the node inclusion policy is set correctly for ContinuousNodeSelector?
2019-04-04 13:14:29.826+0000 INFO Starting RecommendationModule with node Node[1682]
2019-04-04 13:14:29.835+0000 INFO Registering module reco and its context with the task scheduler.
2019-04-04 13:14:29.839+0000 INFO Module metadata loaded.
2019-04-04 13:14:29.839+0000 INFO Starting transaction-driven modules...
2019-04-04 13:14:29.839+0000 INFO Transaction-driven modules started.
2019-04-04 13:14:29.840+0000 INFO There are 1 timer-driven runtime modules. Scheduling the first task...
2019-04-04 13:14:29.861+0000 INFO GraphAware started.
2019-04-04 13:14:29.861+0000 INFO GraphAware Runtime automatically started.
2019-04-04 13:14:39.646+0000 INFO Connected to /$$$##$$:^&%& [raft version:2]
2019-04-04 13:14:58.877+0000 INFO Waiting to catchup with leader... we are 0 entries behind leader at 23472.
2019-04-04 13:14:58.879+0000 INFO Successfully joined the Raft group.
2019-04-04 13:15:05.481+0000 INFO Sending metrics to CSV file at /usr/local/ongdb/metrics
2019-04-04 13:15:05.483+0000 INFO Sending metrics to Graphite server at &^&*(()T:2003
2019-04-04 13:15:05.648+0000 INFO Bolt enabled on 0.0.0.0:7687.
2019-04-04 13:15:07.361+0000 INFO Started.
2019-04-04 13:15:07.552+0000 INFO Mounted REST API at: /db/manage
2019-04-04 13:15:07.569+0000 INFO started
2019-04-04 13:15:07.578+0000 INFO Mounted unmanaged extension [com.graphaware.server] at [/graphaware]
2019-04-04 13:15:07.578+0000 INFO Mounted unmanaged extension [com.xyz.ml] at [/data/insertion]
2019-04-04 13:15:07.634+0000 INFO Server thread metrics have been registered successfully
2019-04-04 13:15:07.729+0000 INFO Google Analytics disabled

And my conf properties related to the pre-computation are as follows :

dbms.jvm.additional=-Dunsupported.dbms.udc.source=tarball
dbms.unmanaged_extension_classes=com.graphaware.server=/graphaware,com.xyz.ml=/data/insertion
com.graphaware.module.reco.1=com.graphaware.reco.neo4j.module.RecommendationModuleBootstrapper
com.graphaware.module.reco.node=hasLabel('jobSeqNo')
com.graphaware.module.reco.engine=com.xyz.graphaware.SimilarJobsProcessor
com.graphaware.runtime.stats.disabled=true
com.graphaware.server.stats.disabled=true
com.graphaware.runtime.enabled=true
com.graphaware.module.reco.maxRecommendations=20
dbms.memory.heap.initial_size=11227M
dbms.memory.heap.max_size=11227M
dbms.jvm.additional=-XX:+UseG1GC

[varunraj1493]

Data nodes are present in the DB but still log is showing warning

2019-04-04 13:14:29.799+0000 WARN Did not find a suitable node in 1000 attempts. Are you sure the node inclusion policy is set correctly for ContinuousNodeSelector?

[varunraj1493]

@ikwattro can i get any help here please....?

[ikwattro]

I will have time to look at this next week.

[varunraj1493]

Thanks @ikwattro

[varunraj1493]

Hi @ikwattro
can you help me with this issue please

[ikwattro]

Hi @varunraj1493 , Can you please send the code of the plugin as well as a representative dataset as a neo4j backup. Thanks.

[aldrinm]

@varunraj1493 typically that error could be due to a label mismatch, so verify the label specified in the config jobSeqNo with the actual labels existing in your graph, case sensitive

[varunraj1493]

Hi @aldrinm , @ikwattro
i sorted the issue, it was because of an another jar file, i removed from folder but didn't change the conf file.
I am facing an another issue, my precomputations are working in my local machine but when i am deploying it in a 3 node cluster, it is not working. Logs didn't throw any error , it only says "Starting RecommendationModule with node Node[1]" and that's it , where as in my local machine which has same data and similar conf properties, it is able to precompute for all the nodes.

Is there anything i have to do to make it work in cluster ?

[ikwattro]

Enterprise Cluster support is only available through a GraphAware Enterprise subscription.