Traverse through all the edges of a vertex in Janusgraph

[sharma-kunal]

I have a large set of data stored in Janusgraph. I have a use case where I need to traverse all the incoming edges of a vertex, below is the gremlin query I am using

traversal = self.educationGraphTraversalSource.V().hasLabel("seeker").has("p_uuid", p_uuid).inE()
while traversal.hasNext():
    edge = traversal.next()

but I keep on getting this error,

Received error message '{'requestId': 'd47158c6-5fa1-4fb1-a3eb-ae92ece57a4f', 'status': {'message': 'Could not call index', 'code': 500, 'attributes': {'stackTrace': 'org.janusgraph.core.JanusGraphException: Could not call index\n\tat org.janusgraph.graphdb.util.SubqueryIterator.<init>(SubqueryIterator.java:70)\n\tat org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1434)\n\tat org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1326)\n\tat org.janusgraph.graphdb.query.MetricsQueryExecutor.lambda$execute$3(MetricsQueryExecutor.java:62)\n\tat org.janusgraph.graphdb.query.MetricsQueryExecutor.runWithMetrics(MetricsQueryExecutor.java:74)\n\tat org.janusgraph.graphdb.query.MetricsQueryExecutor.execute(MetricsQueryExecutor.java:62)\n\tat org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:206)\n\tat org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:69)\n\tat org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:55)\n\tat org.janusgraph.graphdb.query.ResultSetIterator.<init>(ResultSetIterator.java:45)\n\tat org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:68)\n\tat org.janusgraph.graphdb.query.graph.GraphCentricQueryBuilder.lambda$iterables$1(GraphCentricQueryBuilder.java:240)\n\tat com.google.common.collect.Iterables.getOnlyElement(Iterables.java:268)\n\tat org.janusgraph.graphdb.database.StandardJanusGraph$1.retrieveSchemaByName(StandardJanusGraph.java:426)\n\tat org.janusgraph.graphdb.database.cache.StandardSchemaCache.getSchemaId(StandardSchemaCache.java:111)\n\tat org.janusgraph.graphdb.transaction.StandardJanusGraphTx.getSchemaVertex(StandardJanusGraphTx.java:1033)\n\tat org.janusgraph.graphdb.transaction.StandardJanusGraphTx.getRelationType(StandardJanusGraphTx.java:1053)\n\tat org.janusgraph.graphdb.transaction.StandardJanusGraphTx.getPropertyKey(StandardJanusGraphTx.java:1082)\n\tat org.janusgraph.graphdb.relations.AbstractTypedRelation.lambda$properties$1(AbstractTypedRelation.java:188)\n\tat java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)\n\tat java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)\n\tat java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)\n\tat java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)\n\tat java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)\n\tat java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)\n\tat java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)\n\tat java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)\n\tat org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils$3.hasNext(IteratorUtils.java:237)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:44)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.hasNext(ExpandableStepIterator.java:47)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep.processAllStarts(NoOpBarrierStep.java:67)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep.processNextStart(NoOpBarrierStep.java:56)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:55)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:48)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.hasNext(ExpandableStepIterator.java:47)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep.processAllStarts(NoOpBarrierStep.java:67)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep.processNextStart(NoOpBarrierStep.java:56)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:55)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:48)\n\tat org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)\n\tat org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:235)\n\tat org.apache.tinkerpop.gremlin.server.util.TraverserIterator.fillBulker(TraverserIterator.java:69)\n\tat org.apache.tinkerpop.gremlin.server.util.TraverserIterator.hasNext(TraverserIterator.java:56)\n\tat org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.handleIterator(TraversalOpProcessor.java:308)\n\tat org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.lambda$iterateBytecodeTraversal$0(TraversalOpProcessor.java:223)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: org.apache.tinkerpop.gremlin.process.traversal.util.TraversalInterruptedException\n\tat org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:492)\n\tat org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:398)\n\tat org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51)\n\tat org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:578)\n\tat org.janusgraph.graphdb.util.SubqueryIterator.<init>(SubqueryIterator.java:68)\n\t... 54 more\n', 'exceptions': ['org.janusgraph.core.JanusGraphException', 'org.apache.tinkerpop.gremlin.process.traversal.util.TraversalInterruptedException']}}, 'result': {'data': None, 'meta': {}}}'

What am I doing wrong here ? How am I supposed to read all the incoming edges of a vertex using Janusgraph for a large dataset (around 500k edges)

Also, if I use iterate() with my query, it does not raise any error but it returns false for traversal.hasNext() even when the data is present.

traversal = self.educationGraphTraversalSource.V().hasLabel("seeker").has("p_uuid", p_uuid).inE().iterate()
while traversal.hasNext():
    edge = traversal.next()

I tried running the query,

edge = self.educationGraphTraversalSource.V().hasLabel("seeker").has("p_uuid", p_uuid).inE().next()

and it returns the data, but when I use iterate() and do .hasNext(), it returns false ??

[hadoopmarc]

So, you are running a python remote client. The error message starts with: 'Could not call index'. So, does the following give a result:

self.educationGraphTraversalSource.V().hasLabel("seeker").has("p_uuid", p_uuid).next()

[sharma-kunal]

Yes, this returns the result, the Vertex with p_uuid.

[hadoopmarc]

OK, another attempt, I am not at all sure whether the constuct with next()/hasNext() works for sessionless remote connections, such as for gremlin-python. All traversal examples in the TinkerPop docs for gremlin-python are either finished with .iterate() when no returned value is expected or with .to_list() when results are returned. Can you try that, initially with a limit() step for a faster response.

[sharma-kunal]

Ok, so I tried this query

traversal = self.educationGraphTraversalSource.V().hasLabel("seeker").has("p_uuid", "abcd1234").inE().limit(10)
while traversal.hasNext():
    print(traversal.next())

and this time it did not throw the Index error or gave Timeout Error, but traversal.hasNext() returned me False

while, when I ran the query,

query = self.educationGraphTraversalSource.V().hasLabel("seeker").has("p_uuid", "abcd1234").inE().limit(10).toList()

and it gave me 10 results, so why using .hasNext() gave me False in this case ?

The same issue appears when I use .iterate() with the query, it does not throw the error, but .hasNext() returns False when using .iterate() also.

I am connecting to my Janusgraph like this,

connection = DriverRemoteConnection(url='ws://<Janusgraph IP>:8182/gremlin', traversal_source='g')
g = traversal().withRemote(connection)

[sharma-kunal]

I tried this in Java and it is working fine in Java, I have opened a separate discussion regarding this issue, #4126

Closing this one.