[Bug]: Not able to search in a Diskann index

[devanshsa5a]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version:2.5x
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka):
- SDK version(e.g. pymilvus v2.0.0rc2): 
- OS(Ubuntu or CentOS): Amazon Linux 2
- CPU/Memory: 32 vcpu/ 128 GB RAM
- GPU: 0
- Others:

Current Behavior

I am < creating a Diskann index type > on 100M 128 dims dataset.
I am using one node where I am using milvus using docker-compose.

With the following parameters :

            start_time = time.time()
            log.info(f"Waiting for index to be created...")
            collection.create_index(
            field_name=self._vector_field,
            index_params={
                "index_type": "DISKANN",  # Index type
                "metric_type": "L2",    # Distance metric
            },
            sync = True,
            index_name=self._index_name,
            )
            end_time = time.time()
            log.info(f"Index created successfully for collection: {self.collection_name}")
            log.info(f"Time taken: {end_time-start_time}")
        except Exception as e:
            log.error(f"Index creation Failed: {e}")```
            
            
and index is successfully created and using describe_index I can see 


`DEBUG:pymilvus.milvus_client.milvus_client:Created new connection using: 5ce929c8b82a46fc91ffaa638bb06140
{'index_type': 'DISKANN', 'metric_type': 'L2', 'field_name': 'emb', 'index_name': 'vector_index', 'total_rows': 100000000, 'indexed_rows': 100000000, 'pending_index_rows': 100000000, 'state': 'Finished'}`

I am also changing only these parameters in my milvus.yaml to enable diskann  

queryNode.enableDisk.true
queryNode.cache.warmup.async

 
 
 After the index is created 
 I am calling 
 
 ```collection.load()```
 
 **But the then node is crashing.**
 
 After restarting the node with docker compose. I tried with using collection.load()
[milvus.log.zip](https://github.com/user-attachments/files/18167052/milvus.log.zip)

 
 If I don't use  `collection.load()` then the search query
 ```start_time = time.time()
        res = collection.search(
        data=[[0.0, 0.0, 0.0, 1.0, 8.0, 7.0, 3.0, 2.0, 5.0, 0.0, 0.0, 3.0, 5.0, 7.0, 11.0, 31.0, 13.0, 0.0, 0.0, 0.0, 0.0, 29.0, 106.0, 107.0, 13.0, 0.0, 0.0, 0.0, 1.0, 61.0, 70.0, 42.0, 0.0, 0.0, 0.0, 0.0, 1.0, 23.0, 28.0, 16.0, 63.0, 4.0, 0.0, 0.0, 0.0, 6.0, 83.0, 81.0, 117.0, 86.0, 25.0, 15.0, 17.0, 50.0, 84.0, 117.0, 31.0, 23.0, 18.0, 35.0, 97.0, 117.0, 49.0, 24.0, 68.0, 27.0, 0.0, 0.0, 0.0, 4.0, 29.0, 71.0, 81.0, 47.0, 13.0, 10.0, 32.0, 87.0, 117.0, 117.0, 45.0, 76.0, 40.0, 22.0, 60.0, 70.0, 41.0, 9.0, 7.0, 21.0, 29.0, 39.0, 53.0, 21.0, 4.0, 1.0, 55.0, 72.0, 3.0, 0.0, 0.0, 0.0, 0.0, 9.0, 65.0, 117.0, 73.0, 37.0, 28.0, 23.0, 17.0, 34.0, 11.0, 11.0, 27.0, 61.0, 64.0, 25.0, 4.0, 0.0, 42.0, 13.0, 1.0, 1.0, 1.0, 14.0, 10.0, 6.0]],
        anns_field = "emb",
        limit=10, # Max. number of search results to return
        param={"metric_type": "L2","params": {"search_list": 16}}
        )
        end_time = time.time()
        elapsed_time = end_time - start_time
        print(f"Elapsed Time: {elapsed_time:.2f} seconds")
        print(res)```
        
 It takes forever to search and exits with error:
 

ERROR:pymilvus.decorators:RPC error: [search], <MilvusException: (code=103, message=stack trace: /workspace/source/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace
/workspace/source/internal/util/grpcclient/client.go:563 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call
/workspace/source/internal/util/grpcclient/client.go:577 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall
/workspace/source/internal/distributed/querycoord/client/client.go:110 github.com/milvus-io/milvus/internal/distributed/querycoord/client.wrapGrpcCall[...]
/workspace/source/internal/distributed/querycoord/client/client.go:306 github.com/milvus-io/milvus/internal/distributed/querycoord/client.(*Client).GetShardLeaders
/workspace/source/internal/proxy/meta_cache.go:920 github.com/milvus-io/milvus/internal/proxy.(*MetaCache).GetShards
/workspace/source/internal/proxy/lb_policy.go:111 github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).GetShardLeaders.func1
/workspace/source/pkg/util/retry/retry.go:128 github.com/milvus-io/milvus/pkg/util/retry.Handle
/workspace/source/internal/proxy/lb_policy.go:109 github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).GetShardLeaders
/workspace/source/internal/proxy/lb_policy.go:230 github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).Execute: collection not fully loaded[collection=454668483432809547])>, <Time:{'RPC start': '2024-12-17 19:14:16.395207', 'RPC error': '2024-12-17 19:23:23.739230'}>
ERROR:main:Error while creating collection: <MilvusException: (code=103, message=stack trace: /workspace/source/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace
/workspace/source/internal/util/grpcclient/client.go:563 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call
/workspace/source/internal/util/grpcclient/client.go:577 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall
/workspace/source/internal/distributed/querycoord/client/client.go:110 github.com/milvus-io/milvus/internal/distributed/querycoord/client.wrapGrpcCall[...]
/workspace/source/internal/distributed/querycoord/client/client.go:306 github.com/milvus-io/milvus/internal/distributed/querycoord/client.(*Client).GetShardLeaders
/workspace/source/internal/proxy/meta_cache.go:920 github.com/milvus-io/milvus/internal/proxy.(*MetaCache).GetShards
/workspace/source/internal/proxy/lb_policy.go:111 github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).GetShardLeaders.func1
/workspace/source/pkg/util/retry/retry.go:128 github.com/milvus-io/milvus/pkg/util/retry.Handle
/workspace/source/internal/proxy/lb_policy.go:109 github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).GetShardLeaders
/workspace/source/internal/proxy/lb_policy.go:230 github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).Execute: collection not fully loaded[collection=454668483432809547])>

I am attaching the milvus.log file during calling of collection.load() and node got crashed

### Expected Behavior

_No response_

### Steps To Reproduce

_No response_

### Milvus Log

[Uploading milvus.log.zip…]()


### Anything else?

_No response_

[xiaofan-luan]

did you do collection load after index build?
You need to load a collection before search.

[xiaofan-luan]

if collection.load and node crash, you need to check the reason, what may happen is:

1. OOM -> 8GB could host 20m 128dim data
2. You need high peroformance NVMe SSD

[yanliang567]

1. Besides, according to the info of index_describe(), the index building is not completed until the pending_index_rows is 0.
   {'index_type': 'DISKANN', 'metric_type': 'L2', 'field_name': 'emb', 'index_name': 'vector_index', 'total_rows': 100000000, 'indexed_rows': 100000000, 'pending_index_rows': 100000000, 'state': 'Finished'}`
2. the milvus log was not uploaded successfully. please re-upload.

/assign @devanshsa5a
/unassign