Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

创建“区分类目的向量表” 失败 #272

Open
sunwsh opened this issue Oct 16, 2024 · 0 comments
Open

创建“区分类目的向量表” 失败 #272

sunwsh opened this issue Oct 16, 2024 · 0 comments

Comments

@sunwsh
Copy link

sunwsh commented Oct 16, 2024

测试环境是 按照 单机模式部署的havenask。

创建全量表:
/ha3_install/hape create table -t in_offline_category_vec3 -s /ha3_install/example/cases/vector/category_vector1.json -p 1 -f /ha3_install/example/data/vector_category.data -c /ha3_install/hape_conf/vector

schema 文件(category_vector1.json) ,帮忙看看这个配置有错吗? 有没有方法检查 schema 文件 是否有错?

{
    "columns": [
        {
            "name": "id",
            "type": "UINT32"
        },
        {
            "name": "category",
            "type": "UINT32"
        },
        {
            "name": "vector",
            "type": "STRING"
        },
        {
            "name": "DUP_vector",
            "type": "RAW"
        }
    ],
    "indexes": [
            {
            "name": "id",
            "index_config" : {
                "index_fields": [
                    {
                        "field_name": "id"
                    }
                ]
            },
            "index_type": "PRIMARY_KEY64"
        },
        {
            "name": "id",
            "index_config" : {
                "index_fields": [
                    {
                        "field_name": "id"
                    }
                ]
            },
            "index_type": "ATTRIBUTE"
        },
        {
            "name": "category",
            "index_config" : {
                "index_fields": [
                    {
                        "field_name": "category"
                    }
                ]
            },
            "index_type": "ATTRIBUTE"
        },
        {
            "name": "vector",
            "index_config" : {
                "index_fields": [
                    {
                        "field_name": "vector"
                    }
                ]
            },
            "index_type": "ATTRIBUTE"
        },
        {
            "name": "vector_index",
            "index_type": "ANN",
            "index_config": {
                "index_fields": [
                    {
                        "boost": 1,
                        "field_name": "id"
                    },
                    {
                        "boost": 1,
                        "field_name": "category"
                    },
                    {
                        "boost": 1,
                        "field_name": "DUP_vector"
                    }
                ],
                "index_params": {
                    "indexer": "aitheta2_indexer",
                    "parameters": "{\"enable_rt_build\":\"true\",\"min_scan_doc_cnt\":\"20000\",\"vector_index_type\":\"Qc\",\"major_order\":\"col\",\"builder_name\":\"QcBuilder\",\"distance_type\":\"SquaredEuclidean\",\"embedding_delimiter\":\",\",\"enable_recall_report\":\"true\",\"is_embedding_saved\":\"true\",\"linear_build_threshold\":\"5000\",\"dimension\":\"128\",\"search_index_params\":\"{\\\"proxima.qc.searcher.scan_ratio\\\":0.01}\",\"searcher_name\":\"QcSearcher\",\"build_index_params\":\"{\\\"proxima.qc.builder.quantizer_class\\\":\\\"Int8QuantizerConverter\\\",\\\"proxima.qc.builder.quantize_by_centroid\\\":true,\\\"proxima.qc.builder.optimizer_class\\\":\\\"BruteForceBuilder\\\",\\\"proxima.qc.builder.thread_count\\\":10,\\\"proxima.qc.builder.optimizer_params\\\":{\\\"proxima.linear.builder.column_major_order\\\":true},\\\"proxima.qc.builder.store_original_features\\\":false,\\\"proxima.qc.builder.train_sample_count\\\":3000000,\\\"proxima.qc.builder.train_sample_ratio\\\":0.5}\"}"
                }
            }

        },
        {
            "name": "summary",
            "index_type": "SUMMARY",
            "index_config": {
                "index_fields": [
                    {
                        "field_name": "id"
                    },
                    {
                        "field_name": "category"
                    },
                    {
                        "field_name": "vector"
                    }
                ]
            }       
        }
        
    ]
}

vector_category.data 文件, 每行结尾(1F0A))

CMD=add
id=1
vector=0.57350874,0.20913178,-0.28007376,-0.008224763,0.032035876,-0.06109254,0.016208794,0.2546715,-0.5938332,0.2194785,-0.603397,0.095939286,-0.4343296,0.29292983,0.19570276,0.07006806,0.4219676,0.008796717,-0.1956801,0.10365201,-0.18756211,-0.1959297,-0.64994687,-0.0043594907,-0.12155021,-0.37211925,-0.033330534,0.37927315,0.14685613,0.14631498,0.2263155,0.020013737,-0.48688003,0.38279092,-0.14551091,-0.14309452,0.0836372,-0.2602252,-0.68886805,-0.18670991,0.026953692,-0.1361577,0.54825664,0.5495113,-0.11404511,0.020755235,-0.3592109,0.35327643,-0.17384957,0.06951289,-0.33219633,0.32536888,0.4178303,0.06838637,-0.32600296,-0.39717215,0.37040582,0.2577843,-0.022891786,-0.19590716,-0.34784496,0.45056996,-0.16618372,-0.048093718,-0.4501398,0.16741453,0.0021240758,-0.22811683,-0.30895764,-0.29165566,0.51220524,0.26545066,-0.46743664,-0.18925984,0.11317849,0.01674415,-0.41176468,-0.41503182,-0.52814317,-0.52559006,0.5788051,0.12608618,0.62300944,0.20152776,-0.39167175,0.099368036,0.34757188,-0.2860248,0.47917366,0.44290003,-0.3663736,0.074007034,-0.44493687,0.5212625,0.5742631,0.06809457,0.20434684,-0.2527957,0.08273791,-0.5062737,0.30677673,0.0037546167,0.53646725,0.6939553,-0.19357872,0.62397563,-0.25602227,-0.73345435,0.23730282,-0.11466419,-0.22964483,-0.18373214,-0.28414255,0.67296755,0.38798898,-0.0795316,0.050292946,0.51636153,-0.3399461,0.14268692,-0.5273593,-0.055386372,-0.29121432,-0.36689624,0.762627,0.14242941,0.14173277,0.08323621
category=1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant