Skip to content

red-fox-yj/ApiRetrieval

Repository files navigation

ApiRetrieval

Prepare

pip install -r requirements.txt
conda install -c conda-forge faiss-gpu

Results

single-intent

nvidia/NV-Embed-v2

A800 80G

test_size k Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
0.25 1 0.9003 0.9003 N/A
0.25 3 0.9759 0.9759 0.8843
0.25 5 0.9897 0.9897 0.8557
0.5 1 0.8713 0.8713 N/A
0.5 3 0.9647 0.9647 0.8589
0.5 5 0.9877 0.9877 0.8335
0.75 1 0.8800 0.8800 N/A
0.75 3 0.9718 0.9718 0.8169
0.75 5 0.9929 0.9929 0.7374
k test_size Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
1 0.25 0.9003 0.9003 N/A
1 0.5 0.8713 0.8713 N/A
1 0.75 0.8800 0.8800 N/A
3 0.25 0.9759 0.9759 0.8843
3 0.5 0.9647 0.9647 0.8589
3 0.75 0.9718 0.9718 0.8169
5 0.25 0.9897 0.9897 0.8557
5 0.5 0.9877 0.9877 0.8335
5 0.75 0.9929 0.9929 0.7374

注:上述两个表仅排列方式不同,数据完全相同。

tsne_test_size_0 25

BAAI/bge-en-icl

A800 80G

test_size k Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
0.25 1 0.8935 0.8935 N/A
0.25 3 0.9691 0.9691 0.8763
0.25 5 0.9863 0.9863 0.8502
0.5 1 0.8642 0.8642 N/A
0.5 3 0.9700 0.9700 0.8413
0.5 5 0.9806 0.9806 0.7933
0.75 1 0.8541 0.8541 N/A
0.75 3 0.9671 0.9671 0.7671
0.75 5 0.9882 0.9882 0.6828
k test_size Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
1 0.25 0.8935 0.8935 N/A
1 0.5 0.8642 0.8642 N/A
1 0.75 0.8541 0.8541 N/A
3 0.25 0.9691 0.9691 0.8763
3 0.5 0.9700 0.9700 0.8413
3 0.75 0.9671 0.9671 0.7671
5 0.25 0.9863 0.9863 0.8502
5 0.5 0.9806 0.9806 0.7933
5 0.75 0.9882 0.9882 0.6828

注:上述两个表仅排列方式不同,数据完全相同。

tsne_test_size_0 25

dunzhang/stella_en_1.5B_v5

RTX4090 24G

test_size k Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
0.25 1 0.8900 0.8900 N/A
0.25 3 0.9794 0.9794 0.8843
0.25 5 1.0000 1.0000 0.8694
0.5 1 0.8783 0.8783 N/A
0.5 3 0.9788 0.9788 0.8636
0.5 5 0.9929 0.9929 0.8349
0.75 1 0.8659 0.8659 N/A
0.75 3 0.9682 0.9682 0.8078
0.75 5 0.9847 0.9847 0.7219
k test_size Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
1 0.25 0.8900 0.8900 N/A
1 0.5 0.8783 0.8783 N/A
1 0.75 0.8659 0.8659 N/A
3 0.25 0.9794 0.9794 0.8843
3 0.5 0.9788 0.9788 0.8636
3 0.75 0.9682 0.9682 0.8078
5 0.25 1.0000 1.0000 0.8694
5 0.5 0.9929 0.9929 0.8349
5 0.75 0.9847 0.9847 0.7219

注:上述两个表仅排列方式不同,数据完全相同。

tsne_test_size_0 25

muti-intent

nvidia/NV-Embed-v2

A800 80G

test_size k Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
0.25 1 0.7960 0.7960 0.7085
0.25 3 0.8921 0.8921 0.6279
0.25 5 0.9254 0.9254 0.5583
0.5 1 0.7875 0.7875 0.6761
0.5 3 0.9011 0.9011 0.5789
0.5 5 0.9334 0.9334 0.4857
0.75 1 0.7709 0.7709 0.6142
0.75 3 0.8871 0.8871 0.4367
0.75 5 0.9175 0.9175 0.3268
k test_size Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
1 0.25 0.7960 0.7960 0.7085
1 0.5 0.7875 0.7875 0.6761
1 0.75 0.7709 0.7709 0.6142
3 0.25 0.8921 0.8921 0.6279
3 0.5 0.9011 0.9011 0.5789
3 0.75 0.8871 0.8871 0.4367
5 0.25 0.9254 0.9254 0.5583
5 0.5 0.9334 0.9334 0.4857
5 0.75 0.9175 0.9175 0.3268

注:上述两个表仅排列方式不同,数据完全相同。

tsne_test_size_0 25

BAAI/bge-en-icl

A800 80G

test_size k Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
0.25 1 0.7980 0.7980 0.6809
0.25 3 0.8810 0.8810 0.5667
0.25 5 0.9189 0.9189 0.4897
0.5 1 0.7731 0.7731 0.6440
0.5 3 0.8837 0.8837 0.5116
0.5 5 0.9196 0.9196 0.4183
0.75 1 0.7317 0.7317 0.5568
0.75 3 0.8533 0.8533 0.3791
0.75 5 0.8918 0.8918 0.2814
k test_size Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
1 0.25 0.7980 0.7980 0.6809
1 0.5 0.7731 0.7731 0.6440
1 0.75 0.7317 0.7317 0.5568
3 0.25 0.8810 0.8810 0.5667
3 0.5 0.8837 0.8837 0.5116
3 0.75 0.8533 0.8533 0.3791
5 0.25 0.9189 0.9189 0.4897
5 0.5 0.9196 0.9196 0.4183
5 0.75 0.8918 0.8918 0.2814

注:上述两个表仅排列方式不同,数据完全相同。

tsne_test_size_0 25

dunzhang/stella_en_1.5B_v5

RTX4090 24G

test_size k Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
0.25 1 0.7660 0.7660 0.6731
0.25 3 0.8624 0.8624 0.5990
0.25 5 0.9026 0.9026 0.5307
0.5 1 0.7541 0.7541 0.6439
0.5 3 0.8475 0.8475 0.5412
0.5 5 0.8957 0.8957 0.4569
0.75 1 0.7111 0.7111 0.5733
0.75 3 0.8313 0.8313 0.4065
0.75 5 0.8804 0.8804 0.3001
k test_size Overall Accuracy (no dedup) Overall Accuracy (dedup) Overall Precision
1 0.25 0.7660 0.7660 0.6731
1 0.5 0.7541 0.7541 0.6439
1 0.75 0.7111 0.7111 0.5733
3 0.25 0.8624 0.8624 0.5990
3 0.5 0.8475 0.8475 0.5412
3 0.75 0.8313 0.8313 0.4065
5 0.25 0.9026 0.9026 0.5307
5 0.5 0.8957 0.8957 0.4569
5 0.75 0.8804 0.8804 0.3001

注:上述两个表仅排列方式不同,数据完全相同。

tsne_test_size_0 25

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages