A framework for few-shot evaluation of language models.
-
Updated
Apr 18, 2025 - Python
#
evaluation-framework
Here are 115 public repositories matching this topic...
The LLM Evaluation Framework
evaluation-metrics evaluation-framework llm-evaluation llm-evaluation-framework llm-evaluation-metrics
-
Updated
Apr 23, 2025 - Python
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
-
Updated
Apr 23, 2025 - Python
This is the repository of our article published in RecSys 2019 "Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches" and of several follow-up studies.
deep-learning neural-network reproducible-research collaborative-filtering matrix-factorization hyperparameters bpr recommendation-system recommender-system reproducibility recommendation-algorithms knn matrix-completion evaluation-framework content-based-recommendation hybrid-recommender-system funksvd bprmf bprslim slimelasticnet
-
Updated
May 25, 2023 - Python
Data-Driven Evaluation for LLM-Powered Applications
information-retrieval evaluation-metrics evaluation-framework rag llmops retrieval-augmented-generation llm-evaluation
-
Updated
Jan 22, 2025 - Python
AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
-
Updated
Apr 23, 2025 - Python
Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
evaluation-metrics evaluation-framework rag large-language-models llm llms llmops retrieval-augmented-generation
-
Updated
Nov 14, 2024 - Python
Python SDK for running evaluations on LLM generated responses
evaluation evaluation-metrics evaluation-framework llmops llm-eval llm-ops llm-evaluation llm-evaluation-toolkit
-
Updated
Apr 15, 2025 - Python
The official evaluation suite and dynamic data release for MixEval.
benchmark evaluation benchmarking-suite evaluation-framework benchmarking-framework foundation-models large-language-models large-language-model llm-inference llm-evaluation large-multimodal-models llm-evaluation-framework benchmark-mixture mixeval
-
Updated
Nov 10, 2024 - Python
Moonshot - A simple and modular tool to evaluate and red-team any LLM application.
-
Updated
Apr 21, 2025 - Python
A research library for automating experiments on Deep Graph Networks
-
Updated
Sep 9, 2024 - Python
PySODEvalToolkit: A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection
latex metrics evaluation python3 evaluator evaluation-metrics evaluation-framework metrics-visualization mae saliency salient-object-detection saliency-detection co-saliency f-measure pr-curve e-measure s-measure camouflaged-object-detection fm-curve co-salient-object-detection
-
Updated
Sep 27, 2024 - Python
Evaluation suite for large-scale language models.
-
Updated
Aug 15, 2021 - Python
A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.
-
Updated
Apr 22, 2025 - Python
Multilingual Large Language Models Evaluation Benchmark
multilingual nlp natural-language-processing evaluation evaluation-datasets datasets language-model evaluation-framework large-language-models
-
Updated
Aug 21, 2024 - Python
Optical Flow Dataset and Benchmark for Visual Crowd Analysis
tracking computer-vision dataset video-processing synthetic-images video-surveillance optical-flow benchmark-suite motion-estimation multi-object-tracking evaluation-framework trajectories video-analytics crowd-counting crowd-analysis tracking-by-detection tub-crowdflow-dataset
-
Updated
Aug 11, 2023 - Python
Official repository for the paper “The Dawn of Natural Language to SQL: Are We Fully Ready?” (VLDB'24)
awesome evaluation-metrics evaluation-framework text-to-sql text2sql awesome-text2sql awesome-nl2sql text2sql-evaluation nl2sql-evaluation
-
Updated
Mar 26, 2025 - Python
Evaluation framework for oncology foundation models (FMs)
-
Updated
Apr 23, 2025 - Python
The robust European language model benchmark.
european nlp-machine-learning evaluation-framework german-language english-language spanish-language italian-language french-language dutch-language swedish-language danish-language norwegian-language icelandic-language faroese-language llms
-
Updated
Apr 23, 2025 - Python
LiDAR SLAM comparison and evaluation framework
-
Updated
Aug 10, 2021 - Python
Improve this page
Add a description, image, and links to the evaluation-framework topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the evaluation-framework topic, visit your repo's landing page and select "manage topics."