[NeurIPS'24 Spotlight, ICLR'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an …

MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science (EMNLP 2024(Findings))

Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning (ACL 2024)

Official repository for "Mentor-KD: Making Small Language Models Better Multi-step Reasoners", EMNLP 2024

Source code of DRAGIN, ACL 2024 main conference Long Paper

The repository contains the code for our EMNLP 2023 paper "SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts", written by Joon-Young Choi, Junho Kim, Jun-Hyung Park…

An experimental implementation of the retrieval-enhanced language model

Toolkit for downloading and processing Google's AudioSet dataset.

Pytorch implementations of Client-Customized Adaptation for Parameter-Efficient Federated Learning (Findings of ACL: ACL 2023)

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

Large Language Models Are Reasoning Teachers (ACL 2023)

Instruct-tune LLaMA on consumer hardware

Forward-Looking Active REtrieval-augmented generation (FLARE)

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

Pytorch implementations of Co-teaching for noisy label learning

⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚡

Code and documentation to train Stanford's Alpaca models, and generate the data.

KoAlpaca: 한국어 명령어를 이해하는 오픈소스 언어모델 (KoAlpaca: An open-source language model to understand Korean instructions)

list of efficient attention modules

Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network (EMNLP 2022)

PyTorch implementation of "Dynamic Structure Pruning for Compressing CNNs" (AAAI 2023 Oral)

가톨릭대 방학중 SW/AI 캠프

This repository created for the NHN ASR hackathon competition.

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs o…

ALIbaba's Collection of Encoder-decoders from MinD (Machine IntelligeNce of Damo) Lab

Uncertainty-Aware Curriculum Learning for Neural Machine Translation (ACL 2020)

PyTorch implementation of DQN, AC, ACER, A2C, A3C, PG, DDPG, TRPO, PPO, SAC, TD3 and ....

IPython notebook with synthetic experiments for AFLite, based on the ICML 2020 paper, "Adversarial Filters of Dataset Biases".