Jian Wang

Optimizing Queries: Insights from Fine-Tuning LLMs Reading the paper on contrastive fine-tuning for smaller language models has inspired our approach to model upgrades. Although these "smaller" models, like MiniCPM, are actually larger than BERT, the study shows that fine-tuning them can lead to significantly better performance. This suggests that by leveraging larger models, we can achieve superior query results, even if it requires some additional computational resources. The potential improvements in accuracy make this trade-off worthwhile. This paper has motivated us to explore these techniques further, aiming to optimize our models for the best possible outcomes in our applications. https://lnkd.in/ekgM8NjV

1

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

180

2 Comments

🌍 Thrilled to have virtually presented our paper, "Information Parity: Measuring and Predicting the Multilingual Capabilities of Language Models," at EMNLP 2024 last week! This work builds upon and extends our earlier research on Compression Parity (CP), which was presented at the ICML 2024 Theoretical Foundations of Foundation Models workshop. While the metric itself remains fundamentally the same—now renamed as Information Parity (IP)—this new paper introduces significant advancements in analysis and evaluation. What’s new in this iteration? We conduct extensive statistical analysis to validate IP and demonstrate its superiority over alternative metrics, such as tokenization-based measures (e.g., Tokenization Parity and Tokenizer Fertility) and the proportion of languages in training data. Notably, we show that: • IP consistently outperforms these metrics in predicting multilingual model benchmarks performance, such as multilingual ARC and MMLU. • IP overcomes predictive biases inherent to tokenization, such as those associated with different script families, providing reliable multilingual performance predictions. • Tokenization Parity and Tokenizer Fertility are captured within the explained variance of IP, further highlighting its ability to holistically represent multilingual performance. In essence, IP offers a robust, scalable, and task/prompt-agnostic way to evaluate LLMs, even for low-resource languages where existing methods often fall short. A huge thank you to my co-author, Alon Kipnis, for his invaluable contributions and mentorship throughout this work! Read the full paper here: https://lnkd.in/dzreCQ3N

23

1 Comment

Have you wondered how the decision boundary of in-context learning in LLMs compares to traditional models like Decision Trees and KNN? 🤔 Excited to share our new research, which uncovers unexpected irregularities and non-smoothness in LLMs' in-context decision boundaries! 🔍 📄 Paper: https://lnkd.in/gZAKEJH6 🐦 Tweet summary thread: https://lnkd.in/gY9xkk6d 💻 code: https://lnkd.in/gCsgQk8r Here are some key findings 📌 : - Increasing the number of in-context examples can enhance LLM accuracy on both linear and non-linear classification tasks. But how reliable are these in-context classifiers? We probe their decision boundaries to find out. - By visualizing the decision boundaries, we show that SOTA LLMs, ranging from 1B to large closed-source models such as GPT-3.5-turbo and GPT-4o, all exhibit different non-smooth, irregular decision boundaries, even on simple linearly separable tasks. 🔍 - How do these irregularities arise? 🤔 We study various factors that impact decision boundary smoothness in LLMs, including in-context example count, quantization levels, label semantics & examples order. Then, we identify methods to improve the smoothness of the boundaries. 🛠 - First, increasing in-context examples does not guarantee smoother decision boundaries. While classification accuracy improves with more in-context examples, the decision boundary remains fragmented. - Quantization Impact: Reducing precision from 8-bit to 4-bit can significantly affect areas near the decision boundary, especially in regions with high uncertainties, potentially flipping decisions in these regions. - Label Sensitivity: Decision boundaries are sensitive to label names and example order. Shuffling in-context examples and labels can alter the model’s decision boundaries, indicating a dependency on the semantic prior knowledge of the labels. - 🛠️ Can we improve decision boundary smoothness in LLMs through training? We show that fine-tuning on simple linearly separable tasks can improve the smoothness of decision boundaries and generalize to more complex non-linear, multi-class tasks, enhancing robustness. - Further, fine-tuning the token embedding and attention layers leads to smoother decision boundaries, while fine-tuning the linear prediction head alone does not. - We also explore uncertainty-aware active learning: By adding labels for the most uncertain points to the in-context dataset, we can smoothen the decision boundary more efficiently. - 📚 Effect of Language Pretraining: Transformers trained from scratch on synthetic classification tasks can learn smooth in-context decision boundaries for unseen problems. Thanks for reading about our work! 💡 For a more detailed summary, please check out our tweet thread: https://lnkd.in/gY9xkk6d! 🌟 Work done with amazing collaborators 🙌 Tung Nguyen and our advisor Aditya Grover at UCLA!

65

Today, I want to share one of the best deep learning algorithms for cell segmentation that I've worked with in a collaborative project with Constructor University: Cellpose. Described in three papers and offering open code and a UI for data labeling (links in comments), Cellpose is invaluable for medical image analysis. Here are its main features: 1. Dataset. The Cellpose dataset includes 100 fluorescent images of cultured neurons with manual cytoplasmic segmentation. Additionally, there are 508 fully segmented images from various sources, with 69 reserved for testing. Among these, 98 images depict non-microscopy objects such as fruits, vegetables, man-made materials, and various marine creatures, adding to the dataset's diversity. 2. Data Labels. The training data features two types of labels per cell: - Presence/Absence Label: A binary mask identifying whether a pixel belongs to a cell. - Connectivity Map: Encodes the internal structure of the cell by capturing horizontal and vertical changes (gradients) at each pixel. Connectivity maps were created via a diffusion process based on manual cell outlines, and a neural network was trained to predict these gradients alongside a mask indicating cell locations. 3. Model Architecture. Cellpose employs a neural network based on the U-Net architecture, featuring downsampling and upsampling in a mirror-symmetric fashion. Instead of standard U-Net blocks, residual blocks were used for better performance, and the network's depth was doubled. Global average pooling was applied to the smallest convolutional maps to extract the image's 'style.' 4. Loss Function. Cellpose's loss function combines three components: - Gradient Prediction: L2 loss is used to predict horizontal and vertical gradients. - Cell Presence: A sigmoid function estimates the likelihood of each pixel belonging to a cell, minimized using cross-entropy error. Gradients are scaled by a factor of five to balance the L2 and cross-entropy losses. 5. Results. In a comparative study against Stardist, Mask R-CNN, and two U-Net variations using the Cellpose dataset, Cellpose excelled. It achieved an average precision of 0.91, outperforming Stardist (0.76) and Mask R-CNN (0.80). Cellpose's advantage was even more evident at higher Intersection over Union values, showcasing its capability to accurately capture complex cell shapes. Cellpose stands out in the field of medical image analysis for its robust performance and innovative approach.

23

1 Comment

I wanted to share some insights from a recent paper from Amazon Science in which authors tackled an important challenge in language modeling: minimizing sequence truncation. In language models, truncating input sequences might lead to a loss of crucial information, especially when the models are dealing with long-context tasks such as document summarization or story generation. The authors introduce a clever solution by leveraging a heuristic from the Best Fit Bin Packing problem, an NP-hard problem, to reorder input documents and reduce truncation. The proposed approach helps retain more information compared to other strategies by optimizing how sequences are packed into the model's context window. Authors show that the bin packing solution had biggest performance gain reading comprehension (+4.7%), natural-language inference (+9.3%), context following (+16.8%), and program synthesis. They also found that best-fit packing helped prevent closed-domain hallucination. Authors claim that while the experiments primarily focused on LLM pretraining, best-fit packing is broadly applicable to fine tuning as well. https://lnkd.in/gWquw3ud

28

3 Comments

Retrieval Augmented Generation (RAG) is revolutionizing how large language models (LLMs) access and utilize external knowledge, leading to more accurate and timely responses. However, existing RAG methods often rely on complex iterative refinements or additional training steps to improve retrieval outcomes. Our latest research introduces a simpler yet more effective approach. Introducing Speculative RAG, a new RAG framework that harnesses the power of two LMs – a smaller specialist LM for efficient output draft generation and a larger generalist LM for accurate draft verification. By splitting retrieved documents into subsets and generating drafts in parallel, we reduce input length and mitigate potential biases. The result? State-of-the-art performance with significantly reduced latency! 🚀 Speculative RAG redefines the speed-accuracy tradeoff for RAG. It outperforms conventional RAG systems across multiple benchmarks, boosting accuracy by up to 12.97% while reducing latency by 51% on PubHealth. Our framework eliminates the need for complex refinements or additional training, making RAG more streamlined and scalable. Paper: https://lnkd.in/g_7iEa_u Authors: Zilong Wang, Zifeng Wang, Long Le, Huaixiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, Chen-Yu Lee, Tomas Pfister

46

"Are large language models superhuman chemists?" So what these researchers did was make a test -- a benchmark. They made a test of 7,059 chemistry questions, spanning the gamut of chemistry: computational chemistry, physical chemistry, materials science, macromolecular chemistry, electrochemistry, organic chemistry, general chemistry, analytical chemistry, chemical safety, and toxicology. They recruited 41 chemistry experts to carefully validate their test. They devised the test such that it could be evaluated in a completely automated manner. This meant relying on multiple-choice questions rather than open-ended questions more than they wanted to. The test has 6,202 multiple-choice questions and 857 open-ended questions (88% multiple-choice). The open-ended questions had to have parsers written to find numerical answers in the output in order to test them in an automated manner. In addition, they ask the models to say how confident they are in their answers. Before I tell you the ranking, the researchers write: "On the one hand, our findings underline the impressive capabilities of LLMs in the chemical sciences: Leading models outperform domain experts in specific chemistry questions on many topics. On the other hand, there are still striking limitations. For very relevant topics the answers models provide are wrong. On top of that, many models are not able to reliably estimate their own limitations. Yet, the success of the models in our evaluations perhaps also reveals more about the limitations of the exams we use to evaluate models -- and chemistry -- than about the models themselves. For instance, while models perform well on many textbook questions, they struggle with questions that require some more reasoning. Given that the models outperformed the average human in our study, we need to rethink how we teach and examine chemistry. Critical reasoning is increasingly essential, and rote solving of problems or memorization of facts is a domain in which LLMs will continue to outperform humans." "Our findings also highlight the nuanced trade-off between breadth and depth of evaluation frameworks. The analysis of model performance on different topics shows that models' performance varies widely across the subfields they are tested on. However, even within a topic, the performance of models can vary widely depending on the type of question and the reasoning required to answer it." And with that, I'll tell you the rankings. You can log in to their website at ChemBench.org and see the leaderboard any time for the latest rankings. At this moment I am seeing: gpt-4: 0.48 claude2: 0.29 GPT-3.5-Turbo: 0.26 gemini-pro: 0.25 mistral_8x7b: 0.24 text-davinci-003: 0.18 Perplexity 7B Chat: 0.18 galactica_120b: 0.15 Perplexity 7B online: 0.1 fb-llama-70b-chat: 0.05

3

I’m thrilled to collaborate with Josep from DataBites to share a comprehensive guide of top resources in ML. Josep Ferrer is a veteran technical writer of machine learning having published more than 50+ articles in Kdnuggets and towardsDataScience. Together, we've curated an essential list of resources to help you build a strong foundation in ML. The guide is divided into two parts. In Part 1, we share key theoretical resources: • Top-rated online courses • Must-read books. • Popular podcasts and tutorials to get you started. These high-quality resources are designed to equip you with the knowledge and confidence to tackle the challenges ahead in your ML journey. Checkout the article here: https://lnkd.in/gVDS_Uaw Stay tuned for Part 2 on Saturday, where we'll dive into practical, hands-on learning resources, some classic projects, and advanced techniques to help you apply your knowledge and build a standout portfolio. Don't miss out! Check out Part 1 now and take the first step. -- 𝘐𝘧 𝘺𝘰𝘶 𝘭𝘪𝘬𝘦𝘥 𝘵𝘩𝘦 𝘢𝘳𝘵𝘪𝘤𝘭𝘦, 𝘢𝘭𝘴𝘰 𝘤𝘰𝘯𝘴𝘪𝘥𝘦𝘳 𝘴𝘶𝘣𝘴𝘤𝘳𝘪𝘣𝘪𝘯𝘨, 𝑙𝑖𝑛𝑘 𝑏𝑒𝑙𝑜𝑤 𝑚𝑦 𝑝𝑟𝑜𝑓𝑖𝑙𝑒 𝑝𝑖𝑐𝑡𝑢𝑟𝑒 ☝️ #MachineLearning #LearningJourney #MLResources

69

20 Comments

🚀 Excited to share my latest in-depth blog post exploring a cutting-edge AI framework that combines Hyperdimensional Memory Matching (HDMM), Neuro-Symbolic Context Mapping (NSCM), Evolutionary Reinforcement Scoring (ERS) powered by Genetic Algorithms (GAs) and Linear Regression for advanced feature prioritization and decision-making. This framework isn’t just another iteration of AI — it’s a paradigm shift. By leveraging hyperdimensional computing for high-capacity memory representation, symbolic reasoning for human-like interpretation and evolutionary algorithms for dynamic optimization it creates a system that’s scalable, interpretable, and robust. Key highlights: 📈 HDMM enables unmatched scalability for high-dimensional data encoding and matching. 🧠 NSCM bridges symbolic reasoning with data-driven models for enhanced decision-making. ⚙️ ERS with GAs optimizes features in a way that’s both adaptive and efficient. 🧮 Linear Regression integrates seamlessly for actionable insights and predictions. This isn’t just about better AI — it’s about creating systems that mimic cognitive processes, adapt dynamically, and deliver transparent and interpretable results. Let’s push the boundaries of what’s possible in AI! 💡 Read the full article here #ArtificialIntelligence #MachineLearning #HyperdimensionalComputing #NeuroSymbolicAI #GeneticAlgorithms #Innovation #FutureOfAI

1

Revolutionizing Spreadsheet Processing: A New Approach with LLMs After reading the SPREADSHEETLLM paper, I was inspired to develop standalone tools for structural-anchor extraction, inverted-index translation, and data-format-aware aggregation. These tools can be independently utilized and dynamically invoked by an LLM agent based on specific requirements. By integrating these methods with retrieval-augmented generation (RAG), we can significantly enhance token efficiency and maintain clear data representation. This innovative approach promises to revolutionize spreadsheet processing, making it more efficient and intelligent. Combining traditional algorithms with cutting-edge LLM capabilities offers a powerful solution for complex data management and analysis. https://lnkd.in/eFT6SUAq

4

Predictions based on Machine Learning and Statistics often rely on maximal likelihood, which can trap LLMs in biases inherited from the majority opinions of training data. This problem has been substantially addressed through conditional statistics, allowing for mitigation of biases and preventing the cancellation of minority perspectives. https://lnkd.in/gMXKqwyB

7

Interesting article on merging LLMs to fully automate scientific research!

1

LLM Evaluation Metrics: Everything You Need for LLM Evaluation - Confident AI https://bit.ly/3TYhkaP

30

Our paper on "Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models" was also accepted to Neurips! One of the coolest parts of the paper was finding an "en passant" feature in our SAE, which activates whenever an en passant capture is available. I consider this strong evidence that LLMs and SAEs are learning something real in their training process. ChessGPT is trained solely to predict the next character in PGN strings, and an SAE is trained solely to reconstruct ChessGPT activations. Despite the minimal amount of supervision in this process, we find very meaningful features in our SAE. More information here: https://lnkd.in/g6avcMTw

33

2 Comments

🚀Introducing Spatiotemporal Skip Guidance (STG) for Video Diffusion Models ✨ 🎯We're excited to present STG, a new sampling guidance method that enhances video diffusion models. STG can work independently or in conjunction with Classifier-Free Guidance (CFG). 🔬Our approach creates an aligned "weak model" through layer skipping, which guides the main model to produce higher quality outputs by moving away from this model. Key Features: • Improves spatiotemporal quality of generated videos • no quality-diversity tradeoff; Preserves diversity and motion dynamics • Zero training required - plug-and-play implementation • Compatible with any transformer-based video diffusion models (e.g., Mochi, SVD, Open-Sora) Paper: "Spatiotemporal Skip Guidance for Enhanced Video Diffusion Sampling" 📄 arXiv: arxiv.org/abs/2411.18664 🤗 huggingface: https://lnkd.in/g-8CeSiv 🌐 Project Page: https://lnkd.in/gd8FjZPY 💻 Github: https://lnkd.in/gwy92WsM

131

2 Comments

Last week, I presented a seminar on Large Language Models (LLMs), focusing on their insights, abilities, and mechanisms. It was an enriching experience to discuss these topics and engage with participants interested in this fascinating area of AI. For those who couldn’t attend, I’ve shared the seminar recording below. Feel free to watch and share your thoughts or questions in the comments—I’d be glad to continue the conversation. #LLM #AI #Seminar #MachineLearning #Education https://lnkd.in/dVZ8NPxy

53

Have you ever wondered how live camera feeds achieve seamless synchronization? As a Research Assistant at the Indian Institute of Technology, Kanpur, I tackled this challenge using GStreamer. This robust multimedia framework also serves as a backend for NVIDIA Deepstream. From filters and codecs to encoders and decoders, GStreamer proved indispensable in creating robust real-time video pipelines. In my latest Medium blog (part 3), I dive deeper into GStreamer’s functionalities and how it simplifies complex multimedia processing. If you're curious about leveraging GStreamer for your projects, head to my Medium post for all the technical details. 🔑 Key Takeaways: Understanding how filters (encoders, decoders, muxers, demuxers) process data Leveraging queues and tees for asynchronous processing and stream branching Managing data with buffers and understanding GStreamer's event states This post will help you master the essentials of GStreamer and optimize your media workflows! 🎬💡 Check out the full post for more insights #GStreamer #C++ #MediaProcessing #Coding #Tech #SoftwareDevelopment #OpenSource

21

Just wrapped up the Finetuning Large Language Models short course by Lamini—quick but packed with insights! It clearly covered when to use finetuning versus prompting for LLMs, making the basics easy to grasp. In just a few sections, it touched on selecting the right open-source models, data preparation, and the steps for training, evaluation, and iteration. The instruction finetuning overview and the quick start tips were especially helpful and practical. Now, I feel more prepared to apply language model finetuning to specific domains. Highly recommend it for anyone needing a quick dive into finetuning essentials! #LLLMFinetuning #finetuning #prompting #LLMs  #training #evaluation #datapreparation

17

1 Comment