Google DeepMind Unveils ‘SALT’: Optimizing Large Language Model Training with SLMs through Machine Learning

Understanding Large Language Models and Their Challenges

Large Language Models (LLMs) are crucial for a variety of applications, including chatbots, automated content creation, and tasks that require understanding natural language. These models are powerful because they can learn and predict complex language patterns from large sets of data. However, creating LLMs is challenging due to the high computational costs involved. Training these models requires optimizing billions of parameters using vast amounts of data, which demands significant hardware resources and time. As a result, there is a pressing need for new training methods that can overcome these challenges while maintaining or improving the quality of LLMs.

Limitations of Traditional Training Methods

Traditional methods for training LLMs are often inefficient because they treat all data equally, regardless of its complexity. These approaches do not prioritize specific data subsets that could speed up learning and fail to use existing models to aid training. This results in unnecessary computational effort, as simple data is processed alongside complex data without distinction. Additionally, standard self-supervised learning, which involves predicting the next word in a sequence, does not fully utilize smaller, less resource-intensive models that could guide and inform the training of larger models.

The Role of Knowledge Distillation

Knowledge distillation (KD) is a technique commonly used to transfer knowledge from larger, well-trained models to smaller, more efficient ones. However, KD is rarely used in reverse, where smaller models help in training larger models. This represents a missed opportunity, as smaller models, despite their limited capacity, can offer valuable insights into specific data patterns. They can efficiently identify “easy” and “hard” instances, which can significantly impact the training dynamics of LLMs.

Introducing Small model Aided Large model Training (SALT)

Researchers from Google Research and Google DeepMind have developed a new approach called Small model Aided Large model Training (SALT) to address these challenges. SALT uses smaller language models (SLMs) to enhance the efficiency of LLM training. It employs SLMs in two ways: providing additional supervision through soft labels during the initial training phase and selecting valuable data subsets for learning. This method ensures that LLMs focus on informative and challenging data sequences, reducing computational demands while improving the overall quality of the trained model.

How SALT Works

SALT operates in two phases:

Phase One: Leveraging Smaller Models

In the first phase, smaller models act as teachers, transferring their predictive insights to the larger models through knowledge distillation. This process helps align the predictions of the LLMs with the areas where the smaller models excel. Additionally, SLMs identify challenging yet learnable data subsets, allowing LLMs to focus on these critical examples early in training.

Phase Two: Traditional Self-Supervised Learning

The second phase transitions to traditional self-supervised learning, enabling the LLM to independently refine its understanding of more complex data distributions.

Benefits and Results of SALT

Experiments show that a 2.8-billion-parameter LLM trained with SALT on the Pile dataset outperformed a baseline model trained using conventional methods. Notably, the SALT-trained model excelled in reading comprehension, commonsense reasoning, and natural language inference benchmarks, using only 70% of the training steps. This resulted in a 28% reduction in training time. The SALT-trained LLM also achieved a 58.99% accuracy in next-token prediction, compared to 57.7% for the baseline, and had a lower log-perplexity of 1.868 versus 1.951, indicating better model quality.

Key Insights from SALT Research

• SALT reduced the computational requirements for training LLMs by almost 28%, primarily by using smaller models to guide initial training phases.
• The method consistently led to better-performing LLMs across various tasks, including summarization, arithmetic reasoning, and natural language inference.
• By enabling smaller models to select challenging yet learnable data, SALT ensured that LLMs focused on high-value data points, expediting learning without compromising quality.
• The approach is particularly beneficial for institutions with limited computational resources, as it leverages smaller, less costly models to aid in developing large-scale LLMs.
• After supervised fine-tuning, SALT-trained models demonstrated better generalization capabilities in few-shot evaluations and downstream tasks.

Conclusion

SALT redefines LLM training by turning smaller models into valuable training aids. Its innovative two-stage process achieves a balance of efficiency and effectiveness, making it a pioneering approach in machine learning. SALT will be instrumental in overcoming resource constraints, enhancing model performance, and democratizing access to advanced AI technologies. This research highlights the importance of rethinking traditional practices and utilizing existing tools to achieve more with less.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

🚨 Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….