news 
NVIDIA NeMo-Aligner Enhances Supervised Fine-Tuning with Data-Efficient Knowledge Distillation
In a groundbreaking innovation, NVIDIA has unveiled a revolutionary approach to supervised fine-tuning through data-efficient knowledge distillation. The newly developed AI technology, NeMo-Aligner, demonstrates remarkable results in enhancing model performance and efficiency.
For years, pre-training techniques have been widely adopted in the field of artificial intelligence. However, a significant gap remained between these methods and those employed for supervised fine-tuning. NVIDIA’s breakthrough addresses this disparity by introducing knowledge distillation during supervised fine-tuning (SFT). This pioneering approach allows developers to transfer knowledge from a larger teacher model to a more compact student model, thereby achieving comparable accuracy while significantly reducing the need for data.
NeMo-Aligner: A Data-Efficient Approach
NVIDIA’s innovative methodology leverages the concept of “dark knowledge,” which provides a more informative gradient signal by understanding similarities and dissimilarities across classes. By preprocessing teacher models’ predictions, the student model is trained to align with these predictions, resulting in substantial memory savings and faster training times.
The implementation of NeMo-Aligner involves caching teacher models’ predictions and subsequently training the student model to match these predictions. This remarkable technique enables developers to save GPU memory by only storing top-K logits from the teacher model, thus optimizing memory usage while maintaining detailed information transfer.
Empirical Results
Experiments conducted utilizing Nemotron-4 15B as a student model and fine-tuned Nemotron-4 340B as a teacher model have yielded impressive results. The knowledge distillation (KD)-finetuned models surpass vanilla SFT models in numerous benchmarks, including HumanEval, MBPP, and MATH. Notably, the KD-finetuned model necessitates fewer training tokens while achieving exceptional performance across six out of seven evaluation metrics.
In addition, the KD approach excels in the MMLU benchmark, which assesses a wide range of language understanding tasks, surpassing the baseline in both zero-shot and five-shot settings.
Source: Blockchain.News
