Workshop Proposal for ICLR 2025
Contact Us: an-instructive-workshop@googlegroups.com
Post-training methods such as instruction fine-tuning and reinforcement learning from human feedback (RLHF) are critical in unlocking the potential of pre-trained foundation models, enabling them to follow complex instructions and perform a wide range of real-world tasks. For instance, posttraining methods have transformed base language models like GPT-3 into the phenomenal ChatGPT, and adapted models across different modalities like CLIP and Llama into vision-enhanced language models.
Despite significant advancements in post-training research, non-trivial gaps persist between closed proprietary models and open(-weight) models. Due to the limited transparency surrounding proprietary models’ post-training processes, there is a pressing need for an open platform that fosters collaboration and discussion. Building on the success of our inaugural workshop at NeurIPS 2023, we propose a sequel at ICLR 2025 to address the following questions:
In addition to the topics above, this year we propose a special theme: harnessing synthetic data in post-training. Given the high cost of obtaining high-quality human-annotated data, synthetic data has emerged as an effective alternative for model post-training, as evidenced by research on Llama-3 and Qwen-2. While synthetic data reduces annotation costs and enhances performance, it also raises concerns such as narrowing generative diversity and the risk of “model collapse”. We aim to explore the following questions at the workshop:
While the workshop will emphasize the key questions above, we welcome broader discussions and paper submissions related to post-training and instruction-following models. Topics may include but are not limited to (1) effective and reliable evaluation; (2) interpretability and analysis; (3) improving models’ reasoning, mathematical, coding, and decision-making capabilities via post-training; and (4) specific applications and use cases of instruction-following models.