Recent advancements in training large language models (LLMs) to follow “instructions” have significantly increased their ability to comprehend open-ended language commands, encompassing a wide range of needs, preferences, and values.

This remarkable transformation has led to the creation of remarkable industrial models such as GPT-4 and Bard, as well as an increased focus within the open-source and research communities: creating new benchmark and resources [1,2], developing new training methods [3,4], and understanding the limitations of these methods [5]. Furthermore, instruction following powered by LLMs has proven to be effective in multi-modal settings, with applications in image editing [6] and robotic command execution [7].

We organize this workshop to facilitate discussions on advancing instruction tuning methodologies and constructing general-purpose instruction-following models. We believe it is crucial to organize this workshop due to the prevalence of proprietary models with restricted access, thereby creating the need for an open platform to encourage discussions. Moreover, we aim to foster interdisciplinary collaboration by bringing together researchers from diverse fields such as natural language processing, computer vision, robotics, human-computer interaction, AI safety, among others, to share their latest findings and explore potential avenues for future research.

Centering on “instructions,” we invite submissions covering various topics, including but not limited to the list below:

• Modeling: algorithms and pipelines for learning from instructions and human feedback; designing training objectives and rewards; training and inference efficiency
• Data Collection: crowd-sourcing; synthetic data generation; data democratization
• Evaluation and Oversight: effective and reliable oversight over existing models; enforcing guardrails and guarantees for model behaviors; interpretability and analysis
• Engineering and Open-sourcing: best practice in training, evaluation and deployment; open-sourcing efforts; openness and reproducibility
• Applications: long-context, multi-round and personalized instruction-following models
• Multimodal and Multidisciplinary: instruction following models for computer vision, robotics, games, art, etc.
• Limitations, Risks and Safety: bias and fairness; factuality and hallucination; safety concerns arising from instruction-following models
• Other adjacent research topics (e.g., in-context learning, prompting, multi-task learning) that enable better responses to instructions in dynamic environments

Speakers

Check talk details (title, abstract, speaker bio, slides) at this page!

Panel 1

Key Techniques, Insights, and Challenges in Building Instruction-following Models

Time: 10:45-11:30

Panel 2

Open and Collaborative Strategies for Large Language Model Adaptation

Time: 15:15-16:00

Paper Awards

Best Paper

1. Delve into PPO: Implementation Matters for Stable RLHF
   Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin, Yuhao Zhou, Limao Xiong, Lu Chen, Zhiheng Xi, Nuo Xu, Wenbin Lai, Minghao Zhu, Haoran Huang, Tao Gui, Qi Zhang, Xuanjing Huang
2. Learning Interactive Real-World Simulators
   Sherry Yang, Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Dale Schuurmans, Pieter Abbeel

Honorable Mention

1. Understanding Hidden Context in Preference Learning: Consequences for RLHF
   Anand Siththaranjan, Cassidy Laidlaw, Dylan Hadfield-Menell
2. Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
   Sam Toyer, Olivia Watkins, Ethan Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Trevor Darrell, Alan Ritter, Stuart Russell
3. Understanding the Effects of RLHF on LLM Generalisation and Diversity
   Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, Roberta Raileanu
4. Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks
   Lingfeng Sun, Devesh Jha, Chiori Hori, Siddarth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka, Diego Romeres
5. Self-RAG: Self-reflective Retrieval Augmented Generation
   Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, Hannaneh Hajishirzi
6. FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets
   Seonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, Seungone Kim, Yongrae Jo, James Thorne, Juho Kim, Minjoon Seo
   
   

Organizers

Steering Committee

Sponsors