August 7th, 2023

Foundations and Applications in Large-scale AI Models
-Pre-training, Fine-tuning, and Prompt-based Learning

Workshop held in conjunction with KDD 2023

Deep learning techniques have advanced rapidly in recent years, leading to significant progress in pre-trained and fine-tuned large-scale AI models. For example, in the natural language processing domain, the traditional "pre-train, fine-tune" paradigm is shifting towards the "pre-train, prompt, and predict" paradigm, which has achieved great success on many tasks across different application domains such as ChatGPT/BARD for Conversational AI and P5 for a unified recommendation system. Moreover, there has been a growing interest in models that combine vision and language modalities (vision-language models) which are applied to tasks like Visual Captioning/Generation.

Considering the recent technological revolution, it is essential to have a workshop at the KDD conference that emphasizes these paradigm shifts and highlights the paradigms with the potential to solve different tasks. This workshop will provide a platform for academic and industrial researchers to showcase their latest work, share research ideas, discuss various challenges, and identify areas where further research is needed in pre-training, fine-tuning, and prompt-learning methods for large-scale AI models. The workshop will also foster the development of a strong research community focused on solving challenges related to large-scale AI models, providing superior and impactful strategies that can change people’s lives in the future.

We invite submissions of long (eight papers) and short (four pages) papers, representing original research, preliminary research results, and proposals for new work in academia or industry. All submissions will be single-blind and will be peer-reviewed by an international program committee of researchers and industrial professionals and experts. Accepted submissions will be required to be presented at the workshop and will be published in a dedicated workshop proceeding by the workshop organisers.

Topics of interest in this workshop include but are not limited to:

Pre-training:

- Improvements in pre-training: supervised pre-training, self-supervised pre-training with various auxiliary tasks, meta-learning, prompt-based Learning, multi-modal pre-training etc.

- Novel pre-training methods to maximize generalization

- Model selection for pre-trained models

- Pre-training for various application domains, such as computer vision, natural language processing, robotics, etc

Fine-tuning:

- Domain/task adaptive fine-tuning

- Intermediate-task, multi-task, self-supervised, MLM fine-tuning

- Parameter-efficient fine-tuning: sparse parameter tuning, pruning

- Text-to-Text, Text-to-image, Image-to-text, multi-modal fine-tuning, effectively using large autoregressive pre-trained models

- Fine-tuning for various application domains, such as computer vision, natural language processing, robotics, etc

Prompted/Instruction-based:

- Manual Template Engineering

- Automated Template Learning

- Multi-Prompt Learning; Multi-tasks instruction tuning

- Instruction tuning with HF/RLHF

- chain-of-thought (CoT) prompting

Performance:

- Model compression techniques

- Large-scale model deployments

- Efficient and effective training/inference

- Empirical analysis of various pre-training and fine-tuning methods

- Generalization bounds of different pre-training and fine-tuning methods

- Stability, sparsity and robustness strategies

Downstream tasks of large-scale models:

- NLP models for Text Generation,Text Summarization,Question Answering and other downstream tasks

-CV models for Image Captioning, Semantic Segmentation,Object Tracking and other downstream tasks

Applications powered by large-scale models:

-Conversational AI, Conversational Chatbots

- Enhanced Web Search, Search Engine

- Unified, Personalized next generation recommender systems

Call for Papers

Paper Submission Deadline: June 16, 2023, 11:59 PM AoE.

Paper Notification: Jun. 26, 2023, 11:59 PM AoE.

Camera Ready Version: July. 15, 2023, 11:59 PM AoE.

Half-Day Workshop: Aug. 7, 2023

This workshop follows the submission requirement by KDD.

Instructions:

- Long paper (up to 8 pages) and short paper (up to 4 pages). The page limit includes the bibliography and any possible appendices.

- Single-blind peer review

- All papers must be formatted according to ACM sigconf template manuscript style, following the submission guidelines available at: https://www.acm.org/publications/proceedings-template.

- Papers should be submitted in PDF format, electronically, using the EasyChair submission system

- All selected papers will invited for presentation.

Inquiry Email: llmai.workshop@gmail.com

Invited Speakers

Image

Ed H. Chi

Distinguished Scientist at Google

Speaker Bio:
Ed H. Chi is a Distinguished Scientist at Google, leading several machine learning research teams focusing on neural modeling, reinforcement learning, dialog chatbot models called LaMDA, reliable/robust machine learning, and recommendation systems in Google Brain team. His team has delivered significant improvements for YouTube, News, Ads, Google Play Store at Google with ~600 product improvements since 2013. With 39 patents and >150 research articles, he is also known for research on user behavior in web and social media. Prior to Google, he was the Area Manager and a Principal Scientist at Palo Alto Research Center's Augmented Social Cognition Group, where he led the team in understanding how social systems help groups of people to remember, think and reason. Ed completed his three degrees (B.S., M.S., and Ph.D.) in 6.5 years from University of Minnesota. Recognized as an ACM Distinguished Scientist and elected into the CHI Academy, he recently received a 20-year Test of Time award for research in information visualization. He has been featured and quoted in the press, including the Economist, Time Magazine, LA Times, and the Associated Press. An avid swimmer, photographer and snowboarder in his spare time, he also has a blackbelt in Taekwondo.

Image

Tania Bedrax-Weiss

Director of Research at Google

Speaker Bio:
Tania Bedrax Weiss is a Director of Research at Google. Her current focus is on identifying untapped problems at the intersection of Recommender Systems, Natural Language Understanding, and Vision at Google and outlining research agendas to advance the state-of-the-art. During the 15+ years she’s been at Google she has launched transformative products in Google Play, Ads, and Search. Previously, she worked at NASA Ames Research Center and was part of the team that wrote the software used to schedule daily observations for Spirit and Opportunity. She has also worked in industry on automated configuration and pricing systems. She holds a PhD from the University of Oregon in Artificial Intelligence.

Image

Shafiq Joty

Research Director at Salesforce AI Research

Speaker Bio:
Shafiq Joty is currently a research director at Salesforce Research (Palo Alto, USA), where he directs the NLP group's work on large language modeling (LLM) and generative AI. He is also a tenured Associate Professor (currently on leave) in the School of Computer Science and Engineering (SCSE) at NTU. He was a founding manager of the Salesforce Research Asia (Singapore) lab. His research contributed to 20+ patents and more than 120+ papers in top-tier NLP and ML conferences and journals. He severed as a PC chair of SIGDIAL-2023, best paper award committee of ICLR-23, NAACL-22 and a (senior) area chair for all the NLP and ML conferences.

Image

Yikang Shen

Research Staff Member at MIT-IBM Watson Lab

Speaker Bio:
Yikang Shen is a Research Staff Member at MIT-IBM Watson Lab. He obtained his Ph.D. from Mila lab at the University of Montreal, where he worked with Aaron Courville. During his Ph.D., his research focused on studying the fundamental mechanisms that allow deep learning methods to understand the latent structure of human language and model its compositional phenomenons. He received the ICLR best paper award in 2019 for his work on inducing grammar from language modeling. After joining IBM, he worked on building modular foundation models that are flexible, efficient, and extendable. He is also interested in developing instruction-tuning and alignment methods that require minimum human annotation. He is now leading IBM's effort to train a large-scale modular language model.

Accepted Papers

- Retrieval-Augmented Multimodal Language Modeling by Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer and Wen-Tau Yih

- AutoHint: Automatic Prompt Optimization with Hint Generation by Hong Sun, Xue Li, Yinchuan Xu, Youkow Homma, Qi Cao, Min Wu, Jian Jiao and Denis Charles

- Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head Generation by Zhichao Wang, Mengyu Dai and Keld Lundgaard

- Compositional Prompting with Successive Decomposition for Multimodal Language Models by Long Hoang Dang, Thao Minh Le, Tu Minh Phuong and Truyen Tran

- Dr. LLaMA: Improving Small Language Models on PubMedQA via Generative Data Augmentation,Zhen Guo, Yanwei Wang, Peiqi Wang and Shangdi Yu.

- In-Context Learning User Simulators for Task-Oriented Dialog Systems by Silvia Terragni, Modestas Filipavicius, Nghia Khau, Bruna Guedes, André Manso and Roland Mathis

- Challenges in post-training quantization of Vision Transformers by Piotr Kluska, Florian Scheidegger, A. Cristano I. Malossi and Enrique S. Quintana-Ortí

-Extractive Summarization via ChatGPT for Faithful Summary Generation by Haopeng Zhang, Xiao Liu and Jiawei Zhang

- Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion by Haotian Ju, Dongyue Li, Aneesh Sharma and Hongyang Zhang

Workshop Program

Time Speaker Title
8:00-8:10AM, 2023/08/07 (PDT) Host Chair Welcome and Open Remarks
8:10-8:40AM, 2023/08/07 (PDT) Ed Chi [Google] Talk 1:
8:40-9:10AM, 2023/08/07 (PDT) Tania Bedrax-Weiss [Google] Talk 2: Large-scale AI Model Research at Google Pre-training, Fine-tuning, and Prompt-based Learning
9:10-9:25AM, 2023/08/07 (PDT) Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer and Wen-Tau Yih Paper-1: Retrieval-Augmented Multimodal Language Modeling
9:25-9:40AM, 2023/08/07 (PDT) Silvia Terragni, Modestas Filipavicius, Nghia Khau, Bruna Guedes, André Manso and Roland Mathis Paper-2: In-Context Learning User Simulators for Task-Oriented Dialog Systems
9:40-9:55AM, 2023/08/07 (PDT) Piotr Kluska, Florian Scheidegger, A. Cristano I. Malossi and Enrique S. Quintana-Ortí Paper-3 : Challenges in post-training quantization of Vision Transformers/td>
9:55-10:10AM, 2023/08/07 (PDT) Haotian Ju, Dongyue Li, Aneesh Sharma and Hongyang Zhang Paper-4 : Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion/td>
10:10-10:30AM, 2023/08/07 (PDT) Coffee Break
10:30-11:00AM, 2023/08/07 (PDT) Shafiq Joty [Salesforce] Talk 3: NLP Research in the Era of LLMs
11:00-11:30AM, 2023/08/07 (PDT) YiKang Shen[IBM] Talk 4: Modular Large Language Model and Principle-Driven alignment with Minimal Human Supervision
11:30 - 11:40AM, 2023/08/07 (PDT) Hong Sun, Xue Li, Yinchuan Xu, Youkow Homma, Qi Cao, Min Wu, Jian Jiao and Denis Charles Paper-5: AutoHint: Automatic Prompt Optimization with Hint Generation
11:40-11:50AM, 2023/08/07 (PDT) Zhichao Wang, Mengyu Dai and Keld Lundgaard Paper-6: Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head Generation
11:50-12:00PM, 2023/08/07 (PDT) Long Hoang Dang, Thao Minh Le, Tu Minh Phuong and Truyen Tran Paper-7: Compositional Prompting with Successive Decomposition for Multimodal Language Models
12:00PM-12:10PM, 2023/08/07 (PDT) Zhen Guo, Yanwei Wang, Peiqi Wang and Shangdi Yu Paper-8: Dr. LLaMA: Improving Small Language Models on PubMedQA via Generative Data Augmentation
12:10-12:20PM, 2023/08/07 (PDT) Haopeng Zhang, Xiao Liu and Jiawei Zhang Paper-9 : Extractive Summarization via ChatGPT for Faithful Summary Generation
12:20 - 12:30PM, 2023/08/07 (PDT) Closing Remarks

WORKSHOP ORGANIZERS

General Chairs

Image

Ed H. Chi

Google Research
Image

Wei Liu

University of Technology Sydney
Image

Nitesh Chawla

University of Notre Dame
Image

James Bailey

The University of Melbourne

Program Committee Chairs

Image

Linsey Pang

Salesforce
Image

Derek Cheng

Google Research
Image

Dhaval Patel

IBM Research
Image

Sameep Mehta

IBM Research

Contact: llmai.workshop@gmail.com

Program Committee

Chidansh Bhatt, IBM RESEARCH

Wang-Cheng Kang, Google

Ruoxi Wang, Google

Hima Patel, IBM RESEARCH

Abhishek Malvankar, IBM RESEARCH

Jianmo Ni , Google

Abby Xianjing , Appel

Mengyu Dai, Salesforce

Zhichao Wang , Georgia Institute of Technology

XueLi, Microsoft

Sandeep Singh Sandha, Abacus.AI

Sahisnu Mazumder, Intel Labs