�㽶��ý

The Department of Electrical and Computer Engineering presents its seminar series featuring guest speaker Hongkang Li, postdoctoral researcher in the Department of Electrical and Systems Engineering at the University of Pennsylvania, who will give a presentation on “Theoretical Perspectives of Efficient Learning for Large Foundation Models.” This seminar will take place on Friday, October 3, from 12:45–1:45 p.m. over Zoom.

Abstract

The development of large language models has driven a revolution in the learning paradigm, leading to the proposal of many efficient methods designed for large-scale deep learning systems. However, theoretical studies on these learning algorithms remain limited. We conduct an in-depth theoretical investigation of the fundamental framework of the large language model, the Transformer models. In this talk, I first introduce our recent works in theoretical analysis of the training dynamics of nonlinear Transformers and the resulting in-context learning generalization capability on unseen tasks with and without data distribution shifts. I will then present our analysis on the task vector method and show when and how this method can provably achieve desired multi-task learning, unlearning, and out-of-domain generalization performance.

Biography

Hongkang Li is currently a postdoctoral researcher from the Department of Electrical and Systems Engineering at the University of Pennsylvania. He obtained his Ph.D. degree from the Department of Electrical, Computer, and Systems Engineering (ECSE) at Rensselaer Polytechnic Institute (RPI) in 2024. He obtained his bachelor's degree from the Department of Electronic Engineering and Information Science at the University of Science and Technology in China in 2019. His research interests focus on the mathematics of deep learning. He received the MLCommons ML and Systems Rising Star Award in 2025. He received the Founders Award of Excellence from RPI and the Belsky Award from the School of Engineering of RPI in 2023.