Kaifeng Lyu
Assistant Professor
Works on machine learning theory, AI safety and alignment, and optimization in deep learning.
Kaifeng Lyu's lab at Tsinghua works on learning theory & optimization, trustworthy & safe ML and interpretability, evaluation & safety of LMs. 24 papers since 2023, 24 at top venues, mostly ICLR and ICML. 5 PhD students and 1 master's student. Applications close .
At a glance
- papers, 2023–2026
- 6/ year
- at T1 venues
- 100%
- PhD students
- 5
- MS students
- 1
- alumni listed
- —
Papers by yearfrom the lab homepage, 2023–2026
- 7 papers2026 · T1 7 · collecting
- 10 papers2025 · T1 10
- 5 papers2024 · T1 5
- 2 papers2023 · T1 2
T1 venuesOtherVenues: ICLR, ICML, NeurIPS, AISTATS
Research focuspaper titles that use each topic's words
- Learning theory & optimization5 papers
- Interpretability, evaluation & safety of LMs1 paper
- Trustworthy & safe ML0 papers
- interdisciplinary information0 papers
- iiis0 papers
Paper list
From the lab homepage.
20267 papers
- ICML (T1)Beyond Problem Solving: UOJ-Bench for Evaluating Code Generation, Hacking, and Repair in Competitive Programming (opens in a new tab)
- ICLR (T1)Can Small Training Runs Reliably Guide Data Curation? Rethinking Proxy-Model Practice (opens in a new tab)
- ICLR (T1)How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining (opens in a new tab)
- ICLR (T1)Larger Datasets Can Be Repeated More: A Theoretical Analysis of Multi-Epoch Scaling in Linear Regression (opens in a new tab)
- AISTATS (T1)Shift is Good: Mismatched Data Mixing Improves Test Performance (opens in a new tab)
- ICML (T1)SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection (opens in a new tab)
- ICML (T1)The Power of Power Law: Asymmetry Enables Compositional Reasoning (opens in a new tab)
202510 papers
- ICLR (T1)A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules (opens in a new tab)
- NeurIPS (T1)Adam Reduces a Unique Form of Sharpness: Theoretical Insights Near the Minimizer Manifold (opens in a new tab)
- NeurIPS (T1)Data Mixing Can Induce Phase Transitions in Knowledge Acquisition (opens in a new tab)
- ICLR (T1)Efficient Stagewise Pretraining via Progressive Subnetworks (opens in a new tab)
- ICLR (T1)Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (opens in a new tab)
- NeurIPS (T1)How Far Are We from Optimal Reasoning Efficiency? (opens in a new tab)
- ICLR (T1)RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval (opens in a new tab)
- ICLR (T1)Safety Alignment Should Be Made More Than Just a Few Tokens Deep (opens in a new tab)
- ICLR (T1)Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias (opens in a new tab)
- ICML (T1)Weak-to-Strong Generalization Even in Random Feature Networks, Provably (opens in a new tab)
20245 papers
- ICLR (T1)A Quadratic Synchronization Rule for Distributed Deep Learning (opens in a new tab)
- ICLR (T1)Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking (opens in a new tab)
- ICLR (T1)DistillSpec: Improving Speculative Decoding via Knowledge Distillation (opens in a new tab)
- NeurIPS (T1)Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates (opens in a new tab)
- ICLR (T1)The Marginal Value of Momentum for Small Learning Rate SGD (opens in a new tab)
Where alumni went
Alumni are not listed on the lab homepage.
Apply to TsinghuaFall 2027
Computer Science PhD (international applicants)
- Round
- Main
- Opens
- GRE
- See program page
- English
- Per school catalog
- Fee
- RMB 800
Not yet confirmed for this intake.
Similar labsnearest by research summary
- SGNTU
Yeshwanth Cherapanamjeri
- Learning theory & optimization
- USNYU
Allen Liu
- Learning theory & optimization
- CAToronto
J. Christopher Beck
- Learning theory & optimization
- USUC Berkeley
Dawn Song
- Trustworthy & safe ML
- USMIT
Ashia Wilson
- Learning theory & optimization
- USGeorgia Tech
Jesse Thomason
- Multimodal & grounded language
Your notes
Save this lab to keep notes.
Data
Some numbers could not be verified:
- The alumni list could not be found on the lab site.
- Collected
- Verified
- Papers
- The lab homepage
- Members
- Lab homepage
- Alumni
- Not collected
Tiers are venue classifications, not lab rankings.
Tsinghuacloses