Yonatan Bisk
Associate Professor
Studies grounded and embodied language understanding that connects language, vision and robotics.
Yonatan Bisk's lab at CMU works on multimodal & grounded language. 50 papers since 2023, 32 at top venues, mostly ICLR and CoRL. 9 PhD students and 1 master's student. 21 alumni, most often to Microsoft and OpenAI. Applications close .
At a glance
- papers, 2023–2026
- 13/ year
- at T1 venues
- 64%
- PhD students
- 9
- MS students
- 1
- alumni listed
- 21
Papers by yearfrom the lab homepage, 2023–2026
- 12 papers2026 · T1 4 · collecting
- 12 papers2025 · T1 7
- 16 papers2024 · T1 12
- 10 papers2023 · T1 9
T1 venuesOtherVenues: ICLR, CoRL, CVPR, EMNLP, ICRA
Research focuspaper titles that use each topic's words
- Multimodal & grounded language18 papers
- language14 papers
- grounded0 papers
- language understanding0 papers
- technologies institute0 papers
Paper list
From the lab homepage.
202612 papers
- Third Conference on Language ModelingAmplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models (opens in a new tab)
- IJRR (T1)Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation (opens in a new tab)
- Findings of The Conference on Empirical Methods in Natural Language ProcessingContextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models (opens in a new tab)
- Findings of The Asia-Pacific Chapter of the Association for Computational Linguistics \& the International Joint Conference on Natural Language ProcessingData Augmentation for Improving Contextual Faithfulness Under Knowledge Conflict (opens in a new tab)
- ICPRFieldWorkArena: Agentic AI Benchmark for Real Field Work Tasks (opens in a new tab)
- IROS (T1)IndoorR2X: Indoor Robot-to-Everything Coordination with LLM-Driven Planning (opens in a new tab)
- Computer Vision and Pattern Recognition ConferenceMM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning (opens in a new tab)
- NeurIPS Position Paper TrackPosition: Anthropomorphic Language in AI Discourse Can Be Constructive
- Forty- International Conference on Machine Learning Position Paper TrackPosition: Evaluation of {ML} Resource Utilization Requires Model Life Cycle Assessment (opens in a new tab)
- ICRA (T1)RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptation (opens in a new tab)
- ICLR (T1)RobotArena $\infty$: Scalable Robot Benchmarking via Real-to-Sim Translation (opens in a new tab)
- WACVUnsupervised Discovery of Long-Term Spatiotemporal Periodic Workflows in Human Activities (opens in a new tab)
202512 papers
- RA-LAffordance RAG: Hierarchical Multimodal Retrieval with Affordance-Aware Embodied Memory for Mobile Manipulation (opens in a new tab)
- CoRL (T1)Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models (opens in a new tab)
- Annual Conference of the Nations of the Americas Chapter of the ACLDirect Preference Optimization of Video Large Multimodal Models from Language Model Reward (opens in a new tab)
- TMLR (T1)Efficient Hardware Scaling and Diminishing Returns in Large-Scale Training of Language Models (opens in a new tab)
- ACL (T1)Energy Considerations of Large Language Model Inference and Efficiency Optimizations (opens in a new tab)
- ICLR (T1)Explore Theory-of-Mind: Program-Guided Adversarial Data Generation for Theory of Mind Reasoning (opens in a new tab)
- ICLR (T1)Language Models Need Inductive Biases to Count Inductively (opens in a new tab)
- EMNLP (T1)MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Correction (opens in a new tab)
- RA-LMotIF: Motion Instruction Fine-tuning (opens in a new tab)
- Forty- International Conference on Machine Learning Position Paper TrackPosition: You Can't Manufacture a NeRF (opens in a new tab)
- CVPR (T1)Re-thinking Temporal Search for Long-Form Video Understanding (opens in a new tab)
- Second Conference on Language ModelingREM: Evaluating LLM Embodied Spatial Reasoning through Multi-Frame Trajectories (opens in a new tab)
202416 papers
- Conference on Language ModelingAgentKit: Flow Engineering with Graphs, not Coding (opens in a new tab)
- CoRL (T1)ANAVI: Audio Noise Awareness using Visuals of Indoor environments for NAVIgation (opens in a new tab)
- NeurIPS (T1)DiffusionPID: Interpreting Diffusion via Partial Information Decomposition (opens in a new tab)
- Findings of the Conference on Empirical Methods in Natural Language ProcessingGradient Localization Improves Lifelong Pretraining of Language Models (opens in a new tab)
- Findings of the Conference on Empirical Methods in Natural Language ProcessingHow to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models (opens in a new tab)
- ICRA (T1)MOSAIC: Learning Unified Multi-Sensory Object Property Representations for Robot Perception (opens in a new tab)
- ICRA (T1)Open X-Embodiment: Robotic Learning Datasets and RT-X Models: Open X-Embodiment Collaboration (opens in a new tab)
- CVPR (T1)OpenEQA: Embodied Question Answering in the Era of Foundation Models (opens in a new tab)
- ECCV (T1)Situated Instruction Following (opens in a new tab)
- ECCV (T1)Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation (opens in a new tab)
- ACL (T1)SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents (opens in a new tab)
- ICLR (T1)SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents (opens in a new tab)
202310 papers
- ICLR (T1)Computational Language Acquisition with Theory of Mind (opens in a new tab)
- CVPR (T1)EXCALIBUR: Encouraging and Evaluating Embodied Exploration (opens in a new tab)
- CoRL (T1)HomeRobot: Open-Vocabulary Mobile Manipulation (opens in a new tab)
- IROS (T1)Self-Supervised Object Goal Navigation with In-Situ Finetuning (opens in a new tab)
- NeurIPS (T1)SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs (opens in a new tab)
- CoRL (T1)Spatial-Language Attention Policies for Efficient Robot Learning (opens in a new tab)
- NeurIPS (T1)SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning (opens in a new tab)
- EMNLP (T1)The Framework Tax: Disparities Between Inference Efficiency in Research and Deployment (opens in a new tab)
- Thirty- Conference on Neural Information Processing Systems: Competition TrackTowards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge (opens in a new tab)
- TMLR (T1)Training Vision-Language Transformers from Captions (opens in a new tab)
Where alumni went21 listed, 20% to industry
Industry 3Academia 12
Most often to
- Microsoft
- OpenAI
Also to
- Carnegie Mellon University
- Cornell University
- MIT
Apply to CMUFall 2027
CMU SCS Computer Science PhD
- Round
- Single round
- Opens
- GRE
- See program page
- English
- TOEFL / IELTS / Duolingo (F-1/J-1 non-native speakers)
- Fee
- USD 100 (USD 80 by early deadline)
Confirmed on the program page.
Similar labsnearest by research summary
- USUCLA
Cho-Jui Hsieh
- Trustworthy & safe ML
- CNTsinghua
Yuantao Gu
- Learning theory & optimization
- CHETH Zurich
Thomas Hofmann
- Learning theory & optimization
- GBOxford
Ian Horrocks
- Reasoning & neuro-symbolic AI
- USUC Berkeley
Kannan Ramchandran
- Signal processing & information theory
- GBOxford
Giuseppe De Giacomo
- Reasoning & neuro-symbolic AI
Your notes
Save this lab to keep notes.
Data
Checked against the lab homepage.
- Collected
- Verified
- Papers
- The lab homepage
- Members
- Lab homepage
- Alumni
- Lab homepage
Tiers are venue classifications, not lab rankings.
CMUcloses