Weidi Xie
副教授
Works on computer vision, self-supervised video understanding and AI for medicine.
Weidi Xie's lab at SJTU works on video understanding and generative & foundation models. 91 papers since 2023, 10 at top venues, mostly Nature Communications and ACM MM. 9 PhD students and 2 master's students. Applications close .
At a glance
- papers, 2023–2026
- 24/ year
- at T1 venues
- 11%
- PhD students
- 9
- MS students
- 2
- alumni listed
- —
Papers by yearfrom the lab homepage, 2023–2026
- 22 papers2026 · T1 2 · collecting
- 25 papers2025 · T1 3
- 22 papers2024 · T1 4
- 22 papers2023 · T1 1
T1 venuesOtherVenues: Nature Communications, ACM MM, IJCV, JAMIA, Medical Image Analysis
Research focuspaper titles that use each topic's words
- Video understanding20 papers
- Generative & foundation models16 papers
- understanding6 papers
- self-supervised3 papers
- medicine3 papers
Paper list
From the lab homepage.
202622 papers
- Nature Machine Intelligence (T1)A Knowledge-driven Framework for Predicting Single-cell Responses for Unprofiled Drugs. (opens in a new tab)
- In: Cancer Cell (5-Year Impact Factor = 58.5), .A Knowledge-enhanced Pathology Vision-language Foundation Model for Cancer Diagnosis. (opens in a new tab)
- In: Communications Medicine, .Advancing Radiology Foundation Models with Reasoning through Step-by-step Verification from Daily Reports. (opens in a new tab)
- In: Nature (5-Year Impact Factor = 55.5), .An Agentic System for Rare Disease Diagnosis with Traceable Reasoning. (opens in a new tab)
- Nature Communications (T1)Boosting Pathology Foundation Models via Few-shot Prompt-tuning for Rare Cancer Subtyping (opens in a new tab)
- ACM MMDenseStep2M: A Scalable, Training-Free Pipeline for Dense Instructional Video Annotation. (opens in a new tab)
- To appear at Nature Communications, .EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis (opens in a new tab)
- In: International Conference on Machine Learning (ICML) , .FAIL: Flow Matching Adversarial Imitation Learning for Image Generation. (opens in a new tab)
- To appear at Nature Reviews Computing, .Foundation models for radiology.
- In: European Conference on Computer Vision (ECCV) , .OmniStream: Mastering Perception, Reconstruction and Action in Continuous Streams. (opens in a new tab)
- In: European Conference on Computer Vision (ECCV) , .PhenoLIP: Integrating Phenotype Ontology Knowledge into Medical Vision-Language Pretraining. (opens in a new tab)
- To appear at Nature Chemical Biology, .Phenotypic Bioactivity Prediction as Open-set Biological Assay Querying (opens in a new tab)
202525 papers
- In: The Thirteenth International Conference on Learning Representations (ICLR) , .A Sanity Check for AI-generated Image Detection. (opens in a new tab)
- ACM MMCharacter-Centric Understanding of Animated Movies.
- In: Communications Medicine, .Development of A Large-scale Medical Visual Question-Answering Dataset (PMC-VQA). (opens in a new tab)
- IJCV (T1)Diagnosing Human-Object Interaction Detectors. (opens in a new tab)
- In: Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI)Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant. (opens in a new tab)
- In: Npj Digital Medicine (Nature Portfolio), .Large-Vocabulary Segmentation for Medical Images with Text Prompts. (opens in a new tab)
- In: International Conference on Computer Vision (ICCV) , .Learning Streaming Video Representation via Multitask Training. (opens in a new tab)
- MICCAI WorkshopM^3Builder: A Multi-Agent System for Automated Machine Learning in Medical Imaging (opens in a new tab)
- In: The Thirteenth International Conference on Learning Representations (ICLR) , .Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning. (opens in a new tab)
- In: International Conference on Computer Vision (ICCV) , .MRGen: Diffusion-based Controllable Data Engine for MRI Segmentation towards Unannotated Modalities. (opens in a new tab)
- ACM MMMulti-Agent System for Comprehensive Soccer Understanding. (opens in a new tab)
202422 papers
- In: Conference on Neural Information Processing Systems (NeurIPS) , .A General Protocol to Probe Large Vision Models for 3D Physical Understanding (opens in a new tab)
- ACM MMA Large-scale Dataset for Audio-Language Representation Learning. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .Amodal Ground Truth and Completion in the Wild. (opens in a new tab)
- In: European Conference on Computer Vision (ECCV), .Appearance-based Refinement for Object-Centric Motion Segmentation. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .AutoAD III: The Prequel -- Back to the Pixels. (opens in a new tab)
- In: Asian Conference on Computer Vision (ACCV), .AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description. (opens in a new tab)
- In: Empirical Methods in Natural Language Processing (EMNLP), .EchoSight: Advancing Visual-Language Models with Wiki Knowledge. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .Grounded Question-Answering in Long Egocentric Videos. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .InstaGen: Enhancing Object Detection by Training on Synthetic Dataset. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .Intelligent Grimm - Open-ended Visual Storytelling via Latent Diffusion Models. (opens in a new tab)
- In: European Conference on Computer Vision (ECCV), .Knowledge-enhanced Visual-Language Pretraining for Computational Pathology. (opens in a new tab)
- Nature Communications (T1)Large-scale Long-tailed Disease Diagnosis on Radiology Images. (opens in a new tab)
202322 papers
- In: IEEE Robotics and Automation Letters (RA-L), .Aerial Monocular 3d Object Detection. (opens in a new tab)
- In: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), .Annotation-free Audio-Visual Segmentation. (opens in a new tab)
- In: International Conference on Computer Vision (ICCV) , .AutoAD II: The Sequel – Who, When, and What in Movie Audio Description. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .AutoAD: Movie Description in Context. (opens in a new tab)
- In: British Machine Vision Conference (BMVC) , .Boost Video Frame Interpolation via Simple Motion Adaptation. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .Collaboration Helps Camera Overtake LiDAR in 3D Detection. (opens in a new tab)
- In: International Conference on Computer Vision (ICCV) , .Joint-Relation Transformer for Multi-person Motion Prediction. (opens in a new tab)
- In: Big Task Small Data, 1001-AI, MICCAI Workshop (Oral).K-Diag: Knowledge-enhanced Disease Diagnosis in Radiographic Imaging. (opens in a new tab)
- Nature Communications (T1)Knowledge-enhanced Pre-training for Auto-diagnosis of Chest Radiology Images. (opens in a new tab)
- In: Conference on Computer Vision and Pattern Recognition (CVPR) , .Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervision. (opens in a new tab)
- In: International Conference on Computer Vision (ICCV) , .MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training. (opens in a new tab)
- In: International Conference on Machine Learning (ICML) , .Multi-Modal Classifiers for Open-Vocabulary Object Detection. (opens in a new tab)
Where alumni went
Alumni are not listed on the lab homepage.
Apply to SJTUFall 2027
Computer Science PhD (international applicants)
- Round
- Self-funded
- Opens
- —
- GRE
- See program page
- English
- See program page
- Fee
- See program page
Not yet confirmed for this intake.
Similar labsnearest by research summary
- CNPeking
Wei Hu
- 3D vision & neural rendering
- CNTsinghua
Hang Zhao
- Embodied & active vision
- CNPeking
Shanghang Zhang
- Vision-language & multimodal
- GBUCL
Francisco Porto Guerra E Vasconcelos
- 3D vision & neural rendering
- CHETH Zurich
Konrad Schindler
- 3D vision & neural rendering
- CNSJTU
Weiyao Lin
- Embodied & active vision
Your notes
Save this lab to keep notes.
Data
Some numbers could not be verified:
- The alumni list could not be found on the lab site.
- Collected
- Verified
- Papers
- The lab homepage
- Members
- Lab homepage
- Alumni
- Not collected
Tiers are venue classifications, not lab rankings.
SJTUcloses