Tao Jin
教师
Works on multimodal machine learning and speech synthesis.
Tao Jin's lab at Zhejiang works on multimodal & grounded language, speech synthesis & voice and vision-language & multimodal. 77 papers since 2023, 53 at top venues, mostly ACL and CVPR. Applications close .
At a glance
- papers, 2023–2026
- 20/ year
- at T1 venues
- 69%
- PhD students
- —
- MS students
- —
- alumni listed
- —
Papers by yearfrom DBLP, 2023–2026
- 15 papers2026 · T1 11 · collecting
- 31 papers2025 · T1 22
- 20 papers2024 · T1 13
- 11 papers2023 · T1 7
T1 venuesOtherVenues: ACL, CVPR, ACM MM, ICLR, Findings of ACL
Research focuspaper titles that use each topic's words
- Vision-language & multimodal26 papers
- Multimodal & grounded language24 papers
- Speech synthesis & voice17 papers
- speech synthesis1 paper
- software technology0 papers
Paper list
From DBLP.
202615 papers
- ICLR (T1)AlignSep: Temporally-Aligned Video-Queried Sound Separation with Flow Matching (opens in a new tab)
- Findings of ACLBridging the Pose-Semantic Gap: A Cascade Framework for Text-Based Person Anomaly Search (opens in a new tab)
- CVPR (T1)Decoupling Stability and Plasticity for Multi-Modal Test-Time Adaptation (opens in a new tab)
- KDD (T1)DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark (opens in a new tab)
- ACL (T1)DPDV: Dual-Pathway and Dual-View Representation Learning for Bridging Information Asymmetry in Text-Video Retrieval (opens in a new tab)
- TIP (T1)Emphasizing Domain Differences Through Interactive-Augmented Prompts in Continual Audio-Visual Speech Recognition (opens in a new tab)
- Findings of ACLGenerative-to-Discriminative Test-Time Adaptation via Manifold-Aware Diffusion and Bayesian Distillation (opens in a new tab)
- ICLR (T1)MARS-Sep: Multimodal-Aligned Reinforced Sound Separation (opens in a new tab)
- ACL (T1)Rectifying the Emotional Flow: Aligning Priors and Dynamic Guidance for High-Arousal Text-to-Speech (opens in a new tab)
- ACL (T1)SAME: Signer-Aware Mixture-of-Experts for Test-Time Adaptation in Sign Language Translation (opens in a new tab)
- AAAI (T1)Scene-Aware Spatiotemporal Generalization: Towards Robust Temporal Action Detection Across Domains (opens in a new tab)
- IEEE Trans. Big DataTAG: Triple Alignment With Rationale Generation for Knowledge-Based Visual Question Answering (opens in a new tab)
202531 papers
- AAAI (T1)A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter (opens in a new tab)
- NeurIPS (T1)AHa-Bench: Benchmarking Audio Hallucinations in Large Audio-Language Models (opens in a new tab)
- AAAI (T1)Bridging the Gap for Test-Time Multimodal Sentiment Analysis (opens in a new tab)
- EMNLP (T1)Chat-Driven Text Generation and Interaction for Person Retrieval (opens in a new tab)
- CVPR (T1)ConceptGuard: Continual Personalized Text-to-Image Generation with Forgetting and Confusion Mitigation (opens in a new tab)
- ICASSP (T1)Curriculum Learning aided Audio-Visual Speech Recognition with Arbitrary Speaker Number (opens in a new tab)
- Findings of NAACLData-Efficiently Learn Large Language Model for Universal 3D Scene Perception (opens in a new tab)
- ICLR (T1)Diff-Prompt: Diffusion-Driven Prompt Generator with Mask Supervision (opens in a new tab)
- NAACL (T1)Efficient Prompting for Continual Adaptation to Missing Modalities (opens in a new tab)
- ICML (T1)IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Models (opens in a new tab)
- ACM MMISDrama: Immersive Spatial Drama Generation through Multimodal Prompting (opens in a new tab)
- Frontiers Inf. Technol. Electron. Eng.Multi-talker audio-visual speech recognition towards diverse scenarios (opens in a new tab)
202420 papers
- NeurIPS (T1)Action Imitation in Common Action Space for Customized Action Image Synthesis (opens in a new tab)
- EMNLP (T1)AudioVSR: Enhancing Video Speech Recognition with Audio Data (opens in a new tab)
- ACM MMBoosting Speech Recognition Robustness to Modality-Distortion with Contrast-Augmented Prompts (opens in a new tab)
- ACM MMCalibrating Prompt from History for Continual Vision-Language Retrieval and Grounding (opens in a new tab)
- NeurIPS (T1)Classifier-guided Gradient Modulation for Enhanced Multimodal Learning (opens in a new tab)
- NeurIPS (T1)E3: Exploring Embodied Emotion Through A Large-Scale Egocentric Video Dataset (opens in a new tab)
- KDD (T1)EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration (opens in a new tab)
- NeurIPS (T1)Extending Multi-modal Contrastive Representations (opens in a new tab)
- ICML (T1)FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion (opens in a new tab)
- Multim. Tools Appl.GTADT: Gated tone-sensitive acne grading via augmented domain transfer (opens in a new tab)
- ACM MMLow-rank Prompt Interaction for Continual Vision-Language Retrieval (opens in a new tab)
- CVPR (T1)MPOD123: One Image to 3D Content Generation Using Mask-Enhanced Progressive Outline-to-Detail Optimization (opens in a new tab)
202311 papers
- Findings of ACLContrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation (opens in a new tab)
- CVPR (T1)DATE: Domain Adaptive Product Seeker for E-Commerce (opens in a new tab)
- IEEE Trans. Geosci. Remote. Sens.Electromagnetic Imaging Boosted Visual Object Recognition Under Difficult Visual Conditions (opens in a new tab)
- ICCV (T1)Exploring Group Video Captioning with Efficient Relational Approximation (opens in a new tab)
- CVPR (T1)Gloss Attention for Gloss-free Sign Language Translation (opens in a new tab)
- ICCV (T1)MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition (opens in a new tab)
- ACL (T1)OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment (opens in a new tab)
- ACM MMRethinking Missing Modality Learning from a Decoding Perspective (opens in a new tab)
- Findings of ACLSemantic-conditioned Dual Adaptation for Cross-domain Query-based Visual Segmentation (opens in a new tab)
- ACL (T1)TAVT: Towards Transferable Audio-Visual Text Generation (opens in a new tab)
- ACL (T1)Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning (opens in a new tab)
Where alumni went
Alumni are not listed on the lab homepage.
Apply to ZhejiangFall 2027
Computer Science PhD (international applicants)
- Round
- Self-funded
- Opens
- —
- GRE
- See program page
- English
- Per ZJU language requirements
- Fee
- RMB 800
Not yet confirmed for this intake.
Similar labsnearest by research summary
- CNSJTU
Yu Wang
- Large language models
- SGNTU
Shafiq Joty
- Large language models
- USUC Berkeley
Dan Klein
- Multilingual, translation & linguistics
- CNSJTU
Renqiu Xia
- Reasoning & agents
- GBEdinburgh
Alexandra Birch
- Multilingual, translation & linguistics
- USUCLA
Nanyun Peng
- Large language models
Your notes
Save this lab to keep notes.
Data
Some numbers could not be verified:
- The member list could not be found on the lab site.
- The alumni list could not be found on the lab site.
- The lab site and the paper record do not agree.
- Collected
- Verified
- Papers
- DBLP
- Members
- Not collected
- Alumni
- Not collected
Tiers are venue classifications, not lab rankings.
Zhejiangcloses