Nihar Shah
Associate Professor
Studies peer review and evaluation systems using statistics, machine learning and game theory.
Nihar Shah's lab at CMU works on learning theory & optimization, reasoning & neuro-symbolic AI and interpretability, evaluation & safety of LMs. 28 papers since 2023, 14 at top venues, mostly NeurIPS and TMLR. 2 PhD students and 2 master's students. 15 alumni. Applications close .
At a glance
- papers, 2023–2026
- 7/ year
- at T1 venues
- 50%
- PhD students
- 2
- MS students
- 2
- alumni listed
- 15
Papers by yearfrom the lab homepage, 2023–2026
- 8 papers2026 · T1 6 · collecting
- 8 papers2025 · T1 4
- 5 papers2024 · T1 1
- 7 papers2023 · T1 3
T1 venuesOtherVenues: NeurIPS, TMLR, ICML, EMNLP, ICLR
Research focuspaper titles that use each topic's words
- Learning theory & optimization2 papers
- Interpretability, evaluation & safety of LMs1 paper
- Reasoning & neuro-symbolic AI0 papers
- game theory0 papers
- game0 papers
Paper list
From the lab homepage.
20268 papers
- ICLR (T1)Designing Rules to Pick a Rule: Aggregation by Consistency (opens in a new tab)
- NeurIPS (T1)Learning What Evaluators Value: A Reliable Approach to Modeling Evaluator Preferences (opens in a new tab)
- ICML (T1)Policies Permitting LLM Use for Polishing Peer Reviews Are Currently Not Enforceable (opens in a new tab)
- ICML (T1)Position: Peer Review in ML/AI Conferences Should Separate Publication from Presentation and Offer Non-Anonymous Review Tracks (opens in a new tab)
- EMNLP (T1)ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues (opens in a new tab)
- NeurIPS (T1)Smooth Partial Lotteries for Stable Randomized Selection (opens in a new tab)
- SPIE Astronomical Telescopes and InstrumentationUncertainty-aware Ground-based Telescope Observation Scheduling at ALMA (opens in a new tab)
- IAAIWho is a Better Matchmaker? Human vs. Algorithmic Judge Assignment in a High-Stakes Startup Competition (opens in a new tab)
20258 papers
- TMLR (T1)A Gold Standard Dataset for the Reviewer Assignment Problem (opens in a new tab)
- NeurIPS (T1)A Principled Approach to Randomized Selection under Uncertainty: Applications to Peer Review and Grant Funding (opens in a new tab)
- KDD (T1)Benchmarking Fraud Detectors on Private Graph Data (opens in a new tab)
- PLOS ONEDetecting LLM-Generated Peer Reviews (opens in a new tab)
- Publications of the Astronomical Society of the PacificEnhancing Peer Review in Astronomy: A Machine Learning and Optimization Approach to Reviewer Assignments for ALMA (opens in a new tab)
- Communications of the ACMIdentity Theft in AI Conference Peer Review (opens in a new tab)
- PLOS ONEPeer Reviews of Peer Reviews: A Randomized Controlled Trial and Other Experiments (opens in a new tab)
- USENIX Security (T1)Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions (opens in a new tab)
20245 papers
- PLOS ONEA Randomized Controlled Trial on Anonymizing Reviewers to Each Other in Peer Review Discussions (opens in a new tab)
- PLOS ONEHow do Authors’ Perceptions of their Papers Compare with Co-authors’ Perceptions and Peer-review Decisions? (opens in a new tab)
- TMLR (T1)On the Detection of Reviewer-Author Collusion Rings From Paper Bidding (opens in a new tab)
- AAAI WorkshopReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing (opens in a new tab)
- PLOS ONETesting for Reviewer Anchoring in Peer Review: A Randomized Controlled Trial (opens in a new tab)
20237 papers
- WWW (T1)A Dataset on Malicious Paper Bidding in Peer Review (opens in a new tab)
- PLOS ONEA Large Scale Randomized Controlled Trial on Herding in Peer-Review Discussions. (opens in a new tab)
- TMLR (T1)Assisting Human Decisions in Document Matching (opens in a new tab)
- ACM SIGMETRICSBatching of Tasks by Users of Pseudonymous Forums: Anonymity Compromise and Protection (opens in a new tab)
- PLOS ONECite-seeing and Reviewing: A Study on Citation Bias in Peer Review (opens in a new tab)
- NeurIPS (T1)Counterfactual Evaluation of Peer-Review Assignment Policies (opens in a new tab)
- PLOS ONEThe Role of Author Identities in Peer Review (opens in a new tab)
Where alumni went15 listed
Apply to CMUFall 2027
CMU SCS Computer Science PhD
- Round
- Single round
- Opens
- GRE
- See program page
- English
- TOEFL / IELTS / Duolingo (F-1/J-1 non-native speakers)
- Fee
- USD 100 (USD 80 by early deadline)
Confirmed on the program page.
Similar labsnearest by research summary
- USUMD
Sanghamitra Dutta
- Trustworthy & safe ML
- CNTsinghua
Yujiu Yang
- Learning theory & optimization
- CAToronto
Konstantinos N. Plataniotis
- Signal processing & information theory
- GBImperial
Lucia Specia
- Multilingual, translation & linguistics
- CNSJTU
Yibo Yang
- Trustworthy & safe ML
- USUSC
Sai Praneeth Karimireddy
- Learning theory & optimization
Your notes
Save this lab to keep notes.
Data
Checked against the lab homepage.
- Collected
- Verified
- Papers
- The lab homepage
- Members
- Lab homepage
- Alumni
- Lab homepage
Tiers are venue classifications, not lab rankings.
CMUcloses