CMU Crowdsourcing Lunch Seminar: Learning from People



Title: Learning from People

Speaker: Nihar B. Shah, Assistant Professor in Machine Learning and Computer Science, Carnegie Mellon University

Date: October 10, 2017

Time: 12:00-1:00pm

Room: Gates-Hillman Complex 6501


Learning from people represents a new and expanding frontier for data science. Two critical challenges in this domain are of developing algorithms for robust learning and designing incentive mechanisms for eliciting high-quality data. In this talk, I describe progress on these challenges in the context of two canonical settings, namely those of ranking and classification. In addressing the first challenge, I introduce a class of “permutation-based” models that are considerably richer than classical models, and present algorithms for estimation that are both statistically optimal and significantly more robust than prior state-of-the-art methods. I also discuss how these algorithms automatically adapt and are simultaneously also optimal over the classical models, thereby enjoying a surprising a win-win. As for the second challenge, I present a class of “multiplicative” incentive mechanisms, and show that they are the unique mechanisms that can guarantee honest responses. Extensive experiments on a popular crowdsourcing platform reveal that the theoretical guarantees of robustness and efficiency indeed translate to practice, yielding several-fold improvements over prior art.


Nihar B. Shah is an Assistant Professor in the Machine Learning and Computer Science departments at CMU. He is a recipient of the the 2017 David J. Sakrison memorial prize from EECS Berkeley for a “truly outstanding and innovative PhD thesis”, the Microsoft Research PhD Fellowship 2014-16, the Berkeley Fellowship 2011-13, the IEEE Data Storage Best Paper and Best Student Paper Awards for the years 2011/2012, and the SVC Aiya Medal 2010. His research interests include statistics, machine learning, and game theory, with a current focus on applications to learning from people.