Aligning Machine Learning with Society

Abstract: Machine Learning (ML) systems are increasingly integrated into society, but challenges arise when human incentives and expectations are overlooked. In this talk, I will present frameworks for aligning ML with society, focusing on strategic classification and personalization in decision-making. Strategic classification models scenarios where individuals, aware of the deployed classifier, manipulate their observable attributes to achieve favorable outcomes. For example, individuals might apply for additional credit cards to boost their credit score just so they can qualify for a loan, even though it doesn’t impact their ability to repay the loan. I will survey extensions to strategic classification, including sequential classifiers, partial knowledge about the deployed classifier, the problem in the context of large language models, and whether classic learnability implies strategic learnability. In addition, I will discuss multi-objective Markov Decision Processes (MDPs), which involve multiple, potentially conflicting objectives. In classic reinforcement learning and MDPs, policies are evaluated with scalar reward functions, implying that every optimal policy is optimal for all users. However, real-world scenarios involve multiple, sometimes conflicting objectives, necessitating personalized solutions. I will present an MDP framework that accommodates different user preferences over objectives, where preferences are learned via policy comparisons, and the goal is to efficiently compute a near-optimal policy for a given user.