On the Interplay of Transparency and Fairness in AI-Informed Decision-Making

Using artificial intelligence (AI) systems for informing high-stakes decisions has become increasingly pervasive in a variety of domains, including but not limited to hiring, lending, or law enforcement. As the nature of such AI-informed decisions is becoming ever more consequential, numerous examples have shown the potential for adverse effects on humans due to the deployment and operation of such systems. Consequently, researchers and policymakers have articulated the need for heightened scrutiny and initiatives that promote greater transparency and equity in the design and implementation of AI systems. The first contribution of this thesis is to synthesize the research discourse by conducting a structured literature review. The primary focus is to investigate the intricate relationship between transparency and fairness in AI systems. From this review, we infer several desiderata for transparency, such as its potential to enable fairness audits. However, many of these desiderata lack theoretical or empirical support, which casts doubt on the commonly conjectured role of transparency as a panacea in AI-informed decision-making. These insights form an important cornerstone for all subsequent work presented in this thesis. ... mehrThe remainder of this thesis studies ramifications of transparency mechanisms as they pertain to various dimensions of fairness. First, we address fully automated decision-making. Herein, we propose a ranking-based algorithm that is inherently transparent and respects established notions of fairness. Recognizing the complexity of fairness as a concept, we also assess human perceptions towards AI systems through two mixed-method online studies. In the first study, we observe that perceptions are positively associated with the amount of information that is provided about a given system, ceteris paribus. This could be exploited, however, when malevolent system designers seek to manipulate more vulnerable stakeholders into trusting problematic systems. In the second study, we contrast human perceptions towards AI-based versus human-based decisions. Here, we find that perceptions tend to be more favorable towards automation, although for reasons that may not always be justified. We then turn to human-in-the-loop decision-making, that is, scenarios where humans are endowed with discretionary power to override AI-issued decisions. We first establish a framework on the interplay of reliance behavior—how humans adhere to or override AI systems—and decision quality. This framework eventually serves as a blueprint for our third online study. The findings from this study indicate that human reliance behavior varies based on whether transparency mechanisms disclose the use of sensitive information, such as gender or race, by the AI system. Interestingly, these differences in reliance behavior imply opposing downstream effects on the fairness of decisions. Overall, this thesis underscores the complex, multidimensional relationship between transparency and fairness in AI systems. In this respect, it sheds light on the often-overlooked limitations of popular transparency mechanisms and emphasizes the discrepancy between desiderata and empirical evidence. By advocating for a reevaluation of transparency as a more comprehensive concept rather than a monolithic notion, our findings provide valuable insights for researchers and system designers aiming to create genuinely responsible AI systems.