Artificial Intelligence Systems’ Impermanence: A Showstopper for Assessment?

Artificial intelligence (AI) assessors are struggling with an expectation that they currently cannot meet: To guarantee that the AI system will work as intended without changes that can lead to undesired behavior, such as discrimination. Prior research points out the crucial role of AI impermanence due to AI systems’ learning capabilities, and adaptability. However, we lack explanations on why and how AI impermanence impacts assessments. We provide insights into the impermanent nature of AI systems with a focus on its implications for AI assessments based on qualitative interviews conducted with AI experts from industry and academia. Our study reveals changes that can occur by accident, update, or design, and illustrates how these changes can lead to undesired AI system behavior that impacts assessments (e.g., outdating prior assessment results). This study contributes to a better understanding of AI impermanence and its implications on AI assessments.