A novel coronavirus has been spreading around the world since early 2020. Due to its high infectiousness and lethality, countries around the world have adopted drastic policy measures to contain the pandemic— with far-reaching implications for economies and societies. We analyze the effect of two non- pharmaceutical interventions (NPI), namely closure of schools as well as a lockdown, on the development of daily case numbers of COVID-19 patients in several countries and US states. By applying Adaptive AI- based Information Systems (AAIS) based on concept drift detection, we conclude that it takes more than two weeks on average until NPIs show a significant effect on the number of new cases. This information is crucial for policymakers as they are currently confronted with difficult decisions to trade off the reopening of the economy with a strict containment of the virus.