Testing Software for Non-discrimination: An Updated and Extended Audit in the Italian Car Insurance Domain

Context. As software systems become increasingly intertwined with societal infrastructure, the responsibility of software professionals to ensure compliance with non-functional requirements, including but not limited to safety, privacy and non-discrimination, is paramount. Motivation. Ensuring fairness in pricing algorithms allows for fair access to essential services by not discriminating on protected attributes. Method. We replicated a previous empirical study that used black box testing to audit pricing algorithms used by Italian car insurance companies, accessible through a popular online comparator website. In comparison with the aforementioned study, we augmented the number of tests and the number of demographic variables under analysis. Results. The present study corroborates and extends previous findings, highlighting the persistent nature of discrimination across time. Demographic variables significantly impact pricing, with birthplace remaining the primary discriminatory factor against individuals not born in Italian cities. Furthermore, the analysis revealed that driver profiles can determine the number of quotes available to users, thereby denying equal opportunities to all. Conclusion. The study emphasises the significance of incorporating non-discrimination testing into software systems that have a direct impact on individuals’ daily lives. The analysis of algorithms over an extended period facilitates the assessment of their evolution over time. Furthermore, it illustrates the potential of empirical software engineering to enhance the accountability of software systems.