Automating Recommender Systems: Advances in Algorithm Selection, Evaluation, and Sustainability

The rapid digitalization of the information age has led to challenges such as information overload, underscoring the critical role of Recommender Systems (RecSys) in organizing and delivering relevant content. Despite their ubiquity, the development of RecSys remains resource-intensive and labor-intensive. Additionally, the environmental impact of RecSys has emerged as a critical yet underexplored concern as modern approaches increasingly employ resource-intensive architectures. The field of Automated Machine Learning (AutoML) has demonstrated significant success in streamlining model development for general machine learning tasks, lowering the barrier of entry for researchers and practitioners by reducing necessary manual labor and expertise while increasing performance. Motivated by their advancements, we investigate the transfer of AutoML principles to RecSys in the framework of Automated Recommender Systems (AutoRecSys) in this dissertation. We focus on solving the automated algorithm selection problem, due to its high relevance for efficient RecSys modeling, and address significant optimization confounders in AutoRecSys. We propose an algorithm selection framework that addresses dataset limitations through community-contributed metadata while offloading computationally intensive meta-learning tasks to server-side components for efficient client-side deployment. Additionally, we provide the first analysis of algorithm selection for ranking prediction tasks with implicit feedback datasets, showing the correlation with ground-truth performance for traditional and AutoML-based meta-models. We quantify the impact of randomness during data splitting, showing that it leads to significant performance deviations unless mitigated through repeated experiments or cross-validation. Furthermore, we analyze the suitability of top-N metrics for optimization, showing that their use in validation does not introduce confounding effects in RecSys evaluation. This reinforces the reliability of conventional evaluation methodologies. Beyond automation and evaluation concerns, we conduct the first comprehensive investigation of the environmental impact of RecSys experiments. We reveal that modern deep learning-based papers emit 42 times more CO2 equivalents than papers employing traditional approaches. Furthermore, we introduce a software tool for measuring and reporting energy consumption in RecSys experiments, enabling researchers to understand and report their environmental impact. Finally, we synthesize our contributions and demonstrate that automated algorithm selection can amortize its environmental impact through widespread adoption. To summarize, this dissertation lays the foundation for future research in algorithm selection through the AutoRecSys framework, further reduces uncertainties for RecSys evaluation methodologies, and helps the RecSys community to understand and address their environmental impact for a sustainable future.