Dagstuhl Seminar 24211: Evaluation Perspectives of Recommender Systems — Driving Research and Education
Recommender-Systems Evaluation is and maybe always has been a challenge. The Dagstuhl Seminar 24211 attempted to improve the current state of recommender-systems evaluation, and as a participant, I may report that it succeeded ;-).
The Dagstuhl Seminar 24211, held from May 20 to May 24, 2024, focused on “Evaluation Perspectives of Recommender Systems: Driving Research and Education.” This seminar aimed to critically examine and reflect on the state of evaluating recommender systems by bringing together academia and industry professionals. Building on the discussions from the PERSPECTIVES workshop series at ACM RecSys 2021-2023, the seminar sought to understand the diverse and potentially contradictory perspectives on evaluation in this field. The goal was to foster a setting for development and growth in the evaluation methodologies of recommender systems, which are crucial for their advancement and deployment.
Recommender systems, while largely applied, rely heavily on theories from information retrieval, machine learning, and human-computer interaction. Each field offers different theories and evaluation approaches, making the thorough evaluation of recommender systems a complex task that necessitates integrating these diverse perspectives. The seminar provided a platform for experts from these areas to collaborate and discuss state-of-the-art practices. Emphasizing the importance of considering both technical performance and human elements, the seminar aimed to develop comprehensive evaluation metrics, methods, and practices through interdisciplinary collaboration.
The seminar also highlighted the need to prepare the next generation of researchers to evaluate and advance recommender systems comprehensively. By bringing together participants from various backgrounds, including academic and industry researchers and practitioners, the seminar facilitated a holistic understanding of a recommender system’s performance in its context of use. The organizers, Christine Bauer, Alan Said, and Eva Zangerle, emphasized creating a foundation for future research that integrates technical rigor with practical usability and user interaction considerations, thus driving forward the field of recommender systems.
The participant list included many known figures from the RecSys community, and we all together created a report that is to be published soon.
IntRS@RecSys2020: Joint Workshop on Interfaces and Human Decision Making for Recommender Systems
Andrew Ng introduces a new (free) course on Vector Databases by Sebastian Witalec
Which researchers have made the most significant advancements in recommender system research? (ChatGPT)
About The Author
Joeran Beel
I am the founder of Recommender-Systems.com and head of the Intelligent Systems Group (ISG) at the University of Siegen, Germany https://isg.beel.org. We conduct research in recommender-systems (RecSys), personalization and information retrieval (IR) as well as on automated machine learning (AutoML), meta-learning and algorithm selection. Domains we are particularly interested in include smart places, eHealth, manufacturing (industry 4.0), mobility, visual computing, and digital libraries. We founded or maintain, among others, LensKit-Auto, Darwin & Goliath, Mr. DLib, and Docear, each with thousand of users; we contributed to TensorFlow, JabRef and others; and we developed the first prototypes of automated recommender systems (AutoSurprise and Auto-CaseRec) and Federated Meta Learning (FMLearn Server and Client).