-------- Forwarded Message -------- Subject: [wkwi] [CfP] TORS Special Issue on Perspectives on Recommender Systems Evaluation Date: Thu, 12 May 2022 22:10:32 +0000 From: Bauer, C. (Christine) c.bauer@uu.nl Reply-To: Bauer, C. (Christine) c.bauer@uu.nl To: Bauer, C. (Christine) c.bauer@uu.nl
Dear colleagues,
We would like to draw your attention to the CfP for the ************************ Special Issue on Perspectives on Recommender Systems Evaluation at ACM Transactions on Recommender Systems (TORS) ************************
Abstract submission deadline: September 30, 2022 Full article submission deadline: November 30, 2022
You can find the details below - and via the following link: https://dl.acm.org/pb-assets/static_journal_pages/tors/pdf/TORS-CfP-SI-Evalu...
We are looking forward to your contributions.
Best regards, Christine Bauer, Alan Said, and Eva Zangerle (Guest editors)
--------------------------------------------------------------------------------------------------------------------------------------
************************** Call for Papers **************************
Special Issue Perspectives on Recommender Systems Evaluation ACM Transactions on Recommender Systems
Guest Editors: • Christine Bauer, Utrecht University, The Netherlands, c.bauer@uu.nl • Alan Said, University of Gothenburg, Sweden, alansaid@acm.org • Eva Zangerle, Universität Innsbruck, Austria, eva.zangerle@uibk.ac.at
Evaluation is essential when conducting rigorous research in recommender systems (RS). It may span the evaluation of early ideas and approaches up to elaborate systems in operation; it may target a wide spectrum of different aspects being evaluated. This special issue aims to capture the state-of-the-art and current trends of RS evaluation—theory and practice—, and will highlight opportunities to advance RS evaluation. In doing so, we embrace the various perspectives on the evaluation of RS. Thereby, the term perspective may, for instance, refer to the various purposes of a RS, the various stakeholders affected by a RS, or the potential risks sought to be minimized. Further, we consider that various methodological approaches and experimental designs represent different perspectives on evaluation. This special issue seeks to capture the big picture of the current challenges and trends in RS evaluation. Besides identifying insufficiencies in current evaluation practices, it will give perspective on where we need to head towards as a research community.
************************** Topics **************************
While almost every paper on RS will have an evaluation section, we are mainly interested in works that have a particular focus on evaluation aspects, e.g., in terms of methodology. Papers that present a particular evaluation outcome can be in the scope as well, as long as they provide a methodological contribution, e.g., a novel way of evaluating a system in a complex scenario.
Topics of interest include, but are not limited to, the following: • Novel Approaches in Evaluation Methodology - Multi-objective and multi-stakeholder evaluation o Validation of evaluation metrics - User-centric evaluation - Simulation-based evaluation - Meta-evaluations - Novel evaluation methodology • Problems and Challenges - Issues of reproducibility - Showcasing (structural) problems in RS evaluation - Practical challenges in RecSys evaluation from industry - Complex evaluation settings and environments - Divergence between evaluation goals and what is actually captured by the evaluation • Evaluation in Practice - Case studies of difficult, hard-to-evaluate scenarios o Combination of offline and online experiments - (Unexpected) experiences from industry - Longitudinal studies (e.g., on the impact of RS) - Evaluations with contradicting or non-conclusive results (e.g., misalignment between online and offline evaluation)
************************** Important Dates **************************
• Abstract deadline: September 30, 2022 • Abstract notification: October 14, 2022 • Submissions deadline: November 30, 2022 • First-round review decisions: January 16, 2023 • Deadline for revision submissions: February 28, 2023 • Notification of final decisions: April 6, 2023
************************** Submission Information **************************
The special issue follows a three-stage process. First, authors are asked to submit an extended abstract. Second, authors of accepted abstracts will be invited to submit a full article. Third, submitted papers will undergo a normal single-blind review process.
Extended abstracts, together with a description why the paper is considered a good fit for the special issue, must be submitted as a pdf via e-mail to perspectives.recsys@gmail.com. Full articles must be prepared according to the TORS submission guidelines (https://dl.acm.org/journal/tors/author-guidelines) and must be submitted via Manuscript Central (https://mc.manuscriptcentral.com/tors).
When in doubt whether a paper is indeed a good fit for this special issue, we encourage potential authors to contact perspectives.recsys@gmail.com
--------------------------------------------------------------------------------------------------------------------------------------
-
Gustaf Neumann