-------- Forwarded Message --------
Dear colleagues,
We would like to draw your attention to the CfP for the
************************
Special Issue on
Perspectives on Recommender Systems Evaluation at ACM Transactions
on Recommender Systems (TORS)
************************
Abstract submission deadline: September 30, 2022
Full article submission deadline: November 30, 2022
You can find the details below - and via the following link:
https://dl.acm.org/pb-assets/static_journal_pages/tors/pdf/TORS-CfP-SI-Evaluation-1650479094177.pdf
We are looking forward to your contributions.
Best regards,
Christine Bauer, Alan Said, and Eva Zangerle
(Guest editors)
--------------------------------------------------------------------------------------------------------------------------------------
**************************
Call for Papers
**************************
Special Issue
Perspectives on Recommender Systems Evaluation
ACM Transactions on Recommender Systems
Guest Editors:
• Christine Bauer, Utrecht University, The Netherlands,
c.bauer@uu.nl
• Alan Said, University of Gothenburg, Sweden,
alansaid@acm.org
• Eva Zangerle, Universität Innsbruck, Austria,
eva.zangerle@uibk.ac.at
Evaluation is essential when conducting rigorous research in
recommender systems (RS). It may span the evaluation of early
ideas and approaches up to elaborate systems in operation; it may
target a wide spectrum of different aspects being evaluated. This
special issue aims to capture the state-of-the-art and current
trends of RS evaluation—theory and practice—, and will highlight
opportunities to advance RS evaluation. In doing so, we embrace
the various perspectives on the evaluation of RS. Thereby, the
term perspective may, for instance, refer to the various purposes
of a RS, the various stakeholders affected by a RS, or the
potential risks sought to be minimized. Further, we consider that
various methodological approaches and experimental designs
represent different perspectives on evaluation.
This special issue seeks to capture the big picture of the current
challenges and trends in RS evaluation. Besides identifying
insufficiencies in current evaluation practices, it will give
perspective on where we need to head towards as a research
community.
**************************
Topics
**************************
While almost every paper on RS will have an evaluation section, we
are mainly interested in works that have a particular focus on
evaluation aspects, e.g., in terms of methodology. Papers that
present a particular evaluation outcome can be in the scope as
well, as long as they provide a methodological contribution, e.g.,
a novel way of evaluating a system in a complex scenario.
Topics of interest include, but are not limited to, the following:
• Novel Approaches in Evaluation Methodology
- Multi-objective and multi-stakeholder evaluation o Validation of
evaluation metrics
- User-centric evaluation
- Simulation-based evaluation
- Meta-evaluations
- Novel evaluation methodology
• Problems and Challenges
- Issues of reproducibility
- Showcasing (structural) problems in RS evaluation
- Practical challenges in RecSys evaluation from industry
- Complex evaluation settings and environments
- Divergence between evaluation goals and what is actually
captured by the evaluation
• Evaluation in Practice
- Case studies of difficult, hard-to-evaluate scenarios o
Combination of offline and online experiments
- (Unexpected) experiences from industry
- Longitudinal studies (e.g., on the impact of RS)
- Evaluations with contradicting or non-conclusive results (e.g.,
misalignment between online and offline evaluation)
**************************
Important Dates
**************************
• Abstract deadline: September 30, 2022
• Abstract notification: October 14, 2022
• Submissions deadline: November 30, 2022
• First-round review decisions: January 16, 2023
• Deadline for revision submissions: February 28, 2023
• Notification of final decisions: April 6, 2023
**************************
Submission Information
**************************
The special issue follows a three-stage process. First, authors
are asked to submit an extended abstract. Second, authors of
accepted abstracts will be invited to submit a full article.
Third, submitted papers will undergo a normal single-blind review
process.
Extended abstracts, together with a description why the paper is
considered a good fit for the special issue, must be submitted as
a pdf via e-mail to
perspectives.recsys@gmail.com.
Full articles must be prepared according to the TORS submission
guidelines (
https://dl.acm.org/journal/tors/author-guidelines) and
must be submitted via Manuscript Central
(
https://mc.manuscriptcentral.com/tors).
When in doubt whether a paper is indeed a good fit for this
special issue, we encourage potential authors to contact
perspectives.recsys@gmail.com
--------------------------------------------------------------------------------------------------------------------------------------