-------- Weitergeleitete Nachricht --------
======================== CALL FOR PAPERS =======================
EvalUMAP 2016 Workshop
Towards comparative evaluation in user modeling, adaptation and
personalization
http://evalumap.adaptcentre.ie/
================================================================
To be held in conjunction with the 24th Conference on User
Modeling, Adaptation and Personalization, UMAP 2016, July 2016,
Halifax, Canada
Call for Papers
Submission deadline: May 7, 2016
Research in the areas of User Modelling, Adaptation and
Personalization faces a number of significant scientific
challenges. One of the most significant of these challenges is the
issue of comparative evaluation. It has always been difficult to
rigorously compare different approaches to personalization, as the
function of the resulting systems is, by their nature, heavily
influenced by the behaviour of the users involved in trialling the
systems. To-date this topic has received relatively little
attention. Developing comparative evaluations in this space would
be a huge advancement as it would enable shared comparison across
research, which to-date has been very limited.
Taking inspiration from communities such as Information Retrieval
and Machine Translation, the first EvalUMAP Workshop seeks to
propose and design one or more shared tasks to support the
comparative evaluation of approaches to User Modelling, Adaptation
and Personalization. The workshop will solicit presentations from
key practitioners in the field on innovative approaches to
evaluating such systems and will provide a forum to start scoping
and designing tasks for the following year. The resulting shared
task(s) will be accompanied by appropriate models, content,
metadata, user behaviours, etc., and can be used to
comprehensively compare how different approaches and systems
perform. In addition, a number of metrics and observations will be
outlined, that participants would be expected to perform in order
to facilitate comparison.
The planned outcome of the EvalUMAP Workshop 2016 will be a
roadmap to develop initial shared task(s) that will be published
well in advance of UMAP 2017, giving an opportunity for
participants to test and tune their systems and complete the task
in order for comparative results and associated publications to be
prepared for and presented at the EvalUMAP Workshop in 2017. We
envision that EvalUMAP 2017 will be the starting point for an
annual comparative evaluation challenge at future UMAP
conferences.
Workshop topics are evaluation focused and include, but are not
limited to:
- Understanding UMAP evaluation
- Defining tasks and scenarios for evaluation purposes
- Identification of potential corpora for shared tasks
- Interesting target tasks and explanations of their importance
- Critiques or comparisons of existing evaluation metrics and
methods
- How we can combine existing evaluation metrics and methods
- Improving on previously suggested metrics and methods
- Reducing the cost of evaluation
- Proposal of new evaluation metrics and methods
- Technical challenges associated with design and implementation
- Privacy, Ethics and security issues
- Legal and ethical issues
- Workshop format:
This will be an interactive workshop structured to encourage group
discussion and active collaboration among attendees. The workshop
will feature a keynote talk, lightning round presentation session
for position papers, multiple (parallel) breakout sessions, and a
final discussion session to wrap up the event.
Paper Submissions
The workshop is now accepting paper submissions. Position papers
(max 2 pages) describing approaches or ideas / challenges on the
topics of the workshop are invited. Submissions should be in ACM
SIGS format. LaTeX and Word templates are available at
http://www.acm.org/sigs/publications/proceedings-templates.
Papers should be submitted in pdf format through the EasyChair
system
https://easychair.org/conferences/?conf=evalumap2016
no later than midnight Pacific Daylight Time on May 7, 2016.
Submissions will be reviewed by members of the workshop program
committee. Accepted papers will be included in the UMAP 2016
EvalUMAP workshop proceedings, which will be indexed with CEUR.
Authors of select papers may be invited to contribute to a journal
publication which describes the outcomes of the workshop.
Important Dates
May 7, 2016: Deadline for paper submission (midnight Pacific
Daylight Time)
June 1, 2016: Notification to authors
June 7, 2016: Camera-ready paper due
July, 2016: Full-day Workshop during UMAP
Further Information
Further information is available on the workshop website at
http://evalumap.adaptcentre.ie/
or by emailing the workshop organizers at
evalumap@adaptcentre.ie.
Workshop Organizers
Owen Conlan, Trinity College Dublin, Ireland
Liadh Kelly, Trinity College Dublin, Ireland
Kevin Koidl, Trinity College Dublin, Ireland
Séamus Lawless, Trinity College Dublin, Ireland
Killian Levacher, Trinity College Dublin, Ireland
Athanasios Staikopoulos, Trinity College Dublin, Ireland