CFP: Evaluating Personal Search Workshop at ECIR 2011 - in Ireland!
especially the DERI folks will enjoy a workshop within bus/train distance.
Call for Papers
Evaluating Personal Search. An ECIR 2011 [Half Day] Workshop. April 18, 2011. Dublin, Ireland.
http://www.cdvp.dcu.ie/iCLIPS/EPS2011/
Personal Search refers to the process of searching within one’s personal space of digital information, e.g., searching one’s desktop or mobile phone for required data items or information. While some recent advancements have been made in this domain, research acceleration is hindered by the lack of established or standardized baselines and evaluation metrics, and lack of commonly available test collections. There is a clear consensus within the research community of the need for standardized repeatable evaluation techniques in the PS space, perhaps in the form of a TREC track for example. However, there are a number of significant challenges associated with this, not the least of which is the fact that the data associated with this domain is personal to the individual, multimedia in nature, and different users will have different forms of collections, differing information needs and different memories of required information.
This workshop aims to bring together researchers interested in working towards standardized evaluation approaches for the personal search space. Due to the large space that this covers, as a first step towards overall standardized personal search evaluation this workshop will focus on evaluation for the textual elements within personal desktop collections and known item keyword queries for these elements. An interactive forum for researchers to share ideas and initiate collaborations will be provided, with the explicit goal of establishing a means to evaluate personal search algorithms and solutions in a comparable and repeatable way. It is also intended to form a consortium of people at the workshop interested in bringing this evaluation solution to fruition in the subsequent months.
The topics of the workshop will be evaluation focused and include but not be limited to:
- Understanding personal search evaluation
- Interesting target tasks and explanations of their importance
- Critiques or comparisons of existing evaluation methods
- How we can combine existing evaluation methods
- Improving on previously suggested methods
- Making evaluation more realistic
- Reducing the cost of evaluation
- Proposal of new evaluation methods
- Using human computation games for evaluation
- User study approaches
- Simulation approaches
Datasets
A number of resources are available prior to the workshop. We expect interested participants to be able to use these resources in various ways -- analyzing the characteristics of the data, evaluating various retrieval methods using the queries and result sets, suggesting a better data set for evaluating personal search, etc. Further information and download instructions are available at http://www.cdvp.dcu.ie/iCLIPS/EPS2011/datasets.html
Paper Submissions
The workshop is now accepting paper submissions. Short position papers (max. 2 pages) describing creative use of / modifications to provided datasets and approaches or ideas / challenges for the domain are invited. Submissions should be in ACM SIGIR format. LaTeX and Word templates are available at http://www.acm.org/sigs/publications/proceedings-templates (for LaTeX, use the "Option 2" style).
Papers should be anonymised and submitted in pdf format through the EasyChair system http://www.easychair.org/conferences/?conf=eps2011 no later than midnight Pacific Daylight Time on February 21, 2011. Submissions will be subject to double-blind reviewing. Accepted papers will be included in the Evaluating Personal Search Workshop 2011 proceedings.
Important Dates
February 21,
2011: Deadline for paper submission
(midnight Pacific Daylight Time)
March 18, 2011:
Notification to authors
March 25, 2011:
Camera-ready copy due
April 18, 2011: Workshop
Further Information
Further information is available on the workshop website at http://www.cdvp.dcu.ie/iCLIPS/EPS2011/ or by emailing the workshop organizers.
Workshop Organizers
David Elsweiler – University
of Erlangen, Germany (david-0tjEYskydmyx7wk7FZxnQA@public.gmane.org)
Liadh Kelly – Dublin City University,
Ireland
(lkelly-j9mISXBebvq70S6Re0YL1A@public.gmane.org)
Jinyoung Kim – UMass Amherst, USA
(jykim-bjBJFzlPIWP2fBVCVOL8/A@public.gmane.org)
Program Committee
Leif Azzopardi – University of Glasgow, UK
Robert Capra – University of
North Carolina, USA
Sergey Chernov – L3S, Germany
Bruce Croft – Univ. of Massachusetts,
Amherst, USA
Ronald Fernandez – University of Santiago
de Compostela, Spain
Karl Gyllstrom – Katholieke Universiteit Leuven,
Beligium
Donna Harman – NIST, USA
David Hawking – Funnelback, Australia
Gareth Jones – Dublin City University,
Ireland
Noriko Kando – National Institute of Informatics,
Japan
Diane Kelly – University of North Carolina,
USA
David Losada – University of Santiago de
Compostela,
Spain
Ian Ruthven – University of Strathclyde,
Glasgow, UK
Alan Smeaton – Dublin City University,
Ireland
Jaime Teevan – Microsoft Research, Redmond,
USA
Paul Thomas – CSIRO ICT Centre, Australia
<div>
Hi Semantic Desktop People,<br><br>
especially the DERI folks will enjoy a workshop within bus/train
distance.<br><br><span> </span>
<div>
<div>
<div>
<p>
</p>
<div class="Section1">
<p class="MsoNormal"><span lang="EN-US">Call for
Papers </span></p>
<p><span lang="EN-US">Evaluating
Personal Search. An ECIR 2011
[Half Day] Workshop. April 18, 2011. Dublin,
Ireland.</span></p>
<p><span lang="EN-US"><a moz-do-not-send="true" href="http://www.cdvp.dcu.ie/iCLIPS/EPS2011/">http://www.cdvp.dcu.ie/iCLIPS/EPS2011/</a></span><span></span></p>
<p class="MsoNormal"><span lang="EN-US">Personal Search
refers to the process of searching within one’s
personal space of digital
information, e.g., searching one’s desktop or mobile
phone for required
data items or information. While some recent
advancements have been made in
this domain, research acceleration is hindered by the
lack of established or
standardized baselines and evaluation metrics, and
lack of commonly available
test collections. There is a clear consensus within
the research community of
the need for standardized repeatable evaluation
techniques in the PS space,
perhaps in the form of a TREC track for example.
However, there are a number of
significant challenges associated with this, not the
least of which is the fact
that the data associated with this domain is personal
to the individual,
multimedia in nature, and different users will have
different forms of
collections, differing information needs and different
memories of required
information. </span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">This
workshop aims to bring together researchers interested
in working towards standardized
evaluation approaches for the personal search space.
Due to the large space
that this covers, as a first step towards overall
standardized personal search
evaluation this workshop will focus on evaluation for
the textual elements
within personal desktop collections and known item
keyword queries for these
elements. An interactive forum for researchers to
share ideas and
initiate collaborations will be provided, with the
explicit goal of
establishing a means to evaluate personal search
algorithms and solutions in a
comparable and repeatable way. It is also intended to
form a consortium of
people at the workshop interested in bringing this
evaluation solution to
fruition in the subsequent months. </span></p>
<p class="MsoNormal"><span lang="EN-US">The
topics of the workshop will be evaluation focused and
include but not be
limited to:</span></p>
<ul type="square">
<li class="MsoNormal"><span lang="EN-US">Understanding personal search
evaluation </span></li>
<li class="MsoNormal"><span lang="EN-US">Interesting target tasks and
explanations of their importance </span></li>
<li class="MsoNormal"><span lang="EN-US">Critiques or comparisons of existing
evaluation methods </span></li>
<li class="MsoNormal"><span lang="EN-US">How we can combine existing evaluation
methods </span></li>
<li class="MsoNormal"><span lang="EN-US">Improving on previously suggested
methods </span></li>
<li class="MsoNormal"><span lang="EN-US">Making evaluation more realistic </span></li>
<li class="MsoNormal"><span lang="EN-US">Reducing the cost of evaluation </span></li>
<li class="MsoNormal"><span lang="EN-US">Proposal of new evaluation methods </span></li>
<li class="MsoNormal"><span lang="EN-US">Using human computation games for
evaluation</span></li>
<li class="MsoNormal"><span lang="EN-US">User study approaches </span></li>
<li class="MsoNormal"><span lang="EN-US">Simulation approaches </span></li>
</ul>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Datasets</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">A number
of resources are available prior to the workshop. We
expect interested
participants to be able to use these resources in
various ways -- analyzing the
characteristics of the data, evaluating various
retrieval methods using the
queries and result sets, suggesting a better data set
for evaluating personal
search, etc. Further information and download
instructions are available at <span><a moz-do-not-send="true" href="http://www.cdvp.dcu.ie/iCLIPS/EPS2011/datasets.html">http://www.cdvp.dcu.ie/iCLIPS/EPS2011/datasets.html</a></span></span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Paper
Submissions </span></p>
<p class="MsoNormal"><span class="style441"><span lang="EN-US">The workshop is now accepting paper
submissions.</span></span><span class="style411"><span lang="EN-US"> </span></span><span class="style391"><span lang="EN-US">Short
position
papers (max. 2 pages) describing creative use of /
modifications to provided
datasets and approaches or ideas / challenges for
the domain are invited.
Submissions should be in ACM SIGIR format. LaTeX and
Word templates are
available at </span></span><span class="style391"><span lang="EN-US"><a moz-do-not-send="true" href="http://www.acm.org/sigs/publications/proceedings-templates">http://www.acm.org/sigs/publications/proceedings-templates</a></span></span><span class="style391"><span lang="EN-US"> (for LaTeX,
use the
"Option 2" style). </span></span><span lang="EN-US"></span></p>
<p class="MsoNormal"><span class="style391"><span lang="EN-US">Papers should be
anonymised and submitted in pdf format
through the EasyChair system </span></span><span class="style391"><span lang="EN-US"><a moz-do-not-send="true" href="http://www.easychair.org/conferences/?conf=eps2011" title="blocked::http://www.easychair.org/conferences/?conf=eps2011">http://www.easychair.org/conferences/?conf=eps2011</a></span></span><span class="style391"><span lang="EN-US"> no later
than
midnight Pacific Daylight Time on February 21, 2011.</span></span><span lang="EN-US"> <span class="style371">Submissions will be subject to
double-blind reviewing.
Accepted papers will be included in the Evaluating
Personal Search Workshop
2011 proceedings.</span></span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Important
Dates </span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span lang="EN-US">February 21,
2011:</span><span lang="EN-US"> Deadline for paper submission
(midnight Pacific Daylight Time)<br></span><span lang="EN-US">March 18, 2011:</span><span lang="EN-US">
Notification to authors<br></span><span lang="EN-US">March 25, 2011:</span><span lang="EN-US">
Camera-ready copy due<br></span><span lang="EN-US">April 18, 2011:</span><span lang="EN-US"> Workshop
</span><span></span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Further
Information </span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Further
information is available on the workshop website at <span><a moz-do-not-send="true" href="http://www.cdvp.dcu.ie/iCLIPS/EPS2011/">http://www.cdvp.dcu.ie/iCLIPS/EPS2011/</a></span>
or by emailing the workshop organizers.</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Workshop
Organizers </span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">David Elsweiler – University
of Erlangen, Germany (<a moz-do-not-send="true" href="mailto:david@..."><span>david@...</span></a>)<br>
Liadh Kelly – Dublin City University,
Ireland
(<a moz-do-not-send="true" href="mailto:lkelly@..."><span>lkelly@...</span></a>)<br>
Jinyoung Kim – UMass Amherst, USA
(<a moz-do-not-send="true" href="mailto:jykim@..."><span>jykim@...</span></a>)</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Program
Committee </span></p>
<p class="style372"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Leif Azzopardi – University of
Glasgow, UK</span></p>
<p class="MsoNormal"><span lang="EN-US">Robert Capra – University of
North Carolina, USA<br>
Sergey Chernov – L3S, Germany<br>
Bruce Croft – Univ. of Massachusetts,
Amherst, USA<br>
Ronald Fernandez – University of Santiago
de Compostela, Spain<br>
Karl Gyllstrom – Katholieke Universiteit Leuven,
Beligium<br>
Donna Harman – NIST, USA<br>
David Hawking – Funnelback, Australia<br>
Gareth Jones – Dublin City University,
Ireland<br>
Noriko Kando – National Institute of Informatics,
Japan<br>
Diane Kelly – University of North Carolina,
USA<br>
David Losada – University of Santiago de
Compostela,
Spain<br>
Ian Ruthven – University of Strathclyde,
Glasgow, UK<br>
Alan Smeaton – Dublin City University,
Ireland<br>
Jaime Teevan – Microsoft Research, Redmond,
USA<br>
Paul Thomas – CSIRO ICT Centre, Australia</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
</div>
</div>
<div>__._,_.___</div>
</div>
</div>
</div>
RSS Feed