Home DELOS 5. F.P. Blank space Restricted pages Blank space Search Blank space Contact DELOS Blank space
DELOS news
Digital library news
DELOS Digital Library
DELOS Newsletter
Work programme
Research clusters
1: Digital library architecture
2: Information access and personalization
3: Audio/visual and non-traditional objects
4: User interfaces and visualization
5: Knowledge extraction and semantic interoperability
6: Preservation
7: Evaluation
Member institutions
DELOS boards
DELOS events
Other events
DELOS initiatives
Digital library resources
Projects and initiatives
Mailing lists
Conference series
Important publications

Workshop on the Evaluation of Digital Libraries

Department of Information Engineering, University of Padua, Padova, Italy
October 4-5, 2004

General information

For more information on the event, please click here.

Digital libraries are destined to become an essential part of the information infrastructure in the 21st century. Given the rapid development in this field, there is an increasing need for evaluations in order to determine the usefulness, usability and economics of digital libraries. However, there is a lack of appropriate evaluation methodologies. This workshop will target at a survey on the state of the art in DL evaluation, and will identify major issues for further research in the area of DL evaluation.

Specifically, this workshop will try to find answers to the following questions:
- Why are DLs evaluated?
- What are standard approaches for DL evaluation?
- Which aspects/components are evaluated in these approaches?
-- collections, users, usage, systems
-- content, community, services, technology
-- societies, scenarios, spaces, structures, streams
- Which evaluation criteria are used in these approaches?
- What are the strengths and weaknesses of these approaches?
- What should be done (in the future) in DL evaluation?

In addition, two general issues are to be addressed:
- What is an appropriate conceptual DL model to which evaluation could refer to ?
- Are there conceptual frameworks for evaluation?

As a result of the workshop, a white paper will be prepared, giving a brief survey over the state of the art in DL evaluation, as well as a description of open research issues in this area.

Participation in the workshop will be by invitation only.

List of workshop participants


09.00-10.30Dedicated to an Introduction of the Problem, Presentations and Discussions Aimed at Identifying a Digital Library Evaluation FrameworkChairs: Maristella Agosti and Norbert Fuhr
  The DELOS Evaluation Cluster Norbert Fuhr
  DELOS NoE: Previous Work, Experiences and Recommendations from the Evaluation Forum. What Is Next? Ingeborg T. Sřlvberg
10.30-11.00 Coffee Break  
11.00-12.30 Evaluation of Digital Libraries: An Overview Tefko Saracevic
12.30-14.00 Lunch  
14:00-16.00Invited speakers - Facets of DL evaluationChair: S. Kapidakis
  Understanding Users' Experiences: Evaluation of Digital Libraries Ann Blandford
  Evaluating the Uses of Digital Libraries Christine L. Borgman
16.00-16.30 Coffee Break  
16.30-18.30Invited speakers - Facets of DL evaluation (cont.)Chair: Carol Peters
  Towards a Quality Model for Digital Libraries Edward A. Fox
Content Evaluation Derek Law
18.30 Closing  
09.00-10.45Evaluation Requirements from the Perspectives of the Different DELOS Clusters and Current EU DL ProjectsChair: Ingeborg T. Sřlvberg
  User Interfaces and Visualization Evaluation - WP4 Hanne Albrechtsen
  Evaluating Preservation Aspects - WP6 (html file) Manfred Thaller
  BRICKS EU Project Umberto Pernice
  DILIGENT EU Project Donatella Castelli
10.45-11.15 Coffee Break  
11.15-12.30Towards an Evaluation Framework: Position Papers of WP7 Members Chair: Norbert Fuhr
  Evaluation of User Interaction in Digital Libraries Giannis Tsakonas, Sarantos Kapidakis, Christos Papatheodorou
  Test Collection Construction: the CLEF Experience Carol Peters
12.30-14.00 Lunch  
14:00-15.45Towards an Evaluation Framework: Position Papers of WP7 Members (cont.)Chair: Norbert Fuhr
  Accessing XML Documents: the INEX Initiative Mounia Lalmas, Thomas Rölleke, Zoltán Szlávik, Tassos Tombros
  Evaluation of a Digital Library System Maristella Agosti, Giorgio Maria Di Nunzio, Nicola Ferro
  Evaluating Digital Library Information Models Trond Aalberg
  The Evaluation Computer: a Model for Structuring Evaluation Activities Lázlo Kovács, András Miscsik
  An Evaluation Framework for E-books Monica Landoni
15.45-16.15 Coffee Break  
16:15-18.00 Panel and Discussion on Future Directions for the Evaluation of DL Panel Chair: Tefko Saracevic
18.00 Closing  
URL for this web page:
The DELOS web-team:
Last update: 2004-12-03