Home DELOS 5. F.P. Blank space Restricted pages Blank space Search Blank space Contact DELOS Blank space
DELOS news
Digital library news
DELOS Digital Library
DELOS Newsletter
Work programme
Research clusters
1: Digital library architecture
2: Information access and personalization
3: Audio/visual and non-traditional objects
4: User interfaces and visualization
5: Knowledge extraction and semantic interoperability
6: Preservation
7: Evaluation
Member institutions
DELOS boards
DELOS events
Other events
DELOS initiatives
Digital library resources
Projects and initiatives
Mailing lists
Conference series
Important publications

Evaluation (WP7)

Cluster homepage


Cluster objectives

Digital libraries need to be evaluated as systems and as services to determine how useful, usable, and economical they are and whether they achieve reasonable cost-benefit ratios. Results of evaluation studies can provide strategic guidance for the design and deployment of future systems, can assist in determining whether digital libraries address the appropriate social, cultural, and economic problems, and whether they are as maintainable as possible. Consistent evaluation methods also will enable comparison between systems and services.

The evaluation cluster will work both on evaluation methodologies in general as well as on providing the infrastructure for specific evaluations. Thus, the following objectives are addressed:

  • Development of a comprehensive theoretical framework for DL evaluation, which can serve as reference point for evaluation studies in the DL area.

  • Research on new methodologies will be supported in order to overcome the lack of appropriate evalution approaches and methods.

  • Development of corresponding toolkits and test-beds in order to enable new evaluations and to ease the application of standard evaluation methods.

Cluster activities

In order to reach these goals, the following activities will be carried out:

  • Workshops on DL evaluation, for collecting exixting evaluation approaches and methods.

  • Evaluation support to the DL community, by creating an evaluation forum for enabling communication between evaluation specialists and DL developers.

  • Development of new evaluation approaches and methods, in order to overcome the weaknesses of current approaches and the lack of methods for new types of applications.

  • Development of evaluation toolkits, e.g. for collecting and analysing experimental data.

  • Creation of test-beds for new content and usage types in DLs, by starting from the existing test-beds for XML and cross-lingual retrieval and extending these towards new media, applications and usage types.

  • Creation of test-beds for usage-oriented evaluation, by extending existing test-beds or by creation of test-beds of user interactions

Expected results

  • Survey of existing evaluation methodology

  • A conceptual framework for DL evaluation

  • Collection of evaluation approaches and methods

  • Collection of evaluation toolkits and testbeds

  • New test-beds for new content and usage types and for usage-oriented evaluation

Cluster report in DELOS Newsletter, Issue 1, April 2004
Cluster report in DELOS Newsletter, Issue 2, October 2004

Cluster coordinator

Norbert Fuhr
Universität Duisburg-Essen Germany

URL for this web page:
The DELOS web-team:
Last update: 2005-07-13