Executive Summary

The overall aim of the DESIRE project is to make networked research information more easily accessible to European end-users of that information. Work package 8 (WP8) is tasked with producing and delivering packs of training materials to support the subject-specific services for groups identified particularly within Work Packages 3 and 9, ie Social Scientists, Engineers, Orthopaedic Practitioners and Art Historians. Overall, the training materials are intended to raise awareness of, and ensure the most effective use of, the networked information tools and services produced by DESIRE. These information tools and services include state-of-the art reports on organisation of information and quality selection criteria and their application in the demonstrator services. This document describes the second deliverable for WP8, the subject-specific training materials.

Most of the work on this deliverable has been undertaken by Tracey Hooper, Lesly Huxley and Paul Hollands (Institute for Learning and Research Technology, University of Bristol) and Tony McDonald (University of Newcastle) in consultation with members of the subject-specific groups previously referred to. We have also had assistance and support from the Netskills project at the University of Newcastle, particularly with respect to the provision of staff presentations input and facilities for delivery of the workshop.

WP8's first deliverable, D8.1, provided a framework for production of quality training materials including an initial specification of training needs, quality assurance procedures for the production of paper and online training materials and the functional specification for an online tutorial. D8.2 builds on this framework, taking into account the primary training need identified in D8.1 for material to support the training of contributors to the various information services, however their Distributed Cataloguing Models are structured. D8.2 comprises:

Sample training materials for contributors

These materials use the Social Science Information Gateway (SOSIG) as an example of presentation - for a subject-specific audience - of tools and methods developed under DESIRE and based on D8.1's framework, including:

paper-based material
workshop exercises
descriptions of presentations and sample slides
sample content for an online tutorial based on the quality evaluation criteria developed under DESIRE
a sample workshop outline for delivery of these contributor materials

Train the trainer materials and workshop for trainers of contributors

A two-day workshop for a small number of staff from the subject-specific services who will be responsible for training contributors to their services using appropriate elements of the subject-specific material outlined in 1. above. The workshop and materials cover:

discussion of the various Distributed Cataloguing Models (DCMs) operated by the services
issues of day-to-day management of and training for the DCMs
hints and tips on preparing, authoring and using:
- paper-based training materials
- presentations and slides
- static Web-based training and `help' materials
- interactive Web-based training material
an online page of links to services and useful sites, applications and notes

Executive Summary

Description - D 8.2 Subject-based Training Materials

1 Introduction

1.1 Aims and Objectives

2 The Training Materials

2.1 Sample training materials for contributors

2.2 Train the Trainer Materials & Workshop for Trainers of Contributors

3 Evaluation

Bibliography and References

Appendices

Appendix A: Workshop Workbook

Appendix B: Slides for October workshop - main

Appendix C: Slides for ROADS presentation

Appendix D: Slides for TONIC-NG presentation

Appendix E: Section Editors Workshop Outline

Appendix F: Section Editors Slides

Appendix G: Workshop Support Materials

Appendix H: Workshop Commentary

Appendix I: Section Editors Workbook

Appendix J: SBIG Models

Description - D 8.2 Subject-based Training Materials

1 Introduction

1.1 Aims and Objectives

The overall aim of the DESIRE project (EU Telematics for Research project no.1004) is to make networked research information more easily more easily accessible to European end-users of that information. Work package 8 (WP8) is tasked with producing and delivering packs of training materials to support the subject-specific services for groups identified particularly within Work Packages 3 and 9, ie Social Scientists, Engineers, Orthopaedic Practitioners and Art Historians. It was these groups that were identified in Initial Specification of Training Requirements (Section 2 of D8.1 Project Deliverable Report) as being the most likely to benefit from the training materials being produced by WP8. Overall, the training materials are intended to raise awareness of, and ensure the most effective use of, the networked information tools and services produced by DESIRE. These information tools and services include state-of-the art reports on organisation of information and quality selection criteria and their application in the demonstrator services.

The DESIRE WP8 workplan involves the following tasks:

the specification of, and framework for, training materials to support the DESIRE project (known as generic materials) (T8.1)
tailored training materials based on generic materials (above) for specific user subject-groups within DESIRE (T8.2)
design and delivery of a workshop for trainers of users to introduce them to DESIRE training materials (T8.3)
revision of training materials based on feedback from T8.3 (T8.4)
design and delivery of workshops for users with DESIRE subject-specific groups and final revision to provide a pack of training materials to support DESIRE (T8.5)

WP8's first deliverable, D8.1, provided a framework for the production of quality training materials to meet the needs of highly distributed subject-based services where both trainers and those needing to be trained were widely dispersed across Europe. D8.2 builds on this framework, taking as its starting point the Specification of Training Needs (Section 2 of D8.1 Project Deliverable report). Consultation with representatives of the subject-based services from WP3 and WP9 indicated that the focus for service-specific training material should be on the contributors to the services (cataloguers, editors, correspondents, etc). These are people responsible for the identification, evaluation and cataloguing of Internet resources for the various distributed subject-based services and need to be:

given background information about DESIRE
aware of the structure of the Distributed Cataloguing Model operated by a parricular service
trained to use the DESIRE quality evaluation tools and service-specific cataloguing rules
coached in general Internet skills in searching for resources and using browsers

Service administrators and core staff are involved in the training and coaching of these contributors, whatever the structure of their Distributed Cataloguing Model. D8.2 therefore provides both sample content for the training of contributors as described above and content aimed at trainers of contributors, offering hints and tips on preparing and using training materials in different media.

Note:
We have adopted the following convention in this Deliverable report. Wherever reference is made to an appendix of the Deliverable itself, this is denoted in bold face, e.g. (Appendix I). Wherever reference is made to an appendix that is itself part of a deliverable appendix, this is denoted in italics, e.g. (Appendix A: SOSIG Scope Policy).

2 The Training Materials

2.1 Sample training materials for contributors

These materials use the Social Science Information Gateway (SOSIG) as an example of presentation - for subject-specific audiences - of tools and methods developed under DESIRE including:

paper-based material
workshop exercises
descriptions of presentations and sample slides
sample workshop outline for delivery of these contributor materials
sample content for an online tutorial based on the quality evaluation criteria developed under DESIRE

The paper-based materials and workshop exercises are presented in a bound workbook entitled "The Internet for Social Scientists - Cataloguing Quality Resources: A workshop for SOSIG Section Editors" (see Appendix A). The contents of the workbook (see Appendix I) cover:

1 Introduction

Responsibilities of the SOSIG Team
Section Editors' Responsibilities

2 A Quality Controlled Gateway

Introduction to SOSIG
Using SOSIG: an end-user's perspective (including exercises)
Using SOSIG: a Section Editor's perspective (including exercises)

3 Locating and Managing Resources

Introducing Essential Tools (browser hints and tips)
Locating Internet Resources: search engines
Locating Internet Resources: other methods (including exercises)

4 Evaluating and Cataloguing Resources for SOSIG

Evaluating Resources: a summary (including exercises)
Using ROADS to catalogue resources
What is a SOSIG Catalogue Record?
Getting Started
Using the SOSIG Database Admin Centre
Creating a new template (including exercises)
Editing an existing template (including exercises)

5 Further Information

Appendix A: SOSIG Scope Policy
Appendix B: SOSIG Collection Management Policy
Appendix C: SOSIG Evaluation Criteria
Appendix D: SOSIG Cataloguing Rules

This workbook was piloted with SOSIG Section Editors and provides a sample for other services: they can `pick and choose' which elements are most relevant for their needs. The workshop for trainers of contributors used to deliver D8.2 provides an opportunity for trainers to assess the appropriateness of the various sample elements.

2.2 Train the Trainer Materials & Workshop for Trainers of Contributors

A two-day workshop was designed and delivered for a small number of staff from the subject-specific services who will be responsible for training contributors to their services using appropriate elements of the subject-specific material outlined in 2.1 above. The workshop and materials cover, in summary:

discussion of the various Distributed Cataloguing Models (DCMs) operated by the services
issues of day-to-day management of and training for the DCMs
hints and tips on preparing training materials in a variety of media
an online page of links to services and useful sites, applications and notes

The workshop was delivered by staff from SOSIG (including Research and Training staff), Netskills and DESIRE. Participants included representatives from the four DESIRE demonstrator services plus, on day 1, a number of invited representatives from other UK subject-based services (OMNI - Medicine; ADAM - Art, Design, Architecture and Media; EEVL - Engineering; IHR-Info - History and NETEG - training materials). The presence of the other SBIGs was intended to facilitate discussion of the Distributed Cataloguing Models covered on the first day of the workshop.

The workshop was supported by a paper-based workbook (see Appendix B), workshop discussions and presentation slides (included in the workbook) and online materials including a Web page of useful links and sample online tutorial material.

The workshop (and supporting material) covered:

1 Introduction

DESIRE background, workshop programme and introductions

2 ROADS and Cross-database searching

An update on ROADS software used by some of the services and demonstration of cross-database searching mechanisms currently under development.

3 Distributed Cataloguing Models

example from SOSIG
other examples, structures and experiences
day-to-day management issues for the DCM

4 Training for Distributed Cataloguing Models

difficulties and possible solutions
sample training materials and tools for supporting a DCM
- D8.1
- using email pre- and post-workshop
- SOSIG Section Editors workshop materials
- SOSIG support materials: tools and static Web pages
- slides
TONIC-NG (online tutorial framework with sample content)

5 Developing Training Materials for the DCM

hints and tips on creating training materials
authoring training materials for the Web: static and interactive

6 Planning and Running a workshop

planning content, materials and timing (an opportunity to assess appropriateness of sample material and different media for own workshops)
practical issues of running a workshop

7 Online Tutorial
A demonstration of the online tutorial system, TONIC-NG, was given to the subject support staff. This comprised of an overview of the system and a general discussion of how the support staff might create new tutorials or convert their existing paper-based documentation for inclusion into the TONIC-NG framework.

The support staff were then given a brief demonstration of how the quiz engine component of the tutorial system worked and then had a hands-on session where quizzes were created, edited and tested. The initial feedback from this session was very positive.

8 Appendices

Appendix A: SOSIG Section Editors' Workshop Outline
Appendix B: SOSIG Section Editors' Slides
Appendix C: SOSIG Section Editors' Workshop support materials
Appendix D: Tailoring the SOSIG Section Editors' Workshop: commentary
Appendix E: SOSIG Section Editors' Workbook

The online page of useful links and sample materials is available at:

http://www.netskills.ac.uk/DESIRE/materials/oct97-workshop/

3 Evaluation

A full evaluation of the workshop materials will be provided by the peer reviewer, Lucy Smallwood, from WP9. At the end of the second day of the workshop, participants were asked to complete an online evaluation form. This is a generic form used by Netskills for their training workshops and was not tailored for the DESIRE workshop.

Overall the workshop was rated at an average of 3.87 on a scale of 1-5 (where 1=low and 5=high). A summary of the evaluations is given below.

Participants were asked to score the presentations and hands-on sessions on a scale of 1-5: (the numbers shown are arithmetic means of the individual scores)

Presentations:

usefulness: 3.75
clarity: 3.87
interest: 3.75

Hands-on Sessions:

usefulness: 4.12
clarity: 4.12
interest: 4.25

The evaluations were completed on Day 2 of the workshop, which precluded non-DESIRE SBIGs from being represented. There were 8 evaluations completed in all (a 66% response rate from 12 DESIRE SBIG representatives), and it is quite possible that there was a language problem involved. There were 8 non-DESIRE SBIGs involved, and 7 trainers.

Participants obviously found the hands-on sessions more useful and of greater interest than sessions which simply involved presentations, discussions or demonstrations.

The evaluation questionnaire also asked participants for their comments on particular aspects of the workshop. One participant answered `no' to most of these questions: it is likely that this was a participant whose first language is not English and who did not understand what was being asked for.

Wanted more on ...

Participants were asked whether there was anything they would have liked the workshop to cover more:

Two of the five who responded to this question indicated that they would have liked to have had more opportunity to discuss experiences with other services, either on technical issues or day-to-day management issues.

One suggested that more time could have been given to preparation for their own workshops.

One wanted details of specific software and training methods and one wanted to know when the online tutorial, TONIC-NG, would be ready for use.

Wanted less on ...

Participants were asked whether they would have liked the workshop to have covered less:

Of the five respondents to this question, two indicated that they would have preferred to have had less time devoted to the description of Netskills courses (in the general section on preparing training materials).

Two suggested that less concentration on software (general software and online tutorial systems) would have been preferred.

One would have preferred to hear less about the practicalities of running a workshop.

It is difficult to devise a workshop which will meet the needs of all participants, given the wide range of experiences and types of role (trainers of contributors, some of whom were service administrators, others librarians, distributed cataloguers or contributors working at their host site). Two participants would have preferred to have less coverage of software, whilst one, in response to the previous question, wanted to hear more.

Food, accommodation, rooms ...

Participants' reactions to this question (five responded) were generally favourable for both training facilities, food and accommodation as well as social events.

Course organisation ...

Four participants responded: two gave generally positive responses ("organisation was okay. No technical problems!" and "very impressive - very clear workbook and web page pointing to the relevant online resources, and all the WP8 trainers were very professional").

One participant simply noted `No' and another noted that there had been some uncertainty before the workshop - although there is no indication of which area was particularly uncertain.

Pre-course organisation and information ...

Two participants responded: one stated simply `No', one made a positive comment: "very impressive organisation - the web page and the email contact was really excellent".

General Comments ...

Participants were asked to offer any other comment they had about any aspect of the workshop. Six participants responded. Five respondents gave generally positive comments, one responded simply `no':

"workshop was very inspirational"

"I think it was very useful. I got lots of ideas for our own workshop and I also find it important that we could talk (again) about the SBIG idea in general and made plans for some awareness raising activities"

"Very useful, also because it gave a good insight in what is going on in Europe in this field"

"Should be useful"

"I found day 1 of particular benefit - I think all the participants meeting and talking was extremely useful and we can now produce some very beneficial outcomes. I wonder if there was too much show and tell on Day 2, but having said that I learned some good things from it".

On balance the evaluation comments and scores were generally positive, although some issues are raised about methods of delivery of material and how useful and interesting participants found these (hands-on sessions, for example, were more highly rated on average than presentations). This feedback, and that from the peer review, will be fed into T8.4 in preparation for the final task of WP8, the delivery of workshops for contributors to the DESIRE services in their own countries (and languages).

Bibliography and References

none

Appendices

Appendix A: Workshop Workbook

This document is available at the following URL

http://www.netskills.ac.uk/DESIRE/materials/d82/contrib.doc

Executive Summary

Sample training materials for contributors

Train the trainer materials and workshop for trainers of contributors

Contents

Description - D 8.2 Subject-based Training Materials

1 Introduction

1.1 Aims and Objectives

2.1 Sample training materials for contributors

Appendices

Appendix A: Workshop Workbook