This section considers the experiences of the JUSTEIS team in working across strands A and C simultaneously, and the implications for the next cycle of research due to start in October 2000. It then discusses the possible implications of the findings so far upon JISC planning, HE library and information services and HE academic departments. Finally it makes recommendations for future cycles or areas of investigation

The perceived benefits of integrating the two strands were set out in the proposal:

• The use of consistent definitions and a unified taxonomy of EIS for both surveys;
• Methodological advantage from the construction of common survey instruments;
• Resource savings will be made through optimal use of manpower during data collection.

At the conclusion of the first cycle, it is clear that these benefits were manifest and that integration of the two strands should continue in subsequent cycles. An additional benefit was the sharing of data between the two strands which enabled a greater insight into some of the findings.

The next cycle will begin in October 2000 and last for six months. The main changes and recommendations are highlighted and discussed in this section. There will be other minor changes to actual research procedures as each institution has its own procedures and practices which need to be considered carefully when arranging interviews or questionnaire surveys for staff and students, to ensure that response is satisfactory.

The existing methodology, (based on critical incident and critical success factors) works well as a means of establishing a rapport at interviews and eliciting details of a recent search for information. The development of a questionnaire to complement the interview instrument proved successful and will be maintained in future cycles.

A major implication of the Exception Plan is that the team will work through academic staff for organising interviews and distributing questionnaires to students and academic staff to improve the response rate.

The questionnaires and interviews are essentially complementary, and we now feel that a full interpretation of possible trends requires both types of survey. The problems of terminology which may not be obvious in the analysis of the questionnaire data are easier to spot during analysis of the interview data. However, it is important to keep the survey instruments to a manageable length, and thus it is better for the problems of terminology to be dealt with at the analysis stage rather than employing lengthy definitions within the questionnaire.

The adaptation of the interview schedule to a questionnaire format was successful, although minor changes will be necessary.

The general rubric will need some small modification to make it more user friendly. For some reason, it seems that the nature of the critical incidents that are discussed at interview differs from those used by questionnaire respondents. The tendency for questionnaire respondents to speak generally about their use of EIS and to include more than one incident suggests that a tightening of the wording for the main critical incident questions (questions 1 and 2) is also necessary (Appendix 11).

Other minor changes could include the order of categories within the question (question 1) concerned with reasons for undertaking the critical incident search, and consideration of the different terminology used for categories of assessment types at the various institutions.

With regard to student understanding of the various EIS, one possible approach considered was a check list of EIS as opposed to a open-ended questions (questions 2 and 6). Given the lack of awareness and understanding of sources by students, however, it seems likely that asking them to tick lists of EIS would result in highly erroneous findings.

There are issues of terminology that go beyond those associated with EIS which require fine tuning for the second cycle. There are also the differences in terminology use between institutions, disciplines, student groups and individuals to be considered as well. For example, subgroup analysis could cope with the fact that ‘papers’ (as opposed to assignments) will mean one thing to a postgraduate and another thing to some undergraduates.

The JUSTEIS team proposes to target the same number of institutions as in the first cycle, using the same sampling frame to ensure that types of institution and discipline clusters are adequately represented in the sample. As far as possible, the sampling should be random within that framework to avoid any bias. Areas of low response in the first cycle were:

• relatively low number of interviews for the Clinical Medicine discipline cluster
• relatively low number of questionnaires from the Humanities and Arts discipline cluster
• few distance learning students and few part-time students

Given recent government policy initiatives, it is important to ensure that these disciplines and categories of students are investigated in greater depth than was possible in the first cycle.

Given the apparent importance of academic and LIS staff in promoting the use of EIS, the second cycle will investigate in more detail:

• the instruction offered to students about EIS from LIS and academic staff
• the influence of all staff on student use of EIS
• activities such as specific requirements in assessment, presentations and advice and support provided through web pages
• Impact of EIS on library staff and services.

This will require more interviews with staff and students within a single department, something that was not possible at some of the institutions in the first cycle.

An appropriate mix of interviews and questionnaires will be arranged for each participating department, and the team envisage that it should be possible to obtain around 600 questionnaires and 200 interviews overall. The coding framework developed should be applicable to the Second Cycle.

The JISC call required "a survey of resource access provided by individual HEIs" however, apart from highlighting several surprisingly high figures for resources on HEI Web sites (individual electronic journals, informational and institutional Web sites, remote OPACs and gateways), the survey in this form was not very informative, and there seems little point in repeating the exercise in its present form.

For future cycles, the team believes that a survey should go beyond a mere count of links. An examination of the structure and content layout, together with listings of some areas such as gateways or JISC/CHEST resources, of typical HEI information service Web sites, would be more informative. This issue is perhaps more clearly understood from the following examples.

As a first case, most (not here quantified by the Strand C survey) sites offer quite lengthy hierarchical subject trees as well as more general pages. These must have been labour- and time-intensive to complete and presumably remain a significant drain on time to maintain, yet the Strand A interviews and questionnaires show that very little use was made of this kind of resource. It would be more useful to quantify the amount and degree of subject tree development to assist access to EIS.

A second example concerns subject gateways and the possible links from HEI departmental web pages to those of the library (particularly to the subject trees) and vice versa. It would be instructive to determine how often the gateway function was duplicated (with either the same or different links) and how often one gateway simply pointed to, and used, the other.

The team felt that the survey instrument was satisfactory and that it could be used, with minor alterations to reflect current concerns, in the second cycle. During the interviews senior librarians noted that they had found the survey instrument and the interview a useful catalyst to reflect on current issues and problems.

Progress on JUSTEIS was impeded considerably by the variations of interpretation, and apparent variations in practice, of the registered uses of personal student data such as email addresses. It would help for future cycles, and other evaluations of JISC funded services, if JISC could encourage HEIs to set up their registered uses to allow bona fide researchers access to student emails under certain conditions.

There seems a possibility that students will look on their Web-based email address in the same way as they do their mobile phone number – and there may be a merging of the two in any case (WAP technology). For HEIs, there are implications in that students may compartmentalise their personal email/phone number and their ‘HEI’ number. It was clear from the difficulty in contacting students at some institutions that the email address which was checked more frequently was the Web-based email.

Whilst it is good that there is a more extensive use of email, the problems associated with the use of more than one email account (see Section 6.11.2) need to be addressed. There is a potential role for JISC to encourage HEIs to establish a policy with regard to use of and access to email accounts such as the JANET acceptable use regulations to which all users must agree.

There is no doubting the effect of the Internet on information seeking by staff and students at all institutions, and this must be a central consideration in the future provision of EIS. However given the wide range of search engines used and the haphazard nature of much of the searching, some thought might be given to ways of encouraging students to use the Internet more effectively (Section 6.11.2).

JISC policies for encouraging access to resources need to bear in mind the increasing ‘home-based’ use of the Internet, which is likely to be particularly prominent for part-time and mature students. Authentication needs to be by arrangements such as ATHENS, and as simple as possible.

The lack of use of gateways was disappointing, given the fact that use of subject gateways could save time – a precious commodity for staff and students. Whilst we are cognisant of the newly-announced Resource Discovery Network projects, an investigation of the pattern of undergraduate courses and the changing popularity of certain subject areas (media studies, sports science) might be fruitful. This could suggest other areas for subject gateways that fit niche multidisciplinary areas, and areas where resources are likely to be in short supply at institutions that put on degree programmes to meet new demands.

The lack of use of subject gateways, the low take up of EIS providing only bibliographical information by all groups, and the disappointingly low use of JISC negotiated services and Web databases by students and staff, needs to be addressed (Section 6.11.1-2).

The benefits to students of downloading and re-using electronic information should be recognised, but this practice also raises issues of plagiarism. HEIs will need to continue to monitor and address this problem. (Section 6.11.5)

The low use of LIS staff and courses that direct students to appropriate EIS needs to be addressed. (Section 6.11.6)

The marketing of Gateways and other specialised EIS is deficient. Whilst this is not something that can be handled by individual HE information services alone, they do share a responsibility for making potential users adequately aware of the services they are providing. (Section 6.11.2)

Access to resources that cannot easily be located elsewhere, and the ease of manipulating the information (textual and image) into project and assignment materials, are issues that should continue to be a focus for training. (Section 6.11.5)

The default search for information is via a search engine, usually the first that comes up. LIS pages and routes are sometimes ignored completely. This is probably a reflection also of many students’ use of home computers and their own Internet service providers and is a matter deserving consideration.

Although the first cycle survey was not concerned with exploring user education in depth, inevitably there were many valuable insights into this field.

Students perceive academic staff as the first point of contact, should they have problems finding information required for assignments or as the primary medium for general guidance. Structured guidance on EIS with clear expectations for assignments, backed up with training in the use of particular services at the appropriate point in the degree programme should feature in courses (Section 6.11.6).

The quality of information provided by resources appears to be of low concern to students, and departments need to encourage critical evaluation of EIS (Section 6.11.2).

A major implication for academic departments is that staff need to work, as some do, with LIS staff in the development of training programmes which can be timetabled at appropriate points in the degree programme and integrated into the educational programme and informal modes of support. User education packages and tutorials, need to be viewed as part of an overall strategy, not as an independent elements. Liaison between LIS and academic staff is vital to ensure that the latter know about tutorial packages, as well as subject collections and gateways, and that they can manipulate these for their own teaching.

For the pure and applied sciences, there seems to be a well-established system of support in some institutions in the form of formal teaching about particular resources at an appropriate point in the programme, with both LIS staff and academic staff co-operating successfully. Hopefully, cooperation with academic staff to develop EIS skills training could be extended to the other disciplines.

A further implication concerns the use of departmental Web pages. This seems to vary but this does seem to be a route that students (caught early enough) will use as a means of accessing administrative and course-related information such as advice on information sources and services. Departments need to work closely with LIS to ensure that appropriate links are made – human liaison as well as hypertext links.

Experience undertaking the research identified the following issues which cannot be addressed in the next cycle, but which may require further investigations in future cycles or else in supplementary studies.

---

This page last updated 25 August 2000 10:03:19
Research Pages maintained by Chris Armstrong