18.104.22.168 Plans for Deployment, Training, Testing, and Evaluation
A "bare-bones" prototype of the first version of the application will be deployed for initial testing and evaluation at the laboratory in SILS and in the offices of the AAHOM following installation of an ISDN connection allowing access to the Internet. This early deployment by the six month mark will allow some feedback on the system to be incorporated in the first test version of the application, which will be ready for deployment one year from the start of funding. Prof. Durrance has extensive experience in the evaluation of library user services [Durrance, 1991], and the AAHOM also has experience in evaluating computer-based exhibitry using both internal software elements to record specific visitor behaviors as well as evaluative tools external to the computer such as time studies and visitor interviews. This expertise will be transferred to the development of an evaluation strategy for "Windows to the Universe".
At the end of the first year of funding, the first test version of the application will be delivered and testing will begin for several months at SILS and in a test-display at the AAHOM. Results from this evaluation will be incorporated in the second test version of the application including expanded content, to be ready 18 months into the project. At this point, the application will be deployed at an Internet ready and computer equipped library within Michigan for further testing and evaluation (which will continue throughout the project at SILS and the AAHOM) as well as an additional hands-on science museum located in the Mid-West. This same six month cycle for development, deployment, testing, and evaluation will be utilized over the entire three years of the project at an expanding network of libraries and museums. Although preliminary contacts have already identified museums and libraries which are enthusiastic in participating in the project (including the Cranbrook Institute of Science in Bloomfield Hills, MI, the Cincinnati Museum of Natural History, and the Indianapolis Children's Museum), we do not think it wise to firmly fix the participants in the second and third years of the project at this point. Instead, we propose to solicit competitive proposals from interested museums and public libraries during the second and third years of the project. This approach would raise awareness among a number of libraries and museums and therefore the interest in obtaining Internet connections. Overall, in addition to SILS and the AAHOM, we envision deploying the application at one new site in each of the second and third years of the project, attaining 4 official and fully participating deployments by the end of the three year interval, with additional unofficial and unfunded installations likely. We plan to maximize the cost-effectiveness of this project and expand the exposure of "Windows to the Universe" to a wider-audience by working with individuals participating in the recently funded NSF University of Michigan Digital Library Development project, including Prof. Dan Atkins and Dr. Joan Durrance, to provide access to users at sites they provide at the Ann Arbor Public Library and within the Ann Arbor Public Schools.
For each official and fully participating library and museum, the pilot-project will provide a computer platform on which to run the application, a modem connection to the Internet if necessary, partial funding support for a staff member at each location to provide users assistance with the application, participate in resource training sessions at SPRL, and assist in on-going e-mail based user and staff evaluation of the application. Technical support for the application will be available by phone, fax, or e-mail by TCS and by project staff at the SPRL. Each participating library and museum will provide on-site Ethernet connections and an appropriately designated location in the library for the computer platform and associated materials. In addition to floor space, project participants in the second and third years will be selected based on the level of interest expressed, the needs of their particular community, and possible cost sharing or enhancements they may be able to provide. These enhancements might include the use of additional equipment such as CD-ROM readers, printers, and paper, the availability of additional staff release time to focus on the project, the presence of existing Internet connections, the development of services designed to respond to the needs and the resources which will be made available including preferential scheduling assistance and marketing to potential clientele and the availability of space for interested user groups to meet (such as science clubs).
Two-day training workshops will be held as SPRL once a year during which the new test version of the application will be introduced and software and content developers will have an opportunity to discuss problems and suggestions together.
The application and its deployment will be the subject of an on-going research and evaluation effort. This study will: inform the development of the several iterations of Windows to the Universe through an analysis of use; examine the impact on users' approach to using library resources; determine primary access factors and barriers; examine models for providing access to complex digital information; determine change in awareness of digital formats; provide examples of ways this project can enrich learning; and determine ways librarians modify professional practice as the result of the project.
Research Approach: A user-centered evaluation will be conducted to determine the value and the impact of Windows to the Universe on its users and on the institutions that test it. Corrections will be made in the software and the service model based on an ongoing, formative, evaluation process. Research Methods: Qualitative methods will be combined with data which can be statistically analyzed. Site visits, observation, interviews with selected users, focus group interviews, transaction logs, and written evaluation instruments will be used.