5.0. INTRODUCTION

The findings of this evaluation study indicate that GILS has not reached its full potential as a government–wide information locator service. A number of agencies, however, have put their "agency GILS" to work in interesting ways including assisting in public access to government information and broader information management efforts. In addition, the basic concept of GILS as a set of decentralized, agency-based locators containing structured metadata records and accessible via Z39.50 remains a valid architecture to support networked information discovery and retrieval. Yet, what exists in Spring 1997 is not a government–wide information locator service but a set of diverse agency implementations that vary in coverage and scope. The past two years of implementation experience highlight important issues—at both policy and implementation levels. Without this experience, neither agencies nor users would be able to articulate the issues identified in this study.

Networked-based locator services such as GILS should be seen as innovative approaches for providing access to government information. Precisely because of its innovative character, the U.S. Federal GILS initiative has identified and clarified some basic research issues for networked information discovery and retrieval. In fact, aspects of GILS such as the capture/use of metadata and distributed search and retrieval tools are essentially research issues for which scalable and operational solutions have yet to be fully developed. The recently released report on the Canadian GILS pilot project (see Appendix I) parallels many of the findings and recommendations reported in this study of U.S. Federal GILS implementation. A comparison of the two reports indicates that networked-based locator services share systemic problems and common issues. In the view of the investigators of this study, the common threads that run through both reports point out that the architecture of GILS provides a valid approach to a networked-based locator service, yet carrying out that architecture in actual implementation makes visible important research questions.

Two years of agency GILS implementation experience, however, provide both the implementors and policymakers with a firm basis for determining the future shape of a government information locator service. One consequence of this study has been documenting the range of issues, problems, and success factors that are only visible now because of the actual implementation experience. The findings and associated recommendations discussed in Chapter 4 contribute to the discussions among policymakers and implementors in deciding what next steps need to be taken and how those steps should be sequenced. The recommendations in Chapter 4 range in detail and priority from, for example, government–wide information policy integration to specifics about the data elements in GILS records. This chapter provides a summary of the recommendations and places them in a framework for action.  

One of the first actions resulting from the completion of this report will be the responsibility of the GILS evaluation advisory group. The GILS Board charged a group of agency representatives with planning the evaluation study. Members of that committee have served as an advisory group to the investigators during the study. When this final report is submitted to the COTR and distributed to the advisory group, that group will have the responsibility to meet, review and respond to the findings and recommendations in this report, and decide on specific next steps to move the GILS initiative forward. This report will serve as a point of departure for discussions and agreements among the advisory group, and the advisory group’s actions will guide the next phase and the overall success of the U.S. Federal GILS initiative. The recommendations and framework for action reported below provide the advisory group with a beginning point for their deliberations.

In the final assessment, there is much left to learn about networked–based locator services. Throughout the report, the investigators described many issues and problems that are beyond the scope of this report to resolve, and often the recommendations identify areas in need of further research. This final chapter enumerates a series of research areas that require attention if the refocused GILS initiative is to succeed.

5.1. CHARACTERIZING THE REFOCUSED GILS: RECOMMENDATIONS

This report has documented major issues facing the U.S. Federal GILS initiative. If a true government–wide information locator service is to evolve from the agency implementations, the investigators have concluded that major revisions in the approach to GILS are necessary. These revisions are at policy, administrative, oversight, and implementation levels. The investigators recommend that such revisions should occur based on a refocusing of the purpose, goals, and scope of the current approach to GILS.  

A refocused GILS initiative builds on the success of selected agency GILS implementations, but with a clear demarcation from the current approach. The investigators concluded that given the current confusion over what GILS means, it is essential that policymakers and implementors clearly differentiate the refocused GILS from the effort guided by the original OMB 95-01. The new initiative acknowledges the value of many aspects of the original GILS concept, yet policy for a refocused GILS provides a clear line of demarcation between the early GILS implementation period (i.e., 1995-1996) and a refocused GILS. One approach to distinguishing the refocusing of GILS is through a change in the name to reflect, for example, a "second release" of the U.S. Federal GILS service. And important aspect of such a demarcation with the early implementation period is to acknowledge the lessons learn from that experience. To give some indication of what a refocused GILS could include, the investigators offer the following major recommendations.

5.1.1. The Refocused GILS Initiative Clearly Articulates the Purposes and Utility of a Government Information Locator Service

The current GILS is different things to different people and has led to inconsistent implementations and a wide range of expectations of GILS. A refocused GILS must clearly articulate the function of a government–wide information locator, its scope of coverage, what people can legitimately expect it to provide, and the benefits it offers.

The investigators recommend that GILS be refocused and aligned with the following vision: 

The purpose of the refocused GILS is to enable users to discover what government information exists and provide users with direct access to that information. The revised purpose does not include records management. Any additional functions proposed for GILS that extend this initial purpose must be tested and demonstrated prior to raising expectations and to determine whether or not GILS can achieve or support such purposes and functions (e.g. EFOIA).

To support networked information discovery and retrieval, the GILS records (i.e., metadata) will be crucial. Web–based searching or browsing as currently offered by agency Web sites or through implementations such as the White House or Thomas Web sites do not give users a government–wide view nor provide government–wide discovery and access. GILS records are a necessary linchpin to solve the networked information discovery and access problem.

The scope of a refocused GILS should a subset of all government information resources, namely those that are in digital form. Due in part to the ease of Web publishing and the ease of interfacing with existing online databases and services through Web scripting, the amount of network–accessible government information will continue to increase. This scope of coverage is realistic rather than simply reasonable. First, users would know what they could expect to find if the scope of the refocused GILS is "metadata records that describe the publicly accessible electronic resources of the government and provide linkages or access to those resources." A refocused and more limited scope would provide guidance to agencies in their development of GILS implementations, especially by clearly specifying the agency resources GILS records should describe. Also, this scope accounts for the increased expectation of users who want to obtain the actual information rather than just a description of it.  

The GILS that results from a systematic refocusing can clarify to agencies and users what the government information locator service is, how it works, what is covered, and what users can expect from it.

5.1.2. The Refocused GILS Initiative Provides Clear Lines of Authority and Oversight

An essential feature of U. S. Federal GILS is its decentralized approach—at the agency level—for providing locator services to agency information resources. To date, this decentralized implementation responsibility has not been balanced by integrated or coordinated management and administration. The refocused GILS initiative—through policy directive—identifies an appropriate organizational unit that has the responsibility, authority, and accountability for providing government–wide coordination and administration of GILS activities. The refocused GILS initiative, however, safeguards the decentralized character of agency–based locators, where those people closest to the resources are responsible for identifying them, assisting in the creation and maintenance of GILS records, and providing public access to them.

The investigators recommend that two organizational units be charged with separate mandates for the next stage of GILS development and deployment. First, the GILS Board should provide the forum—through various task forces—for determining the revised focus of GILS. The result of discussions by the task force and the GILS Board should be a set of policy recommendations submitted to the Office of Management and Budget as a basis for its revisions to OMB Bulletin 95–01 that expires at the end of 1997. The forum provided by the GILS Board should include representatives of agencies as well as non–Federal stakeholders in GILS such as citizens, librarians, researchers, and public interest groups.

Second, the CIO Council is an appropriate interagency body that can provide ongoing coordination and administration of GILS. The government–wide character of a government information locator service combined with the Council’s mandate makes it a suitable locus for these responsibilities. One important function of the Council will be setting appropriate, realistic, and measurable objectives for agency GILS implementations. OMB Policy directives and goals for the refocused GILS need to be translated into actionable steps that agencies can take. Measurable, specific, and realistic objectives can guide agency actions. In addition, the Council should determine reporting schedules, receive agency GILS progress and implementation reports, and provide information to OMB and the GILS Board on the status of GILS activities. The CIO Council should have representation on the GILS Board to enable communication between the two organizations.

5.1.3. The Refocused GILS Initiative Demonstrates Effectiveness and Benefits Through a Pilot Program

The refocused GILS initiative recognizes that networked information discovery and retrieval (NIDR) is a new and as yet relatively unexplored terrain. Many of the current technical implementation issues are part of the larger research area dealing with NIDR. Systems for the organization and access to information—government or otherwise—have developed over the years, often through experimentation and lessons learned from implementation experience. The networked environment, which is the context for all information handling at the end of the 20th Century, adds new layers of complexity to traditional approaches of information organization and access. The refocused GILS initiative participates fully in utilizing emerging information technologies to improve access to government information. Further, the refocused GILS initiative should acknowledge the immaturity of NIDR by establishing a ongoing pilot program to identify problems and issues in both policy and implementation arenas. 

The investigators recommend that once stakeholders reach consensus on the character and specifics of a refocused GILS, the CIO Council will establish a GILS pilot program. A GILS pilot and demonstration program offers many benefits to implementing agencies and users. Agencies gain the benefit of tested technology, procedures, and best practices. Pilot implementations can demonstrate tangible benefits to those agencies that need convincing that GILS is worth doing, and doing well. Users can experience the utility of a government–wide search and retrieval service. More importantly, users can provide critical input at the design and development phases of the next generation of GILS implementations through a pilot program to ensure that the resulting information locator service meets the requirements of various user communities that need access to government information.  

Working in parallel, OMB, the GILS Board, and the CIO Council should establish policies, goals, and objectives for the refocused GILS. Specific objectives will provide a standard against which the implementations in the GILS pilot program can be measured. Thus, the pilot program serves as a check on the ability of GILS implementations to achieve the objectives, goals, and policies of the refocused GILS initiative.  

A pilot program can serve as a testbed for experimental implementations of any additional functionality that policymakers and agency implementors determine reasonable for the refocused GILS. The important point is that before policymakers or implementors raise expectations of functionality to be supported by a discovery and access service such as GILS, experimentation occurs to test and demonstrate how realistically GILS supports additional functionality. A pilot program also can be effective in determining the prospects of new and emerging technologies, such as the proposed Advanced Search Facility (e.g., single–point search and retrieval) or the current push technology (e.g., for announcing updates of GILS services to cognizant communities). 

Most importantly, a pilot program serves as the focal point for resolving many of the technical and procedural issues identified in Chapter 4 including the appropriate data elements in a GILS record to support discovery and access, the most efficient procedures for capture of metadata (manual or automatic), and the presentation of GILS records to users. To ensure that the results of a pilot program serve all agency implementors, the program should include representative implementors from large and small agencies. In cases where smaller agencies face severe resource constraints for participation, incentives and funding may be necessary.

5.1.4. The Refocused GILS Initiative Includes a Program of Government–Wide Education and Promotion 

The refocused GILS initiative rejects the If we build it, they will come perspective. Policymakers and implementors should promote GILS as the first point of contact for users looking for government information. A clearly articulated purpose for a refocused GILS, aligned with demonstrable utility of it through a GILS pilot program, will be a basis for developing a program of promotion and education.

The investigators recommend that if GILS is worth doing, it is worth promoting! In the networked environment, there are many competitors for the attention of information seekers. GILS is a service that can compete, since it offers the general public with a government–wide search and retrieval service not offered by other online services. The Government has no special advantage in the marketplace with GILS; GILS is a service that offers a product (freely available) to other networked services providers to create value–added products of their own. GILS, however, is a service for which the Government has special responsibilities, since it will be the point at which citizens and the Government intersect for information access and dissemination.

A government–wide program of education and promotion also includes a focus on the agencies themselves. Agencies need to better understand what the refocused GILS can offer them, and a program of education directed at the agencies can build intra–agency support for the next stage of GILS development. A GILS pilot program will demonstrate tangible benefits as well as provide tested practices and procedures to the agencies. That effort may result in improved agency management buy–in, which may in turn result in sufficient and dedicated funds for agency GILS activities.

5.1.5. The Refocused GILS Initiative Emphasizes Continuous Improvement Through Ongoing Evaluation

The refocused GILS initiative acknowledges the evolutionary character of the networked environment, the changing needs, behaviors, and expectations of users, and the need for GILS to evolve to address user requirements and technology changes. GILS should be committed to the goal of continuous improvement with a resulting service and product that is responsive to its users. Its focus on users requires ongoing evaluation and assessment by the users for which it was developed. 

The investigators recommend that ongoing evaluation be a essential component of the refocused GILS. As reported from this study, GILS is a complex, networked service that can be assessed along multiple dimensions and from multiple perspectives. In addition, this study demonstrated tools and procedures for assessing various dimensions of GILS. Ongoing evaluation of GILS must incorporate a user–based approach since the users—internal agency users or external citizen users—are the final arbiters of the success of GILS. The refocused GILS must also identify specific and measurable objectives against which it can be assessed. Therefore, evaluation programs and procedures need to be incorporated during the early discussions about the refocused purpose, scope, functions, and objectives of GILS. GILS policymakers and implementors need to understand the objectives and criteria that constitute a successful GILS, and they must understand the need for and be able to collect appropriate information to conduct useful assessments.

5.2. PRIORITIES FOR ACTION TOWARDS THE REFOCUSED GILS

The characterization of the refocused GILS and associated major recommendations discussed above do not address the details of the issues documented in this report. Rather, they suggest an initial set of actions for government and agency policymakers and implementors to move toward specific solutions and to encourage the success of the refocused GILS across the Federal government. The investigators imagine a number of possible scenarios for the evolution into the next stage of GILS deployment, but all include the identification and prioritization of actions by policymakers and implementors. This section briefly outlines one ordering of priorities based on the findings and recommendations reported in Chapter 4. In a certain sense, the following ordering reflects a commonsense approach in that certain actions and decisions occur logically before to others.

The highest priority for the refocused GILS initiative is to gain consensus on the purposes, goals, and scope of GILS. The investigators have concluded that a major flaw with the current GILS effort is a lack of clearly understood purpose and utility. Chapter 4 noted that the successful agency GILS implementations occurred where agencies determined what GILS would be for them regardless of whether it aligned exactly with the prescriptions of OMB Bulletin 95–01. By defining a clear purpose for GILS, these agencies also identified its utility and recognized the benefits from their GILS implementations. Such agency–centric approaches, however, are unlikely to lead to a coherent government–wide information locator service. The refocused GILS must be directed by policy, but that policy needs to be built upon the consensus of individuals representing the affected agencies and the public that GILS serves. The first and most important step is to articulate the purpose of the refocused GILS, what it is, how it will work, and the potential benefits that will accrue to agencies and the public.

The second priority is for GILS policy to state clearly who has authority, who is responsible, and where accountability will rest for the refocused GILS as a government–wide initiative. This needs to include explicit statements concerning what such authority and responsibility entails. OMB Bulletin 95–01 named several agencies with various responsibilities for GILS, but except for the GILS Board, no organizational unit had government–wide authority, responsibility, or accountability. By devolving all GILS activities to the agencies without overall coordinating counterbalance, the result was very uneven implementations or no implementations at all. Part of this coordination and administration responsibility is to provide a necessary forum where agency implementors and others can work out specific implementation issues, requirements, and strategies.

The third priority is to develop policy goals for the refocused GILS and translate them into specific, realistic, and measurable objectives. OMB develops policy in consultation and with advice from agencies, the GILS Board, and others. OMB voices the information policy goals for the Federal government. OMB, however, does not have the responsibility for micromanaging the agencies, and the translation of policy goals to specific objectives must be carried out by others. One level at which this can happen is the organizational unit responsible for government–wide coordination and administration of the refocused GILS. At the agency level, appropriate objectives for GILS will also be developed. Without the intervening government–wide coordination level, however, agencies may take too much latitude in interpreting OMB goals and translating them—for themselves—into objectives that do not support the government–wide character of refocused GILS.

The fourth and final priority for initiating the refocused GILS effort is to establish a GILS pilot program. The organizational unit responsible for government–wide coordination could be charged with overseeing and administering a pilot program. To maintain a government–wide perspective for the refocused GILS, representative agencies of all sizes and missions should be included in a pilot program. A pilot program does not have to result in technologies, procedures, and practices that are a "one size fits all," and the variance of agency missions and resources must be reflected in the participants in the pilot.

The investigators recommend that these four priorities are critical first steps to move to the next stage of GILS evolution. The investigators also recommend that the GILS Board, with advice from the CIO Council and OMB, establish a GILS Transition Task Force to address these priorities. 

5.3. THE POLICY CONTEXT FOR THE REFOCUSED GILS

As discussed earlier in this report, the Federal policy context for information management has been dynamic and challenging in recent years to say the least. The specific context for GILS, as described in Chapter 2, mirrored this phenomenon. There is some sense that the efforts of the Information Infrastructure Task Force (IITF) on GILS, OMB Bulletin 95–01, NARA’s Guidelines for the Preparation of GILS Core Entries, and other efforts to implement GILS have been a grand experiment from which a significant amount of knowledge has been gained to improve the existing GILS context and refocus the service to accommodate evolving expectations.

There is wide agreement that strong and visible support from the OMB–OIRA, the CIO Council, the GILS Board, the Government Information Technology Services Board (GITSB), and the office of the Vice President as part of the National Performance Review are critical to the successful evolution of GILS (however it might be recast or reinvented). There is equal agreement that such support does not currently exist. Also important is the need for a better understanding of the roles and responsibilities for policy leadership as well as implementation leadership.

The study finds that OMB Bulletin 95–01 was a good first effort to outline a policy context for the development of GILS. Some issues that will require attention in a forthcoming revision to the Bulletin include: 

• Clarifying purpose and objectives of GILS (e.g., relationship with EFOIA, if any)
• Divesting records management responsibilities and activities from GILS
• Clarifying Federal leadership for a range of GILS activities
• Recognizing the extent to which agencies can take on GILS responsibilities in a time of budget reductions and increased demands on productivity
• Indicating realistic and tangible benefits that can accrue from GILS
• Integrating GILS into a broader context of agency information systems (including Web sites), IRM, and general information management missions
• Providing regular oversight and enforcement of GILS policies
• Promoting the development of search and retrieval mechanisms and processes that integrate and coordinate agency components of GILS into a government–wide GILS. 

These areas for policy revision are illustrative only. They do, however provide a flavor of the range and content that will need to be addressed in a revised OMB Bulletin on GILS. The investigators suggest other concerns (see Chapter 4) that will also require attention in a revised Bulletin. 

5.4. FURTHER RESEARCH AND EVALUATION EFFORTS 

The study collected a significant amount of data that describe many GILS–related activities and products. This evaluation also identified areas of policy and implementation needing additional research. Such research should be conducted to improve understanding of how future efforts broadly related to organization, discovery, and access to government information can be improved. GILS policymakers and implementors need to recognize, however, that networked information discovery and retrieval (NIDR) is basically still an evolving research area. This report noted that scalable and operational solutions to issues related to NIDR have yet to emerge. A GILS pilot program offers a valuable opportunity to conduct research on issues specific to U.S. Federal GILS implementation. Some specific research areas in need of additional attention include: 

• Use, Presentation, and Content of Metadata Records: Findings from this study and knowledge of other networked information discovery and retrieval activities suggest that metadata records may be a critical component in utilizing networked resources. For the refocused GILS initiative, a formal assessment should be done of the appropriateness of the metadata, the use of each of the data elements, and the extent to which they can support the goals of GILS. This research should include the use of metadata by human as well as machine processes (e.g., software agents). Alternative approaches for visually presenting the GILS metadata should be developed and tested. 

• Linkages between GILS and Web: This study has identified a number of approaches by which GILS or metadata information can be integrated and linked to agency Web sites. Research into which types of approaches are most effective (from a user as well as developer perspective) should be initiated. Criteria can be developed against which the various approaches can be assessed. 

• Extent of Coverage of GILS: Assessing the existing coverage of agency information resources by GILS was beyond the scope of the current study. Such assessment is necessary, however, to indicate agency compliance with OMB Bulletin 95–01. Research on this topic would examine the appropriate units of analysis for GILS records (i.e., specifying the aggregation/granularity of objects suitable for description), as well as identifying the universe of agency resources that should be described. To assess extent of coverage, however, would require knowledge of all agency information resources. This research would assist in answering the question: how many GILS records are sufficient and appropriate to provide coverage of all agency information resources. 

• Networked Services and User–Performance Variables: Another aspect of networked information discovery and retrieval is isolating user–performance variables relative to the environment (the Web) versus the functionality of system design. In a number of the assessments conducted (e.g., scripted online user assessment), it was difficult to determine whether, for example, poor response time is due to difficulties in Internet routing, technical design of the agency GILS, poor server response at the agency, or other variables. 

• Cost Benefit Studies: Some agencies that participated in this study clearly believed that the current GILS initiative was well worth the effort in light of the costs and other various resources committed to the effort. Others were adamant that no benefits occurred regardless of the costs involved. Additional study into why there are such vast differences in perceptions would be very instructive. Indeed, simply being able to identify specific costs directly associated with the GILS effort on an agency by agency basis would be useful. To a large degree, the investigators have to take at face value participants’ views of costs and benefits with little supporting "evidence." 

• Performance Criteria and Indicators: Neither OMB Bulletin 95-01 nor agency implementors detailed performance criteria and indicators, thus it is very difficult to determine, post–hoc, the degree to which a particular GILS effort can be described as a "success." The next phase of GILS should include a research initiative to determine a number of performance measures that can be used to gauge the success of the effort. Such performance measures require development, testing, and validity assessment (McClure & Lopata, 1996; National Academy of Public Administration, 1996). Further, this research could address how GILS links to the Government Performance and Results Act (GPRA). The Act requires that all agency programs have performance measures. GILS may be seen as an enabling tool for assessing program accomplishment, and it also needs to be assessed as a program itself. 

• Government–Wide Search and Retrieval: Effective searching across agency GILS (and other databases) or otherwise massive amounts of data requires additional research. In part, the failure of GILS to provide an effective cross–agency search facility is limited by the availability of effective search and retrieval tools. Support for distributed search and retrieval technologies such as the Advanced Search Facility (ASF) or some ASF–like effort is essential. Further, recent applications by GPO and FedWorld for cross–agency searching need to be evaluated to measure their utility for government–wide searching. User–based assessments on search and retrieval tools are required. 

• Agency Staff Responsibilities, Accountabilities, and Authorities: The study identified a wide range of agency staff and offices that ultimately had responsibility for "GILS–related activities" during the 1995–1996 effort. However, a systematic identification of who, specifically, had what types of responsibilities (i.e., management, records development, technical design, etc.), other responsibilities of these individuals, and some background information as to their education, experience, knowledge, and degree to which they had training, would be very useful. Such information could help explain, possibly, the significant discrepancies in agencies’ perspectives toward the GILS effort. 

• Comparative Studies of Other Government Information Locator Service Implementations: Numerous state and international GILS implementations are occurring (e.g., Washington State and Canada). Although the U.S. Federal implementation of GILS preceded others’ efforts, analysis of policy and procedures of these non–Federal implementations could provide practices and procedures to improve U.S. Federal activities. 

• Government–Wide Records Management System: This study recommended that NARA be tasked with the responsibility for developing a records management system to better identify, schedule, and ultimately preserve appropriate government information resources, especially resources in electronic or digital form. This effort will require a research component prior to any system design and testing. The component will need to clearly identify system requirements, determine the agency uses and applications of such a system, and describe existing techniques for management of electronic records. 

• Policy Review and Analysis of "Locator" Systems: The literature and policy review provided in Chapter 2 identifies a range of ambiguous, contradictory, and confusing policy language related to government–wide locator systems. Policy research and analysis is needed to develop one coherent statement that organizes policy language from these various instruments. The results of such research can be included in a future revision of OMB Bulletin 95–01. 

This list of additional areas requiring research is not intended to be comprehensive. Rather it is illustrative of key topics from which additional knowledge would be extremely helpful in supporting possible future GILS activities. 

Another thrust of this study was to design, develop, and test assessment techniques. The intent was to provide policymakers and agency officials with tools by which they could deploy a range of assessment techniques and comply with policy such as GPRA. To date there has been little consideration (at least as identified in this study) about agency–based performance assessment and the development of performance indicators for GILS efforts. 

The various instruments developed for this project should be seen as first efforts. Additional research related to these evaluation tools is both necessary and appropriate. Some possible areas for additional effort, for example, include: 

• Log Analysis: Appendix E–4 provides detailed explanations, techniques, and findings regarding the log analysis done for two weeks of HTTP transaction logs from EPA. There are numerous avenues of additional research in this area, some of which are outlined in the appendix. These techniques offer agencies and policymakers an important tool for monitoring and refining Web–based services. While some agencies do make use of basic log analysis techniques, most have yet to explore the techniques described or to develop the techniques proposed in this study. 

• User–Based Assessment Techniques: The study found that the scripted online user assessment approach is a powerful tool for obtaining users’ assessments of GILS information resources and services. Techniques developed here can be modified for use in individual agencies. Additional research should be undertaken on how to simplify the technique and how to better relate the scripting process to specific assessment criteria and performance indicators. In addition, video–taping users and asking them to "think aloud" as they use a particular networked service appears to have great potential as an assessment technique (Eschenfelder, et al., 1997). See Appendix C–5 for a description of the method used in this study and Appendix E–3 for the summary of results. Included in the latter appendix is a list of suggested questions and procedures to improve the method. 

• Metadata Record Content Analysis: The study included a task that identified all GILS records and then analyzed the content of a sample of these records. This was an important first step (see Appendix C–4). An important next step will be to refine the criteria and procedures for assessing the quality of metadata. Especially important is obtaining assessments of these metadata records from users. A scripted assessment technique, similar to that used to assess the agency GILS implementations could be developed for assessing the records. This research can identify primary, secondary, and tertiary metadata elements that support the purposes of GILS. Appendix E–2 identifies a series of questions that could guide research to improve this method. 

Additional research on evaluation methods and data collection tools in the provision and management of networked information services is essential (McClure & Lopata, 1996). Overall, there has been little evaluation research in the area of government electronic networked services (Wyman, Beachboard & McClure, 1997). 

An important benefit from this study is the development, testing, refining, and documentation of research techniques and evaluation tools. Initiatives related to GILS and its evolution should continue the development, testing, and use of assessment tools and methods discussed in this report. Indeed, these tools should help agencies better comply with GPRA for GILS–related activities and programs. 

5.5. REENGINEERING THE GILS EFFORT 

Policymakers must carefully determine the best approach to take for future GILS or GILS–like efforts. As reported in this study, there are serious issues and problems that currently limit the overall usefulness of GILS. The investigators believe that the original vision of GILS was not a clear one. GILS evolved into being different things for different people and agencies—evolving, with some notable exceptions, into an effort with little user or implementor support, limited usefulness, and with confounded purposes. 

Having said this, however, one should not overlook the fact that a number of agencies developed working and successful agency GILS, yet they defined the GILS vision within the context of their agency. These efforts are significant and should be recognized and applauded. Indeed, the knowledge gained by GILS implementations at DTIC, Treasury, EPA, GPO, and Interior, for example, is critical for the next stage of GILS development.  

Despite these successes at a few agencies, the diagnosis is that GILS suffers from multiple–personalities disorder and schizophrenia. Despite its condition, GILS has not received adequate treatment from its "doctors" as they each have different views about the needs and appropriate treatments for the patient. Many individuals and agencies have given up on developing any treatment for it and have gone on to other more pressing problems. But such does not have to be the case for future efforts. 

This study recommends that the existing GILS as developed during 1995–1996 be considered as Phase I. The lessons learned from this experience can contribute significantly to future efforts to develop a discovery and access service for government information. But GILS, as currently constituted and currently implemented, must be refocused and reengineered if it is to be a success. 

A refocused GILS initiative based on the recommendations offered in this chapter is feasible and doable IF there is administrative coordination and commitment to completing such an effort and IF there is agreement as to the specific nature of the effort. The following vision for a refocused GILS could be a basis for such agreement: 

The next phase of GILS development will build on the basic architecture of decentralized, agency-based locators, standardized metadata records, use of Z39.50, and will draw as well upon Web and other technologies and developments in the arena of NIDR. The refocused GILS would provide government–wide search and retrieval capability and it would provide direct links to full–text information when available. It would provide online access to information and not just metadata, and the metadata records could be transparent to users except to provide them with characterization of resources that might be relevant to them. At the administrative level, the refocused GILS provides a balance between decentralized, agency-level GILS activities and government-wide oversight and coordination to result in a coherent and usable government-wide information locator service. This is only a broad brush at what that vision should be, but it offers a direction for the refocused GILS efforts. The investigators believe that the GILS experience can provide a significant number of lessons and information for moving forward with and improving GILS. 

The vision of a tool that allows users to search for, discover, and obtain government information across all agencies in full–text via the network is an important vision to maintain—regardless of the future of GILS. While there is likely to be controversy and debate on how best to reach that vision, efforts should continue to make that vision a reality. Individual agencies cannot reach this vision on their own, however. Central direction, coordination, and some resource support will be needed.  

Users, policymakers, agency officials, librarians, public advocacy groups, and others widely support the vision of a refocused GILS as outlined in this chapter. It is a vision that requires national support. It is a vision that is too important to be ignored. It is a vision that the Administration’s efforts to improve the government’s ability to provide a range of networked information resources and services clearly support. It is a vision that can be reached.

Appendices
Previous Chapter
Table of Contents