4.0. INTRODUCTION

Since the publication of the Office of Management and Budget (OMB) Bulletin 95–01 in December 1994, Federal agencies have undertaken implementations of the GILS Profile to comply with the mandate of the Bulletin. The study identified approximately 45 agencies that have some form of GILS implementation. Study results indicate that for a handful number of agencies, GILS has improved access to and knowledge of agencies’ indexes, catalogs, finding tools, and other "metadata sources."

Yet, for many other agencies, the Federal GILS initiative has been little more than another unfunded mandate that received little administrative support, has not met original objectives, has provided few benefits to agencies and users, and has little visibility either in government or with the public. Further, much confusion exists over what GILS is and should be. The agency GILS that are operational have limited use and study participants assessed them as difficult to use. In addition, the records management component of GILS mandated by OMB Bulletin 95–01 was poorly conceived, and GILS as a records management tool does not assist records managers in meeting their responsibilities related to records management.

For the majority of agencies, their GILS effort and expense has not resulted in adequate or tangible benefits—regardless of how one defines "benefits." Many agencies reported that limited resources were available for GILS, and the lack of resources and effort by some agencies (e.g., one agency had created only a single GILS record) has limited the potential utility of GILS as a government-wide information locator. Nonetheless, the study also finds that agencies and users are positively disposed to the concept of GILS, defined by OMB Bulletin 95–01 as a service that "will identify information resources throughout the Executive Branch, describe the information available, and provide assistance in how to obtain the information" (Office of Management and Budget, 1994b). With a conscientious refocusing, GILS could have great potential to improve access to and use of Federal government information.

The investigators conclude that GILS—as a concept and mechanism—has an important role to play in discovering and accessing government information in the networked environment. The investigators affirm the underlying architecture of GILS: standardized metadata records, decentralized agency-based locators, standard protocols (i.e., Z39.50) for intersystem information retrieval. The U.S. GILS implementation, however, has not achieved fully the vision of a "virtual card catalogue" of government information nor have the agency GILS implementations matured to the extent of providing a coherent and usable government-wide locator service. The investigators conclude that many of the current shortcomings with GILS relate to problems of focus, scope, and administration rather than a fundamental flaw in the concept of GILS.

The investigators recommend the Federal GILS initiative be refocused to clarify both purpose and functions of GILS implementations. A refocused GILS initiative can assist in providing guidance to all agencies as they continue their implementations as well as offering clearer evidence of the utility of GILS to the many agencies that have concluded GILS is neither useful nor beneficial.

A refocusing of the GILS effort provides the next evolutionary step for U.S. GILS development. It will build upon the work accomplished and upon the experiences and lessons learned for improving public access to government information in the networked environment. Policymakers, however, must draw a clear line of demarcation between the early GILS implementation period (i.e., 1995-1996) and a refocused and reengineered GILS. This line of demarcation is essential because it represents an acknowledgement by policymakers and implementors that:

• Many agencies are now unwilling to put additional resources into an initiative of questionable utility
• Lessons have been learned by policymakers and implementors from the early implementation experience
• The refocused GILS will address shortcomings and issues made visible from existing implementations.

One way in which a refocusing of GILS can be underscored is through a change in the name to reflect, for example, a "second release" of the U.S. Federal GILS service.

These general statements of findings and recommendations are detailed in the subsequent sections of this chapter. The chapter has two opening sections that describe the Federal context in which the GILS initiative occurred and the current status of agency GILS implementations. The chapter then organizes findings and recommendations into four primary opportunities which are discussed in Sections 4.3. through 4.7.: 

• Refocus GILS for Clarity of Purpose and Utility
• Improve GILS Efficacy in Networked Information Discovery and Retrieval (NIDR)
• Resolve GILS Relationships with Other Information Handling Functions and Processes
• Increase GILS Awareness.

The opportunities provide policymakers and implementors with a framework for addressing areas where the Federal GILS initiative can be improved. Each section in this chapter that describes one of these opportunity sections includes a table identifying relevant findings, recommendations, and supporting sources of evidence. Table 4-1 summarizes the four opportunities and associated findings and recommendations.

The recommendations reflect the investigators’ analysis, synthesis, and understanding of the data collected during the study and the findings reported here. A number of the findings and issues uncovered during the study, however, presented challenges in devising specific recommendations. This is especially the case in recommendations relating to the second opportunity area, "Improve GILS Efficacy in Networked Information Discovery and Retrieval (NIDR)." NIDR is an active research area; researchers and early implementors have recognized the complexity of many NIDR problems in the past several years. For example, in the areas of metadata and distributed search and retrieval, there are prototype implementations but complete and scalable solutions will await additional research (see Lynch, 1997; Lynch, et al., 1995).

The investigators believe that the recommendations will contribute to determining the next evolutionary steps for the U.S. Federal GILS initiative. Chapter 5 proposes a framework of action that identifies next steps for a refocused GILS effort. Ultimately, however, it is the project’s advisory group, the sponsoring agencies, and the GILS Board that must determine what is to be done with GILS. The findings and recommendations reported here can provide substance as well as points of departure in the deliberations of the advisory group, Federal policymakers, implementors, and the GILS Board. 

In keeping with the charge of the study to examine how the GILS initiative serves users (see Moen & McClure, 1996a, for the study’s Technical Proposal), the findings reported here rely on data collected from the various groups of "users" involved with GILS. The term "users" of GILS belies the complexity of identifying who, specifically, the GILS users are. For purposes of this discussion, user groups appear to be best described in the following terms: 

• Federal agency staff: including agency GILS implementors, agency managers, records managers, policymakers, agency librarians, and others
• State and local government staff: including state and local GILS implementors, state library agencies, records managers, librarians, and others
• Non–governmental individuals: including librarians, public advocacy groups, journalists, the "public," those with special subject interests, and others.

The user–based evaluation designed by the investigators recognized and valued the various special interests and perspectives of all these user communities.

Chapter 3 discussed the multi–method approach used in this study (with complete details of the specific methods in Appendices C–1 through C–6). The data collection and analysis activities carried out during the study produced a significant amount of information from which the study’s findings and recommendations flow (Appendices E–1 through E–4 contain detailed results organized by data collection activity). The findings reported here are based on data collected through the following sources of evidence: 

• Site visits
• Focus groups
• Survey
• GILS record content analysis
• Scripted online user assessments
• Web server transaction log analysis
• Policy and literature review.

For each of these activities, the study team compiled results and produced detailed summaries. For example, the summary for a typical site visit is about 25–40 pages plus appendices. Particular findings may be based on data produced from one or more of the study activities. Often, similar findings emerged from more than one data collection effort. Instead of reporting the results for each data collection activity or instrument, this chapter organizes the findings and recommendations into opportunity areas. When appropriate, the discussion links evidence from specific sources or assessment activities to particular findings. 

4.1. GILS IN THE FEDERAL CONTEXT 

OMB Bulletin No. 95–01, issued in December 1994, formalized the U.S. Federal GILS initiative and provided policy guidance for its implementation. At the same time, the National Institute for Standards and Technology (NIST) released Federal Information Processing Standard (FIPS Pub.) No. 192 that provided technical specifications and implementation guidelines in the GILS Profile. In February 1995, NARA published The Government Information Locator Service: Guidelines for the Preparation of GILS Core Entries to assist Federal agencies in the creating GILS records. Approximately 2 years have passed between the formal announcement of the initiative and this assessment effort. As a government–wide initiative, GILS is relatively young.  

During this 2–year period, a number of key factors affected the Federal government environment and the GILS initiative. First, the government launched GILS during a time of significant downsizing, budget cutting, and reorganization of the Federal government. There was substantial discord between Congress and the Administration regarding the appropriate role and size of many government agencies. Agencies, oftentimes, were under pressure to reduce budgets and reduce staff size, yet also expected to demonstrate greater productivity and "streamline" operations. The closing of the Federal government due to budget disagreements between Congress and the Administration in 1996 also contributed to an already difficult work environment (in fact, the shutdown caused an extension to the first OMB Bulletin 95–01 deadline for GILS implementations). In short, agency morale often suffered.

Second, recent years have seen, perhaps, the greatest amount of activity related broadly to information management issues, policies, and legislation in the history of the U.S. Federal government. The Government Performance and Results Act of 1993, The Paperwork Reduction Act of 1995, The Information Technology Management Reform Act of 1996, and The Electronic Freedom of Information Act Amendments of 1996—to name but a few legislative initiatives—significantly affected the environment of information management in the Federal government (see the policy review section in Chapter 2).

In addition to legislation, a host of policy issues related to encryption, privacy, information technology (IT) procurement, standards, electronic records management, access to government information, the National Information Infrastructure (NII), revision of Federal printing laws (e.g., Title 44 USC), and other topics required the attention of agency information managers. Between legislated mandates and other information management/policy issues, there has been no lack of work or policy issues demanding attention from agency officials in the broad area of information management.

Finally, the emergence of new IT and related applications has also been significant. Since early 1994, Internet—specifically Web—applications have dominated and redefined access to and dissemination of information. Due in part to initiatives related to the NII and the Administration’s interest in utilizing Internet and Web technology, agency use of Web applications for disseminating information and providing electronic information services grew exponentially. One need only examine GPO Access, NTIS’ FedWorld, the Library of Congress’ Thomas legislative search system, and the many agencies that have established Web sites in the past several years to gain an appreciation for the use and interest in Web technology by the Federal government. The well–known Federal Web Locator maintained at The Villanova Center for Information Law and Policy <http://www.law.vill.edu/Fed-Agency/fedwebloc.html> now indexes and provides access to nearly 1,000 Federal Web sites. Truly, the Web changed fundamentally the ways many agencies use the Internet for presenting and publishing information.

The development and use of the Internet and Web technology by Federal agencies is a significant factor that shaped the Federal information management environment in recent years. At the time of the work on the technical and policy specifications that underlie the GILS initiative (1993–1994), the growth and development of Web–based services could not have been foreseen. The Web phenomenon was a surprise to almost everyone, including the designers and developers of the GILS Profile as well as U.S. Federal GILS policymakers. To some degree, the GILS initiative may have been swallowed by Web developments. The latter clearly caught the interest of both the public and government officials much more so than GILS because the Web was concrete and real—people could see it, use it, and understand its potential. The Web now offers agencies a mechanism for easy electronic publishing and dissemination of large amounts of information, and users can access the full–text of documents. 

GILS as a set of metadata records describing government information—or GILS as an implementation of Z39.50—is not nearly as glamorous nor easily understood as the Web. While the Web offers new opportunities to agencies, it has a limited capability to help users discover and locate government information resources, especially on a government–wide basis. GILS metadata records and the use of Z39.50 as a standard mechanism for interoperable search and retrieval across GILS databases, however, has the potential for solving the problems of information discovery in the networked environment.

Several other factors will be identified in this chapter that affected the success of GILS as implemented by Federal agencies. But these three—downsizing government, expanding information management legislation and policy issues, and Internet/Web development—should be recognized as affecting the current status of the U.S. Federal GILS initiative. As a long–time information resources management (IRM) official commented to one of the investigators during the study, "never in my years working for the government have I seen as much change in information management and policy as I have seen during the last three years."

The implementation of GILS took place during a period of significant technological and agency change, uncertainty, political discord, opportunity, pressure, stress, and excitement for Federal information managers. One important finding from the study is that GILS, given this context, simply was unable to compete for the attention, resources, and commitment from most agency administrators.

4.2. EXTENT OF CURRENT GILS IMPLEMENTATION 

An initial analysis of the number of agencies involved with GILS implementations presents a relatively positive picture. During the evaluation, the investigators identified 45 units of government (e.g., executive agencies, independent agencies, commissions, government corporations, etc.) carrying out some type of agency GILS implementation. A closer look, however, reveals the extent to which these agencies have taken ownership of the initiative. Further, certain cabinet–level departments appear not to have undertaken any GILS implementation as of February 1997 (e.g., Departments of Education, Justice, Transportation, and Veterans Affairs). 

Agencies had the option of either mounting their GILS records on an agency server that complied with FIPS Pub. 192 specifications for using Z39.50 or contracting with another agency to make their records available. The study identifies these approaches as "record–source hosted GILS" and "brokered–GILS." The former means that the agency creating the records is also responsible for making those records available via the Internet, and the latter means that an agency creating GILS records contracted with another agency to make those records available. Both the Government Printing Office (GPO) and FedWorld offer this "brokering" service to agencies.

The study identified eighteen "record–source hosted GILS" sites where an individual agency server provides access to that agency’s GILS records. A total of 2,089 GILS records are available from these servers. Table 4–2 presents a summary of records provided by each agency. See Appendix B for a list of agency GILS servers/databases with network addresses.

FedWorld and GPO offer services to agencies in mounting and making agency GILS records accessible, and through this service they have become central points of access to the majority of agency GILS records. Table 4–3 summarizes the brokered records from GPO and FedWorld. 

As of March 1997, GPO hosted a total of 2,815 GILS records from 27 agencies (in addition to mounting the Privacy Act notices database from NARA). It also provides "pointer records" to 7 agencies that have GILS records available but which are not mounted at GPO. In April 1997, GPO began offering a new search application through which a user can submit a search across one or more agency GILS, whether or not the records are mounted at GPO. A user selects which agency GILS databases or servers to search, submits a query, and the search is broadcast to the selected GILS databases and servers. GPO’s recent efforts point to one direction of possible cross–agency, government–wide searching with GILS. 

GPO also offers searches on a database compilation of Privacy Act Notices, an area of GILS coverage mandated by OMB Bulletin 95–01. In August 1995, NARA and OMB agreed that this requirement could be met by allowing NARA to make available its compilation of Federal Register Privacy Act notices on GPO (see Appendix A–4 for the NARA memo). This agreement relieved agencies from the requirement to create GILS records for agency Privacy Act systems and associated notices already published in the Federal Register. The NARA database of Federal Register notices provides coverage of additional agency resources not necessarily reflected in the records in Tables 4–2 and 4–3. GPO mounted the compilation of Privacy Act Notices to meet the requirements of OMB Bulletin 95–01. There are currently 5483 documents listed in the Privacy Act Notices compilation, but these are not in the standardized GILS record structure, and are not calculated into the total number of GILS records available for searching by users. 

Also, as of March 1997, FedWorld served as host for three agencies’ records totaling 353. During the course of the evaluation study, however, FedWorld expanded its listing of GILS records to include those hosted by GPO (excluding the GPO Privacy Act application) and an additional six record–source hosted GILS sites. FedWorld currently offers searches of 35 different agencies’ records. (FedWorld lists 36 agencies’ databases but that number includes Department of Commerce GILS records mounted at FedWorld as well as its records mounted at GPO.) Users of FedWorld GILS can search the three databases mounted at FedWorld as well as following links to other agencies’ GILS records.

* Date associated with the number of records found; these sources were checked in March 1997
** GILS records offered as standalone HTML files rather than in a WAIS or Z39.50 searchable/accessible database  

* Date associated with the number of records found; these sources were checked in March 1997 

Based on the information presented in Tables 4–2 and 4–3, a reliable estimate of the number of available GILS records (as of March 1997) is approximately 5,000. One might immediately ask:  

This, however, is a difficult question to answer. The Information Infrastructure Task Force (IITF) report (Information Infrastructure Task Force, 1994, p. 11), provided the following estimate: 

Although the origin of the "100,000" number is unclear, the goal was to create sufficient GILS records to provide comprehensive coverage of Federal government information resources and assist users in locating those resources. The estimated 100,000 locator records would describe the resources identified in OMB Bulletin 95–01: automated information systems; locators to agency resources; and Privacy Act Systems.  

OMB 95-01 defines GILS Core as "a subset of all GILS locator records which describe information resources maintained by Federal agencies, comply with the GILS core elements defined in Federal Information Processing Standards Publication (FIPS Pub.) 192, and are mutually accessible through interconnected electronic network facilities." The OMB 95-01 definition is less descriptive than that offered in the IITF report which provides additional information about GILS Core including: 

The emphasis in the GILS design document and policy on distinguishing "GILS Core" records from other GILS records, however, has not led to clear distinction in practice. The analysis of a sample of GILS records conducted as part of this study (see Appendix E-2) showed little difference between GILS records identified as "Core" (through the use of the term "U.S. Federal GILS" in the Controlled Vocabulary–Local Subject Index Term element) and those not so described.

If an agency already had locators or inventories that could be described by a GILS record, a few GILS records might be sufficient to address the goal. In the absence of pre–existing locators, however, some agencies have been describing individual documents and publications. In that case, a major information disseminating agency might have to create thousands of records to gain the coverage envisioned for GILS. Measuring the extent of coverage of agency resources by GILS records would require the existence of comprehensive inventories of agency resources (i.e., a baseline against which to measure). Although the study did not attempt such a measure of coverage, the question of whether 5,000 records is sufficient to provide users with the ability to discover and access agency resources needs to be addressed. The question can be framed as follows:

Data from the study suggest that the GILS initiative, as it is currently being carried out, is not likely to improve coverage. Moreover, users, who were the focus of this evaluation, stated that based on their experience with GILS, current coverage of government information resources is insufficient. Users also want GILS to provide direct access to the actual information resources. 

One can claim that the GILS initiative is new, and the approximately 5,000 records created in the past 2 years are a good beginning. Other findings discussed below, however, suggest that many agencies are not likely to be creating new records. Thus, the current 5,000 may be the extent of GILS record creation and government-wide information resource coverage cannot be expected. For example, one major information disseminating agency stated that their approximately 300 records cover what GILS mandated, and it is not likely to be creating additional GILS records. A number of the smaller agencies stated that because insufficient resources were allocated to implement GILS, and because they see little return on investment (ROI), they would not be creating more records and in fact would not maintain the records they had created. The Source Date column in Tables 4–2 and 4–3 is indicative of GILS activities, with many of the databases showing the most recent updating in 1996.

To estimate the universe of GILS records, it was first necessary to identify existing GILS sites. This was a major task to ensure that no agency involved in any GILS implementation was overlooked. Reviewing the steps in that identification process (see below) also demonstrates one of the challenges facing users of GILS and a liability of the current implementation—there is not a single registry of existing agency GILS implementations. Implementing such a registry would provide a user with a source to determine which agencies have GILS implementations, the number of records associated with each implementation, and the network location of each implementation. 

For the evaluation study, the sites listed in Tables 4–2 and 4–3 were discovered through the following activities: 

• Verbal or written mention during the 1996 GILS Conference presentations and in handouts and survey responses
• Linking from the White House Web site’s "President’s Cabinet" (http://www.whitehouse.gov/WH/Cabinet/html/cabinet_links-plain.html) and "Federal Agencies and Commissions" (http://www.whitehouse.gov/WH/Independent_Agencies/html/independent_links-plain.html) to agency homepages, which in turn linked in some cases to FedWorld GILS (http://fedworld.gov/gils)
• Web searches by means of Alta Vista and Lycos search engines for Executive department and agency names
• as delineated in the 1996–97 Government Manual via GPO Access (http://www.access.gpo.gov/su_docs/aces/aaces002.html)
• as comprising the Chief Information Officer Council as specified in Executive Order 13011 of July 16, 1996 Federal Information Technology (http://www.gsa.gov/irms/ka/regs/exo13011/exo13011.htm)
• WEB searches by means of Alta Vista and Lycos search engines for "GILS," and "government information locator service"
• GPO Access GILS server
• Appendix A of Potholes on the Information Bridge to the 21^st Century, the Second Annual OMB Watch report on the U.S. Federal Government Information Locator Service (Henderson, 1997).

This effort was necessary to ensure that all agency GILS sites were identified. Through this effort, the investigators not only confirmed the agencies’ GILS identified at GPO and FedWorld, but also identified 8 other agency GILS implementations not listed by either GPO or FedWorld. Not all of those 8, however, have their GILS records residing on an information retrieval–based platform such as WAIS or Z39.50–compliant server. These agencies offer their GILS records via a Web server, and the GILS records are simply hypertext markup language (HTML) files comprising GILS elements. The fact remains that these agencies are implementing GILS in a fashion, and their records should be included in estimating the universe of GILS records.  

FedWorld’s and GPO’s recent efforts to provide single points of access to multiple agencies’ GILS records move the Federal GILS initiative in the direction of a truly government–wide locator service. The study found, however, a range of responses to and interpretations of what GILS is or should be and how it should be implemented. These responses and interpretations by individual agencies may mitigate against comprehensive coverage of publicly available government information in a manner that is useful to the public and other users trying to discover and access government information. The first "opportunity" that needs to be addressed is how to refocus the GILS effort by clarifying its purposes, goals, benefits, and expected impacts.

4.3. OPPORTUNITY: REFOCUS GILS FOR CLARITY OF PURPOSE AND UTILITY 

Many of the findings reported in this section reflect a need for a clarification of what GILS is, what functions it should support, what agencies are expected to do, and what benefits might accrue. The study found that the original expectations for agency participation in GILS did not adequately acknowledge the resulting burdens upon many agencies nor account for a range of factors that might constrain agency GILS implementations (e.g., the lack of appropriate network and information technology infrastructure). On the basis of these findings, the investigators recommend refocusing the U.S. Federal GILS efforts in the next stage of GILS development. Table 4–4 summarizes the findings and recommendations for this opportunity. 

* CA=content analysis of GILS records; FG=focus group sessions; KP=interviews with key participants;
LA=log analyses of Web servers; SU=survey conducted at the 1996 GILS Conference;
SV=site visits to selected agencies; US=scripted 

4.3.1. FINDING: People Are Confused About GILS Mission, Purposes, and Uses 

Considerable confusion exists among both agency implementors and external users as to the purpose of GILS, what it was intended to accomplish, and just "what exactly the GILS is." One person commented "at 30,000 feet, GILS is a good idea, but implementing this at ground level, it became all things to all people." This problem is exacerbated by different stakeholder groups and audiences who each look at the GILS initiative from different perspectives. Figure 4–1 summarizes some of the competing purposes and audiences that might be addressed by GILS. Clearly, additional possible purposes and audiences could be added to this figure. 

 The confusion over what GILS was intended to be, what it is, and what it might become was a constant theme in the various data collection efforts. As one example, the survey administered at the November 1996 GILS conference asked several questions related to respondents’ understanding and definitions of GILS. Approximately 180 conference participants completed the survey (see Appendix E–1 for details on survey respondent demographics).

Question 1 asked respondents for their definition of GILS. This open–ended question produced a wide range of answers. (Tables E1-7 through E1-10 in Appendix E–1 summarizes the responses.) Respondents’ definitions highlighted four primary perspectives on GILS, but their definitions often addressed more than one: 

• GILS from the perspective of its functions including Finding Aid ("card catalog," "index," "pointers," etc.); Access ("provide access to," "retrieve information," etc.); IRM ("managing resources," "records management," etc.); Collect ("agencies ‘collect’ information via GILS"); Control ("agencies ‘control’ information via GILS")
• The types of information GILS comprises including Publications, Resources, Systems, Records, and Services
• Potential users of GILS including Public, Agency, Private, Library, Researchers, etc.
• The coverage of GILS including "Federal government information," "Important/major/prime information," "Executive information," "Electronic information," "Useful information," and "Other." The category of "Other" includes the following limitations to GILS coverage:

The broad range of responses to this request for a definition of GILS is indicative of competing expectations as well as misconceptions on the part of users and implementors. 

The survey also requested respondents to assess a number of key issues, some of which addressed definitions and purposes of GILS (see Tables E1-11 and E1-12 in Appendix E–1). There was a high level of agreement to the statement: A purpose of GILS is to improve public access to government information (89% of respondents agreed with this statement). Yet only 55% agreed with the statement: A purpose of GILS is to help agency officials better manage agency information. Only 45% of respondents agreed with the statement: I am able to describe GILS accurately and fully to others. In terms of coverage of GILS, only 33% agreed with the statement: GILS records represent the complete information resources of an agency.

The site visits and focus groups also highlighted a lack of clarity about the purpose of GILS. Many of the participants in those activities identified the need to clarify the purpose of GILS so that people (e.g., agency staff and public users) could know what to expect to find when using it. One person in an agency site visit stated that "GILS has an identity crisis—what exactly is its purpose? Is it for public relations? Is it for providing information to the public? Is it for records management?" The need to clarify GILS’ purposes and objectives was also tied to understanding what tangible benefits would accrue to agencies by using GILS.

To a large extent, GILS has become "different things to different people" or, more precisely, people see in GILS what they want to see. Individuals complained that they cannot find quick factual answers to reference questions in GILS. While users might have such exceptions, the fact is that the original design of GILS did not intend it to support that functionality. Others have proposed that GILS be used to manage electronic Freedom of Information (EFOIA) requests and information—again, never a stated goal or purpose of GILS. In both of these instances, the GILS record structure does not support such purposes.

The study found contradictory, confused, ambiguous, and erroneous perceptions of GILS’ intended purposes and GILS’ potential purposes. The investigators were told of instances when GILS policymakers and implementors, during early training sessions, publicly disagreed with each other as to GILS’ purposes.  

Given this situation, the successful implementations were those by agencies that decided for themselves what GILS would be in their setting. For example, EPA, Defense, and Treasury created agency GILS to serve both internal and external users and uses. EPA sees its GILS implementation as a component of its larger information dissemination and access responsibilities. Defense and Treasury see GILS as serving as a useful tool for inventorying and information management. While these are not contradictory roles for GILS, a user looking for information across the government may be confused by the differing levels of coverage, granularity of description, and focus of specific agency GILS.

Study participants and users of GILS judge the service in light of their perceived purposes and expectations of GILS and often are very disappointed. Clearly, some of the cells in Figure 4–1 are not mutually exclusive. But the findings indicate that there is a lack of agreement as to the purposes of GILS and what one can reasonably expect GILS to accomplish in terms of providing access to and management of government information. One person commented that GILS does not provide government–wide information (as advertised in the name of the service); rather it identifies some possible agency sources that might have the information needed if one could get into those other sources. To this person, the name of GILS was a misnomer in itself.

4.3.2. FINDING: Expectations for GILS Are Evolving 

At the 1996 GILS Conference, a number of speakers made an important point by separating the original GILS vision from the manner in which agencies had implemented GILS to date. The GILS Conference survey (and presentations made at the Conference) and other data collection activities indicate substantial support for the original GILS concept of improved public access to government information. Yet only limited support exists for the GILS implementation as outlined in OMB Bulletin 95–01 or as undertaken by most agencies. This may be due, in part, because a "government–wide" perspective on Federal information has yet to emerge from the GILS initiative.

Study participants noted the desire to obtain the "actual" information rather than simply descriptions of information resources. In part, the widespread deployment of Web technology has raised expectations on the part of users in terms of gaining immediate full–text access to government information.

This study found support for what might be termed a refocused GILS concept which can be summarized as: 

Study participants suggested the original GILS concept is being replaced by a belief that a refocused GILS is of greater utility. This refocused GILS concept is not incompatible with the existing concept of GILS, yet it is more limited in scope (e.g., the refocused GILS is not tied to records management; see Section 4.5.1.). In addition, the refocused GILS clearly responds to the desire of users for a single point of access for searching government–wide for information. This can be seen as a positive evolution for GILS.

As noted in Chapter 1, GILS was an ambitious undertaking. The effort should not have been viewed as a panacea for the various issues relating to access and management of government information, and it could be expected that major technical and policy issues would arise during this early implementation period. Learning from implementation experience has been common for many agencies. Further, the technology environment in which GILS has been implemented since early 1995 has changed enormously. The emergence of Web technology has generated new expectations among Internet users, and a simple set of pointers to metadata is no longer sufficient for most users.

These and other factors have created a need for a more focused and consensus–driven conception of GILS that responds to the demands of users, both Federal agency staff and non–government users, interested in discovering what information is available and then being able to access that information directly.

4.3.3. FINDING: Government–Wide Administrative Coordination and Policy Oversight Are Lacking

GILS, as originally conceived, would be a decentralized information service consisting of agency information locators linked and interoperable through the use of common technical and content standards. OMB Bulletin 95–01 identified lead agencies for particular aspects of GILS (e.g., NARA for record creation guidelines and training). The Bulletin, however, was silent on how government–wide coordination and oversight of the GILS initiative would occur. The Bulletin established the GILS Board with responsibilities to "evaluate the development and operation of the GILS," but it has met only once since the publication of OMB Bulletin 95–01. Study participants suggested that a lack of government–wide coordination and oversight is one of the causes for the current state of GILS. Further, a number of participants recognized that the decentralized implementation of GILS needs to be balanced by some level of centralized management and coordination to assure the coherent development of a government–wide information locator service. 

One group that has been active since March 1995 is the Special Interest Group on the Government Information Locator Service (GILS SIG). Its Statement of Purpose (see Appendix A–6 for the complete Statement) includes the following: 

The GILS SIG has been instrumental during the past 2 years of GILS implementation and has provided a forum for information sharing during GILS development. It is not authorized, however, as a policy making or coordinating body for U.S. Federal implementations of the GILS Profile. Since the GILS Profile has application outside of the U.S. Federal implementation, the GILS SIG membership is open to anyone interested in using the GILS Profile (e.g., state and other national governments). Since its responsibilities and participants are broader than U.S. Federal implementation of GILS, it is not an appropriate forum for administrative and policy coordination for the U.S. Federal GILS initiative.

The GILS SIG operates under the auspices of the Open Systems Implementors Environment Workshop (OIW) and assumed in late 1996 the responsibility for maintaining the GILS Profile. The GILS SIG does not provide a formally constituted or authorized forum for discussions of U.S. Federal implementations of GILS. As originally constituted, the OIW groups were places where implementors and users could convene to identify specific application requirements for standards and to arrive at consensus agreements on profiles. Given this, U.S. Federal implementors of GILS are just one user group that would bring their requirements to the GILS SIG (along with Canadians, states, etc.). 

Generally, no administrative unit has provided government–wide leadership, coordination, and development for the Federal GILS. GILS "leadership" that does exist has occurred at the agency level and resulted because of strong administrative interest and commitment by the individual agency (e.g., EPA and Defense).

The 1996 GILS Conference survey asked respondents for their assessment of the following statement: There is adequate policy guidance to direct the development and operation of GILS. Only 39% of the respondents agreed with this statement. To be fair, however, one should note that less than half of the respondents were familiar with OMB Bulletin 95–01 and other GILS documents and policies (see Table 4-10 below). Overall, study participants generally agreed that the existing GILS policy provided too much latitude to agencies, that OMB, Office of Information and Regulatory Affairs (OMB–OIRA) had "shirked its duty" to enforce GILS provisions, that OMB–OIRA provided conflicting messages to agencies about the relative importance of GILS development, that agencies rarely had an internal policy on GILS development and management, and that with the significant amount of information policy issues that have been on the government’s agenda during the past 2 to 3 years, GILS policy and oversight fell through the cracks.

Spokespersons for various agencies—small and large—believe that after OMB finished Bulletin 95–01 it simply "dropped the ball" in terms of administrative leadership and policy oversight. Others, however, believed that such administrative leadership and oversight were not the responsibility of OMB. Whatever one’s point of view, the study found that the lack of administrative leadership and coordination of GILS implementation across agencies and the lack of oversight to determine the degree to which agencies were in fact complying with OMB Bulletin 95–01 contributed to the current limited success of the GILS effort. Centralized leadership, coordination, management, and oversight is critical as a counterweight to the decentralized, distributed implementation of GILS as a networked service.

4.3.4. FINDING: Smaller Agencies Feel Special Burden and Frustration

Participants in the study from small agencies felt burdened and isolated, and believed they were not heard regarding GILS. Smaller agencies were especially frustrated with the lack of leadership, direction, and resources during the GILS implementation process. They expressed significant dissatisfaction with OMB. In particular, they felt OMB had not listened to or acknowledged the burden that GILS would impose on their agencies. Individuals at these agencies translated reinventing government as "doing more with less," and, with GILS, it was doing something more with questionable value. They felt disenfranchised from the process of developing GILS, and viewed GILS as something directed primarily at the larger agencies—those that had the resources to implement GILS. While the larger agencies may hold the bulk of government information, GILS, if it is to be a government–wide information service, must have government–wide coverage. From this perspective, smaller agencies have many important information resources to contribute. 

GILS implementors in many small agencies have responsibilities not only for records management but also computer security, FOIA, etc. The requirement to implement GILS in addition to these other responsibilities seemed unreasonable and many were quite angry about having to manage such a range of responsibilities. Thus, a number of these agencies are barely, if at all, carrying out the directives that govern GILS. While they have created some GILS records, and those records are accessible (usually on a brokered basis by GPO), a number of participants indicated no plans to produce additional records or maintain the records they have created. OMB Bulletin 95–01 required that agencies must create locator records, so some records were created, period.

For many of the smaller agencies, inadequate technology infrastructure or technology resources was a constraining force in accomplishing the GILS mandate. But such infrastructure constraints are not necessarily limited to the smaller agencies. At least one of the larger agencies visited by the investigators described the lack of a robust networked infrastructure (e.g., lack of network access at the desktop by those creating GILS records) and its impacts on implementing GILS. For example, the use of distributed data input procedures and software such as that developed by the Defense Technical Information Center (DTIC) was not an option if a modern information technology and network infrastructure did not exist in the agency. 

Many of the smaller agencies did not believe mounting their few GILS records on a local agency Z39.50 server was cost effective. As an example, more than 20 of the smaller agencies contracted with GPO to mount their records in the interest of resource optimization. Yet this expediency resulted in a quality–control constraint; agency staff that lacked desktop network connection could not access the records once they were sent to GPO, and thus could not update records easily. 

Based on discussions with representatives from small agencies, the investigators found that as a group, the small agencies are unlikely to participate in future GILS activities without significant changes in the existing GILS initiative. Their participation will be contingent upon the degree to which they are involved in future GILS planning, the degree to which they better understand GILS initiatives and benefits, and the degree to which they can marshal resources to be compliant with requirements. The latter will require some demonstration of tangible benefits (and the costs incurred for those benefits) of extending their GILS implementations.

4.3.5. FINDING: Agencies’ Cultures and Missions Promote Different Commitment to GILS 

Where an agency has a history of strongly supporting public access to its information resources, GILS tends to be more enthusiastically embraced and perceived as successful than in agencies without such a history. Where top management has endorsed GILS and provided strong support—especially by dedicating staff and capital—GILS has tended to be much more successful, at least in its implementation if not in its use. Shallow administrative support, no agency champion, and convenience–based decision making (e.g., choosing a GILS record data input/creation software because it was virtually free) severely constrained GILS success. As a corollary, when staff asked to be in charge of their agency’s GILS effort or were already committed to the GILS concept, the agency’s GILS efforts were more likely to be a success.

Some agencies already had some type of a locator or finding tool in place. In these agencies (e.g., EPA, Defense, and GPO) the GILS effort appeared to be better understood and coincided with existing agency culture that was predisposed to support public access. A number of other agencies did not have a culture predisposed to support public access. A participant in a focus group with representatives from Federal agencies declared that except for one or two items, her agency’s information resources contained proprietary or private information that would not be made public; she questioned why she should create GILS records identifying those resources.

Champions who were dedicated to the GILS concept, knowledgeable about locators and public access, and had good credibility in the agency were critical factors in successful agency implementations. One or two competent staff working at the day–to–day level, providing continuous injections of enthusiasm, and helping to solve problems can, and did, make the difference between a successful and unsuccessful effort. The study found only a limited number of agencies where the existing culture, administrative support, and the involvement of champions directly supported the GILS effort. 

OMB Bulletin 95–01 delegated primary responsibility for implementing GILS to the departments and agencies, who then had considerable freedom to determine how they would respond. The findings identify three basic types of agency response to the GILS initiative: 

• Thoughtful and Committed: A small number of agencies carefully planned their agency response to the GILS initiative, had a champion, provided staff and other resources to support the effort, and produced a working GILS.
• Good Faith Effort: In these agencies, someone or some unit emerged to motivate production of at least some GILS records and meet "the letter of the law" even without agency–wide support or commitment to the GILS concept or its implementation.
• Minimal Compliance: For a number of agencies, there was little to no acknowledgment of GILS. These agencies followed the letter of the law (in their interpretation), and did so by producing a handful of records—usually mounted by a brokering agency—and then considered their GILS effort completed. 

These three types characterize those agencies providing some GILS product. It should be noted, however, there are some departments and agencies that have yet to engage in any GILS development (e.g., Departments of Education, Justice, Transportation, and Veterans Affairs). 

Given this wide range of responses, generalizations of the findings from an agency perspective are difficult to make. Indeed, it should be kept in mind that there are a number of different agency–based GILS and not one GILS.

4.3.6. FINDING: Intra–Agency Efforts Reflect Different Levels of Enthusiasm for GILS

Staff responsible for implementing GILS quickly came up against the reality that different agency units had different levels of enthusiasm for GILS. Some individuals who were tasked to "handle" the GILS initiative in their department or agency found the job to be very onerous, especially since the task came without additional resources. Others latched onto the task and were extremely enthusiastic about the GILS initiative as a means to improve access to government information, or for realizing other individual or agency–specific benefits (e.g., the individual had a personal commitment to GILS or GILS was viewed as a useful information management tool). 

Cooperation among staff within departments and agencies tended to vary with individual agency units’ perception of the importance of GILS. Some factors unrelated to GILS worked in its favor, such as when an agency suffering from a negative public image seized upon GILS as a way to improve its image by providing access to information about the agency. In most instances, persons assigned responsibility for GILS had little direct authority over others from whom the person had to obtain records information. Participants in this position reported no enthusiasm for the GILS effort and, in some instances, outright anger about "having to do this on top of everything else that I am supposed to do."

Site visit and focus group participants identified one barrier to implementing GILS as the difficulty in obtaining agency–wide staff involvement in gathering information to create GILS records. Agencies that had preexisting information locator resources found this part somewhat easier because they had already established procedures for locator data collection and input. Most agencies believed that responsibility for GILS records input should reside with the "offices of primary interest" (i.e., the office or staff responsible for an particular information resource) but obtaining these offices’ cooperation was a chronic problem. Often personnel in these offices saw GILS records input as just one more work demand. In some cases, these staff resisted GILS because they believed that putting their names and phone numbers into GILS records as contact persons would increase their workload.

The study finds a significant likelihood that (1) some "minimal compliance" agencies will not create many additional records nor update ones originally submitted, and (2) those agencies that are conscientious about their GILS efforts will find it increasingly difficult to obtain updated information. These findings point to a possibility of overall GILS degradation over time.

4.3.7. FINDING: GILS Benefits Compared to Burdens Are Not Clear

The study found a range of views on the benefits versus the burdens of GILS. Many agency personnel see GILS as a pure burden without benefit. Or worse, they see it as an unfunded mandate for which they had no administrative commitment or resources, and which distracted them from other "more important tasks." In site visits and focus groups, emotions often ran high reflecting the anger and frustration felt by some agency implementors. They were on the receiving end of the mandate to implement GILS and concluded that the entire effort was a waste of time and effort, without regard to obtaining additional resources. They were quick to point out that they believed strongly in improved public access to government information. But, in their view, GILS, as currently conceived, "was certainly not the tool to accomplish improved access, nor did it assist in records management efforts." A number of these respondents argued that GILS was "dead on arrival." 

Another group of respondents thought GILS will return little benefit if it remains an isolated system. This view holds that GILS becomes useful only when integrated into other systems such as agency Web sites, other information systems, or other metadata schemes. Many questioned whether existing levels of GILS use and benefits warrant continued support and development. Others were unable to articulate any specific tangible benefits arising from GILS. On the other hand, these same people often tended to believe GILS should not be eliminated, but rather refocused and improved.  

Yet a final group of agency implementors had a much more positive assessment of GILS and listed a range of specific benefits that had accrued to their agency as a result of their GILS implementation efforts. Benefits mentioned include: 

• Improved public access to electronic and other agency information resources
• Improved agency knowledge and coordination of existing information resources and how to access them
• Better understanding of the importance of metadata and the need for metadata records
• Increased visibility and involvement for the IRM, information managers, records managers, etc. in department/agency/bureau information resources management
• Identification of potential resources that may need to be scheduled for records retention and preservation
• Development of GILS as a "platform" or base from which other systems could be linked into a "one stop shopping" approach for locating and accessing government information.

This group provided the investigators with a number of anecdotes and experiences that supported these benefits. 

Participants of several focus groups believed that GILS is, in fact, serving as a catalyst for "good things" that should get done in the area of information access. One benefit people pointed to was the fact that, as a result of GILS, agencies were indeed taking inventories of their information products, which is something they were expected to do but often had not. On the other hand, some voiced the fear that GILS is "robbing resources from other information access efforts that are more worthwhile."

To some degree, GILS burdens and benefits are in the eye of the beholder. There also was a clear correlation between those agencies that had committed staff, resources, and administrative support to also believing they had gained significant benefits from the effort—the opposite correlation also holding true as well. Since no formal cost–benefit study has been done on the GILS effort, and was not completed as part of the current study, the study finds that perceived benefits are likely to be situational and stakeholder group dependent. 

Notwithstanding the varying purposes and goals discussed earlier in this section, GILS was premised on improving public access to government information, agency information management, and records management. Another way to think about GILS is: what is an appropriate and realistic purpose for GILS whereby it provides tangible benefits to agency implementors and provides a value to users who want to discover, identify, and access government information?

4.3.8. RECOMMENDATION: Focus on Public Access to Government Information

Early in the evaluation study, it became apparent that "GILS" meant different things to different people. While there was some consensus that the U.S. Federal GILS initiative was intended to support and enhance access to government information, there was little consensus on exactly how that would be accomplished. GILS policy statements and implementation goals raised high and varied expectations of GILS. Unrealistic expectations of what GILS could accomplish has in part increased the volume of expressed disappointment and frustrations by both agency staff and users. 

That GILS has been many things to many people is no accident. OMB Bulletin 95–01 identifies several purposes and goals for GILS: 

• Assist users in locating government information by developing core locator records for
  – Information dissemination products
  – Automated information systems
  – Privacy Act record systems
• Scheduling and disposition of records through NARA
• Electronic records management
• Improved agency responses to the Freedom of Information Act (FOIA) requests
• Potential reduction of information collection burden on the public.

The question is: can one mechanism such as GILS serve multiple and diverse purposes and goals?

On the basis of policy goals for GILS as well as what the investigators learned in the study, it is possible to identify purposes that stakeholder groups have assigned to or expected of GILS including: 

• Inventorying of selected agency information resources
• Capturing and creating metadata for those resources
• Making the metadata available for public access
• Using the metadata for records management
• Linking metadata records to actual full–text information resources
• Enhancing public knowledge of and access to government information
• Providing full–text access to government information.

Because confusion exists among the agencies and the public as to what GILS is and why it needs to exist, the investigators recommend that the Federal GILS initiative be refocused and part of the process of refocusing should be a redefinition and clarification of the purpose and goals of GILS. In addition to clarifying the purpose, scope, and expected functionality of GILS, the task of refocusing should address a range of questions such as: 

• What demonstrable benefits result from implementing GILS?
• What strategies are appropriate for marketing the GILS "product" to agencies and users?
• What types of training are required to accomplish GILS objectives?
• How can agencies cooperate to develop one–stop shopping by subject?
• On what basis should agencies establish electronic linkages between GILS and full–text information resources and electronic services?
• What is a desirable level of granularity or units of information described by GILS records?

The experience to date with GILS (as a technology implementation as well as an information policy initiative) suggests that loading any one system with too many expectations reduces the likelihood that it can adequately fulfill any of the expectations. 

The investigators recommend that the primary purpose of a refocused GILS initiative should be to assist users in the discovery, identification, and access of government information (in the broader networked environment this is referred to as networked information discovery and retrieval). The investigators heard from many people in the study that an information locator service should assist people in finding out what information is available from the government and then provide a way for them to link to that information directly.

The refocused GILS can be summarized as: 

The investigators view this refocused GILS not as a radical break with the current GILS initiative but rather as an evolutionary refinement to the concept of GILS.

4.3.9. RECOMMENDATION: Focus Scope of Descriptions on Network–Accessible Information Resources

Discovery and identification are logical prior steps to accessing or acquiring government information. Assuming that agency information resources are described by GILS in a manner that they can be discovered, the next challenge is for users to access or acquire the information described. This problem is compounded by the environment in which GILS is implemented. 

GILS is a networked–based service. Since early 1994 when Web browsers became easily and freely available, Internet users have become conditioned to browsing and retrieving the full–text of electronic documents and being linked to online databases and other information services. The importance of this "conditioning" cannot be underestimated when refocusing the GILS effort to assist discovering, identifying, and accessing government information. User input to the evaluation suggested strongly that simply providing a "virtual card catalog" of government information is not acceptable. A networked locator to resources is of far greater utility when the resources described are immediately available for access (e.g., one or two "mouse clicks" away). 

Currently, GILS records describe both electronic and non–electronic resources. It is highly unlikely that non–electronic resources will be retrospectively digitized and made available online unless agencies see a benefit to doing so (e.g., a report that is in high demand, as a way to reduce the manual handling of documents frequently requested, etc.). One question that must be addressed in a refocused GILS effort is: what should be the scope and coverage of GILS?  

One aspect of the coverage of GILS records is the extent to which GILS records will exist for all agency information resources. OMB Bulletin 95–01 directs agencies to create GILS records for three types of resources: 

• Privacy Act Systems
• Automated information systems (AIS)
• "Locators that together cover all of [agency] information dissemination products."

An agreement between NARA, OMB, and GPO dealt with Privacy Act Systems (see Appendix A–4). A review of GILS records shows that agencies are describing AIS, but this study did not attempt to examine whether implementing agencies had created GILS records for all AIS (the purpose of this evaluation was not to address "compliance" in the audit sense of the word). The GILS record content analysis (see Appendix E–2) addresses the difficulty of understanding—from the description provided by GILS records—what "discrete set of information resources organized using information technology" (from definition of AIS in OMB Bulletin 95–01) comprise a particular AIS.  

A more problematic area for producing GILS records is to list the "locators that together cover all of its information dissemination products" where "locator" is defined in OMB Bulletin 95–01 as an "information resource which identifies other information resources, describes the information available in those resources, and provides assistance in how to obtain the information." OMB Bulletin 95–01 uses the definition from OMB A–130 for information dissemination product as "any book, paper, map, machine–readable material, audiovisual production, or other documentary material, regardless of physical form or characteristic, disseminated by an agency to the public."

The review of GILS records done in this study indicates that some agencies are describing individual information dissemination products (e.g., a discrete publication or database), not simply "locators" that contain listings of those products. For some users, an "item level" description or granularity of the GILS records is much more helpful, especially when the item is in digital form and one can link from the GILS record to the actual item directly. More fundamentally, agency practice of creating GILS records that describe individual items reflects little understanding by agencies of what constituted "GILS Core" records, or possibly reflects the ambiguity of that concept. In addition, such practice may have been a response to the lack of agency locators that policy assumed existed and which were to be described by GILS Core records. 

GILS assumed the existence of agency information locators, but, in fact, many agencies did not have a set of locators that cover "all of its information dissemination products." Agencies were then faced with the question: if no agency locators exist that cover all their information dissemination products, how should they proceed with their GILS implementation? Were they first to create the locators before creating GILS records that describe them? Or, could they simply begin using GILS to describe individual information dissemination products, whereby the GILS record itself became the "locator?" The creation of GILS records (i.e., the capture of metadata) at the item level for all the existing information dissemination products, however, would be resource intensive.

OMB 95-01 directed agencies to compile inventories if they did not exist. "As a first step, agencies should inventory their existing holdings and institute adequate information management practices…. By December 31, 1995, compile an inventory of its 1) automated information systems, 2) Privacy Act systems of records, and 3) locators that together cover all of its information dissemination products. Each such automated information system, Privacy Act system of records, and locator of information dissemination products shall be described by a GILS Core locator record." The policy, however, lacked specificity regarding what and how those inventories should be made available. There was clearly a missing step between the compilation of the inventories and the production of a "locator" to the inventoried items.

The issue of coverage is a difficult one for policymakers and implementors in determining appropriate guidance. Can the scope of a refocused GILS realistically cover all government information resources, especially if agencies do not have existing locators to their information dissemination products? Without additional resources, study participants agreed it is unlikely that the vast holdings of agencies will ever be described at an item level by metadata records.

If a refocused GILS initiative centers on networked information discovery and retrieval, the value of describing resources (locators, databases, automated information systems) that are not in digital form or network accessible is questionable. Focusing the coverage on government resources that can be linked to electronically (i.e., either in digital form or electronically accessible) may be a positive response to the expectations of users conditioned by the Web.

The investigators recommend that the scope of the refocused GILS should be on primarily supporting the discovery, identification, and access to online and networked resources, and preferably resources available or cast in terms of the Web. This recommendation responds to the increasing number of American citizens who operate in the networked environment and who are likely to want immediate, networked access to information described in a refocused GILS. Anything less will create frustrations and raise questions as to the utility of the service. This recommended scope should not constrain individual agencies from describing non–digital resources, but at a government–wide policy and implementation level, GILS would be so focused.

The investigators realize that users will be interested in government resources even if they are not available electronically, but recommend this narrowing of scope for the refocused GILS. An accompanying recommendation, however, is that agencies be required to create, when none exists, network–accessible locators that describe non-digital, non-electronic, and non-network accessible agency resources.

The investigators recommend that the following two parameters guide a refocused GILS service: 

• Purpose: Discovery, identification, and access of government information (i.e., not records management, information management, or other functions) through structured metadata records
• Scope: Descriptions of electronic resources that are publicly accessible, so that users can move from the metadata record to the "actual" resource.

 Following from this, the refocused GILS should: 

• Promote record creation describing existing and new publicly–accessible automated information systems (AIS), with the provision that users can link directly to those AIS via the Web (i.e., implement an interface between the Web and publicly accessible AIS through scripting mechanism such as the Common Gateway Interface [CGI], Java, or other alternatives).
• Require agencies to produce network-accessible locators that are described by GILS records.
• Point users to the GPO compilation of Privacy Act Systems.
• Encourage description of discrete information products where appropriate (e.g., high-value publicly accessible documents such as the President’s budget) and which are not covered by network-accessible agency locators.

This latter recommendation is problematic because of the resources it will take to create such records. There are several options, however, that can provide agencies with some flexibility: 

• If there are machine–readable metadata records of agency resources held on internal, non–networked databases and servers, use an automated procedure to convert those records to standardized or compliant GILS records.
• Identify existing electronic locators to agency information resources and describe those in GILS records with a link from the GILS record to the locator.
• Identify frequently requested information dissemination products and describe those in GILS records, and ensure that those products are in digital form for network access and available via linking from the record.
• Identify all other information dissemination products that are in digital form (including resources available via an agency’s web site) and describe those in GILS records, with links between the record and the information product.

A comprehensive list of government information resources is desirable, but if locators for all agency resources do not exist—especially given the current "do more with less" policy environment—a certain realism must be reflected in the refocused GILS policy.  

Finally, and most importantly, agencies should be directed to create a GILS record for each and every new information dissemination product or ensure that such products are covered by agency locators in a timely manner. Determining the appropriate set of GILS record data elements needed for such item level description to support networked information discovery and retrieval is a question that needs to be addressed (see Section 4.4 for additional discussion of metadata). In addition, government-wide and agency-level policymakers need to identify classes or categories of information dissemination products that deserve item–level description in GILS and develop government-wide guidance for agency implementors. Retrospective cataloging of existing resources may never be carried out in a comprehensive manner. Therefore, the investigators recommend a "from this date forward" policy that would require GILS records for new information dissemination products. This approach will, over time, populate GILS databases with records that reflect increasing coverage of agency products and resources. Further, since these resources and products begin life as an electronic file, an ever–increasing number of GILS records will be linked to digital copies of the products.

4.3.10. RECOMMENDATION: Identify Responsibilities and Authority for Policy Leadership, Government–Wide Coordination, and Oversight

For a refocused GILS effort to emerge and flourish as a government–wide initiative, the decentralized, distributed nature of the current approach needs to be balanced by some level of centralized oversight and coordination. Government–wide leadership of the refocused GILS initiative will be necessary. If OMB is unable to provide the leadership, coordination, and oversight, it must designate an appropriate body with such responsibility, and attendant authority and accountability. The goal is to establish formal mechanisms for addressing the refocused GILS initiative outlined in this report. The investigators view the GILS Board and the Chief Information Officers (CIO) Council as appropriate bodies to lead the refocused GILS effort.

The investigators recommend that the GILS Board—as an established body—has an important role regarding overall policy development and leadership for the refocused GILS effort. OMB Bulletin 95–01 provides a mandate for the existence of the Board, and the Board could be charged with responsibilities in addition to its current charge related to annual evaluation and reports on the progress of GILS. Current language in OMB Bulletin 95–01, "The Board may ask the heads of other agencies to designate representatives to serve on the Board or on task forces established by the Board," enables the Board to create task forces that could assist in the refocused GILS initiative. The GILS Board should have the responsibility, authority, and accountability for formulating the policy direction for next phase of GILS development. OMB may be required to issue policy, but OMB should draw upon the Board’s recommendation for the content of that policy. The investigators further recommend the following: 

• GILS Board membership include representatives from the Small Agency Council and the CIO Council
• GILS Board establish a GILS task force consisting of representatives from Federal agencies as well as public users to refine and articulate the scope, purpose and goals for a refocused GILS.

The recently established CIO Council also has an important role to play in the coordination of GILS activities across the government. As an interagency body, the CIO Council could create one or more technical committees and working groups for discussions related to technical issues and concerns regarding GILS development. The CIO Council could, for example, establish a GILS Committee that would be responsible for government–wide coordination of the refocused GILS effort. Its working groups could address specific issues such as metadata record elements, marketing, ongoing evaluation, etc. The focus of CIO Council activities should be on technical and implementation concerns (as opposed to government–wide policy that the GILS Board would provide). 

A CIO Council GILS Committee would provide a forum for Federal implementors of GILS to discuss and agree upon their requirements for the GILS Profile, which can then be taken to the GILS SIG for action. The CIO Council would be an appropriate unit for agencies to report their GILS implementation progress, and with such information the CIO Council could maintain the registry of known GILS implementation. Given its interagency makeup, the CIO Council would be an ideal forum for the identification and dissemination of GILS "best practices" related to all aspects of GILS implementation.

Identifying a formal body as a home for technical and operational coordination responsibility, authority, and accountability should also provide increased credibility for the refocused GILS effort.

4.3.11. RECOMMENDATION: Implement a Refocused GILS Initiative 

With the passing of the December 1996 OMB 95–01 deadline for GILS implementation and the conclusion of this evaluation study, GILS may be said to have completed its first phase. Pursuing a refocused GILS can be considered a second phase for the initiative. The question that faces policymakers—at both agency and government–wide levels—is how to take the next steps in evolving and implementing a refocused GILS that has the clear purpose of supporting the discovery, identification, and access of government information.

The Federal GILS initiative was driven in part by the Clinton Administration’s efforts at reinventing government and the development of a National Information Infrastructure (NII). GILS, and its use of information technology, had the potential for supporting the accomplishment of agency mission by providing a mechanism for better information management (e.g., inventorying agency resources). Further, GILS was to support enhanced public discovery, identification, and access to government information. In Spring 1994 as the final GILS Profile specifications were being completed and the Information Infrastructure Task Force (1994) report on GILS was released, there were pressures to implement GILS as soon as possible. OMB Bulletin 95–01 directed agencies to begin developing their implementations in 1995.

In retrospect, the implementation would have profited from a GILS pilot program. Many of the issues encountered through this study could have been identified earlier, and with less onerous consequences, had a pilot program experimented with the various GILS requirements. As a case in point, the Canadian government established a GILS pilot project in 1996 and recently completed an evaluation of the pilot (see Appendix I for copy of the report on the Canadian pilot).  

A U.S. Federal GILS pilot program would likely have identified the following issues:  

• Record Creation: How much effort would it take to compile the information needed to create records? What barriers might be encountered? What data input mechanisms could be devised to ease the burden of data collection and data input?
• Z39.50 Software: What was available and what would be the demands for implementing Z39.50? Were the GILS Profile specifications realistic and implementable?
• Record Content: Had appropriate data elements been defined? Were the data elements and the content of those elements clear and usable?
• Records Management: How would GILS records support records management? To what degree did GILS metadata elements satisfy records managers information requirements?
• Usability of GILS: What was the best way to present GILS data to users? To what extent did it satisfy users?

A pilot program could have not only identified problems and issues, but could have served as a testbed to resolve them.  

Many agencies are not only skeptical about GILS after the past two years; some are frustrated and angry from trying to do GILS with no new resources and little realization of tangible benefits from their activities. Exhortations from policymakers will not be enough to overcome resistance to doing anything more with GILS as it currently exists (either at a management or staff level). A refocused GILS must demonstrate that it can solve networked information discovery and retrieval challenges and provide real benefits to agencies and their users. 

Assuming that redefinition of GILS occurs along the lines recommended by the investigators, that the purposes and objectives for a refocused GILS are identified and articulated, and that organizational units are delegated with the responsibility, authority, and accountability for coordinating a refocused GILS initiative, the next step should be the implementation of a phase two GILS pilot program. A GILS pilot program could be used to implement the recommendations offered in this report.

For the refocused GILS, a period of time (e.g., 9–12 months) should be allotted to a pilot program. During this period, a small selected group of agencies could participate in pilot implementations of GILS that address some of the specific issues and problems identified in this evaluation. Agencies should be chosen that reflect differing missions, sizes, information holdings, levels of information management sophistication, etc. Reasons for conducting a pilot program include: 

• Demonstrate that GILS improves public access to government information
• Demonstrate the tangible benefits to agencies
• Demonstrate the costs incurred by agencies
• Demonstrate an approach that improves user satisfaction in discovering and accessing government information
• Demonstrate the appropriate staffing required for successful implementation of GILS
• Demonstrate the technology solutions for record creation, information retrieval, record presentation, etc.
• Demonstrate how GILS can be integrated into other agency information handling processes
• Document how GILS can be implemented and share lessons learned, best practices, etc.
• Showcase the potential of GILS in improving information discovery and access both for agencies and users.

This pilot program assumes that policy leaders, project–management and technical experts, and input from various user communities have refocused goals for GILS, have identified specific and measurable objectives for GILS, and have provided guidelines for implementors to follow. The success of the GILS pilot program can then be gauged against (1) conformance to specified goals, objectives, and requirements and (2) user feedback as to the degree to which GILS "enables" information discovery, identification, and access. 

The refocused GILS policy should communicate clearly the goals or future conditions so that agencies and users can envision the purpose, scope, and utility of GILS. Agencies should support these goals by developing specific, realistic, and time–phased objectives with assigned responsibilities, accountabilities, and authorities (see Appendix G for characteristics of successful objectives). This approach can encourage measurable performance, and the goals and objectives—and procedures for measuring and assessing performance—would provide a basis for agencies to comply with The Government Performance and Results Act of 1993 (GPRA) requirements related to GILS activities.

4.3.12. RECOMMENDATION: Require Agency Reporting on GILS Progress and Reward Agencies That Achieve Stated Objectives

Existing U.S. Federal GILS policy lacks a requirement for agencies to report on the progress of their GILS implementations. The GILS Board is charged with conducting yearly assessments on the progress of GILS and documenting its findings in an annual report. Without any agency reporting requirements, how the GILS Board would gather information for its annual assessment is unclear. In general, neither sticks nor carrots are identified to "encourage" or "reward" agencies for their progress (or lack thereof).

The evaluation study also identified a lack of incentives and benefits to agencies that participated GILS. The incidence of disincentives may be higher than that of incentives. Agencies, especially smaller agencies, view GILS as providing little return on investment (i.e., much burden, few benefits). In some cases, especially where records managers are charged with GILS record creation, there are disincentives for creating GILS records. Not only did the records managers have to create records, but, if those records described unscheduled items, the items then had to be scheduled as well.  

Agencies that demonstrated creativity and innovation in their GILS initiatives received no public recognition. Nor were there any rewards for or acknowledgment of agencies that met the deadlines of OMB Bulletin 95–01.

If the GILS Board and the CIO Council assume specific responsibilities (outlined above) for the refocused GILS initiative, they will need adequate information from agency implementors to manage and coordinate the initiative successfully. The investigators recommend that agencies be required to report at least annually on the status of their GILS implementation. Such reports should include the following: 

• Network address of the agency’s GILS records
• Implementation used for providing network access to the records including type of database and search engine used
• Number of GILS records created in reporting period
• Total number of GILS records created
• Number of GILS site accesses, searches, and record retrievals per agency log analysis
• An estimate of the percent of agency information resources described by GILS records per scope of the refocused GILS initiative
• Identification of any evaluation/assessment conducted by the agency of its GILS implementation
• Identification of mechanisms employed to gain user input into development of the agency GILS.

The investigators also recommend that policymakers (e.g., OMB, the GILS Board, and the CIO Council) explore the creation of incentives for agency compliance and develop a program of rewards or public recognition for those agencies that demonstrate creativity in accomplishing and/or exceeding the clearly stated objectives of the refocused GILS initiative.

The proposed GILS pilot program needs to be authorized or supported with a source of money. The Information Technology Management Reform Act of 1996 (ITMRA) established an Information Technology Fund (including a proposed funding source) consisting of an Innovation Loan Account Fund (to be funded out of existing agency IT budgets) and a Common Use Account Fund (to support multi–agency acquisitions). Some of these funds might be tapped, on a reimbursable basis as required with the Fund, to support innovative development of GILS efforts. This could be done on a proposal basis, whereby agencies could submit short proposals for innovative projects that address significant problems with GILS and solutions for which can have government-wide application, and could be awarded funds to carry out innovative projects. Challenges and problems identified in this report that would be suitable for such pilot program activities include the efficient capture of metadata in conjunction with electronic document management systems, usability studies for options in presentation of GILS records, identification of high-value metadata elements that support discovery and retrieval of government information resources. Government–wide solutions for improving public access and agency information management resulting from the use of the IT Fund appears clearly justifiable. 

4.3.13. RECOMMENDATION: Ensure Ongoing, User–Based Evaluation for Continuous Improvement

The investigators spoke with many agency staff who are committed to GILS and who are making good faith efforts in implementing it even if they do not have adequate resources allocated to their work. Yet, with notable exceptions where agencies (e.g., EPA) actively solicited potential users’ input, users external to the agencies have not been involved in GILS design and implementation. GILS has been dominated by agency, resource, and system–centered considerations. The online user assessments of GILS highlighted that, overall, it is not a user–centered system. 

One key finding from this study is that a number of evaluation and self–assessment tools can be used by Federal agencies to assess the overall success of their GILS efforts. A by–product of the study is the development and testing of techniques and instruments that are reprinted in the appendices. Techniques such as server log analysis, user scripted assessment of a GILS site, record content analysis, as well as more familiar focus groups, surveys, and interviews provide important indications of the overall health of GILS. 

While the investigators heard agency representatives lament the lack of time and resources for assessment, especially user–based assessment, an ongoing evaluation of GILS is essential if it is to improve networked information discovery and retrieval of government information. The study finds that mechanisms will be needed to conduct both government–wide and agency–level assessments in the next phase of GILS. A number of those mechanisms and data collection instruments should be adapted from this study.

The investigators recommend that agencies establish ways of routinely seeking user input on the design and implementation—as well as the criteria for determining success—of the refocused GILS (and other public access activities). The CIO Council, as a newly constituted coordinating and policy body for GILS, can lead this aspect of the GILS initiative by identifying procedures and practices to solicit and capture a wide range of user perspectives. 

User involvement should begin during the phase of clarifying the purpose, goal, and objectives for the refocused GILS (e.g., having public representatives on the GILS Board and its GILS Task Force). The proposed GILS pilot program must build in user involvement (e.g., early input into the design and specification), and user–based evaluation should be ongoing through the pilot program activities and implementations. For example, public and government documents librarians could serve as important sets of users in assessing and evaluating GILS clients that could be developed and tested as part of the pilot program.

4.4. OPPORTUNITY: IMPROVE GILS EFFICACY IN NETWORKED INFORMATION DISCOVERY AND RETRIEVAL (NIDR) 

GILS is a networked–based service that can assist users in discovering and accessing government information. In the early 1990s, the term networked information discovery and retrieval (NIDR) emerged to describe the complex activities and problems—technical, organizational, and users—involved in search and retrieval in the Internet environment. GILS serves as an example of a NIDR system. Findings from the study indicate that GILS utility as a mechanism for users to discover, locate, select, and access government information is limited. Table 4–5 summarizes a series of findings and recommendations related to this aspect of GILS.

Section 4.3. discussed issues that require policy, administrative, and organizational attention. The issues related to NIDR are, however, of a different order. In many respects, NIDR is a research area in which computer and information scientists are framing and addressing difficult challenges related to distributed search and retrieval, the character and utility of metadata, interface design, and others (see Lynch, et al., 1995). The many digital library projects underway provide environments where many of the issues and challenges are becoming more clearly defined. Scalable solutions to some of the problems have yet to become operational. The findings reported here from the implementation experience with U.S. Federal GILS will contribute to the understanding of the some of the NIDR problems. Given this situation, some of the recommendations should be viewed as the investigators’ indication of potential next steps. Further, the findings and recommendations point to additional research that needs to be carried out, and Chapter 5 identify the major research topics related to GILS and NIDR. 

* CA=content analysis of GILS records; FG=focus group sessions; KP=interviews with key participants;
LA=log analyses of Web servers; SU=survey conducted at the 1996 GILS Conference;
SV=site visits to selected agencies; US=scripted online user assessments of GILS

4.4.1. FINDING: Web Technology Has Raised Questions about the Role of GILS 

Web technology has developed rapidly during recent years. The degree to which Federal agencies embraced the Web as a means for providing access to government information resources, disseminating information products, and providing a range of information services during that time could not have been foreseen at the time of GILS development and the writing of OMB Bulletin 95–01. DiCaterno and Pardo (1996) provide an analysis of the ability of Web technology to provide a universal interface to government information. At issue is how GILS can best take advantage of the Web technology while providing an essential service not currently offered by Web technology—namely, a search and discovery service.

Currently, all known U.S. Federal GILS implementations are accessible via a Web interface. Yet the study found a certain amount of confusion, if not contention, between the roles of GILS and uses of agency Web site. Agency officials also had varying opinions as to what GILS records are supposed to describe versus what Web pages should include and describe. To a large degree, agencies are still experimenting with how best to integrate these two approaches for information access and dissemination. Interestingly, 79% of respondents to the GILS Conference survey agreed with the statement: Every agency Web homepage should have a link to the agency’s GILS. Only 16% of respondents agreed that: The World Wide Web reduces the need for GILS.

In part, the confusion stems from a lack of understanding of two key elements of GILS:  

• Structured metadata (i.e., GILS records) that describe agency information resources
• Z39.50, the information retrieval protocol.

The GILS records, as structured metadata, provide a standard way to describe agency information resources in a semantically consistent way (see Appendix F). More importantly, Z39.50 provides for "semantic interoperability" in that it enables client software to precisely express a search query to multiple search engines and supports the retrieval of complex records (Lynch, 1997; see also Lynch, 1992). Z39.50 servers and clients that support the GILS Profile share an understanding for search and retrieval, and according to Lynch (1997), "Z39.50 provides maximum leverage [for search and retrieval] where there is a shared understanding between client and server of rich and specific information semantics." Thus, the GILS records and Z39.50 provide an important basis for searching across multiple databases and servers.

A number of study participants suggested that Web search engines provide sufficient searching power. Yet, Web search engines are limited, based as they are on a simple model of retrieving HTML documents from multiple sites and building large centralized indexes based on the occurrences of words in the HTML documents. The search engines are very powerful and robust for full-text searching of HTML documents. However, users cannot search, for example, for a copy of the document with the title of "Circular A-130, Management of Federal Information Resources" and published by the Office of Management and Budget, and be assured that the results that come back from the search engine do not contain commentary on A-130, email messages about A-130, and bibliographic citations to A-130. Another drawback to the Web search engines is that they do not "see" all electronic resources that may be network accessible. For background on Web search engines and their capabilities, see Koster (1997) and Liu (1996).

A more critical area where Web search engines provide only limited service in discovering and identifying resources is electronic databases. Since Federal agencies’ databases are important and valuable resources, GILS provides an important function by enabling standardized descriptions of these resources that are only slightly "visible" to the Web search engines. In many cases, what is visible to the Web search engine is not the database itself but usually an HTML page (possibly forms-enabled) that the user interacts with to use the database.

The following example helps illustrate this point. The Security and Exchange Commission (SEC) makes available its Electronic Data Gathering, Analysis, and Retrieval (EGAR) database through a Web site www.sec.gov/edgarhp.htm. A web search engine could index the EDGAR web site but would limit its indexing to words appearing on the site. The SEC has created a GILS record for the EDGAR database (available on GPO’s GILS service), and the GILS record provides structured information including time period of content of the database, its purpose, how to request information, and other useful information. Using a well-known Web search engine (AltaVista) and the search terms EDGAR and "Securities and Exchange Commission," the search engine found "8,000 documents matching the query." However, none of the first 20 "hits" pointed to the EDGAR homepage. Further, even if the homepage would have showed up in the result set, the listing would not have provided the type and scope of information contained in a GILS record. One particular hit pointed to "EDGAR Online" <www.edgar-online.com/>, a commercial service provider of SEC information; in fine print at the bottom of the page for EDGAR Online, there is the following disclaimer: "EDGAR ONLINE is a product of Cybernet Data Systems, Inc. and is neither approved by, nor affiliated with the SEC." Compare the lack of results when searching a Web search engine with the results when submitting the same search query on GPO’s GILS. In the latter case, the GILS record for the official SEC EDGAR database was near the top of the result set list, plus it provided additional authoritative information from the originating agency.

Hammer and Favaro (1996) identify a potential synergy between the Web and Z39.50 by acknowledging their separate strengths. The Web provides hyperlinks between systems and documents types, as well as a relatively easy mechanism for publishing and an interface to existing databases. The strength of Z39.50 is structured searching and document discovery, precisely the goal of GILS.

The challenge for the future is to refocus GILS efforts to emphasize the discovery of government information provided through the GILS records and the structured searching provided by Z39.50. Once users discover the information resources by searching GILS, it is necessary to provide seamless links from GILS metadata records to individual documents (in full–text) or other electronic resources accessible that may be available on agency Web sites. 

One critical result of the Web’s influence on GILS is the increase in users’ expectations of being able to access the full–text of documents and other electronic resources. Not satisfied simply with viewing "pointer" or descriptive records, users want access to the "actual" information resource. Users participating in the online assessment of GILS expressed "disappointment," "surprise," and "confusion" to the absence of full–text (i.e., the actual documents) when interacting with GILS implementations.

The investigators maintain that producing quality metadata is an important contribution of GILS. Metadata, however, may not be sufficient to satisfy users’ information needs. The GILS record structure provides data elements to enable linkages to the information resource described in GILS. Many agencies are making an effort to use this feature to take the user from the record to the resource described (e.g., The Budget of the United States Government (OMB) and GPO’s Monthly Catalog). In the record content analysis component of the evaluation, approximately 25% of GILS records examined featured at least one instance of hypertext linkage. While linkages occurred most frequently in the Available Linkage data element (approximately 15%) and thus enabled linkages to the resource described, instances of hotlinks were also present in fields such as the Distributor Network Address and Abstract data elements (as well as some locally defined elements). While the maintenance burden of hypertext is recognized, users’ expectations for it will continue to accelerate for the foreseeable future.

Some agencies have integrated GILS into their Web site by providing a link to the agency’s GILS on the agency homepage. At EPA, GILS records assist Web visitors navigate the Web site to find information, even though the records are not labeled as being part of "EPA’s GILS." Most study participants thought that GILS should have a more discernible relationship with an agency’s web site. They wanted to integrate GILS better with agency home pages and with other information systems and information product catalogs. How specifically this should be done is a matter of some considerable debate.

The study finds that agency Web implementations have not replaced the need for GILS or a GILS–like service. Available Web search engines that index Federal Web sites and search engines on individual agency Web sites do not provide access to government–wide finding tools, catalogs, or indexes across agencies and across related topics. Nor does the Web supplant the power that Z39.50 offers for interoperable search and retrieval. Finally, many resources of the government, such as electronic databases, are not "visible" to Web search engines. Even if a search engine indexes an interactive form page for a database, the value-added, structured information captured in a GILS record is not available for the user.

4.4.2. FINDING: GILS is an Agency–Centric, Rather than Government–Wide, Service

The study finds that the Federal GILS initiative has not resulted in a Government–wide Information Locator Service; rather, it has resulted in separate Agency Information Locator Services (AILS). Agency GILS that have been implemented are confined almost exclusively to resources within a particular agency. Until recently, users could conduct cross–agency search and retrieval capability when searching the GPO and FedWorld GILS sites, but the searches were limited to agencies which had contracted with GPO or FedWorld to mount their GILS records. In April 1997, GPO announced it had implemented an application where a user could submit a query across agencies’ GILS records whether or not GPO had mounted those databases of GILS records on its site.

In the online user assessment sessions conducted as part of the evaluation, users were nearly unanimous in their agreement that all agencies’ GILS should be searchable together, from one Web site as well as all government documents on the Internet should be hotlinked from one electronic card catalog (see summary of user sessions in Appendix E–3, specifically Question S32a). 

OMB Bulletin 95–01 recognized the need for the U.S. Federal GILS to be built from agency components. The vision of GILS reflected a decentralized collection of agency information locators. It specified two approaches, however, for creating a logically centralized albeit physically decentralized government–wide locator. First, GILS servers were to implement the Z39.50 protocol, which would allow a single Z39.50 client to interoperate with all GILS servers and provide an impression of seamless searching and navigation among those distributed servers (see Lynch . While fully compliant Z39.50 GILS servers are being implemented, the incidence of desktop GILS clients has been relatively low.  

Most users connect to GILS servers through a Web interface (e.g., a Web/Z39.50 gateway), which limits users to searching GILS records that are offered through the gateway. Without Z39.50 GILS clients that provide users the capability to search across one or more GILS servers, cross–agency searching has yet to be achieved. (Nor does the user have the control over the display and views of GILS records that Z39.50 affords.) "Integrated" services, such as those offered by GPO and FedWorld, are important, however, as they provide users with some modicum of government–wide searching.

Second, GILS record creators and maintainers could include cross references to other resources that might be of interest to a user, whether from the originating agency or resources at other agencies. The identification of these related resources would allow a user to link to or search for these resources that were themselves described by other GILS records. In the record content analysis carried out as part of the evaluation (see Appendix E–2 for the analysis), the occurrence of cross references in GILS records was negligible. Given the difficulty for many GILS record creators to gather agency information to create GILS records, it was probably unrealistic to assume that agency staff would go the additional step in referencing related resources, especially those of other agencies. However, this capability, along with realistic procedures for maintenance of cross–references, should be a goal of the refocused GILS initiative. One step in this direction would be the development of criteria to help identify suitable resources that could be cross–referenced. 

Other models of cross–agency searching or access to agency resources are available. First is the brokered–GILS model with a single agency (e.g., GPO and FedWorld) providing a single point of access for searching against more than one agency’s GILS record simultaneously. As noted earlier, GPO has mounted 27 agencies’ records and allows a user to search across all records with one query. FedWorld provides searching of three agencies’ records at one time. While this model moves in the direction of a government–wide service, it is based on a model where agency databases of GILS records are hosted at a centralized site. Searching is limited to the agency records available at that centralized site. 

Another model is represented by the Advanced Search Facility (ASF) effort. This interagency initiative has been developed under the leadership of the Department of Commerce and is informed by recent Web models of search and retrieval. Web search engines provide for the centralized and automatic indexing of resources accessible by Web robots (Finin, 1997). The robots "crawl" the Web, pull documents to the indexer, index the documents, and then offer a search service against the centralized indexes. Examples of such Web search engines include AltaVista, Yahoo, and Excite. The user connects to a search engine, submits a search which is then executed on the indexes, and is given a list of resources that "match" the search criteria. The user then links to the resources of interest. This model is also based on centralization of resources—in this case, the centralized indexes built by the web robots and search engines.

The objectives of the ASF initiative address the problems of searching for information across many agencies. The ASF expands the indexing of networked resources beyond the Web resources currently covered by the Web search engines, and distributes indexing responsibilities to the distributed servers. Discussions with staff working on the ASF indicated to the investigators that the ASF appears to have potential to help solve the GILS problem of cross–agency searching. At the time this report is being written, a Request for Proposal (RFP) for the ASF has not been issued. Thus, specifics of the project and how it might assist GILS may be premature. 

A third model is represented by interagency initiatives that use the Web to provide access to topical or subject–oriented collections of government information and services. Examples include: 

• Business Advisor, the one–stop electronic link to government for business <http://www.business.gov/>
• Federal Statistics Initiative <http://www.fedstats.gov>
• National Environmental Data Index (NEDI) <http://www.nedi.gov>.

Except for NEDI, the use of GILS to support such interagency efforts is not clear. These models do not provide a government–wide locator service as envisioned for GILS. And, as noted in the previous section, the Web does not provide a systematic solution to the information discovery problem. Topically–based resources must first be "discovered" by users before they can be used (i.e., describing these resources in GILS records would be appropriate). Further, such topically–based resources "pre–select" resources for users (which may be entirely appropriate). An analogy would be a special collection within a larger library. GILS provides a means to discover what is in the larger library. 

Distributed information search and retrieval in the networked environment is a difficult problem—both technically and organizationally. Like other aspects of the GILS initiative, cross–agency searching using Z39.50 appeared reasonable; to date, effective government–wide searching for government information has not been achieved, either through GILS or any other mechanism. The agency components of GILS, however, are a vital foundation for a government–wide locator. Without mechanisms such as the centralized point of access via a centralized service that actually mounts all agencies’ GILS records (e.g., GPO and FedWorld) or centralized index (e.g., a Web–like search engine or ASF) or the deployment of compliant Z39.50 GILS servers and clients, GILS will remain a distributed, unconnected set of AILS.

4.4.3. FINDING: GILS Metadata Are Difficult to Capture

At the core of GILS is the standardized record with defined data elements that can be used to describe agency information resources. Although a number of study participants indicated a limited understanding of the concept of "metadata," others view the standardized record as offering GILS’ most valuable contribution for enhancing discovery and access to government information. The investigators remain convinced that a standardized set of metadata elements is one of the clear strengths of the GILS initiative.

The term "metadata" has evolved into common usage in the networked environment to describe "data about data." That definition is accurate, but its helpfulness is limited. The investigators determined that while people used the term "metadata" often in regard to GILS, common understanding or agreement on what specifically was meant by "metadata" when discussing GILS was not readily apparent. Many study participants were not clear about or had an appreciation for the role of metadata in networked information discovery and retrieval. A number of study participants suggested that Web search engines replaced the need for GILS and GILS records. Such a view is incorrect and suggests a need for better training about the use and benefits of metadata in NIDR (see Appendix F for brief discussion on the role of GILS metadata in NIDR).

Study participants were concerned with the cost–effective capture of data needed to create GILS records. Although some agencies, such as DoD, have implemented an online process for creating GILS records, a prior step of gathering the information to put into the record is necessary. Agency staff involved with record creation pointed out the difficulty in gathering that information. While the Office of Primary Interest (i.e., the staff or office responsible for a particular agency resource) may have the pertinent information about a resource to be described in a record, agency staff indicated that cooperation from those offices was not always enthusiastic. The effort in gathering GILS record information should not be underestimated. As currently done in most agencies, GILS record creation is time consuming and requires major effort.

One wonders why more agencies did not make use of freely available record creation and data input aids such as DTIC’s electronic input form. Part of the answer may lie in uncertain or unfamiliar lines of communication among agencies (e.g., civilian and military), and part of the answer may be that the aids were not known to be available at the time when agencies had to make decisions regarding input procedures. The technology infrastructure or local expertise within an agency were also constraining forces in using such software applications. Better cross–agency coordination could have led, however, to substantial government–wide efficiencies in records creation.

Some agencies preferred to centralize data entry. These agencies believed that they achieved greater record quality assurance in this fashion. On the other hand, centralization sometimes complicated the process of updating and maintaining records, since the people closest to the information resources would need to go through the central point for record updates. The practice of centralization becomes problematic when the described resource or its descriptive metadata change frequently; however, implementation of an updating schedule to allow periodic incorporation of changes may improve efficiency.

Overall, the study found that record creation at the time of the creation of an information resource is rarely done, that "best practices" for GILS records creation should be identified and publicized, and, overall, that the GILS record creation process should be simplified.

The study also found that agencies lack staff, funds, and other resources to retrospectively "catalog" their information. A number of study participants suggested that retrospective cataloging to create GILS records for "comprehensive coverage" an agency’s information resources is unlikely. In part, this is due to the costs involved. Although this study did not attempt to collect information about costs in creating GILS records, it is reasonable to estimate such costs based on the costs involved in cataloging materials in libraries. Recent data from the Library of Congress suggest that cataloging a single monograph can range from $25 to over $100, depending on the depth and extent of cataloging and classification.

Study participants offered suggestions for some form of automatic capture of metadata at the time of creation of the information resource. New tools and procedures such as electronic document management systems (EDMS) provide one possible scenario for capturing metadata, at least of document-like objects.

4.4.4. FINDING: Limited Updating and Maintenance of GILS Records 

One important question concerning the quality and timeliness of GILS records relates to the maintenance activities of updating, verifying, and ensuring record accuracy and currency. Without ongoing maintenance, the quality (e.g., accuracy and currency) of GILS records will degrade. Inaccurate, out–of–date records will not improve access. 

Updating and maintenance burdens will vary based on a number of factors. One factor is whether the records are mounted locally on an agency server or mounted on a host agency server. For example, GPO staff mount the records "as submitted" and rely on each agency to notify them of record changes, updates, deletions, etc. DTIC, on the other hand, has a procedure in place that "strongly encourages" agency maintenance of existing records on a regular basis. 

Updating and maintenance may also be a function of the agency network infrastructure: do the GILS record creators have network access to check and correct the information contained in records they create? Finally, ongoing maintenance of GILS records will depend on agency staff perceptions of GILS’ value. In a number of agencies, the lack of tangible benefits to date provides sufficient reason for them to say, "we created records, but we aren’t going to put any more resources into the effort or maintain the records we created."

Some agency records officers responsible for GILS activities reported that once they created the original record they believed their job completed. Other GILS records creators are dependent on others in the agency for updates and find little cooperation for obtaining the updated information. Still others told the investigators that they have neither plans nor intent to update the records created to date. Overall, the study found a lack of procedures and a general lack of interest at many agencies in updating and maintaining GILS records. 

4.4.5. FINDING: No Clear Agreement on Adequacy of GILS Record Data Elements

Study participants noted that the GILS records may be the lasting contribution of the U.S. Federal GILS initiative. By this, they meant that across government agencies, staff used a standardized set of data elements to describe agency information resources. This effort is analogous to the evolution of a standard bibliographic entry in library catalogs. Standardized, structured metadata records such as GILS can have a longevity beyond the life cycle of the access systems on which they were initially implemented. The GILS records can be viewed as platform and application independent, and the investment made in creating GILS records can have long term payback. The structured records can be converted and migrated to other systems and other applications.

There was not agreement, however, on the adequacy of the GILS record, and in some cases, study participants questioned the usefulness of the many data elements defined for use in GILS records. Some record creators thought that GILS records required too much information, and they concluded that the cost of collecting the information outweighed the benefits of including that information in the records.

In some cases, the GILS records do not contain adequate data elements to support functions expected of GILS. Specifically, records managers participating in focus groups suggested that the information they need for record scheduling cannot easily be put into existing GILS data elements. They also thought that GILS was not an appropriate records management tool and were not interested in trying to "enhance" the data elements to the extent necessary for GILS records to be useful in records management.

The issue of appropriate metadata elements that support information discovery and retrieval warrants additional research, and such research is part of the larger research issues related to networked information discovery and retrieval (see Chapter 5).

Finally, a number of study participants were unclear as to who had or should have authority for the data elements for GILS records. Currently, the GILS SIG is responsible for maintaining the GILS Profile, in which the data elements are defined, and as such, the GILS SIG is open to any implementors of the GILS Profile and is not limited to Federal agency GILS implementors. No Federal forum for U.S. Federal GILS implementors exists where agency GILS implementors can discuss and review their needs regarding GILS data elements.

4.4.6. FINDING: Different Types of Resources Represented in GILS Records 

Considerable discussion occurred in a number of site visits and focus group sessions regarding the types of resources and the granularity and/or aggregation of agency information resources represented by GILS records. The GILS record content analysis and the scripted online user assessment also identified issues regarding the unit of information described by a single GILS record (see Appendix E–2 for discussion of granularity and aggregation). Should agencies create GILS records for individual maps, publications, and documents? For individual databases, which may aggregate many discrete resources?  

As stated earlier, OMB Bulletin 95–01 identified three types of information resources GILS records should describe:  

Yet, based on discussions with agency staff and the GILS record content analysis, there is a great deal of uncertainty as to the appropriate level of granularity, or extent of aggregation, for GILS records. Some participants, for example, told the investigators that they plan to produce GILS records for individual documents and resources because they were "key items" in their agency. Other agencies are creating records for collections of hitherto individual documents (e.g., aggregating "press releases" to be described by a single GILS record). Without government–wide guidance, agencies now have wide latitude for determining what resources and what level of granularity their GILS records describe. The result—from a government–wide perspective—for users is uneven levels of description and inconsistent representation of resources. This also results in users being uncertain as to the scope and coverage of a particular agency GILS based on the number of records that have been created. 

The number of records created by an agency may or may not be an indicator of the degree of resource aggregation. For example, EPA has created approximately 240 GILS records; the Social Security Administration has created more than 1200. Is Social Security Administration creating too many records (they have many GILS records that describe one form) or is EPA creating too few? Absolute numbers of records are less helpful than understanding two important issues related to the GILS records: 

• The granularity/aggregation of described resources (i.e., the extent to which individual information products are "collected" for description by a single GILS record)
• The overall coverage of information resources (i.e., the extent to which an agency’s GILS records describe all agency AIS, Privacy Act systems, and locators per OMB Bulletin 95–01, or describe individual information dissemination product).

EPA had preexisting locators to much of its information resources, and by creating GILS records that describe those locators, EPA may be able to provide good coverage of its information resources through a relatively small number of records. If, on the other hand, an agency does not have existing locators and it chooses to describe individual documents and publications in individual GILS records, then a larger number of records may be necessary to gain adequate coverage.  

Another consideration is whether agencies use the same definition or criteria to determine what, specifically, constitutes an agency "resource" or "product" that should be described by a GILS record. The evaluation study’s record content analysis developed criteria and procedures for assessing GILS records (see Appendix E–2), and, identified various types of resources described in a sample of approximately 80 records from all GILS sources. Table 4–6 summarizes the findings from this analysis.

Granularity and aggregation are not simple concepts. The record content analysis used the following operational definitions to deal with the issues of record aggregation: 

• Record aggregates object: The GILS record, by virtue of its creation, collects discrete information resources that the record content indicates would not have otherwise been collected or aggregated (e.g., "General Files," "Press Releases," and "Forms").
• Aggregated object represented: The GILS record represents an a priori or purposeful collection of information resources (e.g., "Woodpecker Database" or an agency Web site). The GILS record represents an object that collects, or comprises, two or more discrete information objects, and that object represents a collection of standalone information files or products packaged together on the basis of a common theme or subject for functional convenience (e.g., a CD–ROM of regulations, a system of Privacy Act records, or a voice recording of employment opportunities).
• Discrete object represented: The GILS record describes a standalone document–level entity that does not meet the criteria for "object aggregates metadata" below (e.g., an Annual Report or a videotape).
• Object aggregates metadata: The GILS record describes a pre–existing metadata collection, or "locator," as an information resource (e.g., directory, catalog, or index). 

Based on these operational definitions, Table 4–7 provides a summary of aggregation characteristics of information resources found in the sample analyzed. An important finding from this study is that agencies use GILS to describe collections of information resources not previously described. For example, a GILS record describing an agency’s "press releases" (or some subsets of press releases) provides users with the opportunity to discover the existence of these resources. 

Study participants could not define an optimal or appropriate level of granularity. Many concurred that existing GILS records describe a wide range of resources, of varying levels of aggregation, and that this phenomenon could affect GILS usability. Users indicated difficulty in knowing what to expect to find in GILS. Indeed, most in the online user assessment disagreed with the statement: It is clear to me how agencies choose what to include in GILS. 

Currently, there are differing views of the level of granularity that is appropriate for inclusion in both the GILS records and for the items to be included in the GILS database. The result of these differing views is inconsistency in agencies’ GILS records regarding the types of resources included and the detail of the descriptions for the resources. The study finds that specific guidelines are needed to clarify the types of information resources that should be described by a GILS record.

4.4.7. FINDING: User Reaction to GILS Is Not Positive

Throughout the evaluation study, the investigators heard little in the way of positive experiences from people attempting to use GILS for finding information. To capture user perceptions about and reactions to GILS concepts and serviceability, the evaluation featured an exploratory technique based on a set of scripted service encounters (see Appendices C–5, D–5, and E–3 for a description of the technique, the instrument (script), and results, respectively). In this simulation of how users might use and assess a GILS, 10 undergraduate and graduate students at the University of North Texas and at Syracuse University completed a series of browse, search, and retrieval activities. Overall, users were confused and disappointed with the experience for a number of reasons, including: 

• An inordinately high degree of user sophistication is required to exploit GILS (e.g., one user remarked "shouldn’t have to feel like they’re hacking into a government system" and another asked, "would you turn a twelfth grader loose on GILS?").
• Users were interested in and/or expecting to gain access to full–text.
• GILS records were hard to read, contained unnecessary information, and were not linked to the actual source identified.
• Variance exists in the extent of information contained in GILS records and their display (see Appendix H for two example GILS records that represent this variance).
• The service seemed qualitatively and quantitatively unpredictable and/or uneven. 

While a majority of the users reported that they would use GILS to locate government information in the future, there were enough concerns and criticisms from the users to indicate that they consider GILS an unlikely source to help them identify and locate government information.

If users know of GILS, they make little use of it. When they do use GILS, they find it hard to use at best and inexplicable and frustrating at worst. Even agency staff involved in GILS implementations acknowledge that GILS is "user–unfriendly." Agency staff linked the poor user reception of GILS to difficulties inherent in the search and retrieval system, the lack of full–text information, the limited direct links to the resource when discovered through a GILS record, and deficiencies in marketing GILS.

Users interact with specific implementations of GILS. While they may not recognize the elegance of the decentralized, distributed architecture, the construct of metadata for discovering resources, and the necessity of a robust information retrieval protocol, they do provide specific assessments of systems implementing the architecture and standards. Their assessments provide GILS designers and implementors with actual user requirements for what users want in a locator service. 

GILS is in competition with agency Web servers. A participant in the online user assessment of GILS volunteered during the debriefing, in a positive, enthusiastic voice: "I always start with the agency homepage, and I find what I need about 40% of the time." The data from users indicate that the Web has had a dramatic effect on user expectations when locating and accessing networked resources. Users in the study’s scripted online assessment continued to expect access to full–text of documents or access to services described by GILS records even after they had spent time searching and were exposed to the construct of GILS as a locator. From a user perspective, what GILS records describe is unclear and confusing.

4.4.8. FINDING: GILS Record Display Varies Widely and Is Criticized by Users

Most agency staff and virtually all users commented on the need to improve the content and display of GILS records. There is still considerable discussion and debate about the need for and use of specific data elements and the degree to which those data elements should be presented to users. GILS records were described by one person as "user–ugly." Appendix H presents two actual GILS records that exemplify the variation users may encounter as a result of a GPO Access GILS search on <"social security" AND pensions>. These records show variation in content, format, and display. See also Appendix E–2 for examples of 4 high–quality records from the sample use in the record content analysis.

In the scripted online user assessment (see Appendix E–3), users commented on a number of presentation problems with GILS. First, since developers bill GILS as a "government information locator service," the majority of users suggested that all GILS records should look alike. There was also agreement with the statement: The quality of records I examined varied widely. Users recognized, and were disconcerted by, formatting errors (e.g., a record that did not have line wrap). Finally, there was frustration with not knowing "what they were looking at" on the screen or "what to do with the record." Investigators interpreted these comments to mean that users were not achieving an intellectual comparison between GILS and, for example, a record in a traditional or online library card catalog or a results list from a Web search engine.

The specifications for the GILS Core elements do not limit agencies in making improvements in the presentation of GILS records. Some study participants thought a "GILS–Lite" for presentation purposes is appropriate. A GILS–Lite record would offer the user a scaled down or reduced content record in an easier to read and use format. Additional research could determine the best or most useful collections of GILS record data elements to present to different users. Most of the agency GILS implementors, however, were unaware of how Z39.50 (the information retrieval protocol required by the FIPS Pub. 192) can provide different views of the record. The GILS Profile specifies several groupings of data elements to form "views" of the GILS record. But most implementations currently present the user with the entire GILS record.

4.4.9. FINDING: User Orientation and Instruction Is Inadequate 

During the course of the study, the investigators found some agency online guides that provided basic introductory information to their GILS, but not a guide or manual that describes the GILS as a government–wide service and how best to use it, how best to conduct searches, and what kind of information and output can be expected. Generally, training manuals and guides to assist users in their use of GILS are inadequate or non–existent.  

GPO’s manual Helpful Hints for Searching Federal Databases Online via GPO Access (March, 1996) is an example of the kind of guide that would be extremely useful for users to better exploit the GILS databases. The lack of training manuals, guides, or other such educational matter is part of the GILS marketing and visibility problem. The study finds that the lack of adequate user guides and related training material probably contributes to low use of GILS as well as frustration by those who do use GILS. 

Agency officials, librarians, student users in the online assessment, and others contacted during the study gave low marks for the overall usefulness of GILS as a tool for identifying and accessing government information they needed. This is, in part, because they do not understand that GILS records were intended to describe metadata, not individual source documents. Confounding this is the occurrence of GILS records describing individual publications. Also of interest is the number of GILS implementors, GILS policymakers, and others who are involved in the actual development of GILS who are unfamiliar with its operation and use it infrequently, if at all. Thus, there is likely to be contradictory, confusing, or erroneous information disseminated about GILS.

4.4.10. RECOMMENDATION: Continuously Evaluate GILS Policies and Standards Against Emerging Technologies, Especially the Web 

The emergence of the Web and its embrace by many Federal agencies for presenting information to the public have generated questions as to the role of GILS now that "we have the Web." At the time of GILS development, the Web was only minimally implemented. Given the near ubiquity of Web implementations by Federal agencies, a refocused GILS effort must determine how it can be integrated and evolve with the Web, as well as other emerging technologies (e.g. "push" technologies and natural language retrieval systems). Refocusing GILS to support networked information discovery and retrieval may assist in that goal. 

Many study participants acknowledged that the structured metadata record developed for GILS may be its lasting contribution. GILS should build on this success. Metadata can assist in the discovery and access of information in the networked environment. Standardized metadata is also independent of platforms and applications. Thus, the investment in GILS metadata should not be lost as the GILS evolves. GILS implementors will need to monitor ongoing metadata developments such as work on the Dublin Core and others. 

New mechanisms for automatic indexing of networked information resources as envisioned by the Advanced Search Facility (ASF) deserve close attention. The ASF will provide an efficient means of gathering and indexing Federal information that goes beyond what current Web search engines offer. The complexity and difficulty of distributed search and retrieval of digital information cannot be underestimated. Networked information discovery and retrieval is still in its infancy and many issues and challenges remain (Lynch, 1995; Lynch, et al., 1995). GILS policymakers and implementors must have one eye focused on the future and the emerging technologies, and they must have the other eye focused on current citizens’ needs for discovering and accessing information. The investigators think that effort expended in creating metadata records that support discovery and access will show a return on investment—library cataloging is a case in point. Technological solutions may assist in connecting users with government information, but the solutions must be workable and implementable

The Web is a powerful existing technology for publishing and providing access to digital information. Its principle appeal is the hypertext linking within and between networked information resources to assist users in browsing and navigating full–text documents and how it enables user interaction with databases and online service. The Web’s ability, however, to support networked information discovery and retrieval is limited. Existing Web search engines, while powerful, do not provide users with control and precision in searching across Internet resources. Metadata, in the form of GILS records, can be used to enhance the discovery and retrieval of networked objects. Databases of GILS records can be a source for users to discover the existence of government information. Moreover, information in GILS records provide information not necessarily available to Web search engines for categories of information resources such as databases. Given the recommendation to limit a refocused GILS to online, network accessible agency resources, users can perform searches against GILS records using Z39.50 and then be linked to actual resources (e.g., full–text documents, other electronic resources and services). The investigators recommend that the next phase of GILS effort should strengthen the metadata functions and Z39.50 search and retrieval functions while continuing to explore and research integration with the Web and other emerging technologies.

4.4.11. RECOMMENDATION: Specify Resource Types and Aggregation Levels

To optimize the utility of an information system, a user needs knowledge of what information can be expected to be found in that system. In a library catalog, users can expect to find entries that describe items in a particular library’s collection. An understanding of the unit of analysis (i.e., the granularity) of the items described in the catalog assists in its use. Catalogs usually represent a discrete item (e.g., one book) as the unit of analysis. Users have become accustomed to catalog entries representing books, as well as the scope and functions of the catalog. If users require representations of other units of analysis, they will often use other finding aids (e.g., indexes for journals to identify specific articles within a journal). For GILS to be a reliable and understandable aid in discovery, identification, and access to government information, users need to have a clear understanding of what information resources it includes and the unit of analysis for describing the resource.

The range of resource types and their granularity described in GILS is problematic. The original vision of GILS intended GILS records to represent information resources such as existing locators, which might exist as a single publication or system, as well as aggregating resources not previously gathered or described as a collection (e.g., a set of press releases). The issues surrounding the granularity and aggregation of records and resources are complex, possibly more so because of the electronic nature of some of the resources.

From a user perspective, the issue of granularity and aggregation has several aspects. First, what can the user expect to be described by a GILS record? OMB 95-01 policy prescribes the description of three classes of information resources: automated information systems, locators, and Privacy Act system of records. These, however, are not necessarily mutually exclusive classes since a locator might be cast in the form of an automated information system. The actual practice of the agencies that are creating GILS records reflects the description of classes of resources beyond the three prescribed by policy; this was clearly evident from the record content analysis. If there are too many units of analysis being used, it is difficult for the user to know whether GILS will be useful for specific information needs. For example, can a user expect a GILS record to describe an individual document? Will it be a document that is in fact an index or locator, which the user examines to locate an individual document of interest? An understanding of the nature and scope of the refocused GILS equips users in information discovery. Users will need some understanding of the types of resources that might be discoverable through GILS. Further, the GILS records themselves should clearly identify the type of resource described in terms users can understand.

Throughout the study, some individuals stated that GILS needs to get users to the "real" or "actual" information. The implication of "real" or "actual" is that simply having a GILS record that describes a resource is not enough (although it can be easily claimed that just as a library catalog entry contains "real" or "actual" information, GILS records themselves are informative). Leaving a user with only a pointer is not sufficient, these people argue, especially if the resource itself is in electronic form.  

A user perspective could argue in terms of the "distance" the user is from a resource that addresses or answers his/her information need. For most users, a GILS record is more useful if one can electronically link directly to the information object. For example OMB’s The Budget of the United States GILS record describes a specific document, and, with the link provided, the user can retrieve and (via GPO Access) even search the digital version of that document. 

An information object described by a GILS record, however, may be an online "locator" that the user would, in turn, search for desired information resource. An example of this would be GPO’s online Monthly Catalog GILS record. GPO has a GILS record for the Monthly Catalog. In response to a user’s search, the user may be presented with a GILS record for the Monthly Catalog. To continue the search for information pertinent to the information need, the user is required to do at least one more search—this time searching the Monthly Catalog to discover a citation for a specific resource. Although there may be no GILS record for the item described in the Monthly Catalog, the user is able to discover the item (and access the resource assuming it is in digital form and hotlinked from the Monthly Catalog citation).

These two cases of searching GILS illustrate how a user can move directly to a resource pertinent to an information need via GILS, or in the latter case, the user first uses GILS to identify another locator (e.g., the Monthly Catalog), and then conducts additional information retrieval transactions outside of GILS to find the desired information. One can discuss this in terms of "closeness" or "distance" from information objects, as well as traversing different "information spaces" to get to pertinent information resources.

The Web has been a conditioning force for Internet users. They have become accustomed to the experience of making several "clicks" and having at their disposal the "real" information (e.g., the full–text of a document, access to an online system). A refocused GILS with a more limited scope and coverage can support this type of information access, with the two examples give above offering model approaches to providing this networked access.

To help users understand their "distance" from a resource described in GILS and the nature of the aggregation, an existing GILS data element, Resource Description, could contain a controlled value such as one from the list developed during the study’s record content analysis: 

• Subject matter database
• Publication
• Miscellaneous documents in ad hoc collection
• Agency homepage
• Organization
• Form
• Administrative catalog
• Bibliographic database
• Publications catalog
• System of systems
• Program
• Job line. 

This list can be refined and developed so that a comprehensive list of GILS–described resources is available. As a part of the search results, where the user sees a brief form of the GILS record, the user could be presented with the resource type description along with a title and selected other GILS data elements. The brief form of the record should offer the user enough information to characterize the resource and enable the user to determine whether a particular resource described by a GILS record would be useful.

Further, a brief form of the GILS record should indicate whether the resource is network accessible, and by what means. While current GILS records occasionally include this information (i.e., by Available Linkage), users must read through many elements in the GILS record to discover it.

The investigators recommend that GILS policymakers and implementors should specify and define resource types to be described in the refocused GILS initiative. This determination should be informed by users’ expectation to reach the full–text of a resource or link to another electronic resource. Implementors should highlight the type of resource described by a GILS record and its network accessibility to assist users in making relevance judgments and accessing the needed information.

4.4.12. RECOMMENDATION: Enforce Consistent Use of Metadata That AreEmpirically Demonstrated to Enhance Networked Information Discovery and Retrieval

The investigators encountered many comments related to the content requirements for GILS records and questions about the utility and benefit of the information included in GILS records. Based on these comments, and a refocused scope for Federal GILS implementations, there is a need to review the data elements as used in agency GILS implementations with the goal of optimizing them to support the discovery, identification, and access of government information.

Information organization begins with a selection and filtering process and a distillation of essential features from each information object (Hsieh–Yee, 1996). A point of contention becomes immediately obvious: what is valuable or essential? Those who seek to make "resource discovery and retrieval" possible in the networked environment must determine which information resources are worth describing, a significant initial step. But a second set of decisions may be even more difficult—those concerning the salient features of the information resources that need to be represented and described in a record.

The data elements for GILS records had their genesis in an interagency working group. Different stakeholders within that group identified data elements necessary to support specific functions. Record creators need to collect or capture the information to provide content for data elements, recognizing that each bit of information included in a GILS record has an associated cost. Which are the highest value pieces of information that could be included? How much information should be contained in a GILS record? These are not easy questions to answer, especially given the diffuse goals, purposes, and expectations of GILS discussed earlier.

GILS is a pioneering effort in what has become a major research and development activity (i.e., determining appropriate metadata schemes for networked information discovery and retrieval). The community of interest that defined the initial set of metadata (i.e., government agency staff) had particular requirements for GILS, and these requirements were codified in the appendix of the GILS Profile that identifies and defines the GILS elements. A key question at this point is: what are the salient features of an information resource that need to be represented in a GILS record to support users discovering, identifying, and accessing U.S. Federal government information? An associated question is: do different classes of resource types need different groupings of metadata elements (e.g., if one is representing a document rather than a database rather than a Web site).

The work on the Dublin Metadata Element Set could inform a review of the data elements for a refocused GILS. The goal of the Dublin Metadata Element Set is to devise a simple and minimal metadata scheme to provide descriptions of one class of networked information resources (i.e., document-like objects) for their discovery and retrieval. The 15 elements of that metadata scheme may be sufficient for the revised purpose of a modified GILS—namely the discovery and access of government resources. This approach should be explored in the context of reviewing the existing 67 mandatory and optional GILS elements.

Including metadata elements in GILS records that support objectives other than the public’s discovery, identification, and access of information (e.g., IRM and records management) confounded GILS implementation. In the next stage of GILS development, the overriding criteria for determining mandatory metadata should be driven by the newly articulated purpose and goals of a refocused GILS initiative and the uses to which the metadata records will be put. Policymakers may find a review of the development of Federal Geographic Data Committee (FGDC) metadata helpful; that community identified four criteria for inclusion of specific data elements (Mangan, 1995):  

• Availability: information needed to determine what data exist for a given geographic area
• Fitness–for–use: information needed to determine if a dataset meets a specific need
• Access: information needed to acquire an identified dataset
• Transfer: information needed to process and use a dataset.

Regardless of the criteria for determining GILS data elements, a formal process is needed for discussing and identifying the U.S. Federal requirements for data elements to support users’ needs to discover and access government information resources. The formal process requires identifying an agency or interagency body as the official forum for discussion of U.S. Federal GILS specifications as well as acting as a "steward" of the GILS data elements as used in U.S. Federal implementations (e.g., developing guidelines for record creation, providing assistance in using the data elements, etc.).  

The current process for revising GILS data elements is under the jurisdiction of the GILS SIG, with discussion on the elements occurring in monthly meetings of the GILS SIG and through the GILS Forum, an online discussion group established in 1994. According to the GILS SIG statement of purpose, All recommendations developed at the periodic meetings will be distributed via the listserver [i.e., the GILS Forum] for comment and additional discussion prior to becoming final" (see Appendix A-6). The Forum is open to anyone with access to an Internet email account and is not limited to U.S. Federal government agency staff and associated stakeholder communities. The GILS Profile is a general purpose profile for describing and locating information, not exclusively government information. The U.S. Federal implementation of GILS has specific requirements, and it is appropriate that a formal body (agency or interagency) be authorized with the responsibility for stewardship of the data elements scheme in the next stage of GILS development. Such a forum, however, must coordinate efforts with other government agencies that are promoting one or more metadata schemes (e.g., NARA’s records management data elements, FGDC content standard, etc.).

The investigators recommend that metadata elements should be reviewed within the context of the revised and more focused purpose for GILS, namely discovery, identification, and access of government information. Data elements should be included/excluded in the metadata scheme based on the extent to which they demonstrate support of enhanced discovery and access of government resources. An analysis of the cost/benefit of the current data elements compared with their capability to support of the purpose of a refocused GILS should be done. GILS metadata development should also take into account activities of other major groups that are developing and evolving metadata schemes and the evolving technology that supports distributed search and retrieval. 

Either a single agency or an interagency group (e.g., a GILS Committee of the CIO Council) should be charged specifically with the review, development, maintenance, and revision of GILS data elements as used in U.S. Federal GILS implementation. GILS policy should identify the body responsible and direct it to prepare specific written and easily available procedures and criteria for maintaining and revising the GILS metadata elements. The resulting process will provide agency implementors to determine new elements or modifications to existing elements based on the requirements of a refocused GILS. After Federal implementors identify their requirements and proposals, these can be forwarded to the GILS SIG, which has authority for maintaining the GILS Profile. This process recognizes that U.S. Federal implementors may have requirements different from other communities that use the GILS Profile.

4.4.13. RECOMMENDATION: Improve Presentation of Metadata 

Users, whether agency staff, librarians, public users, or others, noted problems with the presentation of GILS records. They remarked about records containing too much information, or not the right information, difficulty in understanding what the GILS records described, and the unpredictability of element inclusion (e.g., use of nonmandatory elements or locally–defined elements).

The development of the GILS Profile acknowledged that different user groups might need different views on the GILS data elements. Although one might question whether the Profile defined appropriate and adequate data elements in the first place, the issue of presenting GILS data elements in the records is quite separate.

Policymakers chose Z39.50 as the information retrieval protocol to support GILS because of its functionality in providing a uniform interface to different information servers and their associated databases. It also allows Z39.50 clients to request different views of the database record (e.g., a GILS record). Thus, it separates searching records (enhanced by the number of structured access points available) from presenting the records (which can by customized by implementors). For example, the GILS Profile identified several views of the record, where each view presented different amounts of data to the user. The key question remains: what is the appropriate information to present to users, and at what stage in the search/retrieval process? 

Current GILS implementations using Web–based interfaces usually present, in response to a search, a result set of "hits" (i.e., pointers to GILS records that meet the criteria of the search). Users of GILS are first presented with the list of "hits" in the result set, and those hits are generally represented by only the title and a relevance score. When users select a GILS record from the result set, most agency GILS implementations display a view of the complete GILS record. The question is: is the complete GILS record the appropriate or only view of the record to present to the user? As discussed earlier, study participants proposed a "GILS–Lite" record that would present a briefer view of the entire record. Such views can be accomplished using Z39.50.

Experience from the Cyberstacks project at Iowa State University (McKiernan, 1996a) suggests that record creators need not "delineate all relevant elements in describing a resource," but rather should "characterize the resource sufficiently so that users can judge its potential usefulness" (Mckiernan, 1996b). McKiernan recommends that users need only an appropriate characterization to determine whether an item is potentially relevant and deserves a closer look.  

Relevance and selection judgments by users comprise a complex process (Barry, 1994). Agencies need to experiment with providing different views of GILS records to their users to determine which views are appropriate at different stages of the information retrieval process. The investigators recommend that agencies should remember the purpose of a refocused GILS and experiment with presenting users with different groupings of data elements. Such experiments should be evaluated closely, and the experiments themselves should be informed by recent and ongoing research in user relevance judgments, as well as human computer interface design (Schneiderman & Croft, 1997). 

An interagency effort should be mounted to address issues of presentation and use of metadata records (e.g., when they should be presented to the user, when should the use of GILS be transparent to the users, which data elements to present, etc.). Speed and ease of finding the information (e.g., identify a maximum number of "clicks" to get the user to the GILS record and the described information resource) should combine with readability, consistency, layout, and other presentation features of the record to optimize information discovery and retrieval. The GILS pilot program offers a venue for the development and testing of two or more Z39.50 clients that support the function of element selection and processing for customized display to users. 

4.4.14. RECOMMENDATION: Develop Policy and Procedures for Record Maintenance

Although many agencies have created GILS records, the maintenance of those records appears to be less well-supported. As noted earlier, agencies that see no benefits from GILS have little or no incentive for continuing to create more records or to maintain the records they have created. Keeping metadata records current and accurate should become part of the day-to-day fabric of agency information resources management activities. 

GILS will not endure unless agency staff consistently maintain GILS records. This is especially important because of time-sensitive data included in the records. For example, a set of elements in current GILS records hold information about the point of contact, including contact names, telephone numbers, and email addresses. Further, where a GILS record contains a pointer or link in the form of a Uniform Resource Locator (URL) from the record to the described resource, staff must ensure that the link is still operable.

In the decentralized environment for agency GILS record creation, the investigators recommend intra- and inter-agency efforts at devising written policy and procedures for record maintenance. Such policy and procedures should address the varying levels of networked infrastructure in agencies as well as other factors such as intra-agency cooperation from offices of primary interest in record maintenance and updating. There will be the need for mechanisms to automatically remind the record creators that their records need review and/or updating. Different types of agency resources may be more subject to change than others, and thus need more frequent maintenance. Software that tracks the date of last modification of a GILS record could trigger an alert (e.g., in the form of an email message or utilizing "push technology") record creators to review their records and update them if necessary.

The issue of record maintenance must be addressed since the degradation of the currency of GILS records will hinder access to government information. A GILS pilot program offers an opportunity for fine-tuning the policy, procedures, and software for maintaining GILS records.

4.4.15. RECOMMENDATION: Promote Interagency Cooperation and Use of GILS for One–Stop Shopping Functionality

Several agency and interagency initiatives use the Web to provide one–stop shopping to collections of government information resources and services. These include: 

• Business Advisor, the one–stop electronic link to government for business <http://www.business.gov/>
• Federal Statistics Initiative <http://www.fedstats.gov>
• WINGS, Web Interactive Network of Government Services <http://www.wings.usps.gov/>
• Commonly Requested Services <http://www.whitehouse.gov/WH/Services/>
• National Environmental Data Index (NEDI) <http://www.nedi.gov>.

One can think of users needing government information about particular topics. Often these information needs are not formulated, nor need they be, in terms of "what agency should I contact to get the information I need?" Rather, users may think in terms of "where can I find government statistics on unemployment rates and their impact on welfare requirements?" In the latter case, the collection of resources from various agencies (e.g., Department of Labor or Department of Health and Human Services) in a one–stop shopping scenario is more effective than presenting information according to the missions of government departments, agencies, and bureaus.  

Implementors structured the Federal GILS initiative along agency lines, but this basis of agency locators does not preclude interagency one–stop shopping scenarios. GILS provides a mechanism for agency resources to be identified and described. For any particular topic area (e.g., environment, energy, etc.), the relevant GILS records could be gathered and placed in a database for user searching. This, when combined with the Web–based initiatives listed above, provides users with several means of access (e.g., browsing and free–text searching of a Web site, and searchable GILS records for identifying specific resources of interest).  

OMB Bulletin 95–01 language reflects this approach: 

During the evaluation study, the investigators saw some evidence of such cooperation vis–a-vis GILS. For example, EPA participates in NEDI, which "contains" EPA’s GILS records. Given the example of NEDI, it is likely that such coordination may occur only if agencies see they are addressing real information needs and are doing so to accomplish their missions. Identifying exemplary interagency cooperation that builds upon individual agency GILS efforts to serve as models and offering incentives could assist in the development of one–stop shopping.

In addition to cooperative efforts that provide a single point of access to collections of resources thematically or topically related, users also identified a desire for one–stop shopping for searching for government information. This part of the vision of GILS has yet to be realized, in part because of the very real difficulty and complexity of conducting distributed search and retrieval. GPO is experimenting with cross–agency GILS searching, and this effort should be applauded as well as evaluated. Such efforts need to assess if searching and retrieval performed under this configuration increases user satisfaction with results. The Advanced Search Facility (ASF) also may offer a technology solution to cross–agency searching. Based on the information gathered during the study about ASF, GILS implementors should follow its development closely.

The investigators recommend that GILS policy promote interagency cooperation and provide incentives to realize one–stop shopping for government information. A refocused GILS should have as its goal the support of government–wide searching for information. GILS should provide a means for users to discover and access information on a government–wide basis. Distributed searching across all agencies’ information resources (e.g., by searching across agencies’ GILS databases) provides one approach to one–stop shopping. Interagency cooperative efforts should be encouraged to develop collection of government resources, and the refocused GILS initiative can explore how GILS can support such efforts.

4.5. OPPORTUNITY: RESOLVE GILS RELATIONSHIPS WITH OTHER INFORMATION HANDLING FUNCTIONS

Agencies’ management of their information resources involves many different information handling functions (e.g., providing public access, inventorying, records management, etc.). The establishment of GILS added yet another function. This opportunity addresses findings and recommendations related to the role of GILS vis–a–vis other agency information handling processes. One of the challenges in the next phase of GILS development will be to resolve how GILS fits with and can be integrated into these processes. Table 4–8 summarizes the findings and recommendations for this opportunity.

* CA=content analysis of GILS records; FG=focus group sessions; KP=interviews with key participants;
LA=log analyses of Web servers; SU=survey conducted at the 1996 GILS Conference;
SV=site visits to selected agencies; US=scripted online user assessments of GILS

4.5.1. FINDING: GILS Does Not Support Records Management Activities 

OMB Bulletin 95–01 identified a records management component for the Federal GILS initiative. GILS designers and researchers, however, did not consider GILS as a tool to support records management (Moen & McClure, 1994a; Information Infrastructure Task Force, 1994). While GILS policy considered public access and records management mutually supportive, the study identified significant problems with using GILS as a records management tool as outlined in OMB Bulletin 95–01. Identifying a records management component for the Federal GILS initiative led many agencies to delegate GILS implementation to records managers. While records managers have responsibilities related to identifying information resources for scheduling and archival purposes, it is not clear that records managers were in an appropriate position to recognize the broader possibilities and benefits for public access and IRM that the GILS initiative could support.

From a records management perspective, specifically in terms of records scheduling and the information needed for scheduling records, much of the information GILS records describe is not organized in such a way as to be useful. Records managers schedule records in series and do not manage individual publications or documents. They schedule publications (e.g., information dissemination products) as part of a series, often a series that describes the agency’s information dissemination products as a whole. For automated information systems, records managers schedule not only the system itself but its inputs and outputs. As noted in the NARA booklet on managing electronic records, "It is also essential to emphasize that all components of electronic information systems are records: inputs, outputs, digital data stored in a variety of ways, and the related documentation" and each of these different record components of an information system may be on different retention schedules, etc. (National Archives and Records Administration, 1990, p. 5). these are not currently described by GILS records.

Some agencies create GILS records for individual publications such as the Internal Revenue Service’s Catalog of Federal Tax Forms, Form Letters, Computer Generated Letters and Notices or the Department of State’s pamphlet Americans Abroad: What You Should Know Before You Go. Other agencies have one GILS record for all publications such as the Federal Emergency Management Agency (FEMA) GILS record for its FEMA Publications Catalog. The variety of aggregates and information types that GILS records describe makes these records ineffective for records management purposes.

The GILS data elements do not support records management since they do not account for important information such as record retention and disposition in ways that serve records managers. OMB Bulletin 95–01 directed NARA to: 

Yet the records managers interviewed during the study stated flatly that GILS records were not adequate for records management purposes. More importantly, records managers expressed little enthusiasm to "enhance" GILS data elements to carry such information, in part because of the mismatch of records management practices (e.g., scheduling records in series) and the types of resources described in GILS and related granularity/aggregation issues. 

OMB Bulletin 95–01 also included guidelines to agencies in terms of their responsibilities for using GILS in records management functions: 

Policymakers envisioned GILS as a mechanism to discover and identify agency records in need of scheduling. Several study participants mentioned that in practice this did happen occasionally.

But in considering GILS as a tool for NARA to use in monitoring agency resources that had not yet been scheduled, NARA representatives said that GILS is not comprehensive and would not be reliable as the only tool for them to use. GILS may be useful as a finding tool to uncover material that should be scheduled and to enhance the thoroughness of agency records management. But in fact, the investigators identified only a very limited use of GILS for these purposes.

NARA representatives also told the investigators that NARA received few SF 115s from only limited number of agencies because of the OMB Bulletin 95–01 requirement. NARA detected no significant increase in the number of scheduling requests attributable to GILS. They concluded that GILS was not having any major impact on scheduling agency records. While the OMB Bulletin viewed GILS records as carrying data that would make submitting the SF 115 redundant, the SF 115 is the legal instrument used in the scheduling process. Further, the SF 115 is only one part of the scheduling process, a process that includes authorizing signatures, etc. GILS policy on records management seems not to have recognized this fact.

Findings from the study suggest that GILS is not equipped to improve government–wide records management activities and responsibilities. A government–wide system for records management is needed. The current GILS, however, is not the system to accomplish it. Curiously, agencies perceived or suspected a connection forged between GILS and records management as something devised by GILS creators in collaboration with NARA to provide political support for the records management function in Federal IRM and not something that arises out of a natural affinity between GILS and records management.

4.5.2. FINDING: GILS Relationship with Agencies’ Inventories of Information Resources Is Not Clear

Agencies are required by OMB Circular A–130 to develop and maintain inventories of their information resources. A previous study by the investigators (McClure, Ryan & Moen, 1992) identified agency locators, but concluded that agencies did not have in place comprehensive locators to their information resources. One of the assumptions of OMB Bulletin 95–01 was that such agency locators did exist, and that creating GILS records describing these locators would not be a major burden on the agencies. As noted in Section 4.3.9., even though the policy required agencies to inventory their resources, no clear guidelines and prescriptions emerged to guide agencies in how those inventories could become useful network-accessible locators, which in turn could be described by GILS records. 

In discussion with study participants, representatives from the small agencies noted that oftentimes such locators did not exist, and they expressed strong feelings of anger and frustration against the assumption that inventories were in place. This false assumption allowed OMB to assume that GILS record creation would be relatively effortless. Given current realities, the smaller agencies are unlikely to participate in future GILS activities without significant changes in the GILS initiative. 

Other study participants acknowledged that implementing GILS forced some of them to accomplish some inventorying of their resources. Both agency staff and other GILS stakeholders said that this was a positive byproduct of GILS.

It is unclear whether GILS should be seen as the tool to gain agency compliance with developing information inventories. In the discussion of GILS and records management, study participants noted that because GILS does not provide a comprehensive list of agencies’ resources, its utility for that aspect of records management (i.e., discovering what resources might exist that are in need of scheduling) is limited. One can conclude that GILS is not moving agencies, especially the smaller ones, to a comprehensive coverage of their resources and has not become an inventory of agency resources.

4.5.3. FINDING: GILS Relationship with FOIA and EFOIA Is Unclear

The passage of the Electronic Freedom of Information Act Amendments (EFOIA) (P.L. 104–231) in Fall 1996 immediately preceded the 1996 GILS Conference. At that conference, Sally Katzen (1996) of OMB stated: 

Her statement had considerable impact on the audience and was mentioned to the investigators repeatedly during later data collection activities. 

One set of viewpoints identified by the study regarding GILS and EFOIA is, indeed, what if GILS could become the "killer application" and provide the means by which agencies could implement various record keeping provisions of EFOIA? But upon further discussion, specific strategies for accomplishing this objective, how the GILS records data elements would need to be changed, and the level of effort to "shoe–horn" EFOIA provisions into the GILS concept were unclear at best. 

Reactions to Katzen’s statement exemplified the multiplicity of understandings (and misunderstandings) of GILS intent and potential. As one person commented to the investigators, "it’s just another unfunded mandate by OMB that hasn’t a clue as to the level of effort and resources needed to make it happen." At one focus group session of agency records managers, participants laughed at the idea that GILS, as presently constituted, could begin to address the EFOIA functionality that Katzen mentioned. 

The timing of Katzen’s statement provided a catalyst for this discussion to occur during data collection activities. To some extent the debate about the role of GILS in EFOIA is a microcosm of the larger GILS assessment: What is GILS’ purpose versus potential purposes? How will GILS initiatives be funded and implemented at the agency level? What changes in GILS record content will be needed? Who will provide the leadership to develop this "killer application?" Overall, study participants found this "opportunity," as suggested by Katzen, to be but another task for which they had no time, staff, or other resources to address. 

The policy review in Chapter 2 discussed EFOIA and a recent memorandum from OMB that links GILS and EFOIA (Office of Management and Budget, 1997a). Yet the guidance in the memorandum (i.e., agencies should establish a GILS "presence" to address requirements of EFOIA) lacks precision and begs the question as to how—specifically—GILS can assist in handling EFOIA requests.  

Resolving the issues of integrating GILS and EFOIA is beyond the scope of this study and requires additional study before a recommendation could be made. A research effort could examine a range of FOIA requests to determine what information a GILS record would need to contain to assist the user in identifying the object of the sampled FOIA requests. Proposal for using GILS to support EFOIA will require careful assessment and study to determine what, if any, real linkage can be made between GILS and EFOIA. The GILS pilot program would provide one opportunity for such study.

4.5.4. RECOMMENDATION: Uncouple the Refocused GILS—as an Information Discovery and Access Service—from Records Management 

The findings in the study are unequivocal about the lack of utility for records management provided by GILS in its current implementation. While there may be some logical connection between a locator service and the records management responsibilities of agencies, the U.S. Federal implementation of GILS does not justify GILS as a records management tool. 

The discussion of findings above offered reasons why GILS does not support and is not a suitable mechanism for records management (e.g., granularity of records, availability of data elements to carry records management information, etc.). Although there was some evidence that GILS records could be used for identifying resources that need to be scheduled, GILS is limited in utility in this records management function as well since currently GILS cannot be relied on to represent comprehensively the resources of an agency. One potential use of GILS that intersects with records management would be to require agencies to create GILS records that describe and point to agencies’ records schedules. Ideally, the schedules themselves should be network-accessible, and users could discover and locate the schedules, and then uses the schedules to identify agencies’ information resources. 

Three options are possible relative to GILS and records management: 

• Make no changes to GILS related to records management and assume that agencies will try to use GILS to some extent for their records management activities.
• Enhance GILS by adding additional data elements and other specifications to help creators of GILS records provide the information and describe at the appropriate level of granularity to serve records management goals.
• Sever GILS from records management activities. 

While all three of these represent possible directions, the investigators recommend the third option. The evidence from the study was substantial—from the perspective of records managers and NARA—that GILS is not suitable for records management, and in particular for supporting records management processes such as scheduling and communicating scheduling information in lieu of the SF-115. Further, there was no agreement on how GILS could be enhanced or changed to make it a usable tool for records management, nor that the effort in doing so was warranted. 

Uncoupling the Federal GILS initiative and records management will bring several issues to the fore. If policymakers designed GILS as a means by which the "electronic records management" problem could be solved, the study concludes that GILS is not the solution. In fact, the term "electronic records management" can refer to the management of electronic records (simply applying records management activities to resources that are in electronic or digital form) or to using information technology to support processes involved in records management such as electronic submission of SF-115, digital signatures, etc. GILS policy appears to have emphasized the latter aspect, and GILS does not support that aspect of electronic records management. 

Senior staff at NARA are aware that GILS is not serving records management purposes, and also realize that government–wide electronic records management needs a solution. Although GILS policy should not address records management issues, policymakers should expect NARA, in collaboration with the agencies, to develop a workable solution to government–wide electronic records management in a realistic timeframe (e.g., 16–24 months). 

NARA should develop a formal program to implement records management processes and procedures that will allow agencies to submit electronically requests for records scheduling and disposition authority (i.e., an electronic version of SF 115). NARA’s program should be developed with input and advice from policymakers and agency officials, and the program should include specific objectives and time frames for monitoring its progress. 

At the agency level, an impact of the uncoupling of GILS and records management raises the question of who in the agency will be responsible for GILS? Many agencies, especially the smaller agencies, delegated GILS responsibility to their records managers. This was likely due to the records management language in OMB Bulletin 95–01. If GILS does not play a role in records management, it is likely that agencies should and will identify non–records managers with responsibilities for agency GILS efforts. This raises an important question about who, in the agencies, are best positioned to assume the responsibilities for implementing a refocused GILS? 

NARA’s responsibilities per OMB Bulletin 95–01 for developing guidelines and providing training for GILS record creation will need review in the next phase of GILS. Such guidelines and training will be needed in the refocused GILS, and NARA brings appropriate expertise related to content standards and descriptive records. GILS planners must identify training and documentation as key areas for attention, and NARA (or possibly the cataloging expertise at the Library of Congress) could be a resource in the development and provision of training. 

The investigators recommend that revised GILS policy should uncouple the discovery, identification, and access function of a refocused GILS from agencies’ records management responsibilities. GILS will not and should not be used as a mechanism for solving a range of electronic records management problems. There is no apparent natural affinity between public–access NIDR and electronic records management, and both programs must be sufficiently mature before viable connections between them will be made. 

4.5.5. RECOMMENDATION: Derive GILS Metadata from Other Information Handling Processes

An important aspect of a refocused GILS effort will be to identify how GILS can be and should be integrated into agency information handling processes. In particular, the refocused GILS effort should identify ways to prevent agency GILS activities from dis–integration with other information handling and dissemination processes. For many agencies, GILS implementation has been a standalone add–on, which weakens its benefits and buy–in. For the new effort to be successful, it must be integrated into other information handling processes. 

One of the primary benefits of GILS to date is the creation of standardized, structured records for describing agency information resources. These metadata records are essential for the discovery and retrieval of information in the networked environment. One important area for development is to determine how GILS metadata can be captured automatically for each new information resource created by an agency. 

The refocused GILS initiative must address how GILS metadata can be captured in the most effective and efficient way. Discussions with agency staff responsible for creating GILS records noted the significant level of effort to collect content for the records. The actual inputting of the GILS record is relatively trivial in terms of labor, but the collection of adequate, accurate, and appropriate information is extremely time consuming. Too often the people in charge of creating the records did not have the information available to them, and too often the record creators did not gain cooperation from agency staff who were primarily responsible for the information resources that needed to be described in GILS. 

The retrospective character of GILS record creation is a problem. Agency resources exist; data must be collected retrospectively about those resources prior to the creation of the GILS record. While it is possible to do such retrospective cataloging of agency resources, the cost of creating GILS records for all agency information resources may far outweigh the benefit. Clear guidance is needed on what existing resources agencies should describe in the refocused GILS. 

Existing agency resources are just part of the government information universe GILS addresses. Each day, new agency resources are created and developed and added to the information universe. Since most current agency information resources begin as an electronic file in the information life cycle, electronic document management systems (EDMS) may contribute a solution to GILS record creation for at least some categories of new information resources. Metadata for document-like electronic resources can and should be captured at the point of creation, and EDMS provides mechanisms to do this. Agencies could use the captured metadata for creating item-level GILS records or could compile item-level metadata into agency locators, which in turn can be described by a GILS record. 

The refocused GILS effort should examine the most efficient means for capturing basic metadata whenever a new information resource is created or initiated. For this to succeed, a refocused GILS cannot be a standalone system but rather GILS records need to be derived from metadata captured in the process of creating and managing an information resource through its life cycle. EDMS should be exploited by agencies to manage their electronic information resources (e.g., document version control, reduced duplication of effort, inventory reporting, etc). The system can incorporate a module whereby metadata about the resource being created can be derived. 

The metadata to be captured, and when, should be informed by the purposes the metadata serve. A refocused GILS should determine the appropriate metadata to support information discovery, identification, and access. Appropriate and accurate metadata can be more easily determined and assembled during the process of creating the information resource than after. As part of managing an information resource through its life cycle, agencies may need to capture metadata that serve purposes in addition to information discovery and access. The focus for the next phase of GILS, however, is to identify the metadata needed to serve purposes of a refocused GILS and identify effective ways of capturing the metadata. 

While discussions of electronic document management systems are outside the scope of this report, policymakers and agency information managers need to make the systematic management of electronic documents a priority policy area. To manage agency electronic resources systematically requires an understanding of an agency’s information processes and flows (i.e., an architecture) and a focus on information life cycle management (Ambur, 1996). One component of the architecture will be the capture of appropriate data about electronic information resources, and the capture of GILS metadata information can be accomplished within such a scenario. 

The investigators recommend that policymakers and implementors explore and assess various practices to integrate GILS into existing or emerging information handling processes and systems. Without integration, GILS may be subject to lack of attention as a standalone activity. A critical aspect of its integration will be in determining the best practices for capturing GILS metadata at the time of creation of new information resources, and EDMS can serve as one model for automatic capture of metadata. Based on comments by study participants, little increase in the number of GILS records is likely unless the process of capturing metadata is less labor intensive and more cost–effective. 

While there are many reasons for the uneven character of agency GILS implementation (e.g., lack of tangible benefits, cost of creating records, lack of management support, etc.), an approach that ties GILS into other information handling processes may assist agencies in reconsidering the utility of GILS. If GILS activities are not integrated, and if metadata capture cannot be made less burdensome, even a refocused GILS effort may be threatened by current resistance to GILS—it will remain "one more thing" agency staff have to do separately from other activities. 

4.6. OPPORTUNITY: INCREASE GILS AWARENESS 

Except for a relatively small number of study participants who have been intimately involved in GILS activities and implementations, the study found the majority of participants lacking in basic understanding of what GILS was intended to be and how it was to function. Outside the "beltway," the investigators found minimal awareness that GILS existed, even among important user communities such as government documents librarians. 

For any product or service to succeed, a program of promotion and education is necessary. The following findings and recommendations address the need for a refocused GILS to increase awareness about the service, but to do that the purpose and goals of GILS must be clarified so a coherent message can be delivered about the service. Table 4–9 summarizes the findings and recommendations for this opportunity. 

* CA=content analysis of GILS records; FG=focus group sessions; KP=interviews with key participants;
LA=log analyses of Web servers; SU=survey conducted at the 1996 GILS Conference;
SV=site visits to selected agencies; US=scripted online user assessments of GILS 

4.6.1. FINDING: No Program for GILS Promotion and Education Exists 

The study explored the extent to which a coordinated promotional effort for GILS exists, and who, specifically, was charged with responsibility for that effort. By and large the answer is that there has been no government–wide campaign effort for GILS. 

A number of champions and spokespersons have come forward to talk about and support the GILS efforts. The Special Interest Group on the Government Information Locator Service (GILS SIG) developed and disseminated a one–page brochure describing GILS, but, according to one of the participants in that effort, gaining consensus on what the brochure should contain and how to state the purpose of GILS was a challenge and involved several months’ discussion. 

In addition, different spokespersons "marketed" GILS differently, leaving contradictory messages of why GILS was important and what GILS was intended to accomplish. One result of the lack of "marketing the product" and "keeping on message" about GILS could be the varying expectations of GILS encountered in the course of the study. The absence of a central and coherent message allowed GILS to become "different things to different people." 

NARA provided training sessions for GILS implementors, specifically for those involved in record creation and the use of the NARA Guidelines. Such training sessions, however, answered only one aspect of the education needed by agency staff to understand why GILS is important and how it can be used to improve public access and agency management of information resources. Efforts for systematic training for GILS users were minimal. The notable exception has been GPO’s training of documents librarians on GPO Access, which now includes a hands-on session for GILS. 

Many agency staff that participated in the study criticized the lack of government–wide or other systematic promotion of the Federal GILS initiative. Study participants remarked that no single and unified voice came forward in the past two years to market GILS. The lack of a program promoting GILS resulted in a low level of awareness and limited acceptance of and support for GILS.

Advocacy of GILS was difficult because of the confusion over its purposes—what it was supposed to offer, how it worked, what people could expect to find in it. This finding reveals the need first to define GILS and then develop a strategy for promoting it. Refocusing the GILS effort by identifying an understandable scope and function of GILS will be an important first step. 

One site visit participant suggested that there was need for a public marketing campaign for educating people about GILS—something they saw had not been done to date. Another person suggested that "GILS should be promoted as ‘this is how you find information about the government’ and make it a central and first point of contact for finding government information or more general information about the government." Given the various problems with GILS, the absence of a promotional campaign is probably not significant. It could become a deciding factor, however, in the success of a refocused GILS. 

Positive promotion of a refocused GILS can reap benefits within an agency. An agency site visit participant suggested that "many agencies do not grasp the significance or potential for GILS and an all–out marketing effort by OMB and NARA needs to address this. Such a dual marketing strategy would assist in getting more top management support that would, hopefully, filter down to the bureaus and department levels of agencies." Senior agency management needs to make the refocused GILS a priority if it is to be successful, and a promotional campaign directed at agency managers could be effective in garnering additional resources and commitment. Moreover, a GILS pilot program could demonstrate how GILS works and the benefits from using GILS. Thus, the program of promotion will serve to educate agency managers and staff as well as non–government users about GILS. 

At an agency level, study participants identified a number of potential benefits from systematic promotion of Federal initiatives: 

• Greater senior management buy–in
• Active demonstrations of the utility
• Extension of participation (e.g., creating and maintaining more GILS records)
• Enhancement of applications (e.g., data gathering and input for GILS records)
• Improved training to agency units
• Development of marketing tools such as brochures. 

Systematic promotion is thus essential for GILS—from a policy perspective as well as management, implementation, and use perspectives. Such marketing has not happened to date in the GILS initiative, nor was any one agency charged specifically to develop and carry out such a marketing program. Lack of product marketing reflects an if we build it, they will come attitude. 

4.6.2. FINDING: Potential User Communities Lack Familiarity with GILS 

The study found very low visibility and limited knowledge about GILS outside a core group of champions, policymakers, and agency implementors. The survey distributed at the Fall 1996 GILS Conference asked respondents to rate their familiarity with GILS documents and policies. Of this group of people that could be considered knowledgeable or at least interested in GILS, less than 50% claimed familiarity with some of the basic GILS documents and specifications. (In contrast, a majority of respondents claimed familiarity with the World Wide Web.) Most problematic is the lack of familiarity by this selected group of people at the GILS Conference with three basic GILS documents: OMB Bulletin 95–01 (policy); NARA’s record creation guidelines (for implementation activities); and FIPS Pub. 192 (for technical specifications and guidance). Table 4–10 summarizes the responses (see also Tables E1-5 and E1-6 in Appendix E–1 that contain the complete survey results). 

* Total does not equal 100% due to rounding.

Potential users contacted by the study team often did not know of, nor had they used, GILS. Indeed, a primary audience and potential user group, government document librarians participating in an early focus group, revealed very little knowledge of GILS. Further, this user group’s interest is also limited, if assessed on the fact that only one person attended a focus group of documents librarians at the American Library Association Midyear Conference in February 1997. GPO staff, finding little awareness of GILS in their training of depository librarians, developed and are delivering training on GILS as part of the overall training on GPO Access. This targeted training effort should improve awareness among the documents librarian community. 

4.6.3. FINDING: GILS Usage Is Limited

The study identified a generally low level of awareness of GILS. As part of the study, several tactics were used to gauge current use of GILS. One technique was the GILS Conference survey; another was the transaction log analysis—the data from the latter requiring caution in interpretation. The findings from the survey pointed out quite clearly that GILS Conference participants, who, after 2 years into the initiative, should be knowledgeable and aware of GILS (see Table 4–10), revealed that their actual use of GILS is very low.

The survey asked GILS Conference respondents to indicate their uses of GILS in a series of True/False statements. A large majority of respondents neither use GILS frequently nor do they refer people to GILS for finding information. A majority (54%) do not find useful information when using GILS. Responses to these questions—especially given the nature of the respondents (i.e., primarily Federal agency staff with some interest in GILS)—raise the question as to the usefulness of GILS for these respondents. Table 4–11 summarizes the responses to these statements. 

Another perspective on GILS comes from transaction logs for searches and hits against Web servers. Most of the GILS implementations sit behind Web (i.e., HTTP) servers, and often are mounted as a database on a WAIS server. It is possible to capture search and retrieve transactions on both the WAIS and Web servers to obtain an indication of GILS use.  

GPO compiles and publishes a summary of monthly GPO Access GILS Usage Statistics (available from <gopher://gopher.cni.org:70/11/cniftp/pub/gils/forum>). Table 4–12 presents a summary of searches on its GILS site since April 1996. The numbers reflect searches against the GILS database, and GPO provides the following to indicate what these numbers include: "the database listed as ‘GILS’, represents searches performed when a user chose to search all agencies' records." 

With Table 4–12, one should note that the "Difference from Average Searches" (and similarly for retrievals) needs to be interpreted carefully because of the variance of minimum and maximum searches and retrievals. In addition, the average number of searches and retrievals reflects the strength of the extreme maximum and minimum outliers (i.e., April 1996 and December 1996, respectively) on the average. One can also look at these numbers for an indication of the trend of GILS usage on GPO Access.  

GPO provides statistics not only for its GILS database, but for all agency GILS databases it hosts, and a similar table could be generated for each of those databases. Individual agencies, as well as GPO, can use such statistics to analyze access to and use of their GILS records.

*Source read "1,260"; assumed correction as shown.
Note: GPO provides the following definitions of search and retrieval:
"A search is counted each time a particular database is queried. A retrieval represents a file actually being transferred onto a local machine, as opposed to frequently reported "hits" statistics, which represent each mouse click or change of a Web page."

The evaluation study also used log analysis procedures of HTTP server transaction files. Appendix E–4 summarizes this analysis, which was an exploratory procedure. The data reflect a two–week period of transactions on one agency’s server. Since the agency’s GILS database sits behind the HTTP server, the server statistics capture all HTTP transactions (i.e., transactions against all the resources, including the GILS database, that are accessible through the HTTP server). This configuration allowed the study team to estimate the amount of GILS usage as a percentage of total Web transactions on the agency’s server. Table 4–13 summarizes the analysis and indicates that GILS activities accounted for less than 1% of all Web transactions.

The data in Table 4–13 should not be compared to the numbers for GPO GILS searches in Table 4–12, as the two sets were collected in and reflect entirely different contexts. GPO’s data resulted from the context of searches against its WAIS server, while Table 4–13 reflects numbers of hits and accesses in the context of an agency’s HTTP server.

Further, the numbers in Table 4–13 should be interpreted with caution. The agency’s HTTP server provides access to a rich collection of documents and other online resources and services. One possible type of analysis would be to identify the "percentage" of resources on the HTTP server represented by the GILS records and then see if the percentage of GILS hits is commensurate with the percentage of resources represented by GILS. While it is possible to state from this log file data that GILS hits and accesses relative to overall server hits and accesses comprise a very small percentage, it would be unwise to conclude that the small percentage of GILS hits and accesses represents low utilization of GILS compared with utilization of other resources accessible via this HTTP server. Longitudinal data over months could, however, reflect whether use of GILS resources on the HTTP server is increasing, decreasing, or remaining steady. 

* A hit is any file from a web site that a user downloads. A hit can be a text document, image, movie, or a sound file. If a user downloads a page with 6 images on it, then that user "hit" the web site seven times (6 images +1 text page).
An access, or sometimes called a page hit, is an entire page downloaded by a user regardless of the number of images, sounds, or movies. If a user downloads a web page that has 6 images on it, then that user just accessed one page of the web site. 

As noted above, from the transaction log analysis carried out during a two week test period, hits to the one agency’s GILS constituted less than 1% of all hits to that agency’s HTTP server. The DTIC GILS locator page on their Web indicated some 34,000 hits during October, 1996. DTIC officials estimated that hits on GILS are also less than 1% of all DTIC Web server hits (the URL for DoD web statistics is: <www.dtic.mil/dusage/>). These individual agency estimates of GILS usage cannot be generalized to overall GILS use, but they do provide one indication as to its use.

The study could not identify any reports of GILS use by agencies except for the published GPO statistics and some basic HTTP log analysis statistics. This suggests that as part of agencies’ responsibilities in a refocused GILS, they will need to report as one performance measure an estimate of GILS use (in whatever ways they measure it).

One anecdote regarding use is especially instructive. The librarian at a Federal agency responsible for inputting GILS records told the investigators that she rarely if ever used the tool for identifying and accessing government information—especially since she knew the type of records that were being input! Interestingly, most agency participants in the study, including this librarian, agreed with the list of problems identified in the user assessment. They favored letting people first become familiar with it, completing an assessment (such as that reported here), and then deciding how to improve GILS.

The findings from the study also indicate that without a substantial investment of time and resources in education and promotion, the investigators question the extent to which people will become knowledgeable about GILS to say nothing of them becoming familiar with it and using it regularly. 

4.6.4. RECOMMENDATION: Develop and Formalize GILS Promotion, Education, and Training Strategies

The original GILS effort as outlined in OMB Bulletin 95–01 charged NARA with responsibility for training—primarily in the area of record creation and maintenance. NARA developed guidelines for record creation and offered a number of training sessions. The study found the need for more than training on creating records. In fact, the investigators found a need for an education and awareness program directed at agency management and agency implementors that would describe and explain how GILS could assist them and what benefits would accrue from participating in GILS. To do this, however, would have required a clearer articulation of the purposes and goals of GILS, which can be achieved in a refocused GILS initiative.

The investigators were unable to identify a government–wide marketing plan or program for the development and implementation of GILS. The investigators did, however, identify some informal efforts within some agencies

First, policy leaders should understand that education and marketing efforts have two very different target audiences: Federal agency staff and users/potential users of GILS. Findings from the study show clearly that neither of these two groups understand the purpose, importance, and potential benefits of GILS. The educational, training, and promotion objectives for each of these groups need to be customized both in content and in delivery. Further, the category of Federal agency staff includes different subgroups such as agency management and actual GILS implementors (and potential GILS users), and education and marketing for these subgroups may have different emphases.

Second, these promotional efforts cannot be planned and implemented until GILS policymakers articulate a clear, achievable purpose, define specific objectives, and agree to implementation procedures that would constitute a refocused GILS. A major problem that developed during the early period of Federal GILS implementation was that various GILS spokespersons oftentimes provided differing visions and purposes for GILS efforts.

In addition, clear lines of responsibility for these efforts need to be established. There are a number of options related to developing a coordinated effort. Assuming the CIO Council takes on (or is charged with) overall GILS development, a Council GILS Committee or interagency task group should have responsibility for a program of planning, implementing, and evaluating a promotional effort. The term program is used to stress the importance of this effort being ongoing and credible.

While there are numerous ways to cast a refocused GILS marketing, education, and training effort, such efforts require careful attention. A formal mechanism should be established to plan and implement a marketing, education, and training effort as described above. Specific objectives should be developed for specific target audiences: 

• Establish a procedure for an "official" spokesperson for the refocused GILS efforts with "official" oversight as to the content of news releases, brochures, etc.
• Document and demonstrate to government officials "best practices" implementation of GILS.
• Demonstrate to government officials specific benefits that will result from a refocused GILS implementation.
• Ensure that the public and more specifically, targeted user communities of GILS have accurate expectations of its products and services.
• Develop brochures promoting the use and importance of GILS.
• Encourage agencies to mount prominently on each GILS site standardized statements that clearly articulate the GILS mission, operability, limitations, and instructions for use. 

These objectives are illustrative only. Discussion among policy leaders will need to occur as to how best to plan and implement a program of promotion for a refocused GILS effort. Regardless of the approach taken, these efforts should stress the refocused GILS as a government–wide tool and not simply an agency–based tool. 

4.7. TRANSITIONING TO THE NEXT STAGE OF GILS DEPLOYMENT 

In Access America: Reengineering Through Information Technology, Vice President Gore states "Information Technology (IT) was and is the great enabler for reinvention. It allows us to rethink, in fundamental ways, how people work and how we serve customers" (National Performance Review and the Government Information Technology Services Board, 1997, p. 1). The original vision of GILS, while appropriate at the time, is in need of reengineering. The degree to which this reengineering process is tied to clear purposes and objectives for GILS will dictate the success of the refocused GILS effort. 

This chapter reported a number of findings concerning the "success" of GILS. These findings indicate that the GILS vision as outlined in OMB Bulletin 95–01 has not been reached despite some individual agency successes. Beyond that vision, however, the study found a desire to articulate a refocused GILS vision, more in keeping with the networked environment in which GILS is deployed. The refocused GILS builds on the basic architecture of decentralized agency-based databases of structured metadata records accessible via Z39.50. The refocused GILS is clearly an evolutionary step in GILS development. 

The findings and recommendations offered in this chapter cover a very broad range of topics and issues. These findings and recommendations describe an initial GILS implementation effort that has had mixed results as of this writing. But these mixed results provide a richness in lessons learned that can guide a refocused GILS effort. Indeed, one might suggest that an initial period such as that during 1995–1996 is inevitable when implementing a complex and multifaceted program such as GILS. The findings and recommendations, while important, may be less important than the resolve to learn from them and develop a clear path to the next stage of GILS deployment. 

Clearly there are issues yet to be resolved in refocusing GILS. Yet those issues are the catalyst to move GILS forward and continue to learn and improve subsequent efforts. There has been a significant amount of knowledge gained during this GILS implementation effort. This report, and more specifically this chapter, documents that knowledge and offers recommendations to build on this knowledge. The investigators believe that the notion of a U.S. GILS is still very powerful, one that if refocused appropriately has the potential to make significant improvements in accessing government information and managing government information resources. 

Chapter 5 will discuss the nature of this transition period, and offer some possible strategies for sequencing recommendations to deal with those that are most important. During this transition period, however, a carefully developed plan with individuals or agencies clearly responsible for project management of the transition is needed. The transition team that manages the transition will need to investigate a number of additional research questions that Chapter 5 outlines. Until some of these research questions are addressed, resolution of key GILS issues will be problematic. 

The knowledge gained from the initial GILS implementation is significant and useful. This knowledge, as outlined in this report, should inform the refocused GILS initiative. The investigators expect discussion and debate about the findings and recommendations offered in this chapter. Clearly, not everyone will agree with all the recommendations offered. More important, however, is that there is a clear and agreed upon vision of the refocused GILS effort and that careful planning guide the transition to the next stage of GILS. This chapter informs that transition process, and Chapter 5 offers some guidelines and research questions to be considered in moving to a refocused GILS.

Next Chapter
Previous Chapter
Table of Contents