APPENDIX C-2
FOCUS GROUP METHODOLOGY
Table of Contents
1.0. Introduction..............................................................................................................1
2.0. Focus Groups as a Research Technique in Support of Project Goals..........................1
3.0. Selection of Focus Groups........................................................................................1
4.0. Selection of Focus Group Participants.......................................................................1
5.0. Focus Group Demographics......................................................................................2
6.0. Focus Group Data Collection and Data Analysis Activities.........................................2
6.1. Content Codes Used for Data Analysis...............................................................3
6.1.1.
GILS Related Codes................................................................................3
6.1.2.
Policy–Related Codes..............................................................................4
6.1.3.
Codes about Users..................................................................................5
7.0. Limitations and Conclusion.......................................................................................5
7.1. Limitations.........................................................................................................5
7.2. Conclusion.........................................................................................................5
Table C2-1 Focus Group Participant Demographics........................................................2
Table C2-2 Focus Group Participant by Type of Work...................................................2
1.0. INTRODUCTION
The investigators conducted seven focus group sessions designed to elicit perceptions from various stakeholder groups involved with GILS. These focus groups occurred throughout the first six months of the study. A total of 84 participated in these sessions. Table C2–1 summarizes the types of participants at each focus group. These focus groups were in addition to focus groups conducted as part of the agency site visits.
2.0. FOCUS GROUPS AS A RESEARCH TECHNIQUE IN SUPPORT OF PROJECT GOALS
Focus groups brought together GILS stakeholders who shared interests in common themes (e.g., public interest, technology). The focus groups provided opportunities to explore shared beliefs and goals with respect to GILS. The investigators included selected individuals at each focus group to ensure content–rich qualitative data from perspectives that would encompass the range of user and stakeholder beliefs and concerns (Krueger, 1988). The investigators chose participants in the focus groups to meet the information needs represented by the evaluation framework’s five dimensions and three perspectives.
3.0. SELECTION OF FOCUS GROUPS
The seven focus groups, dates they were held, and the number of people attending are as follows:
The investigators selected these seven groups to provide opportunities for discussions about GILS implementations with identified categories of stakeholders. Four focus groups represented GILS policy activists and public users, two groups were agency staff, and one group consisted of GILS implementors at governmental agencies other than the Federal level.
4.0. SELECTION OF FOCUS GROUP PARTICIPANTS
The investigators held the first focus group in the North Texas area by interviewing a group of government document librarians at area public and university libraries. The investigators developed focus group questions for this group to assess their awareness and use of GILS. This group of users were intermediaries between the general public and the Federal agencies in that they regularly assisted users in finding government information and would likely be a group for whom GILS would have significant value.
The investigators arranged the next five focus groups at the November 1996 GILS Conference. Conference planners provided the investigators with names of pre–registrants. Investigators analyzed the list of names as to their common characteristics relative to GILS and selected individuals whose responsibilities or expertise matched with information needs of the evaluation study. The investigators invited 12–15 people for each focus group, recognizing that the actual focus group might end up being smaller in number.
The final focus group with the Small Agency Record Officers and IRM Representatives consisted of stakeholders who were involved with GILS implementations at agencies whose staffing resources and information resources were of a significantly smaller size than the site visit agencies. This group’s experience with GILS provided the researchers with additional perspectives from the agency and government–wide perspectives of the evaluation framework. The Small Agency Council assisted in arranging this session.
5.0. FOCUS GROUP DEMOGRAPHICS
These focus groups reflected a purposeful sample of GILS stakeholders. The investigators interviewed a total of 84 participants in the focus groups. Of the 84 participants, 68 completed a profile sheet. Tables C2–1 and C2–2 provide summary demographic information of the focus group participants.
Table C2–1
Focus Group Participant Demographics
|
Name of Focus Group |
Number of Participants |
Average Years of Experience in |
| North Texas Government Documents Librarians |
|
|
| Public Interest |
6 |
5.5 |
| Future Issues |
11 |
4.9 |
| Record Managers |
6 |
5.5 |
| State/Local |
9 |
4.1 |
| Vendor/Technologists |
11 |
3.4 |
| Small Agency Council |
15 |
6.1 |
Table C2–2
Focus Group Participant by Type of Work
| Type of Work (Job Title) |
Number of Participants in All Focus Groups |
| Administrators/Managers |
6 |
| Academics |
1 |
| Analysts |
4 |
| Archivists |
4 |
| Librarians |
16 |
| Lobbyist |
1 |
| Other |
6 |
| Public Affairs |
1 |
| Record Managers |
14 |
| Systems/Computer Specialists |
15 |
6.0. FOCUS GROUP DATA COLLECTION AND DATA ANALYSIS ACTIVITIES
Participants interviewed during the focus group sessions completed a profile sheet which included quantitative and qualitative questions related to GILS. The profile sheet asked respondents to assess, in a quantitative manner, familiarity with GILS policy documents. Focus group participants used a Likert–type scale (from 1–5 in which 1 indicated Very Familiar and 5 indicated Not Familiar) to assess familiarity with GILS policy documents.
The investigators developed separate profile sheets for each of the focus groups in order to match the information needs of the research project with the various stakeholder groups. The profile provided the investigators with assessments about participant knowledge of GILS policies and attitudes, as well as qualitative information concerning expected user benefits, lessons learned, and perceived barriers or threats to the success of GILS. Included on the profile sheet was a question which asked respondents to identity a favorite online alternative to GILS when trying to locate government information and the reason why this particular alternative was a favorite. These questions enabled the investigators to identify online sources of government information which were used in addition to GILS and would reflect a user–based choice for accessing Federal government information.
Prior to each of the focus groups, the investigators developed interview questions for each session which were pretested by investigators and selected participants. These questions guided the interview process, though the investigators varied from the focus group protocols when interviewee responses opened new avenues for data collection. Focus groups lasted from 45 to 90 minutes. Each focus group had facilitators and recorders. The recorders at each session wrote a detailed description of comments made by participants and an analysis of the issued discussed.
The investigators created a database from these seven summaries and used database management software to manage the data collected. The investigators defined a set of coding categories based on the actual data; the evaluation framework sensitized the investigators to the broader categories. The coding factors represented content found within the narrative summaries. Specific coding categories included categories for GILS Issues, Information Policy Issues, and Users. The investigators used coding as a means of analyzing the data obtained from this data collection technique. Once analyzed, the coding scheme provided a data reduction technique for project researchers. As a result of this analysis, investigators were able to query the database for specific incidents of particular factors without losing the ability to focus on the data content from a holistic perspective.
6.1. Content Codes Used for Data Analysis
There were three general categories of codes developed for the focus group database. These three categories centered around GILS, the policies associated with GILS, and users of GILS.
6.1.1. GILS Related Codes
These codes identify GILS–related issues.
GILS–Alt: Alternatives
This code is used for specific references to Web sites that are alternatives to GILS.
GILS–B: Benefits
This code is used for statements about GILS benefits (as opposed to negatives). It includes stated advantages or positive outcomes as a consequence of GILS.
GILS–Cham: GILS Champion
This code describes someone who is a GILS champion or identifies for the need for such. It includes references to individuals who have acted as product champion. This code may also include reference to the lack of a champion.
GILS–Def: GILS Definition
This code includes comments about how users define GILS and what users think GILS is.
GILS–FDLP: Federal Depository Library Program
This code includes comments about the role/relationship of depository libraries to GILS.
GILS–IR: Information Retrieval
This code includes aspects of GILS which lend themselves to information retrieval. The code includes both functional/mechanical and general design aspects of GILS for IR value. It also includes comments as they pertain to the value of information content within GILS for public use.
GILS–N/P: Negative Aspects and Problems
This code includes comments about GILS which were of problematic nature and included real or perceived negative consequences of GILS implementations.
GILS– P: GILS Purposes
What is the purpose of GILS? This code includes agency and end users understanding in this area. It includes misperceptions of purposes as well as intended GILS purposes. If the comment is primarily directed at the confusion caused by OMB as to the purpose of GILS, the code P–OMB is likely to be preferred. This code is general in scope.
GILS–WEB: GILS Web
The code includes comments about the relationship between the Web and GILS. It includes agency staff and user comments about the value of Web pages and GILS.
6.1.2. Policy–Related Codes
These codes identify policy issues.
P–AG/IAG: Agency Issues and Interagency Issues
This code includes implementation issues across agencies, AILS, agency–related aspects of GILS policy, decentralized,/centralized issues about GILS, and the question of who is in charge among the agencies (or lack of interagency leadership). Other codes are used for specific issues such as record management or public access if the comment about agency activity is limited to that topic.
P–F/NS: Future or Next Steps
This code includes comments relating to desired next steps or future actions with respect to GILS and general future needs from a policy perspective, Discussions limited to more narrow improvements are classed under U– Improvements. This code includes both desired future actions and limitations on future action.
P–GILS: GILS Policy
This code includes general comments relating to GILS and a multitude of policy issues too general to be coded elsewhere. It includes both positive and negative aspects of the issues. Generally other codes are preferred if they are more specific.
P–GPO: Government Printing Office
This code includes comments relating to the role of GPO with respect to GILS implementation efforts.
P–INTER: International
This code refers to international uses of GILS.
P–IRM: IRM
This code includes policy aspects of technology, information management, IRM, life cycle management of information, and CIOs .
P– MAR: Marketing
This code includes the need for GILS marketing both to the public and to the agencies. It includes the lack of marketing as a policy problem.
P–NARA: NARA
This code includes specific comments about NARA’s role with GILS. General comments about record management are coded elsewhere.
P–OMB: GILS and OMB
This code include specific references to OMB’s actions or lack of actions with respect to GILS. Anger directed at OMB and confusion about GILS purposes directed at OMB are coded here.
P–PA: Public Access
This code includes general information about the role of the public’s access to information and the use of the phrase public access as an issue. It includes both negative and positive support for public access to agency information.
P–RC/RC: Record Content And Record Creation
This code includes comments both about record content and record creation, comments about the nature of information data within GILS records, discussion about levels of granularity, decisions regarding which records to create in GILS, and discussions about who handles record creation. It includes issues connected with the use of metadata records unless the emphasis is on metadata as a standard.
P–RM: Records Management
This codes comments about policy with respect to GILS and record management.
P–STAND: Standards
This codes includes issues about both technology and metadata standards.
P–ST/LO: State and Local
This codes includes comments predominantly made in the state and local GILS implementors focus group. It provides an review of GILS implementations at these levels and attitudes about implementing GILS among this group of stakeholders.
6.1.3. Codes about Users
These codes identify user issues among the focus groups.
U–EVAL: Users–Evaluation
This codes includes judgmental views of GILS made by users or judgmental views of GILS as GILS pertains to users. It does not include descriptive views.
U–EXP: Users–Experience
This code includes descriptive information about the experiential use of GILS. It is not evaluative, and may make mention of time or number of GILS records examined
U–IMPROVE: Users–Suggested Improvements
This code includes specific suggestions as to needed improvements to GILS. These references are more specific than the P–F/NS code and pertain to users.
U–TECH: Users– Technology Aspects
This code concerns user technology needs when accessing GILS.
7.0. LIMITATIONS AND CONCLUSION
Because of the role that stakeholders hold within the context of GILS, participants at the focus groups actively aided the investigators by providing insight and analysis appropriate for data collection. An advantage of using focus groups for this type of evaluation is that individuals are generally interested and pleased to provide their opinions.
7.1. Limitations
The investigators identified two limitations connected with the use of focus groups in this project. Some participants in focus group sessions asserted their positions emphatically causing the focus group facilitators difficulty in balancing the contributions made by such individuals compared with others in the group. The goal of the sessions was a group discussion rather than a discussion focused on the interests of one or two individuals. Secondly, the focus group facilitators were aware that existing political or administrative relationships which existed between individuals outside the focus groups caused some participants to self–censor remarks given the presence of others in the group
7.2. Conclusions
The use of focus groups provided the investigators with access to groups
of individuals who were key stakeholders in GILS. The investigators benefited
from the opportunity to reach such a significant number of these people
at the GILS Conference. The focus groups complemented the site visits conducted
during the study in that the site visits predominantly represented opportunities
to gain information about agency implementation while the focus group sessions
provided opportunities to learn about GILS from users, policy makers, vendors,
state and local GILS implementors, and librarians.