About Methodologies

Secondary data is information previously compiled by another organization. Analysis of secondary data is based on drawing inferences from statistics. The underlying assumption of the approach is that it is possible to make useful estimates of needs and social well being by analysing statistics. In conducting an assessment there are two key types of data: social indicators and service utilization statistics.

These are descriptive statistics such as census data. The factors most commonly used as indicators are as follows:

• demographic and economic characteristics such as age, gender, ethnic origin, education, and family income
• social, behavioural and well-being patterns as they relate to substance abuse, crime, family relations, and morbidity and mortality rates
• general social conditions in which people find themselves, such as the quality of housing, access to services and overcrowding (United Way of America, 1982)

Service utilization statistics are also referred to as rates-under-treatment. The underlying assumption with these types of data is that people's needs are reflected by the use of demand for services (Warheit et al., 1984). Numbers of clients, client characteristics, referrals to the agency, registrations and waiting lists are types of service information that is often collected from agencies and organizations during an assessment.

• they are less invasive and intrusive
• they provide a good basis for discussion by serving as a reality check and raising non-obvious and/or difficult issues

• without information from other sources to back them up, assumptions made on the basis of secondary data can be risky (Edmonton Social Planning Council, 1988)
• the use of secondary data doesn't involve community participation and, therefore, is best used in conjunction with a community participation process to gather information

A survey is a formal systematic method of gathering information from a defined population in specified geographical area. This method of information gathering includes interview surveys, telephone surveys, and mailed (self administered) questionnaires. These are among the most common and reliable methods of obtaining data for a needs assessment.

Surveys are only successful when they are carefully designed and administered. The main steps to conducting a survey includes the following:

• choosing your survey approach
• designing a questionnaire
• pilot-testing the questionnaire
• distributing/administering the questionnaire
• collecting data
• analyzing data

The survey instrument should begin with a statement of purpose which includes why it is being done, who is conducting it and how the information will be used. The introductory statement should also ensure respondents of the confidentiality of their responses. Survey instructions and questions should be clearly stated and easily understood.

Unlike focus groups and key informant interviews which utilize open ended questions, survey questions should be followed by a check list or a rating scale so they are easier and less time consuming to answer, as well as easier to compile and analyze.

If surveys are the primary means of obtaining community perspectives, respondents should be selected using a random sampling technique to reduce the likelihood that segments of the population are left out. If surveys are to complement focus groups and key informant interviews then random sampling is some what less of a concern (Samuels et al., 1995).

Surveys should not be used to collect demographic information in and of itself as this can be done through other sources such as census information. Instead, demographic information gathered via surveys should be used to gain insight into the demographic characteristics of the respondents (i.e., age, income, gender, education, ethnicity) (Samuels et al., 1995).

The methods of dissemination and return of the survey are important to consider. Each of the three survey techniques including personal interviews, telephone surveys and mailed questionnaires have advantages and disadvantages as outlined below (Northern Development Council, 1989).

• they provide a better response rate than mailed surveys
• they provide better coverage of a more representative sample
• they minimize responses of "don't know" and "no answers" because the interviewer can probe for answers
• they allow clarification by the interviewer if respondents do not understand the questions; but this must be done according to strict specifications

• they are costly to administer
• they are time consuming to administer
• the personal safety of interviewers may be in jeopardy
• extra time must be taken to ensure that interviewers are properly trained in reading and recording the respondents' answers so that they are not biased

• they are less costly than interview surveys
• they are less time consuming than interview surveys
• respondents may be more honest about socially sensitive issues if they don't have to look the interviewer in the eye

• there will be sampling difficulties caused by the fact that some people don't have a phone and others have unlisted numbers
• there may be difficulty determining who should be interviewed at the telephone number called
• there is less opportunity to probe for answers
• there may be suspicion of respondents regarding the intentions of interviewer
• there may be poor cooperation by respondents
• this method is not considerate of people's time (i.e.,calls may be made be at inconvenient times for the respondents)

• they are the most efficient means of distribution
• they enable respondents to complete the survey when it's convenient for them
• the anonymity of mailed surveys often allows people to be more honest about personal issues
• they are less costly and time consuming compared to the other survey methods

• they often elicit a low response rate (usually 25 - 35% at best);
• they can be misplaced by the recipient; and,
• there is a lack of opportunity to examine complex issues with this method.

Key informant interviewing is an information gathering technique which uses open-ended questions and is conducted with people who have extensive first-hand knowledge about the community. Key informants either live and/or work in the community. They are normally selected on the basis of the longevity and nature of their involvement in the community. Thus, key informant interviews often include people such as a community health nurse, members of community agencies, long-time residents, a school principal, police personnel, local business people and local clergy. Typically 10 to 15 key community members should be interviewed (Siegel et al., 1987). Key informant interviews are best suited for two types of individuals including those who are uncomfortable participating in group discussions and those who may have a tendency to dominate group discussions (Samuels et al., 1995). Key informant interviews must include people who represent the diversity of the community in terms of characteristics such as income, political beliefs, religion and cultural background.

There are several steps to ensuring that key informant interviews are a success (Samuels et al., 1995). The following briefly describes these steps.

• Formulate a list of open-ended questions that should take no longer than one hour to answer. Otherwise, interviews tend to lose their focus.
• Pilot test the questions to ensure their integrity and clarity and to identify new questions to add to the interview.
• Develop a script for interviewers to use in requesting an interview with potential participants. The script should summarize the initiative (who is conducting the interview, why the information is being gathered, how the information will be used and how the information gathered may ultimately benefit participants of the process and the community).
• Develop a list of potential interviewees. Input from the planning group for the assessment process will help ensure that the list is representative of the community.

Contact the potential interviewees and solicit their participation using the previously developed script. If they agree, an appointment should be scheduled. A good technique to successfully solicit participation is to have a planning group member, who is acquainted with the potential interviewee, make the initial contact. Another effective strategy is to fax or mail the list of questions to interviewees prior to the interview as a reminder and so they can prepare for the interview.

Prior to the interviews taking place, all those conducting interviews to should meet to thoroughly discuss the questions. Everyone should have a common understanding of the intent of the questions as well as how to collect the information.

Possible key informant interview candidates include:

• politicians and other government officials
• police chiefs
• church officials
• school principals
• social workers
• family counsellors
• probation officers
• physicians
• leaders of community groups and volunteer organizations
• at-risk youth
• low-income parents (Samuels et al., 1995)

• this approach is relatively simple and inexpensive
• it permits the input of a variety of individuals
• it allows interaction for clarification, follow-up questions and probing
• it allows the interviewer to probe unanticipated issues
• participants can qualify responses
• the interviewer can monitor non-verbal responses
• the interviewer can acquire deeper levels of meaning, make important connections and identify subtle nuances in expressions/meaning
• this approach is one of the few methods which can be used to obtain data from children and people with low literacy
• the results are easy to understand
• this approach may help relationships among human service agencies

• this approach has built-in bias, since it is based on the views of those who would tend to see the community's needs and assets from their own individual or organizational perspectives
• the findings cannot be generalized to the entire population due to the small number of participants
• open-ended responses often make summarization and interpretation difficult, especially when participants modify or reverse positions
• the interviewer may bias results
• the data may be difficult to analyze and synthesize

The purpose of a focus group is to explore issues by soliciting opinions, anecdotes, experiences and impressions from a group of people. They usually do not aim to reach consensus, provide recommendations nor make decisions. To ensure success, several factors need to be considered.

Groups should be small enough for everyone to have an opportunity to share insights, yet large enough to provide a diversity of perspectives. Typically there should be 7 to 10 participants in each group.

Each focus group should have a skilled facilitator and a recorder, both of whom should be neutral parties and should be perceived this way by the participants.

In general terms, participants should be told before the session why it is being held, who the sponsor is and how the information will be used. When the topic of discussion is complex, participants may be asked to consider the issues prior to the focus group. In most cases, however, the specifics of the topic are introduced within the focus group session.

Providing child care and offering snacks or other incentives will enhance attendance and participation. Invitations from someone within the community who is known to the participants, will also improve participation.

A focus group meeting should begin with an introduction of participants, a statement summarizing the initiative (who is conducting the focus group, why the information is being gathered, how the information will be used and how the information gathered may ultimately benefit participants of the process and the community), an overview of the topics to be discussed and ground rules for discussion (Samuels et al., 1995). It is also important to thank the focus group participants for offering their time and knowledge.

Efforts should be made to ensure a representative mix of residents and other stakeholders in the community. Because familiarity tends to inhibit disclosure, participants shouldn't know each other. It's sometimes impossible to include only complete strangers in a group, but those that work together should not be in the same group nor should close friends. On the other hand, participants in individual focus groups should be selected on the basis of some commonality (i.e., adolescents, parents, seniors, business people).

The success of a focus groups depends not only on the quality of the questions asked but the breadth and depth of the answers. Open-ended questions should be asked and the facilitator should create a context with general questions prior to asking specific questions.

• they are quicker and less expensive than individual interviews
• they allow interaction for clarification, follow-up questions and probing
• the format allows the facilitator to probe unanticipated issues
• participants can qualify responses
• the facilitator can monitor non-verbal responses
• the facilitator can acquire deeper levels of meaning, make important connections and identify subtle nuances in expressions/meaning
• the format allows participants the opportunity to react to and build upon the responses of other group members
• data and ideas are uncovered that might not have been uncovered with individual interviews
• they are very flexible in that they can examine a wide range of topics
• they are one of the few tools that can be used to obtain data from children and people with low literacy
• the results are easy to understand

• the findings cannot be generalized to the population due to the small number of participants
• the researcher has less control compared to individual interview situations
• the results may be biased by a dominant or opinionated member of the group
• the more reserved group members may be hesitant to talk
• open-ended responses often make summarization and interpretation difficult, especially when participants modify or reverse positions
• the facilitator may bias results
• the approach requires trained facilitators with special skills; in working with groups
• the data may be difficult to analyze

A community forum is an open meeting for all members of the community. Its purpose is to provide a setting for members to express their opinions. Generally, forums last about three hours and may include a wide range of activities such as information exchange, communication of details about programs or projects, introduction of various community members and more general social interaction. However, the major function is to elicit views from as many people as possible. Incorporating small group discussions as part of a forum or meeting can be considered as a way of encouraging greater participation (Warheit et al., 1984).

• Steps for organizing and undertaking a successful community forum include the following:
• preparing key questions and an agenda to give some structure to the meeting
• finding an appropriate location to host a forum
• publicizing the meeting in ways which encourage attendance from all sectors of the community (Posters, letters, news announcements should include the purpose of the meetings, who is sponsoring them, as well as the time, place and date.)
• designating a moderator and recorders
• developing a written survey which can be distributed at the meeting to allow participants to express their own ideas regardless of whether or not they speak publicly
• tabulating and summarizing the information identifying needs/resources/barriers and highlighting priorities or items with strong consensus
• arranging transportation and babysitting, if at all possible, in order to encourage all members of the community to attend

• they are easy to arrange and inexpensive to conduct
• they can potentially provide input from many sources
• they offer an opportunity for dialogue between community members
• they provide those responsible for the assessment with an opportunity to identify individuals and organizations that may have an interest in getting more involved in the process

• they are often dominated by the most outspoken, or self-serving individuals
• attendance can be small and will not be representative of the community's entire population
• they can become a forum for airing personal complaints
• they may heighten expectations that cannot be met
• the information gathered is highly impressionistic; this method is not considered a highly reliable data gathering tool

The nominal group process is a method of decision-making in which individuals work in the presence of one another but initially do not interact. This technique is for planning/priority setting rather than information gathering alone. At its most basic level, community members vote for issues that they think are most important.

The votes are tallied and the issues are ranked on the basis of votes received (Samuels et al., 1995).

There are several steps to ensuring that the nominal group process is a success.

A selected group of community members are invited to attend the session.

A question or a series of questions are posed by a facilitator to small groups (8 to 10 people in each group) and each member is invited to write down brief responses to question(s) (Siegel et al., 1987). Questions may seek solutions to a problem or seek opinions. Allocate 10 - 15 minutes for members to write down their individual response.

All ideas are shared with the group in a round robin format, one idea or response at a time. At this time participants are asked to refrain from discussing the ideas generated. Responses are recorded and displayed on a flip chart with every effort to record the ideas exactly. Depending on the questions asked, this procedure may take 1 to 2 hours (Seigel et al., 1987).

Following the round robin process, there is a discussion period where ideas are added, deleted or combined and participants are free to clarify and elaborate. Participants are not asked to defend or substantiate an idea.

Once the list is clarified through discussion, each participant votes for the five (or more) ideas they consider as the most important. This is done individually and anonymously.

Votes are "tallied "and a group rank ordered list is achieved.

• it solicits minority or unpopular viewpoints;
• it is non-competitive;
• it combines the advantages of individual survey with group discussion;
• it encourages innovation and pooling of resources of the group;
• all participants work and contribute to the process; and,
• it is particularly useful in a heterogeneous group as the process doesn't allow one person to dominate (Siegel et al., 1987).

The disadvantages of the nominal group process are as follows:

• it is an imprecise data collection tool because votes/ranking are made without thorough or careful sorting out of all ideas; and,
• some participants may feel manipulated by the highly structured format.

Note:

For the priority setting component of the community assessment process an abbreviated version of nominal group process is suggested. Community members will have already had an opportunity to provide their perspectives about community issues during the information gathering phase. Consequently, rather than seeking opinions about the core issues, the nominal group process simply begins with a presentation of what the issues are.

Steps for this nominal group process include:

• presentation of findings regarding each issue
• answering any questions raised by the group
• voting on the importance of each issue
• tallying votes
• presenting the results of the votes
• categorizing each issue requiring immediate, short-term, or long-term action (Samuels et al., 1995)

This process is a non-interactive group process which allows for systematic solicitation and collation of informed judgements (Siegel et al., 1987). The Delphi Technique is usually composed of a set of carefully designed sequential questionnaires. With each subsequent questionnaire, information and feedback from earlier questionnaires is provided and in this way the process takes the form of a structured dialogue between persons who do not meet but whose opinions are valuable to the issue at hand. This method has three distinguishing characteristics which are:

1. participants respond anonymously to the questions
2. there is controlled feedback of the various stages of the information collection process
3. there is statistical analysis and formulation of collective responses.

This technique is more of a planning/priority setting tool rather than a tool which is used for information gathering.

The Delphi Technique consists of five steps:

• a questionnaire is developed concerning a key issue or set of issues
• questionnaires are distributed to key individuals, usually by mail
• questionnaires are returned and the results tallied to determined areas of agreement and disagreement
• when disagreement occurs, a second questionnaire containing the various reasons for the initial judgements of participants is distributed
• the above steps are repeated until agreement can be reached (Siegel et al., 1987)

• participation is anonymous and, therefore, one person cannot dominate the group
• the negative influences of individual vested interest is controlled because feedback is controlled in a systematic manner
• anonymous participation reduces the group pressure to conform
• the process allows for efficient use of the participants time

• there is a lack of certainty in guidelines on its use or design
• extreme positions may be dropped in order to get agreement on an issue which may result in the loss of creative ideas