The idea for this Guide originated during research meetings in Africa and, in particular, in the course of conversations with Prof. Gabriel Mwaluko of the Health Systems Research (HSR) Programme, Prof. Stephen Kinoti, then working at the Commonwealth Regional Health Community Secretariat (CRHCS) in Arusha, Dr. Luis Sambo of the World Health Organization’s Africa Office, Dr. Yvo Nuyens of the Council on Health Research for Development (COHRED), and Prof. Raphael Owor of the Essential National Health Research (ENHR) Africa Secretariat.

The need for a practical booklet was confirmed by discussions held at the international workshop, “Linking Applied Research with Health Policy,” organized in 1996 by the Applied Diarrheal Disease Research (ADDR) Project, in collaboration with other international agencies.

We would like to give sincere thanks to the following people who took the time to read earlier drafts of the Guide, giving invaluable guidance and suggestions: Stephen Kinoti of the Lesotho Highands Development Authority (LHDA), Gabriel Mwaluko of HSR, Raphael Owor of ENHR, Jon Simon of the Child Health Research Project, Mary Bachman of Africa Consultants, Inc., Carol Baume, Joe Destefano, and Valerie Uccellani at the Academy for Educational Development, and Judy Brace, Sambe Duale, Ellen Piwoz, and Peter Spain of the SARA Project.

Special thanks must go to Hope Sukin, the Manager of the HHRAA Project at USAID Africa Bureau’s Office for Sustainable Development. Her vision and encouragement has been key to the development of this Guide, as well as other SARA products.

Foreword
Stage 1: Defining the Research Question
Stage 2: Developing the Proposal
Stage 3: Conducting the Study
Stage 4: Communicating Research Results
Conclusion
Bibliography
Annex—Selected Resource Materials on Participatory Research and Evaluation

Foreword

This booklet is a guide for researchers who aim to have an impact on policy and program decisions. It is intended for researchers attached to government services and researchers located in academic institutions, as well as researchers working as consultants in the private sector.

The approach presented here is designed to help seasoned researchers to achieve greater impact from their own work. The Guide also can be used as a teaching tool with students and younger colleagues. In addition, the Guide should help relatively new researchers to select research questions and conduct studies with program and policy relevance. Although the examples used for illustration purposes are drawn mainly from the health sector in Africa, we hope that researchers working in other sectors and in other regions of the world will find useful ideas in the Guide.

There is a clear need for good quality research in order to improve policies and programs in all sectors. It is sometimes thought that lack of funding is the main constraint to truly useful research. However, a closer look reveals a more complex problem. Money and energy is being spent on research that is not relevant to practical decisions, and even when research is relevant, it is often ignored by decision makers. The waste involved cannot be afforded by most countries.

As illustrated in the figure on the following page, the basic premise of this Guide is that research informs policies and programs most effectively when there is an extended, three-way process of communication linking researchers, decision makers, and those most affected by whatever issues are under consideration. The traditional audience for most researchers is other researchers. Yet to have an impact outside our own research communities we have to learn other points of view and other ways of communicating. Better communication can increase the relevance of research to potential users and improve the chances that research findings will be heard and acted upon.

A number of practical recommendations for the design and implementation of research projects flow from this proposition. These aim to facilitate a process of communication at key moments in the research process—from the initial choice of research problems to the presentation of final results. Effective communication is a conversation that takes place over time and allows for a sharing of interests, needs and concerns on all sides.

Communication often evolves more or less informally as the work of research proceeds, but it is better to plan ways to encourage it systematically from the beginning.

Most of the considerations that follow have been treated by others elsewhere, yet they are widely scattered throughout the literature of several disciplines. This booklet simply brings many of these ideas together.

The Guide is organized around four basic stages in the research process:

Stage 1: Defining the research question
Stage 2: Developing the research proposal
Stage 3: Conducting the study
Stage 4: Communicating research results

These four stages provide a framework for organizing a series of recommendations intended to focus researchers’ attention from the outset on the desired end-use of the research.

The Guide is not meant to be a handbook on research methodology, or on techniques of data collection and analysis. Although it offers some ideas on research planning, on the choice of research methods, and on the presentation of research products, the objective is always to generate information that is usable and to increase the likelihood that it will be used.

In summary, this booklet examines how researchers can:

• involve decision makers, program managers, front-line workers, and intended beneficiaries in identifying research needs, defining research questions, and reviewing research proposals and plans;
• develop a relationship of trust with those involved in policies and programs by:
  • taking an interest in their work,
  • understanding their problems and constraints,
  • respecting and being responsive to their points of view;
• use similar methods to establish trust with communities or beneficiaries”;
• involve program staff and community members whenever appropriate in carrying out data collection and analysis;
• review the progress of the research periodically with managers, health workers at implementation level, community representatives, and other “stakeholders,” people who have a stake in the research and will use it or be affected by it;
• involve managers, staff, and communities in drawing conclusions and formulating recommendations;
• focus research reports specifically on program and policy problems and concerns;
• suggest specific actions based on research findings;
• prepare different types of reports and presentations for different audiences;
• keep reports and presentations brief; use direct, non-technical language; and package them attractively; and,
• use time effectively and strategically—respect deadlines set by decision makers and managers.

Stage 1: Defining the Research Question

KEY STEPS
Research will be relevant to policies and programs if, in defining the research question, the researcher takes the following steps to involve the potential users, or stakeholders:

1. Reviews published and unpublished literature on the research topic;
2. Identifies the potential users of the research;
3. Makes connections with these potential users;
4. Finds out how they define the issues;
5. Engages them directly in helping to define the research question.

“Defining the research question” has to do with choosing what to study and how to talk about the research concerns. If research is to be relevant to the people who make decisions about policies and programs, and to communities, it must address the problems they face and help them in their search for solutions. Clearly, the way in which they define problems in the first place has considerable bearing on what they believe to be plausible solutions.

Most researchers begin with an idea or general topic area that they wish to study. However, the process of moving from a general topic to a definition of specific research questions is often carried out in relative isolation, without involving those who the researcher hopes will eventually use his/her work.

To ensure that research is on target—relevant, helpful, and practical—it is important to begin to communicate with stakeholders when the researcher is beginning to think seriously about research issues and questions. If, for example, the resulting research products are to be useful to health system managers and decision makers, their information needs must be part of the research from the start. This is when the process of communication between researchers and decision makers should begin.

There may have been an ENHR (Essential National Health Research) priority-setting process in the country of study. If so, the documents pertaining to this process should be extremely useful in laying out the points of view of different groups on priority research issues. These will, of course, be most useful if they have been defined or updated recently.

The first step in defining the research question is to look at the issues relevant to the topic that currently concern decision makers at different levels.

In a growing number of African countries, organized attempts to formulate a broad health research agenda are already under way. The Essential National Health Research movement is bringing health researchers, policy and program managers, health care providers, and community representatives together to identify priority research problems. One of the overall aims of this activity is to establish and promote a national research agenda that:

• pays particular attention to the most vulnerable population groups (women, children, the poor);
• strengthens the links between research, action, and policy;
• cuts across and brings together diverse disciplines and social sectors.

It is at this early stage that the researcher must think seriously about how to involve the people—health workers, communities, other groups of individuals—who may be most affected by the policy or issues that the research is to address.

Sometimes, even decision makers and managers themselves do not have a clear sense of what research questions should be asked and, indeed, what issues are researchable, but they are aware of their operational needs and constraints. In dialogue with researchers, managers and communities can articulate these needs in terms that research can address; through this dialogue, researchers can understand how to design the research to be most useful.

Reviewing the literature
A good place to begin is with the relevant existing research and literature written in journals, monographs, government reports, research agendas, abstracts of professional association meetings, news articles, newsletters, etc. Although “grey” literature—concept and position papers, program evaluations, and other unpublished reports in circulation that touch on program issues and field experiences—are often given less attention, they can be especially valuable in helping to identify how problems are being defined by those involved in decision making, program management, and implementation.

Begin with the most recent publications and work backwards. It is important to read the literature for what it says about technical issues and problems, but also for what it says about the views and interests of other researchers as well as decision makers, program managers, health workers in the field, and communities.

The literature can take a researcher only so far. To identify more immediate issues and problems and, in particular, the up-to-date preoccupations of decision makers and program managers, it is best to speak with them directly. The ways in which issues and problems in the topic area are defined and debated by potential users offer the most important clues about what kinds of research questions to address and what type of research products are most likely to be used.

Identifing the potential users of the research
“Potential users” can be anyone in a position to make a decision or alter policies and activities in response to new information. Users can be at different levels of the health system, and range from policy makers and managers to program implementors and community leaders. Categories of users will depend on the topics studied and the location of the research.

To identify potential users, it is important to be systematic in mapping out who these individuals and groups are. This will entail identifying those at different levels of the health system involved in decision making and implementation, as well as those at the community level who may be affected by decisions within the topic area.

The following questions will help to identify the key “users”:

• What organizations are working in the topic area?
  • government—planning and policy units, at both the ministry and regional levels
  • private organizations and practitioners
  • university departments
  • nongovernmental and voluntary organizations
  • community-level organizations
  • media—news organizations
  • donor agencies
• Which organizations might become interested in the topic if appropriately approached?
• Who, in these various organizations, is most directly involved in managing the issue or problem?
• Who would be involved in implementing any changes at the program level?
• Which communities or individuals are most affected and/or concerned by the issue?
• What are the relationships between relevant organizations and individuals?

Though it may not be possible to contact all the people identified as potential users of the research, this process should give a clear idea of who is involved in the topic area. Then the researcher can locate, interest, and interact with them directly whenever possible.

Making connections
It is probably easiest to start with immediate professional colleagues, and to ask them for referrals to relevant others. Begin with individual researchers, field workers, community members, program managers, and decision makers and then move outwards to groups directly affected by issues to be raised in the study. In some instances this exercise may forge new connections. In other cases established networks may be tapped.

This kind of networking and consultation has a number of important benefits.

• It provides a more complete map of the ‘community’ of professionals engaged, in some important way, with the issues.
• It helps a researcher to understand more clearly the constraints under which potential users are working, and to determine what aspects of policies and programs they have the authority to change.
• It can point to potential collaborators, sources of funding, and other institutional support.
• It can help locate relevant literature—especially “grey” lierature—and other documentation useful for developing research design. (Libraries and information centers are, of course, very useful here.)
• Most significantly, it helps the researcher understand the current ideas, issues, and trends that touch upon the issues to be studied.

If the research is focusing on program issues, limiting interviews to just researchers and high level managers will provide only a partial perspective. It is also important to consult with middle-level personnel, health workers in the field, and members of the community in defining the research question.

In the Gambia: Oral rehydration therapy (ORT) use rates declined after a series of intensive educational efforts. Researchers and program managers at the central level developed several hypotheses about mothers' resistance to ORT, such as forgetting mixing instructions. However, by talking to field workers and community members, researchers identified another problem that had not been recognized—the Julpearl bottles that had served to measure out a liter of water were no longer widely available in the community. One important focus of the research therefore became an assessment of the availability of alternative liter measures.

P. Sankar. Qualitative Study of the Determinants of Oral Rehydration Therapy Use in The Gambia. 1986.

These groups usually have intimate knowledge of operational issues and can be particularly helpful in sharpening problem definitions and suggesting possible courses of corrective action. Health workers and others who are implementing programs have an understanding of problems that managers more removed from everyday practice may miss completely. Community members also may have quite different perspectives on problems, constraints, and issues that concern them directly. These various points of view will all be important when refining the research questions.

Finding out how these potential users define the issues and problems of greatest interest to them

The interviews with people and groups that have been identified should be informal, open-ended, and flexible, yet they should systematically cover a core set of topics. In order to focus discussions, it is advisable to develop and refer to a short list of directed questions to guide the conversation. This kind of interviewing requires preparation, and is very much like using focus group discussion methods.

People actively engaged in policy and program decisions bring a particular set of vested interests to the management or administrative process, and may feel that they are jeopardizing their position by revealing too much. Information is power. Community authorities, members, and groups also may have their reasons for filtering the information given to outsiders. Still, this kind of interviewing remains the best way to gain an understanding of ongoing policy debates and program issues. The challenge is to build a level of trust and confidence that enables the researcher to have discussions that are as frank and open as possible.

At this exploratory stage, it is important to understand the topic area from the point of view of the actors involved at all levels of policy and programs. This understanding will help ensure that the research is relevant. In addition, the process of making connections with potential users should encourage them to take an active and supportive interest in the research.

Research that more clearly defines the dimensions of a problem and identifies its underlying causes can be a catalyst for change, even if the researchers themselves do not recommend a specific course of action.

In Ghana: A study of government health services revealed widespread patient unhappiness about staff attitudes, lack of privacy, length of waiting time, and absence of drugs. Although the research did not offer specific recommendations for solving these problems, simply documenting them was enough to make a difference. The study resulted in a conference of regional directors of health services who introduced a series of practical and sensible policy changes based upon their knowledge of the situation.

International Health Policy Program. Policy Brief. 1996.

When establishing connections with these individuals and groups, a more conceptual task also must begin—moving from general topics and issues to a more specific definition of the research problem or issue. Seeing how others define the problem makes it easier to determine how research interests best fit into ongoing processes of policy making and program management.

Specific questions that may assist in interviewing include:

• What are the origins or causes of the problem as they see it?
• How do they talk about and prioritize specific issues?
• How serious is the problem, what are its consequences?
• How widespread is the problem? Is it getting better or worse? What are the relevant indicators?
• What populations or groups are most affected?
• What is being done to solve the problem? Are solutions (really) being implemented? What impact are they having?
• What are the obstacles and constraints to implementing effective interventions?
• What questions are being asked?
• What research has been done/is known about/is planned to answer these questions?
• Are alternative or competing solutions/interventions under consideration?

These questions will vary, depending on whether the issue under consideration is programmatic or policy-linked.

Also, the language used to formulate the questions and discuss related issues should vary according to the person being interviewed. Listening attentively to how each person responds to initial questions will help to establish a good rapport, adopt an appropriate tone, and adjust the language to the level and orientation of the person. It also lets the person understand that the information that he/she is giving is valued.

Engaging the potential users of the research in defining the research question

Contacts and interviews will enable the researcher to understand how different stakeholders view the issues of concern. Once good communication and good rapport have been established with individuals and/or groups from the different categories of potential users—decision makers, managers, field workers, communities—the researcher can also share with them how he or she is starting to define research questions and solicit their input directly.

In Senegal: regional, district, and health center personnel participated in the design of a community nutrition intervention to improve infant feeding practices in the Fatick Region. Discussions with these field workers helped to identify personnel and groups in the community with the most potential as educators. The operations research design was then built around the field-workers’ sense of which interventions would be most feasible for them to try out and later replicate.

S. Diene, “Use of Participatory Approaches to Design, Implement, and Evaluate a Sustainable Community-based Nutrition Education Delivery System in the Fatick Region of Senegal.” Ph.D. dissertation, Cornell University, 1995 and PRITECH Reports, 1994.

No single one of these various audiences should necessarily have the determining voice in defining the research question. Indeed, sharing some of the different perspectives on problem definition among individuals and groups may be valuable for all concerned and contribute greatly to formulating a comprehensive and balanced sense of the problem. Research questions derived through such a process will have a greater chance of being considered important by potential users at different levels.

The Primary Health Care Operations Research (PRICOR) Project supported workshops in several selected African countries to develop operations research projects. Participants included researchers and the health managers working in the programs where the research would be conducted. The health managers stayed for the first week of the two-week workshop and collaborated in the design of all but the technical aspects of the proposals. Their participation greatly strengthened the relevance of the proposals and the likelihood that the studies would be implemented and the results used.

A. Brownlee. A guide for planning training and research programs in health systems research. 1986.

Stage 2: Developing the Proposal

KEY STEPS
Research will be more relevant to decision makers if, in developing the proposal, the researcher:

1. Clarifies at the outset what decisions are to be influenced, and who the potential users are;
2. Chooses the appropriate research methods to inform those decisions;
3. Chooses research implementation strategies that involve potential users of the findings;
4. Includes a research dissemination plan in the proposal;
5. Decides who, among the potential users, should be involved in reviewing the proposal.

A good research proposal will not only help to obtain funding and academic approval; it is equally important as a tool to involve the potential users of the research.

Clarifing the decisions that the researcher wishes to influence, and who the potential users are

If research is to serve as a guide to policy or program management, it must concentrate on those policy and program variables that can, in principle, be measured and acted upon. Proposals that fail to convince sponsors that they will lead to practical outcomes are not likely to be funded. Research that suggests ‘solutions’ that are impossible to implement will simply be ignored—and with good reason.

Here the researcher can refer to the potential users that were identified when defining the research questions, and name in the proposal the individuals and groups at all levels that have indicated interest in using the research.

Choosing the appropriate research methods to inform the decisions that the researcher hopes to influence

The likelihood that research findings will be used is also directly related to the credibility of the research process. This includes its perceived accuracy and objectivity, as well as the appropriateness of the research design and methods.

High-quality research—that is, research that meets the scientific criteria of the chosen discipline—enhances credibility. But high quality does not in itself guarantee credibility or use. Other factors, such as whether findings are comprehensible to potential users, relevant to their problems, and timely, may well have a more direct bearing on their use.

In Burkina Faso: To document the pharmaceutical distribution system and the use of drugs for diarrheal diseases at the clinic level, a limited number of health facilities at different levels of the health system were visited. Their records were examined to establish prescription practices, stock outs, etc. The data collected was not statistically representative of the whole health system, but the information generated was sufficient to be fed back to drug supply system managers who were able to make recommendations for action to address many of the problems that the study identified.

C. Geslin and M. Zabramba, Etude sure le circuit de distribution des sels de réhydration orale au Burkina Faso. 1989.

Timing can be particularly crucial. Planning and budgeting cycles often demand that many kinds of policy and program decisions be made according to set deadlines. If research results are not ready at the time a decision is to be made, then the research may not be used.

Time constraints also can have very real implications for research design by affecting sampling plans, the selection of data collection instruments, and the use of specific analytic techniques. Research designs that offer greater precision may have to give way to designs offering more timely results if research is to be available on time for decision makers. When confronting such a trade-off, the researcher may wish to consider breaking the project down into smaller, more discrete studies that can be completed rapidly—even if they do not cover all the research issues as exhaustively as one might prefer. It is important to keep in mind that complexity does not necessarily correspond to credibility.

In Nigeria: Instead of a full health-facility survey, a small sample of 20 or 30 health workers was observed for a day each to gather information on the types of interaction that was taking place between health workers and mothers. This information was sufficient to make recommendations for the content and orientation of a training course to improve these interactions and, therefore, use of services.

Data collection methods that provide complete population coverage or employ statistically representative samples yield more precise information, but are often much more costly in terms of money, time, and labor. Approaches to data collection that make use of non-random samples are not as precise, but they are less costly and can generally be implemented more quickly. For example, rapid assessment methods such as focus group discussions, key informant interviews, structured observations, and participatory appraisals are increasingly being used in policy-related research. By capturing perspectives of key stakeholders and illuminating the processes of program performances, these more qualitative methods can provide a valuable complement to the statistical data and analyses that dominate conventional policy research. These methods can also provide rich, in-depth understandings of program and policy effects, and permit greater opportunities for the involvement of local communities and program staff.

The table below gives a schematic idea of how different data collection methods may be perceived by policy makers, program managers, and communities.

Major Methods of Data Collection and Probable Perceived Utility for Decision Making
Study Units	Methods of Data Collection	Utility for Policy Makers	Utility for Program Managers	Utility for communities
Complete population coverage	Census Regular statistical records	Very High	High	Moderate
Statistically representative sample of the population	One-time survey Longitudinal survey	Very High	High	Moderate
Purposive sample of the population	Key informants Ethnographic and case studies community or focus group interviews Participatory appraisals	Moderate	High	Very High

Policy makers tend to be interested in accurate assessments of sectoral performance, problems, potential solutions, and the likely impacts of policy shifts and direct interventions. Although among many policy researchers there is still a strong inclination towards statistical survey methods, a wide range of data collection methods are potentially relevant and useful.

Program managers tend to be particularly interested in timely feedback to guide operational or planning decisions. Transparency of methods, opportunities for staff to identify indicators and participate in data analysis, as well as quick turn-around are important. All of the more local-level data collection methods may be appropriate, but timeliness and cost may tend to outweigh other considerations when designing research for program management.

Community leaders and others participating in local health projects tend to be particularly interested in information that will increase their capacity to organize for effective action. Community interviews, inventories of local resources, and rapid appraisals can be especially helpful to these audiences.

Even where large-scale, quantitative surveys are feasible and preferred, the addition of qualitative components may benefit the research process in several important ways. Focus groups and in-depth, open-ended interviews with representative members of target populations can guide the development of survey instruments and may be critical for the interpretation of quantitative findings. Qualitative data are also useful to illustrate points and to support conclusions grounded in statistics, and can be used to enhance and extend the meaning of numbers. Whatever the research design, paying attention as the research unfolds to the systematic collection and recording of narratives that illustrate the significance of the issues being studied can make the later interpretation and communication of research results to end users much more effective.

In Ukraine: A study using qualitative and quantitative methods to look at women’s health care issues was conducted for presentation to policy makers. The study provided statistical evidence of shortcomings in care, but also documented, in women’s own words, their feelings about how they have been treated in the health system. A page of direct and sometimes moving quotes was presented on each of several topics. This presentation caught the attention of policy makers and motivated them to draw up plans for sweeping changes in service delivery.

C. Baume, et al. Ukraine Maternity Exit Survey. 1994.

Choosing research implementation strategies that involve potential users

There are considerable advantages to involving potential users—health workers, administrative staff, personnel from relevant agencies, and community members—in the research process, as investigators, interviewers, coders, analysts, or in other roles, as appropriate.

Different strategies will be appropriate for the different types of stakeholders. Higher-level decision makers may only be available for occasional consultations as the study proceeds, whereas middle-level managers may welcome the opportunity to take a more active part in data collection and/or analysis.

Involving community representatives or members in the research may influence the research design, since different data collection methods can either hinder or facilitate community involvement. For example, a census or the collection of vital statistics has limited potential for meaningful community input, whereas statistical surveys can accommodate some community involvement, although they require formal technical expertise in sample design, questionnaire development, and data analysis. Nonstatistical methods, particularly those employed in rapid appraisals, are much more amenable to active community participation in planning, implementation, analysis, and interpretation.

Discussions with the different individuals and groups will help to formulate some initial strategies for their continued involvement.

Strategies for involving decision makers and other stakeholders include:

• enlisting the help of local managers in choosing study sites and field workers;
• training local health staff so that they can play a role in data collection;
• identifying school leaders in the community who can play a role in data collection;
• holding discussion sessions with community leaders and groups at different stages of preparation and implementation of the study;
• organizing seminars to give feedback on preliminary results to decision makers and managers.

Including a research dissemination plan in the proposal

Effective dissemination of research findings is a challenging process, and requires a strategy to determine who should receive the information, what form the information should take, and how to maximize the potential influence the information can have. (Also see Stage 4.) The research proposal should demonstrate thoughtful consideration of the following points on which a dissemination strategy should be based:

• knowledge of the decisions the researcher wants to influence, i.e., what would ideally happen as a result of the research;
• identification of the potential audiences for the research findings;
• understanding of where these users generally go to get information;
• consideration of how to structure the research so that it will lend itself to various presentation formats, such as:
  • a reader-friendly report with a concise executive summary tailored to specific target audiences,
  • newspaper articles,
  • magazine/journal articles,
  • synthesis packages (fact sheets),
  • workshops/seminars/conferences,
  • radio programs;
• inclusion in the budget of the costs for selected formats;
• identification of who will be responsible for translating the findings into these formats;
• identification of forums and other appropriate channels to disseminate the findings.

If lack of commitment and funds prevent the findings from being disseminated to the people who can use them for policy changes and programmatic improvements, the research cannot fulfill its potential.

Deciding who, among the potential users, should be involved in reviewing the proposal

Researchers traditionally submit research proposals to a formal research review committee in their institution or region for technical comments. The potential users of research results, however, are not usually involved in such technical reviews. Input from them will, however, strengthen the proposal. While mapping individuals interested in the research area, it is useful to identify some key potential users of the research who could be added to the review process.

If creating a formal advisory group is too difficult, it may be possible to develop an informal advisory relationship with a core set of knowledgeable and interested persons selected from among professional colleagues, program managers, community members, and decision makers.

Whether on a formal or informal basis, it may be desirable to consult with or involve the advisory group members in:

• defining the research question;
• selecting policy and program issues to be addressed;
• identifying potential funders for the research;
• deciding on research methods and implementation strategies;
• addressing the day-to-day work of data collection and analysis;
• discussing modifications of research design because of unexpected events during the implementation;
• reviewing preliminary interpretations of research findings;
• drawing conclusions from the data and making recommendations for action; and
• developing approaches to communicate the research results and advocate for action.

Proposal Development Checklist

Does the proposal clearly indicate how the research is relevant to the concerns of decision makers, program managers, or communities?

Is the research design clearly described and justified? Is it really adequate to the research problem, or will it lead to equivocal or unconvincing results?

Are any trade-offs between measurement rigor, on the one hand, and policy and program relevance, on the other, acceptable?

Will the proposed research provide results when decision makers need them?

Will the kind of data collected translate into compelling stories and presentations?

Does the research proposal contain a dissemination plan?

Does the proposal indicate how to involve potential users, especially those responsible for implementing recommendations?

Does the proposal show who will be responsible for implementing recommendations resulting from research findings? Has sufficient effort been made to involve them in planning, carrying out, and interpreting the research?

Does the proposal provide for appropriate review?

Have periodic briefing and progress reports been planned for during the course of the study to involve decision makers and managers?

Stage 3: Conducting the study

KEY STEPS
Support for the research will increase if, in conducting the study the researcher:

1. Involves decision makers and managers in implementing, monitoring, and interpreting the study;
2. Spends time and effort to build trust with local workers and communities;
3. Involves managers, local health workers, and communities in data collection and analysis.

A growing body of evidence indicates that research use increases significantly if relations between researchers and clients are participatory and collaborative. Participatory approaches can also lead to more rigorous research, through better data collection and more insightful interpretations of research results.

Involving decision makers and managers in implementing and monitoring the study

Involving potential users of the research directly in the day-to-day work of data collection and analysis helps them understand what the research is all about. It also gives them a chance to step out of their customary roles (and out of their offices) and confront on-the-ground realities—a valuable experience in and of itself.

Actively participating in the study also gives them a personal stake in the research results. And this kind of direct, personal involvement in data collection and analysis often turns potential users into active supporters and advocates.

Higher-level decision makers and managers themselves may not be able to participate in data collection, but they may well appreciate being consulted about the involvement of lower-level personnel for whom they are administratively responsible. They may delegate people from their own staff at central or regional levels, or suggest district and field- level personnel who can be involved. Asking them to express their support—through the correct administrative channels—for the participation of staff at lower levels may facilitate contacts later on in the field.

A complementary strategy for involving high-level potential users is to invite them to regular briefings during the course of the study to discuss, for example, modifications in research design, the progress of data collection, and preliminary interpretations of findings. This may lead to important modifications in the focus of the inquiry and improve the usefulness of the final research product. But, most significantly, such briefings also will engage the interest of decision makers and program staff themselves who are not able to participate directly in core research activities.

It is particularly important to involve decision makers and managers when coming to a stage of drawing conclusions from the data and making recommendations for action. Engaging this group of potential users in discussions at this point will increase the chances that recommendations will be relevant and practical. It will also foster a sense of “ownership” of the recommendations and, indeed, of the research itself.

In Nigeria: HEALTHCOM researchers organized a workshop with directors and administrators from primary health care departments to review preliminary findings from recent studies of ORT and immunization behaviors. First, district health care managers presented what they knew about women’s ORT usage and immunization coverage rates. Then the researchers presented their preliminary findings and participants discussed the findings' implications in light of their understanding of the local situation. Finally, the group made joint, research-based recommendations on new strategies for promoting vaccinations and ORT.

Relevance, in this workshop, was enhanced by starting with what health care managers already recognized as important, and then using research findings to update, confirm, and—in some instances—revise these understandings. The result was a health communication plan which directly addressed, for the first time, local barriers to expanded immunization coverage and ORT usage.

S. Yoder., in HEALTHCOM Notes from the Field in Communication for Child Survival. 1993.

Spending time and effort to build trust with local workers and communities

Conducting interviews within decision-making networks or with various levels of program staff is a key step in defining research problems. A similar process of networking can be followed to identify and establish contact with other stakeholders in local communities and organizations. The process should begin prior to and continue throughout the course of field work.

The “field,” or social context for the research, is determined by the nature and focus of the research. It may be a community or other bounded population, a local organization or facility of some kind, or a set of inter-related service providers and health care agencies.

• When first entering the field, it is important for researchers to explain who they are and what they are there to do. This is an obvious recommendation but it is surprising how often this basic step is overlooked.
• The researcher will need to figure out who makes the rules. Every organization has a formal chain of authority and, as an outsider, it makes sense to start with the formal leadership. But there inevitably exists a less explicit, though no less effective, informal network of influence. When getting into the local community or organization, look for people who are held in particularly high regard or people to whom others turn for advice and counsel.
• Once the leaders and decision makers are identified, the researcher can follow the social protocols appropriate for conducting business with each, interacting with them on their own turf and on their own terms.

This may mean attending community meetings or arranging personal briefings to explain the purpose of the research and to discuss its potential benefits and uses, first steps in mutually exploring possible vehicles for future collaboration.

Although building trust takes time and effort, it is an essential step in engaging potential users at community as well as other levels. In conducting field research, the researcher and field team come into continuous contact with local communities and operational program staff. While managing these interactions and developing collaborative relations, it will help to:

• be attentive to the interests and concerns of community members and program staff;
• convey respect and helpfulness;
• show interest in and learn about specific activities and operational tasks carried out within communities and programs;
• try to understand the dynamics of local interests, including factions, power blocs, and centers of decision making;
• communicate in a way that is both informative and active—explaining, illustrating, and demonstrating.

Involving local health workers and communities in data collection and interpretation

Collaboration in the field may take various forms. The researcher may wish to consider the following options:

• Mixed fieldwork teams composed of researchers, program staff, and people from local communities who work together at nearly every stage of data collection and analysis;
• Field-based consultative committees, composed of program staff and representatives from local communities or other interested groups, who help to translate research findings into recommendations for action;
• Community action projects in which researchers’ expertise is put directly to work in support of the goals and efforts of a local, community-based organization or group.

Mixed teams of researchers, program staff, and members of local communities usually have a significant advantage over outside researchers working alone. Team members with program or community ties may be in a good position to explain why collecting certain information is important and motivate program staff or community residents to cooperate with reasonable research requests. At the same time, they can help to adapt data collection instruments and procedures to local conditions and settings. Given their inside knowledge of local social organization they can also help to clarify and interpret findings. This kind of active, collaborative involvement often fosters a personal commitment to seeing the research findings used.

Field-based consultative committees or advisory groups made up of program staff or representatives of local communities also can assist researchers who do not feel comfortable when independently offering policy or programmatic prescriptions. In this situation, differentiating roles may be helpful. The researcher’s primary responsibility lies in communicating research findings clearly and accurately. The advisory group’s responsibility is to apply research to policies and programs. This kind of collaboration relieves researchers of responsibility for making judgments in areas where they may not be particularly competent, but still keeps them involved in seeing that data is appropriately linked to action.

In community action projects, research planning and implementation involves much more joint decision making. Researchers and local collaborators together decide on what is best and then form a partner- ship to translate research into actions that further community agendas. A community, however defined, usually encompasses a range of complex interests, people, or groups with very different priorities who have a stake in what happens. It does not follow that everyone affected has an equal say. Partnerships occur when a number of different interests willingly come together to achieve some common sources, but they need to trust each other and share some commitment.

In the Gambia: Researchers using participatory research methods have assisted local communities in identifying and addressing even highly sensitive problems. In one village, a series of focus group discussions identified teenage pregnancy outside of marriage as a serious community issue. A group of girls in the village developed a plan to form an association to discuss the problem of teenage pregnancy, advise each other, and “try to put a stop to it” (meaning “try to stop the men harassing us.”) When their idea was presented at a community meeting, some villagers were surprised and shocked, others were supportive. Still, the community as a whole was ready to debate the issue in an open forum, and ultimately agreed that the girls' initiative was a positive move.

Here, the participatory research process offered the village community a chance to hear, perhaps for the first time, the voices of those who are often ‘invisible’ and removed from decision making.

E. Kane, et al. Girls Education in the Gambia. 1996.Such collaboration can either be employed independently or combined at various points in conducting field research. Whatever strategy is employed in fostering collaboration at the field level, the overall objective is to create ongoing channels for mutual feedback and learning.

Stage 4: Communicating Research Results

KEY STEPS
The likelihood of the research being used will increase if, in communicating the results, the researcher:

1. Uses a systematic dissemination strategy for reaching different audiences of potential users;
2. Writes timely reports in direct, non-technical language, using a style appropriate for various potential users;
3. Gives individual and group briefings as part of the dissemination strategy.

Even the greatest research breakthroughs mean very little unless they are successfully communicated to decision makers. The eventual users of research are often not professional scientists. The researcher's responsibility is to translate the technical results of the research into concepts and language that make sense to the users. Turning findings into compelling narratives that can capture the significant implications of the research is a much better communication strategy than making presentations based primarily on a series of tables and other quantitative displays.

Using a systematic dissemination strategy for reaching different audiences of potential users

Although the researcher may have outlined a dissemination strategy at the time the research proposal was developed, it will probably be necessary to revise this strategy in the light of the findings that have surfaced during the research, the level of interest and participation of potential users during the research process, and the present status of the issues being studied.

The strategy should clearly identify:

• individuals and groups targeted as potential users of the research;
• the types of information that are appropriate for each targeted user group;
• the barriers to accepting or implementing the results, and strategies for addressing them (motivating factors need to be identified and emphasized); and
• the most promising channels for transmitting information to each user.

In updating the strategy, one can also prioritize the decision makers, managers, and community leaders who will be most influential and best able to use the results of the research.

The researcher also may wish to identify and choose people and channels that are best able to reach these decision makers and managers—the researcher might not be the best person to carry the message in many circumstances.

In any case, be prepared to communicate the research results, either oneself or through others, in a variety of different settings and formats for different audiences. Types of publications and events to be considered include:

• summary policy/program briefs on key findings and recommendations;
• personal presentations to decision makers, managers, health workers, and community organizations;
• articles for newspapers;
• press releases;
• radio or television interviews;
• roundtable debates;
• presentations at workshops or symposia;
• research reports for academic or scientific audiences.

The Union for African Population Studies (UAPS) has learned, through its Small Grants Program, that simply publishing research findings is usually not enough to make an impact on decision makers. UAPS encouraged its researchers to enlist the help of journalists and representatives from end-user service organizations to organize a series of dissemination activities, using different channels and approaches for clearly defined target audiences. Activities included a day-long symposium for the scientific community, decision makers, and representatives of civil society, special presentations and working sessions with different end-user institutions, a press conference for the news media, and radio and television round-table sessions. Multi- channel events of this kind were successfully organized in Benin and Nigeria, and are being considered for other settings.

Union for African Population Studies. Note No. 8, Reunion du Comite Scientifique. 1996.

Writing timely reports in direct, non-technical language, using styles appropriate for various potential users

Communicating results in academically oriented research projects is typically limited to writing a final report, followed at some later date by publication in appropriate professional journals. This, however, is not a very effective way of communicating research results to decision makers. All too often, too much information is provided, in the form of lengthy analyses, far too late to be of use to decision makers (who have probably moved on to other concerns).

The best strategy, in most cases, is to begin communicating preliminary findings early in the research process. Since much attention may be given to these preliminary findings, care should be taken to present only those that are unlikely to change much with further analysis of the data, or else to follow up by disseminating corrected findings as soon as possible. Of course this is much easier to do if decision makers are actively collaborating in carrying out the research. The network of contacts established at earlier stages of the research process can be used for advice on the best times and occasions to present the work.

Research findings and specific recommendations should be presented in clear, simple, jargon-free terms—ideally, in the language of information users.

Reports should be brief and to the point, and packaged in a format familiar to the audience the researcher is trying to reach and influence. They should include:

• a short executive summary that includes key findings and recommendations,
• a relatively brief text, and
• a thorough and detailed appendix documenting methods, data, and analysis.

This format offers the best chance that a policy or program manager will read the summary while giving more research-oriented staff the opportunity to work through longer analyses in the body of the text and the appendix.

Few top-level decision makers have time to read more than several pages of any single report, and may not even read that much if the writing is overly technical or complicated and recommendations for action are vague.

Particular attention should be paid to formulating recommendations. These should:

• flow clearly from the conclusions of the study;
• be stated clearly in terms of specific action steps; and
• suggest actions that seem feasible and relevant to decision makers.

One way of ensuring that the recommendations are appropriate is to ask the network of program staff, decision makers, and community representatives to give feedback and, where possible, join in working on the recommendations before completing them for presentation.

Giving individual and group briefings as part of the dissemination strategy

Findings and recommendations should be personally communicated to key decision makers. Personal briefings differ from written reports in that they create a forum for discussion. Briefings have a number of advantages: they are highly visible, they allow intensive interaction, they can be individually tailored for a specific decision maker and a specific decision, and they encourage action. To facilitate discussion, briefings need to clearly convey research results, answer questions, and offer recommendations oriented towards concrete action.

Consider these guidelines for personal briefings:

• Understand the audience. Ideally briefings should be given to small, select audiences. For example, there may be one key decision maker to reach, accompanied by colleagues. Learn about their particular concerns and the questions they are likely to ask by talking with people who are close to them or by reviewing anything they may have written recently.
• Summarize findings. Prepare a one-page briefing summary that emphasizes actions that follow from research findings. Identify three key messages from the work that are related to specific actions or decisions, and make sure they are repeated to each audience clearly and frequently.
• Carefully select the information to present. Briefings are short, so include only those research results that really matter to the audience. Start with the briefing summary of the written report, adding only necessary details. Provide concrete, specific recommendations as to what is to be done, by whom, and when. Decision makers are usually far less interested in primary data and extensive analysis than in the specific recommendations based on them (though they also want assurance that the research meets professional and scientific standards).
• Choose the presenters. A successful briefing depends largely on those presenting. One presenter is usually best, assisted by someone who can handle technical questions or who may have a relationship with members of the audience and can serve as a valuable liaison.
• Prepare audio-visual aids. All briefings should employ appropriate audio-visual aids. Overheads and slides are effective, but they can malfunction, so be sure to practice with them in advance. Some seasoned presenters recommend briefing charts that are large enough to be read from 40 feet away. Visual aids should be concise, with no more than five to seven lines on any one overhead or chart. Each line should be informative in and of itself. Be sure, however, never to display something that will not be explained fully. Everybody attending the briefing should receive a handout to aid in taking notes. Handouts may duplicate larger briefing charts or overheads, but also can display more detailed information.
• Establish a written agenda for the briefing. Developing a written agenda and distributing it at the start of the briefing helps to establish a structure for the discussion and to ensure that important points are covered in a timely fashion.
• Practice. The best way to ensure an effective briefing is to practice. The more practice the better. Start with informal briefings to small, supportive audiences. Then practice with larger, neutral audiences. Finally, practice with a critical audience that will point out weaknesses in the presentation.
• Conduct the briefing. An effective speaking voice, appropriate eye contact, and confidence and poise in handling distractions all influence the audience's perception of the presenter and, by extension, the briefing. Delivery can be all-important—what the audience actually sees and hears determines whether the purpose is accomplished.
  • Use notes to make sure all key points are discussed, but avoid reading from scripts. Simply reading a written report makes it impossible to speak with conviction and enthusiasm and thus inevitably results in a boring presentation. Even an interesting subject fails to attract listeners when it is delivered in a monotone.
  • Interact with the audience freely and frankly to generate discussion.
  • Bring the findings to life. Analytical findings can easily obscure the human realities they rest on. Bring findings to life with quotes, personal stories, and accounts of particular situations, as appropriate.
• After the briefing. Send all participants unofficial minutes of the discussion, including decisions regarding action. Also send participants follow-up reports on the research to keep it visible.

These suggestions for individual or small-group briefings are also relevant to lectures or presentations given to large audiences. Regardless of audience size, a good presentation is well planned, complete (yet brief), interesting, easy to follow, and relevant to the audience. Although there may be no set formula for a perfect, or even a good speech, these are the characteristics that listeners look for in any presentation. In public speaking, the point of view of the listener is all-important.

Conclusion

The key concept at the core of the discussions and recommendations of this Guide is illustrated schematically in the figure in the Foreword. Research informs policies and programs most effectively when there is a three-way process of communication linking researchers, decision makers, and communities.

The Guide has attempted to promote a practical, collaborative approach, suggesting specific actions that researchers can take to communicate more effectively with potential users (the other two corners of the triangle) at each step in the research process. This will ensure that communication with potential users is not limited to presenting research findings, as has often been the case.

There are, of course, many reasons for the lack of communication shown in the triangle in the Foreword. Groups and individuals at each corners may not know how to approach and speak to the others in their own terms. They may be unwilling to spend the time and energy needed to establish a dialogue. Each group may have multiple reasons that make them suspicious of and resistant to overtures from the others. They may not even see the use of trying to communicate.

It is clearly not always an easy matter to overcome resistance and it will probably never be possible to establish ideal communication. However, without some level of dialogue, research findings are unlikely to be used. Understanding this fact will, hopefully, motivate researchers to approximate the ideal as far as is practical and affordable.

Choosing specific communication strategies cannot be reduced to a simple set of rules or procedures. The levels and types of dialogue that are appropriate and feasible will vary according to the research being undertaken, as well as the political environment, the cultures of the organizations involved, and individual personalities. However, applying the guidelines in this booklet will help to increase the relevance and utility of research products. The successful translation of relevant information into action is often beyond the control of individual researchers, and there cannot be any final guarantees. Yet following the steps outlined in this Guide can improve the odds—help- ing each researcher who aims to have an impact on policies and programs to pursue the art of the possible.

Bibliography

References marked * refer to the case study boxes.

*Baume, Carol, et al. (1994). Ukraine Maternity Exit Survey. Ukraine Ministry of Health and Wellstart International, Washington, DC, USA.

*Brownlee, Anne. (1986). A Guide for Planning Training and Research Programs in Health Systems Research. Geneva: WHO, Division of Strengthening of Health Services.

Chambers, Robert. (1992). “Rural Appraisal: Rapid, Relaxed, and Participatory,” Discussion Paper 311, Institute of Development Studies. Brighton, England: University of Sussex.

Chambers, Robert. (1994). “The Origins and Practice of Participatory Rural Appraisal,” World Development, Vol. 22, No. 7.

Chambers, Robert. (1994). “Participatory Rural Appraisal (PRA): Challenges, Potential, and Paradigm,” World Development, Vol. 22, No. 22.

Cornwall, Andrea, and Rachel Jewkes. (1995). “What Is Participatory Research?” Social Science and Medicine, Vol. 41, No. 12.

*Diène, Serigne. (1995). “Use of Participatory Approaches to Design, Implement, and Evaluate a Sustainable Community-based Nutrition Education Delivery System in the Fatick Region of Senegal.” A dissertation presented to the faculty of the graduate school of Cornell University.

Fetterman, David, ed. (1993). Speaking the Language of Power: Communication, Collaboration, and Advocacy (Translating Ethnography into Action). London: Falmer Press.

*Geslin, Colette, and Michel Zambramba. (1989). Etude sur le circuit de distribution des sels de réhydration orale au Burkina Faso. PRITECH.

HEALTHCOM. (1985). Field Notes: Communication for Child Survival. Washington, DC: Agency for International Development and the Academy for Educational Development.

HEALTHCOM. (1993). Notes from the Field in Communication for Child Survival. Washington, DC: Agency for International Development and the Academy for Educational Development.

Herman, Elizabeth, and Margaret Bentley. (1993). Rapid Assessment Procedures (RAP): To Improve the Household Management of Diarrhea. Boston: International Nutrition Foundation for Developing Countries (INFDC).

The International Bank for Reconstruction and Development. (1996). The World Bank Participation Sourcebook. Washington DC: The World Bank.

*International Heath Policy Program (1996) Policy Brief. Washington, DC: The World Bank.

*Kane, Eileen, et al. (1996). Girls Education in the Gambia. Occasional paper, Institute of Development Studies. Brighton, England: University of Sussex.

Langer, Ana, Mario Bronfman, and James Trostle. (1996). Linking Research to Decision Making: Case Studies of Family Planning, AIDS, Immunization, and Cholera Policies in Mexico (Summary Findings and Recommendations). Presented at the International Workshop on Linking Applied Research with Health Policy, Cuernavaca, Mexico, February 25–28, 1996.

Ndeki, Sidney S. (1997). A Regional Assessment of Health Systems Research Training in East and Southern Africa. Washington, DC: The Joint WHO/Joint Health Systems Research Project and the Support for Analysis and Research in Africa (SARA) Project.

*PRITECH. (1988). Health Facility Survey with a Focus on the Control of Diarrheal Diseases. PRITECH.

Reich, Michael. (1994). Political Mapping of Health Policy: A Guide for Managing the Political Dimensions of Health Policy. Boston: Data for Decision-Making Project, Harvard School of Public Health.

Rochefort, David, and Roger Cobb. (1994). The Politics of Problem Definition; Shaping the Policy Agenda. Lawrence: University Press of Kansas.

Rothman, Jack. (1980). Using Research in Organizations: A Guide to Successful Application. Beverly Hills, CA, USA, Sage Publications, Inc.

Task Force on Health Research for Development. (1991). Essential National Health Research: A Strategy for Action in Health and Human Development. Geneva: UNDP.

Sankar, Pamela. (1986). Qualitative Study of the Determinants of Oral Rehydration Therapy Use in The Gambia. PRITECH.

Slocum, Rachel, et al. eds. (1995). Power, Process and Participation—Tools for Change. London: Intermediate Technology Publications, Ltd.

*Union for African Population Studies. (1996). Note No. 8, Reunion du Comite Scientifique. Abidjan, Côte d'Ivoire: UAPS.

*Yoder, P. Stanley. (1993). “Reporting Research Results as Information Exchange: A View from Niger State, Nigeria.” In Notes from the Field in Communication for Child Survival. Washington, DC: Academy for Educational Development.

Annex—Selected Resource Materials on Participatory Research and Evaluation

*1Aubel, Judi. (1994). Participatory Program Evaluation: A Manual for Involving Program Stakeholders in the Evaluation Process. 70 pp. Catholic Relief Services - USCC, 1011 First Avenue, New York, NY 10022 (also available from PACT Publications). (Addressed to CRS staff, particularly program managers working in MCH, based on the author’s experiences in Cameroon, The Gambia, Senegal, Sierra Leone, and Ecuador.)

Bennet, F. J., ed. (1979). Community Diagnosis and Health Action—A Manual for Tropical and Rural Areas. 190 pp. Macmillans Tropical Community Health Manuals. Available from Teaching Aids for Low Costs (TALC), P.O. Box 49, St. Albans, Herts, AL14AX, United Kingdom.

Casey, Dennis, and Krishna Kumar. (1988). The Collection, Analysis, and Use of Monitoring and Evaluation Data. 174 pp. A World Bank Publication, published by the Johns Hopkins University Press. Available in English and Spanish. (Provides “simple, practical methods of collecting and analyzing,” both quantitative and qualitative. Companion volume to Project Monitoring and Evaluation in Agriculture.)

de Koning, Korrie, and Marion Martin, eds. (1996). Participatory Research in Health: Issues and Experiences. 242 pp. Published by National Progressive Primary Health Care Network (NPPHCN), P.O. Box 32095, Braamfontein 2017, Johannesburg, South Africa. Als available from Zed Books Ltd, 7 Cynthia Street, London N1 9JF and 165 First Avenue, Atlantic Highlands, New Jersey, 07716.

Ellsworth, Lynn, Fadel Diamé, Soukeyna Diop, and Daniel Thieba. (1992). Le Diagnostic Participatif (Participatory Rural Appraisal): Astuces et Aides Memoires Pour les Participants d’un Atelier d’Initiation. 200 pp. PRAAP, C.P. 13, Dakar-Fann, Sénégal. Also Manuel de l’Animateur (trainer’s guide). (An extensive collection of PRA tools useful for training.)

Feldstein, Hilary Sims, and Janice Jiggins, eds. (1994). Tools for the Field: Methodologies Handbook for Gender Analysis in Agriculture. Jumarian Press, 630 Oakwood Ave. #119, West Hartford, CT 06110. (According to review in Monday Developments 9 May 1994, this handbook provides real-life examples on how to assemble and use research tools needed to collect gender-sensitive data. It consists of 39 original cases illustrating a range of techniques from gender-sensitive interview guides to PRA with gender dimension.)

*Feurstein, Marie-Thérèse. (1986, 1993). Partners in Evaluation: Evaluating Development and Community Programmes with Participants, 196 pp. MacMillian Ltd, London and Basingstoke. Available from TALC, Box 49, St. Albans, Hertfordshire AL1 4AX, United Kingdom. (Highly recommended! Available in Portuguese, Arabic, and Vietnamese, as well as English.)

Kumar, Krishna. (1993). Rapid Appraisal Methods. 218 pp. The World Bank, Washington, DC. (According to a review by Charles D. Hale in Evaluation Practice, June 1995, this volume presents five rapid appraisal methods: key informant interviews, focus group discussions, group interviews, structured observation, and informal surveys; with eight detailed case studies.)

Mardsen, David, Peter Oakley, and Brian Pratt. (1994). Measuring the Process: Guidelines for Evaluating Social Development. 178 pp. PACT Publications, 777 United Nations Plaza, New York, NY 10017. (According to PACT, “It is intended primarily as a practical guide for undertaking the evaluation of social development projects and combines a theoretical overview of the concepts involved, with insights into planning and implementation of evaluation.”)

Mardsen, David and Peter Oakley, eds. (1990). Evaluating Social Development Projects, Development Guidelines 5, 162 pp. Oxfam, 274 Banbury Road, Oxford OXO 7DZ, United Kingdom. (Based on conference held in Swansea, September 1989. Covers qualitative indicators, methodologies for social development, partnerships between funders/donors and recipients, and the role and position of the evaluator.)

Narayan, Deepa. (1993). Participatory Evaluation: Tools for Managing Change in Water and Sanitation. 123 pp. PACT Publications, 777 United Nations Plaza, New York, NY 10017. (“Structured around a framework of key indicators that can be measured to determine progress toward objectives of sustainability, effective use, and replicability in water and sanitation programs. Available in English and French.”)

O’Brien, Dan. (1989). Internal Participatory Evaluation: A Model. PHC Report #10, CARE, Primary Health Care Unit, 151 Ellis Street, Atlanta, GA 30303. (One in a series of reports by CARE, this one focuses on participatory evaluation.)

*Pfohl, Jake. (1986). Participatory Evaluation: A User’s Guide. 81 pp. PACT Publications, 777 United Nations Plaza, New York, NY 10017. (Includes key concepts in defining participatory evaluation, a section on training and sample techniques for extending the work of self- evaluation to project staff and community members.)

Srinivasan, Lyra. (1993). Tools for Community Participation. 179 pp. PACT Publications, 777 United Nations Plaza, New York, NY 10017. (The SARAR approach used by PROWWESS/UNDP to expand women’s involvement in water and sanitation projects, but can be adapted to almost any training situation.)

*Thies, Joachim, and Heather M. Grady. (1991). Participatory Rapid Appraisal for Community Development: A Training Manual Based on Experiences in the Middle East and North Africa. 150 pp. IIED, 3 Endsleigh Street, London WCIH 0DD, United Kingdom. Also available in Arabic from Center for Development Services, 4 Ahmed Pasha Street, Citibank Building (6th Floor), Garden City, Cairo, Egypt.

Support for Analysis and Research in Africa (SARA) Project Health and Human Resources Analysis for Africa (HHRAA) Project U.S. Agency for International Development Africa Bureau, Office of Sustainable Development

In collaboration with:
Joint Health Systems Research (HSR) Programme
Essential National Health Research (ENHR) Africa Secretariat
Council on Health Research for Development (COHRED)

The SARA Project is operated by the Academy for Educational Development with subcontractors Tulane University, JHPIEGO, Macro International, Morehouse School of Medicine, Population Reference Bureau, and Porter/Novelli. SARA is funded by the U.S. Agency for International Development through the Health and Human Resources Analysis for Africa (HHRAA) Project (AFR/SD/HRD) under Contract AOT-0483-C-2178-00 (Project number 698-0483).

For more information or copies, contact:
Support for Analysis and Research in Africa (SARA) Project
Academy for Educational Development
1255 23rd Street NW
Washington DC 20037, USA
Tel: (202) 884-8700
Fax: (202) 884-8701
E-mail: sara@aed.org