PASSIA
Seminars
|
 |
|
Home > Seminars > Civil Society Empowerment Through Training & Skills Development |
|
Chapter 2
Monitoring and
Evaluation as an Integral Component of the Project Planning and Implementation
Process
Monitoring and evaluation are integral components of
the program/ project management cycle. Used at all stages of the cycle,
monitoring and evaluation can help to strengthen project design, enrich quality
of project interventions, improve decision-making, and enhance learning.
Likewise, the strength of project design can improve the quality of monitoring
and evaluation. It is important to remember that poorly designed projects are
hard to monitor or evaluate. The following section summarizes the logical
framework approach to project planning, implementation, and monitoring and
evaluation.
2.1
The Logical
Framework Approach to Project Design, Implementation and Evaluation
The logical framework
approach provides a structure for
logical thinking in project design, implementation and monitoring and
evaluation. It makes the project logic explicit, provides the means for a thorough
analysis of the needs of project beneficiaries and links project objectives,
strategies, inputs, and activities to the specified needs. Furthermore, it
indicates the means by which project achievement may be measured.
The detailed description of the processes of designing a program/ project using the logical framework is beyond the scope of this report. However, the following section provides a summary of the milestones and main concepts and definitions[1]:
·
Problem analysis represents the
first step in project design. It is the process through which stakeholders
identify and analyze the problem(s) that the project is trying to overcome. The
result of this analysis is usually summarized in a tree diagram that links
problems with their causes.
·
Next, project goals and objectives
are developed and structured in a hierarchy to match the analysis of problems.
They can be represented as a mirror image of the problem tree diagram. While
projects are usually designed to address long-term sectoral or national goals,
objectives are specific to the project interventions. They should also be
clear, realistic in the timeframe for their implementation and measurable for
evaluation. Examples: school dropouts (in a geographical area or for a target
group) will be reduced by 10% (within a specific timeframe), agricultural
products (in a geographical area or for a target group) will be increased by
15% (within a specific timeframe), etc.
·
Outputs are the immediate
physical and financial results of project activities. Examples: kilometers of
agricultural roads constructed, number of schools renovated, number of farmers
attended a training course; number of textbook printed, etc.
·
Activities and inputs are
developed to produce the outputs that will result in achieving project
objectives.
The product of this analytical approach is usually
summarized in a matrix called the logical frame matrix, which summarizes what
the project intends to do and how, what kind of effects are expected, what the
project key assumptions are, and how outputs and outcomes will be monitored and
evaluated (see below).
The columns of the logical frame matrix represent
the levels of project objectives (hierarchy of objectives) and the means to
achieve them. There are four levels in the logical frame and each lower level
of activity must contribute to the achievement of a higher level. For example,
the implementation of project activities would contribute to the achievement of
project outputs. The achievement of the project outputs would lead to the
achievement of project objectives. This is called the vertical logic. The rows
indicate how the achievement of objectives can be measured and verified. This
is called the horizontal logic. Assumptions (situations needed to promote the
implementation of the project) must be systematically recorded.
The Logical Frame Matrix Structure
|
Project Description |
Indicators |
Means of Verification (MOV) |
Assumptions |
|
Goal: The broader development impact to which the project/ program
contributes- at a national and/or sectoral level. |
Measures of the
extent to which a contribution to the goal has been made. Used during
evaluation. |
Sources of
information and methods used to collect and report it. |
|
|
Purpose: The development outcome expected at the
end of the project. All components will contribute to this. |
Conditions at the end of the project
indicating that the Purpose has been achieved. Used for project completion
and evaluation. |
Sources of information and methods
used to collect and report it. |
Assumptions concerning the purpose/
goal linkage. |
|
Component Objectives: The expected outcome of producing each
component's outputs. |
Measures of the extent to which component
objectives have been achieved. Used during review and evaluation. |
Sources of information and methods
used to collect and report it. |
Assumptions concerning the component
objective/purpose linkage. |
|
Outputs: The direct measurable results (goods and
services) of the project which are largely under project management's
control. |
Measures of the quantity and quality
of outputs and the timing of their delivery. Used during monitoring and
review. |
Sources of information and methods
used to collect and report it. |
Assumptions concerning the output/
component objective linkage. |
|
Activities: The tasks carried out to implement the
project and deliver the identified outputs. |
Implementation/work program targets.
Used during monitoring. |
Sources of information and methods
used to collect and report it. |
Assumptions concerning the activity/ output linkage. |
A brief description of the
terminology is given below:
Project description
provides a narrative summary of what the project intends to achieve and how. It
describes the means by which desired ends are to be achieved.
Goal refers to the sectoral
or national objectives for which the project is designed to contribute, e.g.
increased incomes, improved nutritional status, reduced crime. It can also be
referred to as describing the expected impact of the project. The goal is thus
a statement of intention that explains the main reason for undertaking the
project.
Purpose refers
to what the project is expected to achieve in terms of development outcome.
Examples might include increased agricultural production, higher immunization
coverage, cleaner water, or improved local management systems and capacity.
There should generally be only one purpose statement.
Component Objectives Where
the project/program is relatively large and has a number of components, it is
useful to give each component an objective statement. These statements should
provide a logical link between the outputs of that component and the project
purpose. Poorly stated objectives limit the capacity of M&E to provide
useful assessments for decision-making,
accountability and learning purposes.
Outputs refer to
the specific results and tangible products (goods and services) produced by
undertaking a series of tasks or activities. Each component should have at
least one contributing output, and often have up to four or five. The delivery
of project outputs should be largely under project management's control.
Activities refer
to all the specific tasks undertaken to achieve the required outputs. There are
many tasks and steps to achieve an output. However, the logical frame matrix
should not include too much detail on activities because it becomes too
lengthy. If detailed activity specification is required, this should be
presented separately in an activity schedule/Gantt chart format and not in the
matrix itself.
Inputs refer to
the resources required to undertake the activities and produce the outputs,
e.g., personnel, equipment and materials. The specific inputs should not be
included in the matrix format.
Assumptions refer
to conditions which could affect the progress or success of the project, but
over which the project manager has no direct control, e.g. price changes,
rainfall, political situation, etc. An assumption is a positive statement of a
condition that must be met in order for project objectives to be achieved. A
risk is a negative statement of what might prevent objectives being achieved.
Indicators refer
to the information that would help us determine progress towards meeting
project objectives. An indicator should provide, where possible, a clearly
defined unit of measurement and a target detailing the quantity, quality and
timing of expected results. Indicators should be relevant, independent and can
be precisely and objectively defined in order to demonstrate that the
objectives of the project have been achieved (see below).
Means of verification
(MOVs). Means of verification should clearly specify the
expected source of the information we need to collect. We need to consider how
the information will be collected (method), who will be responsible, and the
frequency with which the information should be provided. In short MOVs specify
the means to ensure that the indicators can be measured effectively, i.e.
specification of the indicators, types of data, sources of information, and
collection techniques.
2.2
Link Between the
Logical Frame and Monitoring and Evaluation
The horizontal logic of the
matrix helps establish the basis for monitoring and evaluating the project by
asking how outputs, objectives, purpose and goal can be measured, and what are
the suitable indicators. The following table summarizes the link between the
logical frame and monitoring and evaluation.
|
Logical frame hierarchy |
Type of monitoring and evaluation activity |
Indicators |
|
Goal |
Ex-post evaluation |
Impact indicators |
|
Purpose |
Program Review |
Outcome indicators |
|
Component Objectives |
Periodic and final evaluation |
Outcome indicators |
|
Outputs |
Monitoring/periodic evaluation |
Output indicators |
|
Activities/Inputs |
Monitoring |
Output indicators |
It is worth noting that the above table represents
a simplified framework and should be interpreted in a suitably flexible manner.
For example, ex-post evaluation assesses whether or not the purpose, component
objectives and outputs have been achieved. Project/program reviews are
concerned with performance in output delivery and the extent of achieving
objectives.
|
Indicators Indicators provide the
quantitative and qualitative details to a set of objectives. They are
statements about the situation that will exist when an objective is reached,
therefore, they are measures used to demonstrate changes in certain conditions
or results of an activity, a project or a program. In addition, they provide
evidence of the progress of program or project activities in the attainment
of development objectives. Indicators should be pre-established, i.e. during
the project design phase. When a direct measure is not feasible, indirect or
proxy indicators may be used. Indicators should be directly
linked to the level of assessment (e.g. output indicators, outcome indicators
or impact indicators). Output indicators show the immediate physical and
financial outputs of the project. Early indications of impact (outcomes) may be obtained by surveying
beneficiaries’ perceptions about
project services. Impact refers to long-term developmental change. Measures of change often involve complex
statistics about economic or social welfare and depend on data that
are gathered from beneficiaries. They should also be clearly
phrased to include change in a situation within a geographical location, time
frame, target etc. A popular code for remembering the characteristics of good
indicators is SMART. S: Specific M:
Measurable A: Attainable
(i.e., can be checked) R: Relevant
(reflect changes in the situation) T: Trackable
(can be tracked over a specific period of time) Source:
ITAD, Monitoring and the Use of Indicators, consultancy report to DG VIII,
European Commission, Brussels, 1996. |
Notes and Comments:
1.
Classifying project objectives into different levels requires that management
will need to develop systems to provide information at all levels, from basic
accounting through sophisticated studies, in order to measure project outcomes.
2. There
are different means for measuring project indicators:
·
Input indicators can be
provided from management and accounting records. Input indicators are used
mainly by managers closest to the tasks of implementation.
·
Output indicators are directly
linked to project activities and inputs. Management of the project have control
of project activities and their direct results or outputs. Those outputs can be
verified through internal record keeping and analysis.
·
By contrast, the achievement of
project objectives normally depends on a number of factors. Some might be
controlled by the project, other cannot. For example, the response of
beneficiaries to project services is beyond the control of the project. Responses
of beneficiaries regarding benefits brought to them by the project require
consultation and data collection.
·
Project outcome are often
measured through the assessment of indicators that focus on whether
beneficiaries have access to project services, level of usage and satisfaction
with services. Such evidence can also be provided easily and accurately through
impact research, e.g. changes in health status or improvements in income.
3.
Exogenous indicators focus on general social, economic and environmental
factors that are out of the control of the project, but which might affect its
outcome. Those factors might include the performance of the sector in which the
project operates. Gathering data on project indicators and the wider
environment place an additional burden on the project's M&E effort.
4. The
importance of indicators could be changed during project implementation. For
example, monitoring and evaluation focus at an early stage of the project is on
input and process indicators. Emphasis shifts later to outputs and impact. In
other words, emphasis is first placed on indicators of implementation progress,
and later on indicators of development results.
M&E designers should
examine existing record keeping and reporting procedures used by the project
authorities in order to assess the capacity to generate the data that will be
needed.
5. Some of the impact indicators, such as
mortality rates or improvement of the household income, are hard to attribute
to the project in a cause-effect relation. In general, the higher the
objective, the more difficult the cause-effect linkages become. Project impact
will almost certainly be a result of a variety of factors, including that of
the project itself. In such situation, the evaluation team might use comparisons
with the situation before the project (baseline data), or in areas not covered
by the project.
6. To maximize the benefits of M&E, the
project should develop mechanisms to incorporate the findings, recommendations
and lessons learned from evaluations into the various phases of the program or
project cycle.
Suggested Reading:
Appleton, Simon, "Problems in Measuring
Changes in Poverty over Time," IDS Bulletin, Vol. 27, No. 1, 1996.
Brighton, UK: Institute for Development Studies.
Casley, Dennis J. and Krishna Kumar, Project
Monitoring and Evaluation in Agriculture. Washington, D.C.: World Bank,
1987.
Operations Evaluation Department, World Bank,
"Building Evaluation Capacity", Lessons & Practices No. 4,
November 1994. --- "Monitoring and Evaluation Plans in Staff Appraisal
Reports in Fiscal Year 1995". Report No. 15222, December 1995.