Monitoring Rural Development in East Asia
SWP439
Wbrld Bank Staff Working Paper No. 439 o.. 2
October 1980
Prepared by: Guido Deboeck and Ronald Ng
Agriculture and Rural Development Department
Copyright � 1980
The MMrld Bank
1818 H Street, N.W.
Washirgton, D.C. 20433, U.S.A.
The views and interpretations in this document are those of the auth
and should not be attributed to the Worid Bank, to its affiliated
organizations, or to any indvidual acting in their behalf. 9 =
The views and interpretations in this document are those of the authors and
should not be attributed to the World Bank, to its affiliated organizations
or to any individual acting in their behalf..
WORLD BANK
Staff Working Paper No. 439
October 1980
MONITORING RURAL DEVELOPMENT IN EAST ASIA
This paper presents the results of discussions during an eight-day
workshop on monitoring and evaluation of rural development projects in East
Asia and Pacific. Held in Kuala Lumpur in December 1979, the workshop was
attended by over thirty project managers and monitoring/evaluation staff from
seven countries, as well as members of the Bank staff and representatives
of international agencies. The workshop provided a forum for exchange of
ideas and field experiences.
Since most of the projects represented at the workshop are still
under implementation, the discussions concentrated on the managerial, tech-
nical and institutional aspects of monitoring. Participants realized that,
while there are no easy recipes for information systems which will provide
quick feedback to management, a sharing of their individual experiences with
different types of projects, under diverse environmental and cultural condi-
tions, did offer insight into how the problem could be tackled. The contents
of this paper deal with many of the issues confronting the designers of
efficient monitoring systems, and are therefore of value to those concerned
with promoting the cause of project specific monitoring and with improving
project management.
The workshop discussions concentrated on the topics of expectations
and disillusions about monitoring; data collection and analysis; presentation
of results; staff and resources requirements; and the role of consultants and
the use of external expertise for monitoring. The emphasis on these subjects
complements the experience of an earlier workshop held in Nairobi for the
field staff of rural development projects in East Africa, a report of which
has since been published under the title of "Managing Information for Rural
Development: Lessons from Eastern Africa" as World Bank Staff Working Paper
No. 379.
Prepared by: Guido Deboeck and Ronald Ng
Agriculture and Rural Development Department
Copyright e 1980
The World Bank
1818 H Street, N.W.
Washington, D.C. 20433, U.S.A.
ACKNOWLEDGEMENT
The authors wish to express their special gratitude
to all participants at the Workshop, on whose contributions
in the form of both their case studies and their discussions,
this paper is based. We have also benefited from the opera-
tional experience of Altaf Hussain and Doug Forno of the Bank,
Ralph Retzlaff from the Agricultural Development Council, and
Claes Lindahl from UNDP.
In the organization of the Workshop and in the prep-
aration of this paper, the authors are deeply indebted to
Ted Davis, Musa Ahmad, Michael Cernea, Nualnapa Buranavanichkit
and Consuelo Carson. The editorial patience of Ella Wright is
greatly appreciated.
I
iii
TABLE OF CONTENTS
MONITORING RURAL DEVELOPMENT IN EAST ASIA
Summary of Conclusions and Workshop Findings .................. iv
I. Introduction .................................................. 1
II. The Managerial Aspects of Monitoring .......................... 5
A. Why Should Projects Be Monitored? .................... 5
B. Expectations and Disillusions about Monitoring ...... 7
III. The Technical Aspects of Monitoring . . ......................... 10
A. Data Collection .. 10
B. Data Processing and Analysis . . 14
C. Presentation of Monitoring Results . . 16
IV. The Institutional Aspects of Monitoring . . ..................... 17
A. Organization of Monitoring and Evaluation . . 17
B. Role of Consultants .................................. 19
C. Staffing Requirements . . 21
D. Other Resource Requirements ....22
V. The Design of Monitoring and Evaluation Systems: The
Field Exercise at Muda ...................................... 26
Annexes
I. List of Participants 35
II. Opening Address by the Secretary General, Ministry of
Agriculture, Malaysia 41
III. List of Papers Submitted to the Regional Workshop on
Monitoring aand Evaluation of Rural Development
Projects in East Asia 45
IV. Abstracts of Case Studies 47
V. Selected References 89
I
-iv -
SUMMARY OF CONCLUSIONS AND WORKSHOP FINDINGS
A Regional Workshop on Monitoring and Evaluation of Rural Develop-
ment Projects in East Asia and the Pacific was held in Kuala Lumpur, Malaysia,
in December 1979. Participants were 36 project managers and monitoring/
evaluation officers, who submitted 21 case studies on monitoring and evalua-
tion of rural development projects in this region. The case studies provided
the basis for the exchange and cross-fertilization of operational experiences.
Abstracts of these studies are included in Annex IV of this report, and a selec-
tion of case studies by workshop participants is available in a separate
publication.
Based on the experience conveyed in the case studies, three major
themes were identified for the workshop sessions:
(1) the managerial aspects of monitoring,
(2) the technical aspects of monitoring, and
(3) the institutional aspects of monitoring.
This report synthesizes the conclusions and findings from the discussions
held by the participants during the workshop. As a part of the workshop, a
field exercise was conducted which involved the participants in the design of
monitoring and evaluation subsystems for selected major project components of
the Muda Irrigation Project in Malaysia. The final section of this report
describes this practical experience.
Workshop participants realized that there are no ready-made recipes
for information systems that will provide quick feedback to management from
complex poverty-oriented rural development projects. However, well-designed
projects with clear and specific objectives can be effectively monitored by
means of relatively simple monitoring and evaluation systems that collect the
minimum amount of information based on carefully selected "key indicators."
On the other hand, although information systems cannot be expected to compen-
sate for poor project design and vague objectives, even in these less
satisfactory situations, project-specific monitoring and evaluation can be a
valuable tool for identifying problems of implementation and recommending
solutions. Participants felt that "monitoring" was a tool for managers
to assist them in what was termed "smooth" implementation and in reducing
bottlenecks and problems that might arise. "Evaluation" was considered
necessary primarily for noting the lessons learned from experience. Project-
specific information systems are therefore required in rural development
projects to determine what the benefits are and who receives them.
Some participants argued for more flexibility in project design so
that project managers could modify operational plans according to the findings
of ongoing evaluation. Others believed that this responsibility belongs to
a hierarchy of decision makers, including project managers, policy planners,
and higher authorities. Information derived through monitoring and evaluation
should be directed through the appropriate institutional channels to higher
levels of management. All decision makers involved in a project should take
an active role in the design, development and execution of the monitoring and
evaluation system. The technical standards should be sufficiently high to
generate confidence and to provide information that is both timely and reli-
able for use in management decisions.
Based on their experiences, participants observed that information
resulting from monitoring activities of rural development projects in East
Asia and the Pacific is seldom used effectively. Such information is generally
of low quality, too elaborate, long delayed, or irrelevant to the needs of
vi -
project management. Resources available for monitoring are often inadequate
to do the work efficiently. Furthermore, results from monitoring are poorly
communicated to the project managers. As many project managers still lack an
appreciation for the benefits that can be derived from effective use of feed-
back information produced through monitoring, few of them are highly committed
to monitoring. In addition, it can be extremely difficult to communicate
sensitive information to project management and to the responsible agencies
in the Asian cultural context.
To avoid duplication of expensive monitoring efforts, information
available from alternative sources should be integrated into the monitoring
and evaluation system. The workshop participants urged that the first step in
the design of any data collection exercise should be to identify a minimum set
of indicators that satisfy the immediate needs of project managers. Appro-
priate design of the sample, proper construction of the questionnaire and
adequate training and supervision of the enumerators are some of the means to
both reduce costs and ensure quality. Delays in feedback to management can
often be traced to the procedures for data processing and analysis. Data
collection and analysis should not be overly ambitious and should be closely
linked to project needs. The capacity for processing and analysis should be
adequate for the data collected. Serious problems can arise because of lack
of staff, skills and experience, processing equipment and analytical
methodology. Clear identification of information requirements for different
purposes at specific times is essential to assure timely processing and
analysis of the data collected.
In the presentation of results, the workshop participants advocated
the importance of brief, concise, specific, and action-orienied reports. The
- vii -
output of the system should be relevant to the requirements of management.
They also stressed the need to recognize who would be the end-users of the
information.
The participants recommended that culturally acceptable channels
of communication be identified to transmit sensitive monitoring and evaluation
findings to project managers and higher authorities without compromising the
substance of the conclusions. The effective use of the monitoring and/or
evaluation results for improving project performance requires a significant
degree of cooperation, commitment, and support by project managers and
planners. Skepticism, suspicion, and defensiveness on the part of the
authorities can make the monitoring and evaluation system completely
ineffective.
As to organization, many participants felt that a two-tiered
M&E system, consisting of a national-level unit which would be linked to
smaller project-specific cells, may be the most efficient means of making
optimal use of the scarce resources available for monitoring and evaluation.
To overcome the critical shortage of skilled manpower for project-specific
monitoring and evaluation -- a shortage which is prevalent in most of the
countries in East Asia and the Pacific -- a number of projects may have to
rely heavily for some length of time on external resources, such as local
universities, research institutions, consultant firms or individual con-
sultants. The participants, after discussing at length the trade-offs
between building an in-house capacity versus the use of external inputs,
concluded that training provided by consultants could be a benefit to
institution building for the country concerned. Meanwhile, incentives and
career prospects would have to be improved to attract the necessary caliber
- viii -
of personnel for the tasks of monitoring and evaluation. The career structure
of individuals should be viewed in a framework larger than that of a single
project.
Although agreeing that most projects require some monitoring and
evaluation activities, the participants noted that a significant proportion of
the World Bank-assisted projects in East Asia and the Pacific do not contain
cost and staff specifications for project-specific M&E. They therefore urged
that the Bank give attention to making specific provisions for the M&E systems
during project appraisal.
Currently, most rural development monitoring and evaluation systems
for rural development projects are in their early stages of development in
East Asia and the Pacific. According to participants, the serious difficul-
ties experienced are mainly due to lack of detailed M&E design at the time of
appraisal or lack of staff and funds for proper conduct of M&E. However,
participants felt that monitoring, if properly conducted, was a valuable
management tool for providing timely information on project progress and
performance.
The tenor of the discussions and the summary of the findings at the
Kuala Lumpur Workshop are broadly in agreement with the experience from
previous workshops in Nairobi, Kenya and San Jose, Costa Rica. The exchange
of experience and opinion at Kuala Lumpur was greatly expedited by the fact
that the participants were almost exclusively either project managers or
monitoring/evaluation officers who have to deal with the practical problems
of implementing M&E systems.
- ix -
During the field exercise, participants had an opportunity to design
various monitoring and evaluation components for the ongoing Muda Agricultural
Development Project. The results of the different working teams were discussed
in detail in a plenary session. Their realistic proposals for M&E of the
project components reflected partly the full cooperation of the field staff at
Muda and partly the excellent briefing of the participants on the existing
system. More significantly, their proposals demonstrated that the participants
had a clear understanding of the functions of monitoring and evaluation and
that they had benefited from the cross-fertilization of experience during the
preceding sessions of the workshop.
In conclusion, the participants in the Kuala Lumpur Workshop were
of the view that "project management would be deprived of indispensable
information in the absence of an M&E system. Investment in M&E is, there-
fore, justified and should receive equal attention relative to the other
project components." Furthermore, workshop participants felt that "investment
in project monitoring and evaluation [would be] recovered many times over in
terms of improved project performance."
SECTION I: INTRODUCTION
Poverty-oriented rural development projects are generally complex,
containing several complementary components. For this type of project to be
implemented successfully, a high standard of project management is required.
Management information systems can make a valuable contribution to efficient
implementation of such projects. Evaluation and review of projects can result
in experience that will lead to improving the design and execution of follow-
up projects of a similar nature.
To promote more effective implementation of rural development pro-
jects, the World Bank in 1979 started a series of Regional Workshops on Monitor-
ing and Evaluation of Rural Development Projects. The main purposes of these
workshops are to promote exchange of experiences and to pass on to others the
lessons learned from design and implementation of monitoring and evaluation
systems.
The first Regional Workshop on Monitoring and Evaluation was orga-
nized in April 1979, in Nairobi, by the World Bank-s Agriculture and Rural
Development Department, in collaboration with the Bank-s East Africa Projects
Department. A summary of the discussions and conclusions of that workshop can
be found in "Managing Information for Rural Development: Lessons from Eastern
Africa" by Guido Deboeck and Bill Kinsey (World Bank Staff Working Paper,
no. 379, March 1980). A selection of the case studies presented at that
workshop is available from the Agriculture and Rural Development Department. 1/
1/ G. Deboeck and D. Rubin (eds.). "Selected Case Studies on Monitoring and
Evaluation of Rural Development Projects: Volume I: Eastern Africa,"
Agriculture and Rural Development Department, World Bank, 1980, 118
pages.
-2 -
The Bank's Latin American Projects Department organized a similar
workshop in San Jose, Costa Rica (in October 1979) on monitoring and evalua-
tion of rural development projects in Latin America. A technical review of
that workshop is available. 1/
In collaboration with the East Asia Projects Department, the Agricul-
ture and Rural Development Department organized another Regional Workshop on
Monitoring and Evaluation of Rural Development Projects in East Asia and the
Pacific in December 1979 in Kuala Lumpur, Malaysia. The workshop was attended
by 36 participants from seven countries: Fiji, Indonesia, Korea, Malaysia,
Papua New Guinea, the Philippines, and Thailand. Among the participants were
19 project managers, 10 monitoring and evaluation officers, 5 national program
officials, a representative from the Agriculture Development Council, a con-
sultant, and 5 staff members of the Bank. 2/
The Regional Workshop on Monitoring and Evaluation of Rural Develop-
ment Projects was officially opened by Datuk Arshad bin Ayub, Secretary
General of the Ministry of Agriculture of Malaysia. In his opening address,
the Secretary General emphasized the great significance and high priority that
is given to monitoring and evaluation of rural development projects in Malaysia.
"In Malaysia [wel realize the importance of a monitoring and
evaluation system within the framework of the planned cycle.
A special unit, the Implementation Coordination Unit, under
the Prime Minister-s Department has been given the task to
undertake this. A computerized system has already been
created to monitor and evaluate the development plans and
projects for the whole country. The creation of the system
is a turning point to institutionalize monitoring and
1/ R. van Oven and K. Swanberg, "Workshop on Monitoring and Evaluation of
Agricultural Projects: Overview," Latin American Projects Department,
World Bank, March 1980.
2/ A complete list of the participants in this workshop can be found in
Annex I.
- 3 -
evaluation as a component of the planning cycle in the
development process. Within this overall framework, the
Ministry of Agriculture is in the process of developing a
monitoring and evaluation system for all its projects and
is also introducing special M&E components in all new major
projects to improve overall management." 1/
In preparation for this Regional Workshop all participants were
invited to submit case studies on their experiences with monitoring and/or
evaluation. Twenty-one case studies were submitted by the participants in
advance of the workshop. A full list of these case studies can be found in
Annex III, and abstracts of all case studies presented at the workshop appear
in Annex IV. A selection of the case studies is available in a separate
volume. 2/ A synthesis of the case studies, prepared by Ronald Ng, was pro-
vided at the start of the workshop for use by the participants.
The first four days of the workshop were devoted to a detailed
review and discussion of monitoring and evaluation experiences; the remain-
ing four days were taken up by a practical exercise, based on a field trip.
The Regional Workshop was organized around three broad aspects of
monitoring and evaluation:
(1) The Managerial;
(2) The Technical; and
(3) The Institutional.
In view of the relative recency of most of the projects represented at the
workshop, very few projects have yet attempted any form of ongoing evaluation.
1/ The full text of the opening address by Datuk Arshad is reproduced
in Annex II.
2/ G. Deboeck and D. Rubin (eds.). "Selectied Case Studies on Monitoring and
Evaluation of Rural Development Projects: Volume II: East Asia and the
Pacific," Agriculture and Rural Development Department, World Bank', 1980,
181 pages.
- 4 -
Only one of the case studies related to an ex-post evaluation experience. The
monitoring aspects of management information systems therefore received
correspondingly much more attention than either ongoing or ex-post evaluation.
However, so far as possible, especially where the methodologies for monitoring
and evaluation shared some features in common, the topics were discussed
simultaneously.
Participants also engaged in workshop exercises that were designed
to focus discussions on these three themes in small working groups. The
results of the exercises and conclusions reached were presented by the work-
shop moderators to provoke debate on alternative issues within the themes.
The results of discussions by the small working groups were summarized, with
graphic presentations by the group rapporteurs at the end of the individual
sessions. A synthesis of the opinions and conclusions on these themes can be
found in Sections II, III and IV of this report.
In addition to these exercises and discussions, relating to elements
of the broad themes, the design of a monitoring and evaluation system for a
typical rural development project was undertaken by means of a field trip.
Workshop participants were briefed on the Muda Irrigation Project by
Mr. Jegatheesan, the senior official responsible for monitoring and evaluation
of the project. Next, at the project site in Alor Star, participants --
accompanied by the technical staff of the project -- designed a monitoring
and evaluation component for each of the five key areas of project operations:
(i) irrigation water supply; (ii) agricultural credit; (iii) agricultural
input supply; (iv) agricultural extension services; and (v) socioeconomic
aspects of the project. The major results of this field exercise are
presented in Section V of this report.
5-
SECTION II: THE MANAGERIAL ASPECTS OF MONITORING
In the case studies submitted for the workshop, monitoring is
recognized as an important management tool to provide timely information on
the progress and problems of a project in the process of implementation. When
it is based on several cases, monitoring should provide information on devia-
tions from the project objectives, on problems that are encountered, and on
ways in which project management can take remedial action promptly.
The issues relating to monitoring as a management tool were:
(i) why project management needs monitoring; (ii) what support project
managers expect; (iii) what information is required from monitoring; (iv) why
some expectations are not met; and (v) why information produced by monitoring
units is not used effectively. The deliberations on these issues at the
workshop led to certain observations and conclusions that are summarized in
the following two sections.
A. Why Should Projects Be Monitored?
Poverty-oriented rural development projects usually consist of
multiple components, including directly productive investments, such as irri-
gation, agricultural credit, and extension, combined with investments in
infrastructure and social services. Most projects discussed at the workshop
were multi-component rural development projects, for which there are par-
ticular needs for monitoring.
The participants felt that the primary reasons for monitoring rural
development projects were:
- 6 -
(1) to keep track of project progress;
(2) to provide feedback to project management on the achievement
of project objectives;
(3) to serve as a "warning" mechanism for project management; and
(4) to help prevent or solve problems encountered during
project implementation.
Thus, monitoring was seen by the workshop participants as a tool
for managers, to assist them in what was called "smooth" implementation; to
help them keep implementation on schedule; to provide for budget control; to
measure physical achievements; and to reduce impediments and problems.
The rationale for monitoring is quite different from the rationale
for undertaking evaluation of rural development projects. The workshop
participants felt that evaluation of rural development projects was necessary
to measure the effects and impact of projects. Evaluation, which consists of
an analysis of the causes of success or failure of a project, is necessary to
improve both the present progress and the future planning of rural development
efforts. In short, participants felt that evaluation is needed primarily to
note the lessons learned from experience.
The rationale for evaluation raised some interesting issues concern-
ing the extent of flexibility currently built into rural development projects.
Some participants felt that more flexibility should be incorporated in project
designs and that project managers should have more authority to respond to the
results from ongoing evaluations undertaken during project implementation.
Greater flexibility could promote more effective implementation of rural
development projects.
-7-
The nature and composition of rural development projects -- in
particular the growing complexity of poverty-oriented projects -- was seen
by the workshop participants as a major rationale for building an information
system within project. Such an information system, which can consist of
either monitoring or a combination of monitoring and evaluation, should
primarily support the project manager. Other potential beneficiaries or
users of the information system would be higher levels of management at
regional and national levels as well as private organizations involved in
the implementation of the project, planning and policy bodies, and external
agencies. Information systems need to be included in rural development
projects, particularly when an objective is to directly or indirectly benefit
the poverty target groups. Without some special system it is virtually
impossible to determine the benefits (i.e., project effects and impact) and
who receives them.
B. Expectations and Disillusions about Monitoring
Following the discussion of the justification for monitoring, a very
lively discussion emerged among the participants on the expectations from
monitoring systems. Various working groups reported that project managers
expect monitoring systems to produce information that is
(1) accurate;
(2) objective;
(3) reliable;
(4) relevant;
(5) timely; and
(6) action-oriented.
The most important of all features were considered to be the timelinessland
relevance of the information for decision making.
- 8 -
It is therefore. not surprising that many monitoring and evaluation
systems do not meet these expectations. From a review of the experiences
presented in the case studies, it appeared that information resulting from
monitoring of rural development projects in East Asia has seldom been used
effectively.
The workshop dealt at length with the major causes of this ineffec-
tive use of existing monitoring systems. The reports from the working groups
indicated several reasons.
First, few project managers are highly committed to monitoring.
Since some project managers.had neither been involved in the design of the
project nor in the design of its monitoring and evaluation component, they
felt little commitment to it. Some of those who were committed had a poor
relationship with the M&E staff. Others had serious-doubts about the-quality
of the information presented by the monitoring unit.
Second, the monitoring system-itself may be at fault. Information
produced by the monitoring unit may be poor, too. elaborate, or irrelevant to
the needs of project management. Alternatively, the resources made available
for monitoring may be inadequate. The products or results from monitoring may
have been poorly communicated to the project manager. Experience of some
participants was that monitoring results were presented with a "take it or
leave it" attitude. Particularly sensitive information that is poorly com-
municated can raise questions as to the integrity of the project manager or
of the agency involved in the implementation of the project.
Finally, workshop participants felt that ineffective use was being
made of monitoring information because project managers often lacked the
-9-
necessary qualifications, authority, and power to reallocate resources during
project implementation. In general, it was felt that project managers often
lack a basic appreciation for the monitoring efforts and for the benefits that
can be derived from more effective use of feedback on the project.
- 10 -
SECTION III: THE TECHNICAL ASPECTS OF MONITORING
In the discussion on why information on monitoring and evaluation
has not been used effectively by project management, participants frequently
referred to delays in supplying appropriate information to management. In
this, the experience of East Asia and the Pacific has much in common with
that of other regions. The delays that cause M&E to be ineffective can be
traced to the lack of adequate manpower and financial resources allocated for
the.purpose and to deficiencies in the technical aspects. Data collection,
processing, and analysis, as well as the reporting format, are discussed in
this section, while the institutional aspects are considered in Section IV.
A. Data Collection
Several working groups referred to the problem of collecting too
much information. Often information is collected in such detail that (a) it
extends beyond the relevant needs of the project; (b) it exceeds the capacity
of the staff or the available processing equipment; (c) it imposes intolerable
demands on the beneficiary farmers; and (d) it duplicates information already
available..to project management. Participants realized, that.much more effort
should be devoted to clearly identifying the specific objectives of rural
development.projects in order to arrive, at a limited number. of meaningful
indicators- that would facilitate.decision making in relation;-to the project
activities. The provision of-staff'for the purpose of.-data collection is.aan
institutional issue that will be.discussed in Section IV. The question,of
appropriate equipment for dealing-wi.th:the collected data will be discussed.
in connection with data:processi-ng,. as. will the question of the relationship
between the format of the questionnaire and the method of dTata proces.sing;..
Nonetheless, these are important design considerations. Data can be speedily
processed only if the collection instrument is compatible with the method for
processing the data. The use of cumbersome questionnaires may negate the
goodwill and cooperation of the beneficiaries in providing the information
necessary for efficient M&E of projects. There is also a great need for
reviewing the information being obtained by other agencies. As much as
possible, such regularly available information should be integrated into the
system to avoid unnecessary duplications and inconveniences to the benefi-
ciaries, as well as unnecessary cost.
The working groups urged the exploration of forms of data collection
other than structured questionnaires. Possible methods include field observa-
tions, in-depth case studies, windscreen surveys, staff records, official
diaries, and the like.
Most of the case studies describe sample surveys as the basic
design for data gathering. However, little thought has so far been given to
estimating sampling errors. Sample sizes are often chosen arbitrarily, with-
out prior statistical analysis on means and variances of known parameters.
The adequacy of relatively smaller sampling proportions in situations where
the target population is either very large or fairly homogenous has not been
sufficiently considered.
Nonsampling errors also occur when the interviewing procedure is
designed by untrained staff. Reliability and quality of the data can further
be compromised by inexperienced enumerators and lack of sufficient field
supervision. Training of field staff has not been given enough attention, nor
has the organization of M&E surveys provided adequate resources for effective
- 12 -
control and verification. These problems are manifest particularly in ela-
borate baseline surveys, where enumerators are hired temporarily to supplement
the core staff. Lack of commitment to the exercise, and lack of familiarity
with the purpose of the survey and the project, often result from limited
budgetary resources. All these factors hinder efficient collection of reli-
able information from the field.
The use of supplementary information from reports by the partici-
pating agencies in multi-component projects is another problem area. Several
of the case studies highlighted the difficulty of acquiring data and informa-
tion from these sources in a timely manner.
The M&E staff in larger projects with extensive geographical cover-
age often have been assigned the additional function of collecting regional
data not specifically relevant to project operations, without the provision
of commensurate resources. In these cases, the meager resources for M&E are
often exhausted, so that information gathering has little meaning for project
implementation.
In the plenary session on data collection, participants recommended:
-- A limited set of indicators to satisfy the immediate needs of
project management should be identified. Clearly, this would
imply a thorough understanding of the project-s-objectives and
goals. It is also imperative that the end-users, particularly
the project manager, should be fully involved in evolving these
indicators. It would also be helpful if some of the indicators
paved the way for both ongoing and ex-post evaluations.
-- The designers of the M&E system should carefully review the
range of information that is already being gathered to
- 13 -
avoid duplication of effort. To the extent possible,
existing reporting channels should be examined, with a view
to integrating these procedures into the M&E setting.
Data collection over extensive areas of fairly homogenous
conditions could profitably be conducted on the basis of a
random sample. The choice of sample size depends, however,
on sound estimates of the variance in the population, based on
professional advice and prior knowledge of the basic parameters.
Quantity is often not a good substitute for quality. Where
project effects can be readily seen, a relatively small sample
may be sufficient to determine the impact of the project.
If a questionnaire is used, possible alienation of the
respondent through time-consuming interviews or the soliciting
of delicate information should be avoided. Persons with field
experience in rural sociology could be very helpful in this
respect. The format should facilitate data processing of the
questionnaire.
Quality of the data can be assured only if the interviewers
are given sufficient training. Personnel hired temporarily
for data collection tend to be inexperienced and require
more training than those who have performed similar duties
in the past. In all cases, supervision must be provided,
preferably by personnel of the M&E Unit.
- 14 -
-- Methods other than large-scale surveys to collect data
need to be explored. The combination of several data
collection techniques could often be advantageous. Formal
and informal approaches, systematic and ad hoc studies,
intensive interviews, surveys, and case studies are often
compatible and can be used simultaneously.
B. Data Processing and Analysis
Participants generally agreed that the transforming of raw data
into information is vital for effective monitoring and evaluation. Data
that are not promptly processed cannot be used by management for decision
making and therefore waste both effort and resources.
It was also recognized that processed data should be interpreted
analytically to produce a basis on which to take management action. Although
all participants agreed that the processes of data collection, processing, and
analysis should be action-oriented, there was considerable disagreement on
whether M&E should make specific recommendations for management action. Some
participants thought that this might lessen the authority of the project
manager, while others argued that recommendations for remedial action enhance
the value of M&E as a management tool. This difference of opinion should not
constitute a serious problem when the project manager takes an active role
in all stages of the design, development and execution of M&E. The technical
standards of M&E should be sufficiently high to generate information that is
both timely and reliable for use in management decisions.
- 15 -
Problems relating to data processing and analysis as revealed in
the case studies were discussed at length. Critical constraints that were
identified were: limited staff, with relatively limited analytical skills
and experience; inadequate data processing equipment; uncertainty of require-
ments; and lack of user confidence in the data base.
Important suggestions given by the participants to resolve
these problems were the following:
-- The volume and format of data that need to be collected
should be compatible with the capacity for data processing
analysis, in terms of both manpower and equipment.
-- Short-term involvement of personnel external to the
project might be required to overcome the shortage of
experience and competence in interpretative analysis of
data. In any case, training of data processing staff is
essential to ensure effective M&E. (The issue of using
external resources is discussed in more detail in
Section IV of this report).
-- The selection of appropriate data processing equipment
depends on the volume of data to be processed, the avail-
able time, the capacity of the staff, the accessibility to
appropriate equipment, the operating environment, and the
required analytical depth. The actual equipment could range
from the traditional abacus to modern electronic data
processing tools. Equipment that does not satisfy the
demand for rapid analysis, for efficient storage, and for
retrieval at a later date can cause considerable problems
and reduce the effectiveness of the M&E system.
- 16 -
-- It is important to distinguish between requirements for
monitoring and those for evaluation to improve future
design of projects. Speed in presenting the findings
is of the utmost importance in the area of monitoring
so that remedial actions affecting project implementation
can be undertaken on a timely basis. Identification
of the end-products of various stages of data analysis
is essential.
C. Presentation of Monitoring Results
Participants in the workshop recognized that there is an endemic
failure to bridge the communication gap between M&E staff, project management,
and higher-level decision makers and planners. Problems of report presen-
tation and information flows can often be traced to voluminous reports with.
vast amounts of inadequately analyzed information. Salient findings and
implications are often obscured by detail. The inability to present a
succinct report of the changes in the project environment and the effects of.
project activities often arises from a lack of perception and understanding
of the objectives and.goals of the projects. To solve these problems, the
participants advocated the importance of brief, concise, specific and action-
oriented reports. The use of tables, charts or other forms of visual pre-
sentation of data can be very helpful in making the narrative more interesting.
In determining the content of the various reports, it may be helpful
to recognize the types of users and their specific requirements for informa-
tion. The output from monitoring and evaluation should be organized in such
a way that advance reports precede the routine ones, in order to assure
timeliness. -In addition, ad hoc reports requested by management should be
produced, if possible, from material that is efficiently stored.
- 17 -
SECTION IV: INSTITUTIONAL ASPECTS OF MONITORING
A. Organization.of Monitoring and Evaluation
In discussing the lessons learned from national experiences in
the East Asia and Pacific region, the participants recognized that the design
of M&E Units should be adapted to the specific institutional and administra-
tive framework of the country. In general, introduction of M&E as a new and
formalized activity into an existing bureaucratic structure will be more
effective if prevailing rules, norms, and social and cultural values of the
established administrative system are taken into consideration.
All working groups emphasized that a high degree of sensitivity
toward criticism and an entrenched respect for senority and age are implicit
in most Asian cultures. These basic beliefs may intensify the cultural
difficulties confronting project evaluators. Communication of sensitive
evaluation findings to project managers, and especially upward through the
organization, should be made within the local cultural context without com-
promising the substance of the conclusions. To achieve this, it was suggested
that
(1) direct personal communication was more appropriate
than public exposure;
(2) small group discussions were better than large meetings;,
(3) self-evaluation of shortfalls should be encouraged;
(4) officials should be educated about the role of
evaluators; and
(5) culturally appropriate channels need to be identified.
To ensure the effectiveness of monitoring and evaluation as
an important management tool, project managers should recognize the contri-
- 18 -
bution that the system could make in improving management efficiency. Project
designers, managers, and evaluation personnel strongly endorsed the view that
project planners and managers should be involved in the design of the M&E
system so that the function of M&E as a management and planning tool will
be more fully appreciated and the output of the system will be more relevant
to management needs. The feedback will therefore more readily be used in the
decision-making process. Although project managers should not be expected to
be conversant with the techniques of M&E, they should clearly see the symbiotic
relationship between project management and M&E. In other words, project
managers should be able to depend on the results of M&E as a sound basis for
decision making, while the M&E unit should rely on project management for full
cooperation as well as for logistic and institutional support. The effective
use of the results from the M&E for improving project performance requires a
significant degree of cooperation, commitment, and support by project manage-
ment. Skepticism, suspicion; and defensiveness on the part of project man-
agers can render the M&E system completely ineffective.
Both monitoring and evaluation should be of a broad technical or
economic character and should assess and report on the reaction of the project
beneficiaries to project inputs and activities. Evaluation should take into
account not only objective limitations on the resources of the beneficiaries
but also constraints derived from traditional patterns of subsistence, social
organization, structure of authority, and other factors, such as religious
beliefs and practices, that may hamper project implementation.
Monitoring should result in a diagnosis of problems and in an
attempt to prescribe solutions and recommendations within the project
- 19 -
framework. The implementation of M&E findings is the responsibility of
project managers, higher management levels or policy making bodies. There is
usually a hierarchy of decision-making levels, and information derived from
the M&E should be directed through the appropriate institutional channels to
each level where action is required. In this manner, the question of the
flexibility and the authority of the project manager prescibed by project
design could be overcome. The feedback from M&E can provide project managers
with a relatively limited scope for reallocation of resources and with the
necessary evidence for requesting additional inputs or necessary policy
changes from higher authorities to ensure effective project implementation.
M&E units can be area-specific or sector-specific. The model of the
unit should vary according to the nature of the project and the subsector,
such as irrigation, livestock, credit, extension, or fisheries, to which the
project belongs. For optimal use of scarce human resources and skills, a
two-tiered M&E system may be necessary for development projects. A two-tiered
system consists of a central unit, well-staffed and well-equipped, linked to
smaller project-specific units and supporting them with technical backup.
For rural development, individual components are often implemented
by the field staff of different specialized agencies, in which case the
project management performs essentially a coordinating role. In these
instances, participants strongly recommended that the flows of information
should be channeled to such agencies.
B. Role of Consultants
To overcome the critical shortage of skilled manpower for project-
specific M&E -- a shortage that is prevalent in most of the countries in
East Asia - a number of projects rely on external resources, such as local
- 20 -
universities, consultant firms, or individual consultants for implementing
M&E. The workshop participants discussed the trade-offs between the building
of in-house expertise and the use of external experts. It was recognized that
independent consultants can, in the short run, provide critical technical
skills, as well as detached judgment and objectivity, particularly in more
complicated and innovative types of projects. The freedom of such independent
consultants from day-to-day operational involvement can have the advantage of
furnishing project management with unbiased advice. Flexibility in the
matching of consultants with the specific tasks to be performed is also an
advantage. However, these benefits should be weighed against some disadvant-
ages that are difficult to avoid. The sporadic and temporary involvement of
consultants in the project is not conducive to the establishment of the
necessary rapport between evaluation personnel and the project manager. In
the case of contracts with universities, M&E may turn into academic research
with slow feedback and little operational relevance to project management.
Most workshop participants felt that, on balance, the building up of in-house
M&E expertise may entail delays but is still preferable. This latter choice
would not only provide vital continuity throughout the life of the project,
but also give more direct feedback for management and planning at signifi-
cantly lower costs, and could be viewed as an investment in the development of
indigenous expertise. The benefits of the built-up expertise often outlast
the project and can thus be considered as an "institution-building" effort for
the country.
Given the critical shortage of expertise in many of the countries in
the East Asia and Pacific region, participants acknowledged that consultants
may play a role in the immediate future, especially in the design of the
- 21 -
systems for data collection, processing and analysis. Inputs to supplement
and guide the project personnel in implementing the M&E system can be benefi-
cial through transfer of skills.
The issue of establishing institutional safeguards for objectivity
and creditability of in-house evaluation was also discussed, and possible
solutions were suggested. The consensus was that M&E staff should be part
of project management, but individuals should be able to pursue a career
in a larger framework, independent of the project but dependent on the per-
formance of the task assigned.
C. Staffing Requirements
Most participants emphasized the need for a multi-disciplinary
team in the M&E units, combining economic, socioanthropological, and technical
competence. During the discussion of the desirable characteristics of the
monitoring and evaluation staff for the units, a workshop exercise was carried
out to heighten the ability of project managers in selecting the appropriate
personnel for the task and that of M&E candidates in presenting their poten-
tial contributions to management. The specific purpose of this role-playing
exercise on Interviewing Candidates for the Position of Monitoring/Evaluation
Officer" was to extract from the participants the set of professional and
ethical characteristics necessary for performing the function of M&E
effectively.
Some of the desirable characteristics of monitoring and evaluation
personnel that were identified through this exercise include:
-- training in economics and sociology and experience in field
studies;
- 22 -
-- a high degree of sensitivity to the needs, values and ways
of expression of rural people;
-- commitment and courage to signal shortfalls in project
performance or other sensitive findings, even if risks
are involved; and
-- awareness of the importance of the integrity and credit-
ability of the evaluator's status.
The discussion also revealed that the incentives and career pros-
pects that are usually offered have generally been inadequate to attract
personnel of the necessary caliber for monitoring and evaluation. During the
role-playing exercise, it was observed that some project managers could not
express their own expectations for the M&E staff and could not describe the
working environment. On the whole, all participants appreciated the oppor-
tunity of serious discussion of this human aspect of recruiting M&E staff of
acceptable quality.
D. Other Resource Requirements
Participants at the workshop also exchanged ideas of specific
provisions for M&E on the projects with which they were associated. Although
most projects require some monitoring, it was noted that a significant pro-
portion of World Bank-assisted agricultural and rural development projects in
East Asia and the Pacific do not contain specific provisions for monitoring
and evaluation. This observation on the part of the participants is further
illustrated in Table 1, which lists all agriculture and rural development
projects supported by the World Bank in East Asia and the Pacific over the
past five years. Out of 64 projects, some 70% had references to monit6ring
and evaluation in their project appraisal reports. However, only 17 out of 45
- 23 -
projects with M&E contained specific resource provisions. The amounts allo-
cated for M&E, as well as the percentages these represent in the total base
costs of the projects, are also shown in the table. This analysis shows that
11 out of the 17 projects with specific provisions for M&E allocated less than
1% of the base project costs; 3 projects allocated between 1% and 2% of the
base costs; and 3 projects allocated a range of 2% to 5% for monitoring and
evaluation. The establishment and effective operation of M&E units is
administratively more difficult, if not impossible,,if specific provisions for
M&E are not included in the project design.
MONITORING AND EVALUATION OF
AGRICULTURE AND RURAL DEVELOPMENT PROJECTS IN
EAST ASIA
FISCAL TOTAL TOTAL M/E
YEAR COST OF PROJECT BASE AS %
PROJECTS M/E COST COSTS OF BASE
COUNTRY NAME WITH M/E US$'000 US$ MIL USJ MIL COSTS
75 AGRICULTURE
INDONESIA AG RESEARCH & EXTENSION 0 0 46.5 30.50 0
MALAYSIA MARDI AGR. RESEARCH 1 100 108.6 72.80 0.1
PHILIPPINES TARLAC IRRIGATION 0 0 34.0 21.60 0
RURAL DEVELOPMENT
INDONESIA IRRIGATION VI 1 600 165.0 95.50 0.6
MALAYSIA KERATONG LAND SETTLEMENT 0 0 98.7 65.30 0
PHILIPPINES RURAL DEVELOPMENT 1 0 50.0 31.00 0
76 AGRICULTURE
FIJI SUGAR DEVELOPMENT 1 0 26.0 18.50 0
KOREA, REPUBLIC OF LIVESTOCK II 0 0 24.5 17.50 0
RURAL INFRASTRUCTURE 1 300 143.5 126.10 0.2
PHILIPPINES SECOND FISHERIES 0 0 23.5 17.60 0
SECOND GRAIN PROCESSING 1 0 28.5 21.80 0
SECOND LIVESTOCK 1 50 41.3 32.70 0.2
THAILAND IRR. VI-PHITSANULOK 1 0 210.0 121.80 0
LIVESTOCK DEVELOPMENT 0 0 11.5 8.50 0
RUBBER REPLANTING I 1 0 148.0 124.60 0
RURAL DEVELOPMENT
INDONESIA IRRIGATION VII 0 0 60.0 41.40 0
NATIONAL FOODCROPS EXT. 1 0 44.2 33.10 0
MALAYSIA NORTH KELANTAN RURAL DEV. 0 0 48.0 31.70 0
PHILIPPINES CHICO IRRIGATION 1 900 84.0 53.20 1.7
MAGAT MULTIPURP. STAGE I 0 0 84.0 51.80 0
THAILAND N E RURAL DEVELOPMENT 0 0 45.0 32.40 0
77 AGRICULTURE
KOREA, REPUBLIC OF AGRIC. WATERSHED DEV. I 0 0 75.0 43.70 0
AGRICULTURAL CREDIT II 0 0 41.2 32.30 0
IRRIG. YONG SAN GANG II 0 0 167.0 97.80 0
PHILIPPINES AGRIC. CREDIT IV 1 100 91.3 80.30 0.1
THAILAND NATIONAL AGRIC. EXTENSION 1 400 56.5 42.60 0.9
2ND CHAO PHYA IRRIG. IMPR. 1 200 112.0 76.40 0.3
RURAL DEVELOPMENT
INDONESIA IRRIGATION IX 0 0 64.0 44.10 0
IRRIGATION VIII 1 0 118.0 76.50 0
TRANSMIGRATION I 1 1700 56.8 41.00 4.1
MALAYSIA NATIONAL SMALL-SCALE IRR. 1 0 89.0 63.60 0
PAPUA NEW GUINEA AGRICULTURAL DEVT. IV 0 0 18.5 11.90 0
MONITORING AND EVALUATION OF
AGRICULTURE AND RURAL DEVELOPMENT PROJECTS IN
EAST ASIA
TOTAL TOTAL M/E
FISCAL COST OF PROJECT BASE AS %
YEAR PROJECTS M/E COST COSTS OF BASE
COUNTRY NAME WITH M/E US$'QQQ US$ MIL US$ MIL COSTS
77 RURAL DEVELOPMENT
PHILIPPINES IRRIGATION V (NISIP I) 0 0 107.2 73.30 0
JALAUR IRRIGATION 0 0 34.0 20.90 0
RURAL DEVELOPMENT II 1 0 32.6 24.10 0
78 AGRICULTURE
INDONESIA RURAL CREDIT I 1 500 60.0 43.50 1.1
KOREA, REPUBLIC OF OGSEO AREA DEVT. PROJECT 1 0 76.0 53.50 0
RURAL INFRASTRUCTURE II 1 300 232.0 182.30 0.2
RURAL DEVELOPMENT
INDONESIA IRRIGATION X 1 0 216.0 131.00 0
IRRIGATION XI 1 0 47.4 31.30 0
NUCL. EST. & SMLHDRS. II 1 0 100.5 60.55 0
NUCLEUS ESTATES & SMLHDR. I 1 1900 134.0 88.70 2.1
LAO, P.D.R. AGR.REHAB. & DEVT. 1 400 11.9 8.80 4.5
MALAYSIA LAND SETTLEMENT FELDA VI 1 0 92.3 66.90 0
NATIONAL EXTENSION 1 0 46.5 33.10 0 u
NW SELANGOR RURAL DEVT. 1 0 60.0 41.70 0
PHILIPPINES IRRIGATION MAGAT II 0 0 346.0 276.40 0
IRRIGATION VII - NISIP II 1 900 140.0 93.90 1.0
RURAL INFRASTRUCTURE I 1 0 59.0 43.20 0
SMALLHOLDER TREEFARMING 1 0 16.0 12.90 0
79 AGRICULTURE
LAO, P.D.R. AGR. REHAB. & DEVT. II 1 0 14.4 11.70 0
PHILIPPINES NATIONAL EXTENSION 1 400 70.1 55.95 0.7
VIET NAM IRRIGATION I 0 0 110.0 90.20 0
RURAL DEVELOPMENT
INDONESIA CIMANUK LOWER BASIN FLOOD CTL. 1 0 77.0 52.40 0
IRRIGATION XII 1 0 118.5 82.70 0
TRANSMIGRATION II 1 0 242.0 148.90 0
MALAYSIA KRIAN/SUNGEI MANIK IRRIG 1 0 60.2 43.90 0
MUDA IRRIGATION II 1 200 69.0 52.00 0.4
--- SMLHLDRS COCONUT DEVT. -1 0 44.2 32.80 0
PAPUA NEW GUINEA RURAL DEVELOPMENT I 1 0 32.2 25.40 0
PHILIPPINES MAGAT RIVER IRRIGATION 1 0 62.0 47.80 0
SMALL FARMER DEVT. 1 0 37.0 32.60 0
THAILAND NE IRRIGATION II 1 0 80.0 56.40 0
NORTHERN RURAL DEVT. 1 110 47.5 35.50 0.3
TOTAL REGION 45 9060 5289.6 3713.90
- 26 -
SECTION V: THE DESIGN OF MONITORING AND EVALUATION SYSTEMS:
THE FIELD EXERCISE AT MUDA
During the working group and plenary sessions in Kuala Lumpur, the
workshop participants had an opportunity to discuss in detail the various
aspects of M&E and to share their specific experience in monitoring different
kinds of rural development projects. A field exercise was therefore a logical
follow-up of the working session in order to focus on the issues in a practical
manner. The need for such an exercise was suggested by the participants at the
workshop in Nairobi, and the field trip organized in connection with the San
Jose workshop was highly appreciated by the participants.
The venue of the field exercise for the Kuala Lumpur Workshop was
the Muda Irrigation Scheme 1/, located at Alor Star. The project is an
1/ The irrigation area, situated 500 km north of Kuala Lumpur, occupies a
flat alluvial coastal plain about 20 km wide and 65 km long. There are
60,000 farm families in the area, with an average holding size of 1.6 ha,
but about a third of the farms have less than 1.0 ha. These farmers are
approximately equally divided into the three land tenurial status cate-
gories of owner-operators, tenants, and owner-tenants. The project aimed
at providing the necessary infrastructure for introducing double-cropping
over the service area, previously devoted to a single annual rice crop.
Agricultural research and extension, as well as agricultural credit, re-
ceived support. This first phase of development was a notable success,
since it led to an increase in the cropping intensity from about 95% to
almost 185% between 1969 and 1974. In constant preproject terms, farm
incomes had doubled in the same period. Almost all of the income increases
were accounted for by production increases as a result of adopting double
cropping and of sowing high-yielding rice varieties. The completion
report for the project was made much easier because of the monitoring
and evaluation system included in the project design.
The Muda Scheme has recently been the recipient of a second loan from the
World Bank to further improve the efficiency of the irrigation system
through the provision of tertiary canals and other on-farm development
measures. Apart from the planned improvements in infrastructure, which
will lay the foundation for another major breakthrough i'n rice production
and farm incomes, the second phase of the project includes a more
intensive extension system, an improved credit system, an effective input
suply system, and an improved operations and management system. The
project also has provision for continuing the monitoring and evaluation
system. The measurement of the secondary effects and impact of the
project was the subject of a special FAO/World Bank Research Project
("The Muda Study: FAO/World Bank Cooperative Programme," Rome,
September 1975, 2 volumes).
- 27 -
important Malaysian attempt to increase rice production in what originally was
a very poor area of the country. The M&E activities at Muda are generally
considered to be among the most successful, and Muda's experience has made a
significant contribution to the advancement of project-specific M&E in the
East Asia region. A number of reports on the results of monitoring and
evaluation have been prepared. A detailed analysis of the impact of the Muda
project has also been published. 1/ The Muda project was the subject of a
special FAO/World Bank research project to measure the secondary effect and
impact. 2/
The objective of the workshop field exercise was not the designing
of a monitoring and evaluation system for the project per se, nor merely to
observe the actual activities in Muda. The purpose of the exercise, given the
limited time available, was to involve the participants in designing, as
realistically as possible, practical M&E subsystems for the various project
components. The results of these exercises were compared with the actual M&E
system, so that the M&E staff of Muda could derive some benefits from alter-
native approaches suggested by the participants. With these possibilities in
mind, the participants were reorganized into working teams according to their
experience and expertise in monitoring and evaluating specialized types of
rural development projects.
In preparation for the workshop field exercise, a detailed briefing
on the Muda Project was given by S. Jegatheesan, Head of the Planning and
Evaluation Division of the Muda Agricultural Development Authority (MADA).
The working teams were then assigned the task of devising, as an exercise, a
monitoring and evaluation system for the five key areas of project operations:
1/ See, for example, "The Green Revolution and the Muda Irrigation Scheme:
An Analysis of its Impact on... .The Distribution of Rice Farmer Income,"
by S. Jegatheesan, Office of the General Manager, Muda Agricultural
Development Authority, 1977.
2/ "The Muda Study: FAO/World Bank Cooperative Programme," Rome,
!pntp mhpr 1975- 2 vol]lmef:.
- 28 -
(1) irrigation water supply;
(2) agricultural credit;
(3) agricultural input supply;
(4) agricultural extension services; and
(5) socioeconomic dimensions.
Although the monitoring and evaluation of the Muda project is a well-integrated
system, it was expedient for the working teams to concentrate on the particular
subsectors within the limited time available for the exercise. The intimate
interrelationship among these subsectors was, however, stressed,at the briefing,
so that the participants could maintain this perspective during the design,
field work, and post-exercise presentation session.
Each working team considered the project component that it studied
as an integral project in -itself -- identifying its objectives and targets,
assessing it information needs, the.quality of the existing reporting system,.
the need..for additional monitoring,information,.and the data requirement for
impact evaluation, designing data collection.tools,.and recommending an
institutional model for a project-specific M&E Unit, with appropriate funding
for staffing.
The Irrigation Water Supply Team identified the optimal utilization
of the scarce water resources, particularly at the beginning of the growing
season, as the objective of its component. The target is naturally the
maximim degree of cropping intensity under an appropriate water supply sched-
ule. The irrigation system is, to a large extent, controlled by the computer
center, but accurate information on the agricultural activities throughout the
service area is needed to compute the water requirements at various locations.
The information from the field starts with the irrigation overseers and goes
- 29 -
through the irrigation inspectors to the district operations engineers.| The
summary of the data is communicated to the MADA management through a report on
the performance of the irrigation plans and problems encountered, as a means
of coordination with other cooperating agencies, such as rural credit, input,
and extension agencies. The M&E model suggested by the working team was
broadly similar to the system actually being used by the project, but the team
had also identified some likely problems in the existing system of which
project personnel should be aware. The discussions between the participants
and the field staff had been very fruitful and mutually beneficial.
The Agricultural Credit Working Team saw as the objective for its
component the provision of an adequate service of rural credit for promoting
modern agriculture, with the social advantage of placing purchased inputs
within reach of the poorer farmers. The target is to maximize the use of
institutionalized credit in order to achieve both the technical goal of in-
creased agricultural productivity and the social objective of equality in
access to development opportunities, as well as to avoid exploitation of one
section of the community by another. The team recognized the importance of
making available the loan proceeds in 'close coordination with the stages in
the cycle of agricultural activities, and therefore timeliness in the supply
of rural credit was considered a critical indicator for management. Informa-
tion on actual loan requirements for fertilizers, farm chemicals, land re-
demption, land purchase, and supplementary hired labor for plowing, trans-
planting, and harvesting is also useful for planning a project-wide supply of
rural credit. Credit supervision would be carefully monitored to ensure
efficiency and prevent unacceptable delinquencies. The participants felt
that as loans for the MADA scheme cover only some types of credit, mainly for
- 30 -
paddy and tobacco cultivation and for purchase of farm equipment and machinery
-- while other agencies provide loans for agribusiness, land purchase and
redemption, estate development, fisheries, and cooperative development --
monitoring and evaluation had an important role to play in ensuring close
cooperation and coordination among all the participating agencies. Drawing on
the experience of their own countries, the team members had a valuable exchange
of opinions with the staff of both MADA and the other operating agencies.
The Agricultural Input Supply Working Team considered the maximiza-
tion (according to technical recommendations) of farm inputs and their supply
through institutionalized channels as the primary objectives of its compo-
nent. Members of the team reviewed the present supply of chemical fertili-
zers, farm chemicals, improved seeds, farm machinery, diesel fuel oil for
farm equipment, and hired labor. The team then identified the information
that would be required by project management to ensure adequate supply of
these inputs on a timely basis. The importance'of close cooperation with the
credit institutions operating in the project area was recognized. The
team then examined the existing reporting system critically, indicating areas
where additional data would be required and suggesting the method for collect-
ing such data. Suggestions were also made for designing appropriate data
collection instruments and for relating them to data processing equipment and
procedures. The team proposed a concise reporting format and discussed the
possible uses of the information by project management. Staffing needs for
monitoring and evaluating the input supply were also proposed.
Agricultural extension is an important link in the MADA strategy for
achieving the goals of increasing cropping intensity and raising rice yields.
The aim of this project component is therefore the promotion of modern farming
- 31 -
methods and farm inputs. To achieve this objective, the project is to install
the training and visits system of agricultural extension as a means of
accelerating the adoption of proven practices. If this approach is to be
successful, it is essential to strengthen the agricultural extension service,
improve the performance of the field staff, and coordinate with the supporting
agencies in matters of input and credit supply. The target is the adoption
of improved practices by 20,000 farm families cultivating 24,500 ha by 1983.
Additional agricultural technicians and subject matter specialists will be
recruited; vehicles will be purchased to increase the mobility of the field
staff; equipment and buildings will be augmented. To monitor the progress of
this component, the working team identified a range of indicators for the
inputs into the system and the activities of the extension workers. The
selection of contact farmers (an essential element in the training and visits
system) and the training of the necessary agricultural technicians are other
key areas in which information is needed by management. The working team
also recommended that the reporting system should be more formalized, with
further effort to verify the information supplied by the field staff. A
sample survey by means of a structured questionnaire would form the basis for
collecting information on the target population. For information that does
not lend itself to regular reporting, ad hoc studies (for example, on the
selection of contact farmers and the impact of training) could be useful. For
evaluating the impact of the whole extension component, crop cutting surveys
should also be considered. A more detailed questionnaire to be administered
to a small sample would provide some of the information required for evalua-
tion. The use of micro-computers to facilitate data entry and analysis was
also considered advantageous. Monitoring would assist management in taking
- 32 -
day-to-day operational decisions; ongoing evaluation would provide for
remedial action involving changes of project-specific objectives and strate-
gies. Ex-post evaluation would provide the basis for planning future projects.
The Socioeconomic Working Team concentrated on evaluating the impact
of the project in terms of increased income through better farm production by
means of irrigation and institutionalized input supply and in terms of the
equitable distribution of such income increases. The information required
would include measurement of farm outputs, farm gate prices, and costs of
labor and inputs, as well as subsidies and charges. The data would be derived
from a sample of farmers representative of various tenurial groups and farm
sizes. The team reviewed the existing data base and proposed that additional
data be required. Apart from the survey questionnaire, the incorporation of
supplementary and ancillary sources of information was also recommended. The
team estimated the staffing needs and stressed the training of the enumerators.
The installation of micro-computers and mini-computers for rapid data process-
ing was considered. The team suggested external inputs to increase the depth
and detail of analysis, discussed efficient use of the information by manage-
ment, and considered an appropriate format for reporting.
During the field exercises, the participants were confronted, in a
practical way, with typical questions that arise in the design and implementa-
tion of project-specific monitoring and evaluation systems. This practical
experience enhanced the value of the regional workshop methodology.
In the workshop evaluation, participants were unanimous in endorsing
the value of the field exercise and appreciative of the cooperation and hos-
pitality of the host country in providing the opportunity of conducting field
work in such a meaningful fashion.
- 33 -
ANNEXES
- 35 -
ANNEX 1
Page 1
LIST OF PARTICIPANTS
Name and Title Address
Mr. Nemani BURESOVA Division of Economics, Planning
Chief Economist and Statistics
Ministry of Agriculture
P.O. Box 358
Suva
FIJI
Mr. Michael CERNEA Rural Operations Review & Support Unit
Sociologist Agriculture and Rural Development Dept.
World Bank, 1818 H Street N.W.
Washington, D.C. 20433
U.S.A.
Mr. Ah Kiow CHAN National Extension Project
Agriculture Officer Extension Branch
Department of Agriculture
Jalan Swettenham
Kuala Lumpur
MALAYSIA
Dr. Suthiporn CHIRAPANDA Research and Planning Division
Director Agricultural and Land Reform Office (ALRO)
Ministry of Agriculture and Cooperatives
166 Pradipat Road
Sapan Kwai, Bangkok
THAILAND
Mr. Yong-Sang CHOI Minyon Experiment Farm
Head Agricultural Development Corporation
P.O. Box 12
Anyang, Yonggi-do
KOREA
Mr. Ted DAVIS Rural Operations Review & Support Unit
Chief Agriculture and Rural Development Dept.
World Bank, 1818 H Street N.W
Washington, D.C. 20433
U.S.A.
Mr. Guido DEBOECK Rural Operations Review & Support Unit
Economist Agriculture and Rural Development Dept.
World Bank, 1818 H Street N.W.
Washington, D.C. 20433
U.S.A.
- 36 -
ANNEX 1
Page 2
Name and Title Address
Mr. Rodrigo N. de GUZMAN Input-Output Monitoring Unit
Project Leader National Irrigation Administration
N.I.A. Building
E. de los Santos Avenue
Diliman, Quezon City
PHILIPPINES
Mr. Candido DoIZON' Agricultural ProJect Preparation Unit
Chief Supervising Consultant Ministry of Agriculture
Elliptical Road, Diliman
Quezon City
PHILIPPINES
Mr. Doug,FORNO World Bank Mission
Agriculturalist Udom Vidhya Building
956 Rama IV Road
Saladaeng
Bangkok 5
THAINLAND
Mr. Phadoongkarn HUNGSAVAISAYA Second Chao Phya Irrigation Improvement
Project Manager Project
Royal Irrigation Department
Samsen Road
Bangkok
THAILAND
Mr. Altaf HUSSAIN World Bank
Chief, Agriculture and Rural P.O. Box 324/JKT
Development Division Jakarta
INDONESIA
Ms. Normah Wan ISMAIL North Kelantan Rural Development Project
Monitoring Officer Lot 1751, Jalan Sultan Yahya Petra
Kota Bharu, Kelantan
MALAYSIA
Mr. Zaharudin bin JAAFAR North Kelantan Rural Development Project
Project Manager Lot 1751 Jalan Sultan Yahya Putra
Kota Bharu, Kelantan
MALAYSIA
Mr. Abdul Wahid JALIT National Extension Project
Director Department of Agriculture
Jalan Swettenham, Kuala Lumpur
MALAYSIA
- 37 -
ANNEX 1
Page 3
Name and Title Address
Mr. Hyeon-Seok KOH Agriculture Credit Project
Deputy Manager, Technical Unit National Agricultural Cooperatives
Federation
75 lst-ka, Chungjeong-ro, Jung-ku
Seoul
KOREA
Mr. Claes LINDAHL UNDP
Economist State and Rural Development Project
Kuala Lumpur
MALAYSIA
Mr. Leopoldo MAGPALE Rural Banks & Savings & Loan Associates
Assistant Director Central Bank
Manila
PHILIPPINES
Mr. Ronald NG School of Oriental & African Studies
Consultant University of London
Malet Street
LONDON WC IE 7EP
UNITED KINGDOM
Ms. Josefina NUNEZ Second Rural Development-Land Settlement
Monitoring & Evaluation Officer Project
Ministry of Agrarian Reform
CCA Building
38 Timog Avenue
Quezon City D-3008
PHILIPPINES
Mr. Pelagio PASTOR Central Project Management Unit
Planning & Monitoring Officer Second Rural Development-Land Settlement
Project
CCA Building
38 Timog Avenue, Quezon City
PHILIPPINES
Dr. Antonio PERLAS Bancom Health Care Corporation
Managing Director Bancom III Building, Rada Corner
Le Gaspi Street, Le Gaspi Village
Makati, Metro Manila
PHILIPPINES
Mr. Francisco G. RENTUTAR Bureau of Agricultural Extension
Director Ministry of Agriculture
Elliptical Road, Diliman
Quezon City
PHILIPPINES
- 38 -
ANNEX 1
Page 4
Name and Title Address
Dr. Ralph RETZLAFF Regional Research & Training Program
Director Agriculture Development Council
P.O. Box 11-1172
Bangkok 11
THAILAND
Ms. Edith SEGGAY Farm Level Monitoring and Evaluation
Chief Division
Development Bank of the Philippines
Makati, Metro Manila
PHILIPPINES
Mr. SIMIRIN Yogyakarta Provincial Planning Board
Head, Evaluation and Bappeda, Propinsi DIY
Coordination Division D.I. Yogyakarta
INDONESIA
Mr. Ram Karan SINGH Seaqaqa Sugar Development Project
Project Manager P.O. Box 106
Lubasa
FIJI
Ms. Supha SINGINTARA Royal Irrigation Department
Economist Economic Branch
Project Planning Division
Samsen Road
Bangkok
THAILAND
Mr. SOEJITRO Directorate General of Rural Development
Subdirector of UDKP, DGRD Ministry of Home Affairs
Pejaten, Pasar Minggu
Jakarta, Selatan
INDONESIA
Mr. Soegeng SOEMARTO Directorate General of Rural Development
Subdirector of Village Pejaten, Pasar Minggu
Planning, DGRD Jakarta, Selatan
INDONESIA
Mr. Mario SONGCO Agriculture Project Department II
Manager Development Bank of the Philippines
P.O. Box 800, Commercial Center 3117
Makati, Metro Manila
PHILIPPINES
Mr. Wirat SOOKYING Phitsanulok Irrigation Project
Chief, Project Monitoring Department of Land Development
Bangkhen, Bangkok 9
THAILAND
- 39 -
ANNEX 1
Page 5
Name and Title Address
Dr. Adisak SREESUNPAGIT Monitoring and Evaluation Unit
Head Department of Agriculture Extension
Ministry of Agriculture and Cooperatives
Bangkhen, Bangkok
THAILAND
Ms. SRI DJUARINI Y. I. S.
Research and Evaluation Officer Nutrition Improvement Pilot Project
Department of Health
Jakarta
INDONESIA
Mr. Bai-Yang SUNG Rural Infrastructure Project
Leader Evaluation Study Team
Korean Rural Economic Research Institute
4-102 Hoigi-Dong, Dongdaemoon-ku
Seoul
KOREA
Mr. Wilfredo TIANGCO UPRIIS
Operations Manager Integrated Irrigation Systems
Cabanatna City
PHILIPPINES
Mr. John WALLIS Southern Highlands Rural Development
Provincial Project Manager Project
Office of Project Coordination
Department of Finance
P.O. Box 98
Mendi
PAPUA NEW GUINEA
Dr. Michael A. H. B. WALTER PNG Institute of Applied Social and
Senior Research Fellow Economic Research
P.O. Box 5854
Boroho
PAPUA NEW GUINEA
Mr. Fatt Way WONG National Extension Project
Agriculture Officer Department of Agriculture
Jalan Swettenham
Kuala Lumpur
MALAYSIA
Mr. In-Sop YOM Administration and Marketing
Chief Agriculture and Fisheries Development
Corporation
13-8 Noryangjin-Dong, Kwanak-KuI
Seoul
KOREA
- 40 -
ANNEX 1
Page 6
Name and Title Address
Mr. Keun-Hak YU Irrigation Division
Farmland Management Bureau
Ministry of Agriculture and Fisheries
Seoul KOREA
- 41 -
ANNEX II
Page l of 4
OPENING ADDRESS BY YB. DATUK ARSHAD BIN AYUB
SECRETARY GENERAL, MINISTRY OF AGRICULTURE, MALAYSIA
Chairman of the Organizing Committee, Distinguished Guests and Par-
ticipants in the Regional Workshop, Ladies, and Gentlemen:
I would like to take this opportunity to thank the World Bank, and
in particular the organizers of this workshop, for inviting me to make the
opening address to this distinguished international group, which includes
representatives from seven countries in East Asia. While it is an honor for
me to be given this opportunity, it is an even greater honor for Malaysia to
be chosen as the venue of this international workshop. On behalf of the
Government of Malaysia, I welcome you to the conference and trust that your
stay here will prove not only pleasant but fruitful.
The subject of this workshop -- the monitoring and evaluation of
rural projects -- has great significance to all of us. For me, this parti-
cular subject is of high priority. The proper identification and the monitor-
ing of developments projects is an important tool in policy formulation,
program planning, and program management. It is appropriate that Malaysia was
chosen as the venue for this workshop for we are in the midst of reviewing and
appraising the performance of our development efforts during the Third Plan
Period in preparation for selecting those programs and projects to be included
in the Fourth Plan. Moreover, we are halfway through the New Economic Policy
period. Right now we are evaluating the extent to which our policies have been
successful in implementing our New Economic Policy goals to reduce poverty and
to break the identification of race with vocation and location. But even
more, we are not only interested in measuring progress but in understanding
the causes of failure. That is the critical concern - to discover why poli-
cies and projects fail, so that we can change future programs to avoid these
same pitfalls.
Over the past thirty years - the only period since the founding
of the science of economics that attention has been paid to the process of
economic development - our understanding of the process of economic growth has
made considerable progress and has slowly become more sensitive, more subtle,
more realistic, and more complex. This progress has not been without its
shortcomings. The recent concern by development economists reveals but one
blind spot in the past formulations of new insights into the development
process. But when all is said and done, there is no doubt that our tool
box for guiding and stimulating the growth process is infinitely richer today
than it was in the 1950s. However, one area in which much progress has to be
made is the field of monitoring and evaluation.
- 42 -
ANNEX II
Page 2 of 4
We in Malaysia realize the importance of a monitoring and evaluation
system within the framework of the planning cycle. A special unit, the Imple-
mentation Coordination Unit, under the Prime Minister's Department, has been
given the task of undertaking this. A computerized system has already been
created to monitor and evaluate the development plans and projects for the
whole country. The creation of the system is a turning point in institutiona-
lizing monitoring and evaluation as a component of the planning cycle in the
development process.
Within this overall framework, the Ministry of Agriculture is in
the process of developing a monitoring and evaluation system for all of its
projects and also is introducing special monitoring and evaluation components
in all new major projects to improve overall management. This task would not
be accomplished without the assistance from the advisers attached to the
Ministry under the UNDP/World Bank Sponsored State and Rural Development
Projects. I would, therefore, like to take this opportunity to record my
appreciation to the UNDP group of advisers in the Ministry of Agriculture for
their invaluable contributions.
I would now like to discuss with you some of the issues we experience
in developing and utilizing monitoring and evaluation systems.
Firstly, monitoring and evaluation of specific projects is a rather
new thing to most of us. The need for such a management function emerges for
a number of reasons. We have seen many ambitious development projects fall
short of the, expectations we had for them. Why did it happen? We need more
information on how small farmers behave and on their reasons for adopting or
not adopting certain technologies the projects introduce. Development projects
are also becoming increasingly complex, trying to reach poverty groups pre-
viously neglected. Development is a learning process; we need to get feedback
on what works, what does not work, and how we can readjust our programs and
strategies to become more effective in reaching small farmers. We need to
know more about to what extent the benefits of the projects are reaching those
we aim them for and to what extent the rural poor in reality see any changes
in their incomes and standard of living.
The awareness of the importance of monitoring and evaluation systems
is rapidly increasing within our countries. Not always is this, however,
translated into allocating manpower and resources for this purpose.
In this context, we appreciate the initiative by the World Bank and
other agencies to focus on these problems, their emphasis on monitoring and
evaluation components in projects they are financing, and their stimulation of
sharing of experience on the problems in workshops like this.
I think that the best means to convince planners and decision makers
of the need to allocate time and resources to monitoring and evaluation is
to demonstrate where such functions have provided substantial benefits to
a project through examples I hope this workshop will highlight in the many
case studies prepared.
- 43 -
ANNEX II
Page 3 of 4
Once we are convinced of the usefulness of monitoring and evalua-
tion, the question arises: how to do it? - the key question of this wo;rkshop.
Most of us are in an early learning stage of developing appropriate monitoring
and evaluation systems. We need to learn more about systems for data collec-
tion and for data processing, means of reporting, the usefulness of various
technical devices such as computers for information processing, how to strike
a good balance between costs, amount of data collected, timing of data, the
organizational set-up, etc. In short, we need to learn more about the technolo-
gies and the management of monitoring and evaluation.
I think our experience in Malaysia might be indicative for most of
the countries participating in this workshop. We are happy to share the know-
ledge that we have with others. We should approach the development of the
monitoring and evaluation system collectively. We have a long way to go until
we have a satisfactory system to monitor and evaluate all development projects,
especially rural development programs and projects.
One bottleneck in this process is shortage of experienced manpower.
The number of officials with experience in this field is very few, thinly
spread. I think this is a problem we share with most other developing coun-
tries. Another bottleneck is our limited knowledge about the technologies
and the practical steps involved. I thus hope that the participants of this
workshop will have sufficient time to look into the practical problems of
monitoring and evaluation.
The third question I would like to touch upon is what criteria we
are applying to assess development efforts. Monitoring and evaluation-systems
should ideally cover a broad range of information, from financial records data
to engineers need to control progress in construction, agriculturalists'
need for information on use of agrochemicals, credit, etc. - i.e., directly
measurable tasks related to information, answering questions such as: Are the
rural poor benefiting from the development efforts? Are they receiving the
increases in income we expect? Do we achieve the changes in distribution of
income and wealth we strive for? It is when we come to the latter types of
questions that our difficulties begin. The means of collecting data is be-
coming more difficult, particularly if we want to go further than pure economic
measurements.
In this context -I would like to bring up the need to tailor monitor-
ing and evaluation systems to the needs of the particular country where they
are being utilized. Even though countries in the East Asia region share
common objectives, they are at the same time at different levels and stages
of development. Therefore, their strategies and policies to achieve their
objectives vary. Take Malaysia, for example; the objective of the New'
Economic Policy is national unity through poverty eradication, irrespective
of ethnic origin, and the restructuring of society so as to eliminate the
identification of race with vocation and location. The indigenous population
is targeted to acquire at least 30% of the national wealth. Hence, there is
a need to develop socioeconomic indicators to constantly and periodically
monitor progress on how income and wealth are distributed and to evaluate
reasons for any discrepancies between targets and reality.
- 44 -
ANNEX II
Page 4 of 4
It is here that monitoring and evaluation systems come into their
own. It is only through the feedback provided by these tools, and through
the newly emerging branch of evaluation research, that we can fully understand
how actual plans, programs and projects are implemented in reality.
Finally, my skepticism that we know enough about how plans, pro-
grams and projects will actually behave, as opposed to how they behave
in studies, is best illustrated by quoting from a rather lengthy poem by
Kenneth Boulding. He says, in part:
The cost of building dams is always underestimated.
There's erosion of the delta that the river has- created,
There's fertile soil below the dam that's likely to be looted,
And the tangled mat of forest that has got to be uprooted.
There's the breaking up of cultures with old haunts and habits loss,
There's the education programme that just doesn't come across,
And the wasted fruits of progress that are seldom much enjoyed
By expelled subsistence farmers who are urban unemployed.
For engineers, however good, are likely to be guilty
Of quietly forgetting that a river can be silty,
While the irrigation people too are frequently forgetting
That water poured upon the land is likely to be wetting.
In brief - no matter how careful our analysis -- we always should take heed
of the multiple effects of implementation. This is why monitoring is so
critical - not to see that everything is done on time but to help us under-
stand how plans, programs and projects really change the area in which they
operate.
Let me conclude by saying that, despite my skepticism, I am a
strong believer in proper selection techniques as the first step in the
successful implementation and execution of a development strategy. I would
not want to leave this conference hall before having made it very clear again
that my full support lies behind those who wish to improve the planning cycle
in our various countries. But to do so, the economists among us must recognize
the shortcomings of the technique they so fully put forth and which, given
its impressive strengths, still must be adopted and used with caution. Any
evaluation system must be put forward in simple, plain, straightforward
language so that technocrats, bureaucrats and, above all, the legislators
fully understand what it tries to show and use it effectively in their daily
work. In other words, any system so developed should not be a deterrent to
successful implementation of development projects and shy people away from
putting it into practice, but rather a facilitator.
I have been given to understand that you all have a busy schedule
ahead of you for the next week. Nevertheless,.avail yourselves of all the
good things in this country; and with that I now officially declare this
workshop open.
- 45 -
ANNEX|III
Page 1 of 2
LIST OF PAPERS SUBMITTED TO THE REGIONAL WORKSHOP ON
MONITORING AND EVALUATION OF RURAL DEVELOPMENT PROJECTS IN EAST ASIA
Number Name Title
RWME/Ol/Ol YOM In-Sop A Case Study on the Monitoring and
Evaluation System of the Integrated
Agricultural Products Processing
Project, Korea
RWME/01/02 CHOI Yong-Sang A Case Study on the Monitoring and
and YU Keun-Haak Evaluation System of the Kumgang-
Pyungtack Agricultural Development
Project, Korea
RWME/01/03 KOH Hyeon-Seok A Case Study on the Monitoring and
Evaluation System of the Agricultural
Credit Project, Korea
RWME/01/04 SUNG Bai-Yung A Case Study on the Evaluation of the
Rural Electrification Project, Korea
RWME/01/05 BURESOVA, N. The Reporting, Monitoring and Evalua-
tion System of the Fiji Sugar Develop-
ment Project
RWME/01/06 SINGH, R. K. A Case Study on the Monitoring and
Evaluation System of the Seaqaqa Sugar
Development Project, Fiji
RWME/01/07 NUNEZ, J. P. A Case Study on Monitoring and Evaluation
in the Rural Development-Land Settlement
Project, The Philippines
RWME/01/08 PASTOR, P.A. A Case Study on the Monitoring and Evalua-
tion System of the Capiz Settlement
Project, The Philippines
RWME/Ol/09 TIANGCO, W. S. A Case Study on the Operation, Monitoring,
and Evaluation System of the UPRIIS Project,
The Philippines
RWME/Ol/10 DE GUZMAN, R. N. The Input-Output Monitoring Progtam for
the Upper Pampanga River Integrated
Irrigation Systems (UPRIIS),
The Philippines
-46 -
ANNEX III
Page 2 of 2
Number Name Title
RWME/Ol/ll SONGCO, M. M. A Case Study on the Monitoring and
Evaluation System for the Livestock and
Poultry Project of the Development Bank
of the Philippines
RWME/01/12 RENTUTAR, F. G. A Philippine Experience on an Agricul-
tural Extension Reporting, Monitoring
and Evaluation System
RP.E/01/13 MAGPALE, L. J. A Case Study on the Philippine Fourth
CB-IBRD Rural Credit Project: Monitoring
and Evaluation Aspects
RWME/01/14 DIZON, C. S. A Proposed Monitoring and Evaluation
System for the Rainfed Agriculture
Project I (Iloilo), The Philippines
RWME/01/15 PERLAS, A. A Proposal for the Design of an Evalua-
tion System for the Second IBRD Rural
Development Project in the Philippines
RWME/01/16 CHIRAPANDA, S. A Case Study on the Monitoring and
Evaluation System for the Agricultural
Land Reform Program of Thailand
RWME/01/17 KRISHNAMRA, J. A Report on the Monitoring of the
Phitsanulok Irrigation Project,
Thailand
RWME/01/18 BIN JAAFAR, Z. A Case Study on the Monitoring and
Evaluation of the North Kelantan Rural
Development Project, Malaysia
RWME/Ol/19 CHAN, A. K. A Preliminary Proposal for a Monitoring
and Evaluation System for the National
Extension Project in Malaysia
RWME/01/20 SAMIRIN A Case Study on Monitoring and
Evaluation in the Yogyakarta Rural
Development Project, Indonesia
RWME/01/21 WALLIS, J. Aspects of Monitoring and Evaluation
of the Southern Highlands Project,
Papua New Guinea
- 47 -
ANNEXIIV
PaRe 1 of 42
ABSTRACTS OF CASE STUDIES
A Case Study on the Monitoring and Evaluation System of the
Integrated Agricultural Products Processing Project, Korea
Yom In-Sop
THE PROJECT
The project aims to increase the income of 5,000 poor farmers in the
southern part of Korea through modernization of cultivation and processing of
agricultural products. The project is for five years (1975-79), with a total
cost of US$20 million, of which US$13 million is funded by the World Bank.
The Project Manager is responsible to the Vice-President of the
Agricultural and Fishery Development Corporation. He has a staff of 20
members in three Divisions: Technical, Marketing and Appraisal.
REPORTING, MONITORING, AND EVALUATION
The major objectives of the reporting, monitoring, and evaluation
system are: (1) to supervise ongoing subprojects; (2) to determine the proj-
ect-s impact on farm income, creation of rural employment, and increase in
export earnings; and (3) to assist in the preparation of follow-up projects.
The cost of operating the system is approximately 0.1% of total
project costs. The performance of the project is measured by comparing the
expected and the actual figures for yields, incremental production, number
of farmers benefited, incremental income, exports, and the financial rate
of return. These indicators were identified with reference to the inputs,
activities, and goals of the project. The selection of indicators aims at
satisfying the needs of the Technical and Marketing Divisions at the project
level, as well as the information needs at the ministerial level.
Information required before project commencement included production
capacity, level of production, availability of raw materials, labor foice,
managerial and technical staff, markets, and financial status of the pros-
pective target groups.
Monthly progress reports on both the processing and on-farm develop-
ment subprojects are prepared; repayment performance records are provided
quarterly. The annual reports cover such topics as total production, inventory,
sales, employment, capacity utilization, and profits. Any substantialichanges
in collaterals, replacement of facilities, relocation of sites and transfer
of ownership are reported as they occur.
- 48 -
ANNEX IV
Page 2 of 42
The information has been used on a regular basis by management for
determining future investment needs. Repayment performance information is
used for making decisions with regard to legal procedures for debt recovery.
Information regarding sales and profits is usually understated,
just as production costs are overstated for tax reasons. Information from
sub-borrowers is sometimes lacking, due to their unwillingness to cooperate
and to the complexity of the original questionnaire (which has subsequently
been simplified).
The activities of the system consist of collecting information from
(1) processor sub-borrowers, using mailed questionnaires, and (2) farmers in
the development subprojects, through interviews by the field extension staff,
and (3) processing and analyzing the data.
One of the most important findings revealed by the monitoring
system was that many of the sub-borrowers had expanded their productive
capacity beyond the original loan request, or had added new lines of enter-
prises, resulting in serious shortage of working capital. The unit had
accordingly recommended that project management provide additional working
capital for them.
- 49 -
ANNEXIIV
Page 3 of 42
A Case Study on the Monitoring and Evaluation Systlem
of the Kumgang-Pyungtack Agricultural
Development Project, Korea
Choi Yong-Sang and Yu Kuen-Haak
THE PROJECT
This is the first large-scale integrated agricultural development
project in Korea. The project began in 1970 and was completed in Kumgang in
1976 and in Pyungtack in 1977.
The objectives of the project were (1) to rearrange the production
basis of 31,000 ha through improvements in irrigation and drainage, land
consolidation, land reclamation, and farm road realignment; (2) to accelerate
rural modernization by means of raising the farm income through increasing
the level of agricultural support services; and (3) to lead effectively
similar large-scale agricultural development projects in the future.
During project implementation, 9,500 people were employed and 87.5%
of the 32,000 farm households in the area received some benefit from farm
development activities.
The original cost estimates amounted to US$100 million but the
completion costs escalated to twice the total owing to inflation and other
reasons.
To operate and maintain the project effectively, the Ministry of
Agriculture and Fisheries established a Division of Special Area Development
and a Kumgang-Pyungtack Project Area Department was instituted in the main
office of the Agricultural Development Corporation. Management field offices
were also established in each project area, and several supervising offices
were subordinated to the management field offices.
REPORTING, MONITORING, AND EVALUATION
The aim of the reporting, monitoring and evaluation,was to provide
accurate and timely information collected from actual project'achievement
to project management and a rapid delivery of management decisions to the
farmers.
The organization for reporting, monitoring and evaluation was
changed several times during the lifespan of the project to adjust to
variable circumstances.
Three main types of information were collected by the unit. The
first consisted of administrative reports and the second was reference mate-
rials relevant to project development collected from external1sources.1 The
- 50 -
ANNEX IV
Page 4 of 42
third--and by for the most important--was the information concerning proj-
ect implementation collected through field surveys and farmer interviews.
Much of the information need was identified by the requirements of the World
Bank and the Ministry of Agriculture and Fisheries. These included soil
surveys, cropping pattern surveys, farm household economic surveys, crop yield
surveys, agricultural supply surveys, farmer organization surveys, and crop
status surveys.
The frequencies of these surveys depended upon the situation each
year. The Agricultural Development Corporation and each management field
office had qualified manpower to take care of the surveys, and the amount of
manpower available was dependent on the size and condition of the subprojects.
Owing to the shortage of funds and manpower, information for a com-
plete evaluation of the project was lacking. Materials available were used
mainly to set up a long-term program for agricultural development in the
country and for short-term annual planning of the project.
As a result of the information collected, modifications were intro-
duced in respect of changing the land utilization ratio and increasing the
target yield. The project, however, was subject to substantial modifications
because of overall government policy changes.
- 51 -
ANNEX IV
Page 5 of 42
A Case Study on the Monitoring and Evaluation System
of the Agricultural Credit Project, Korea
Koh Hyeon-Seok
THE PROJECT
This project is a continuation and expansion of the First Agricul-
tural Credit Project, completed in 1976. The aim is to provide medium-term
and long-term loans through the Participating Gun (County) Agricultural
Cooperatives (PGCs) to about 8,000 farmers for investments in apple orchard
development, sprinkler irrigation, on-farm storage of fruits, greenhouses for
vegetables, and silkworm-rearing houses. It is envisaged that, through this
development, there will be stabilized farming and marketing, as well as
increased production at lower costs, which will ultimately increase farm in-
come while meeting domestic and export demands for fruits, winter vegetables,
and cocoons.
This project, begun in 1977 and to be completed in 1980, will cost
US$41.3 million, of which US$20.0 million will be financed by the World Bank.
The National Agricultural Cooperative Federation (NACF) is respon-
sible for coordination of the project. Project management is subordinated
to the Technical Unit for the Agricultural Credit Project (TU) and has eight
supporting sections. At the provincial level, the General Mlanagers of the
Provincial Branches of the NACF have charge over the PGCs, whose workers in
the field are the Loan Appraisal Officers (LAOs).
MONITORING AND EVALUATION
The major activity of the Technical Unit for Project Monitoring
and Evaluation is the preparation of the annual Project Implementation Plan.
These plans are based on information collected from the PGCs regarding loan
demands, as well as information on government development policies, taking
into consideration the availability of project funds and project implementa-
tion procedures. A difficulty in the formulation of these plans is the
danger of overlapping with other government projects which are implemented
by different agencies with their own monitoring and evaluation systems.
Implicit in the preparation of the annual References and Criteria
for Loan Appraisal for each of the subprojects, another function of the TU,
is a cost effectiveness analysis, which takes into account the changing prices
of inputs and outputs and the changing farm technology. Thislinvolves the
use of statistical and reference materials, as well as intensive on-farm
surveys undertaken by the TU staff, which give a realistic impression Of
project progress and implementation problems.
The main source of information for the monitoring and evaluation
system is the documentation and reports of the PGCs. These are then collected,
- 52 -
ANNEX IV
Page 6 of 42
cross-checked and analyzed by the central TU staff. Twice monthly, infor-
mation on loans made and funds needed is transmitted to the TU by telephone
or telex, for checking on progress in general; more complete information is
mailed every month to the TU for detailed checking. Owing to delays caused
by manual processing, the information often arrives too late for action to be
taken. Furthermore, some PGCs have been unwilling to supply the required
information and have continued to use an older reporting system. Data pro-
cessing by electronic means has been instituted in some pilot areas. Field
trips are occasionally undertaken by TU staff to acquire information on the
quality of PGCs in implementing the project.
Ex-post evaluation will require information on trends of project
loans, farm management performance, institutional efficiency, and general
agricultural change. This information is being gathered by the LAOs and
summarized in the PGC annual review. On completion of the project, a sample
survey, with a carefully designed questionnaire, will be undertaken.
- 53 -
ANNEX IV
Page[7 of 42
A Case Study on the Evaluation of the
Rural Electrification Project, Korea
Sung Bai-Yung
THE PROJECT
The Rural Electrification Project formed part of the Rural Infra-
structural Development Project (1976-78) in Korea, within the framework
of the Saemaul Undong (New Community Movement). The total cost of the
project as a whole was US$167 million, of which 35% was financed by a World
Bank loan.
The Economic Planning Board (EPB) was the agency for monitoring
the progress of the various project components implemented by different
technical agencies. Quarterly reports from the executing agencies of each
subproject on the physical status and the expenditure were forwarded to the
EPB. The Project Monitory Unit of the EPB checked the reports for their
financial consistency and prepared a summary of overall performance.
The EPB was also responsible for the ex-post evaluation of the
project, and in consultation with the World Bank and the Korean Rural
Economics Institute (KREI), drew up a three-year program of evaluation studies.
Study teams from KREI undertook evaluation of each of the project components,
including rural electrification.
The Korea Electricity Company (KECO), under the supervision of the
Ministry of Energy and Resource, was responsible for the survey, design,
operation, and maintenance aspects of the rural electrification component.
The areas to be electrified were selected by the local government.
The total cost of US$55.9 million was provided by a combination of
a, Bank loan through the government (77%), KECO's own investments (11%) and
consumers- contributions (12%). Loan repayments were collected by the KECO.
PROJECT EVALUATION
The purposes of evaluation of the Rural Electrification Project were
(1) to measure the cost-benefit of electrification; (2) to analyze the tech-
nical and operational problems of project implementation; (3) to measure the
improvement in the quality of rural life and changing attitude toward the
project; and (4) to analyze the impact of electrification on the rural
economy.
A sample survey of 200 rural households, half of which were electri-
fied during an early phase in 1970-75 and the other half durinlg 1976-77, was
conducted. Forty village leaders were also interviewed. In addition, a mail
survey was conducted for KECO branch offices on the technical!and operational
- 54 -
ANNEX IV
Page 8 of 42
problems. Local government offices concerned were either interviewed or sur-
veyed by mailed questionnaire for information on regional problems relating to
implementation of, and further plans for, rural electrification.
Analysis of project feasibility and the internal rate of return
was carried out by two alternatives: (1) economic benefits of cost savings,
time savings, productivity gains, and material savings; and (2) actual pay-
ments and consumer surplus. The cost streams included investment, energy
supply, and consumers costs, as well as indirect costs and social costs.
The internal rates of return for the two alternatives were calculated to be
44.26% and 14.20%, respectively.
It is concluded that the reporting and monitoring of the project,
undertaken by the executing, planning, and budget authorities, and the
evaluation, undertaken by an independent research institute under harmonic
coordination and support from the implementing agencies, appear to be very
effective.
- 55 -
ANNEX:IV
Page 9 of 42
The Reporting, Monitoring and Evaluation System
of the Fiji Sugar Development Project
Nemani Buresova
THE PROJECT
The project aim is to increase the sugar production of Fiji by
400,000 tons over a period of five years beginning in 1976, through the
clearance of 8,000 acres of rolling scrub and forest land and the rehabili-
tation of another 26,000 acres affected by sea water intrusion and water-
logging conditions. It has several components, including tramline extension,
road construction and maintenance, seawall reconstruction, land clearance
and drainage improvement, to be implemented by different technical agencies.
The total project cost is US$26 million, of which 46%, representing
the foreign exchange costs, will be financed by a World Bank loan.
The two subprojects of the Seaqaqa Sugar Development Project and
the Drainage Improvement Project are under the overall direction of the
Permanent Secretary for Agriculture and Fisheries. A project manager,
appointed by, and reporting to, the Permanent Secretary in his capacity as
Chairman of the Central Coordinating Committee, would be responsible for
the Seaqaqa land settlement; the Public Works Department would be responsible
for the construction and maintenance of project roads; and the Drainage and
Irrigation Division and the local Drainage Boards would be responsible for
drainage improvement.
REPORTING, MONITORING, AND EVALUATION
The Economics, Planning, and Statistics Division is charged with
the responsibility of conducting the reporting, monitoring and evaluation
activities, for and on behalf of the Permanent Secretary for Agriculture and
Fisheries. This responsibility is directed toward coordinating, analyzing,
reporting, and evaluating aspects of the project's progress,-such as the
rates of land clearance and farm development, deployment of farm advisers,
progress of drainage improvement, yield progression, farm incomes, loan
repayments, and drainage levy collections. The Division works closely with
the Fiji Sugar Corporation, which routinely collects most of the data that
are supplied to the Division for analysis, recording, and information.
As several agencies and ministries are involved in the implementa-
tion of the project, specific information directly related to each has to be
obtained, analyzed, integrated, and agreed upon before project components
can be implemented.
The information available on a monthly basis is thelactivities
closely associated with the Seaqaqa Settlement Project. Quarterly inf6rma-
tion is acquired from the agencies and ministries involved. The information
- 56 -
ANNEX IV
Page 10 of 42
required for the annual reports tends toward analytical information, such as
the comparison between estimated and actual expenditures, volume and value of
agricultural production, inputs and costs of agricultural production, and the
farm budgets for a typical 15-acre farm at Seaqaqa and one of the same
size in the drainage improvement area. The prices of agricultural inputs
are also presented.
Problems with the information system seem to be related to those
parts pertaining to ot'aer agencies and ministries rather than to the Seaqaqa
Settlement. Late arrival of the necessary information from these sources
and personnel changes in these agencies are some of the problems confronting
the system. Informal systems are used in discussions with the reporting
officers from the agencies and ministries, including consultations over
the telephone with, and obtaining viewpoints from, the various agencies
and ministries about a project component first before decisions are made.
The system has done little to monitor and evaluate the socioeconomic
changes, especially when farm incomes have risen faster than expected.
- 57 -
ANNEX IV
Page 11 of 42
A Case Study on the Monitoring and Evaluation System
of the Seaqaqa Sugar Development Project, Fiji
Ram Karan Singh
THE PROJECT
The project is a component of the Fiji Sugar Development Project.
It is concerned mainly with the Seaqaqa Settlement. The project entails
the clearing and development of some 12,000 acres of land to enable it to
produce about 200,000 tons of cane by 1980 and the settling of some 800
farmers, each planting about 40 acres of crops, of which about 15 acres are
for sugar.
While the Ministry of Agriculture and Fisheries is responsible
for the project and manages it through a project manager, several of the
organizations are responsible for different areas of input. The project
manager controls and coordinates all project activities, including land
subdivisions, land surveys, land clearing, cane development, road construc-
tion, and social services in the settlement area.
The five-year project is expected to cost about US$16.0 million,
funded by the Fiji Government, the World Bank, the Fiji Development Bank,
and the Fiji Sugar Corporation. Funds are on-lent to individual settlers
for on-farm development on a 15-year term at subsidized interest rates.
REPORTING, MONITORING, AND EVALUATION SYSTEM
The project is a very ambitious one. The problems associated with
it are manifold and varied. The success of the project depends to a large
extent on the management's ability to keep to schedule, to remain within
budget limits, and to forestall problems and difficulties by taking remedial
actions without delay. The system is therefore designed to monitor progress
in these important areas, provide comparisons with the estimates, and high-
light deviations.
A Monitoring Unit has been set up in the Economics, Planning, and
Statistics Division of the Ministry of Agriculture and Fisheries, headed by
the Chief Economist, who reports directly to the Permanent Secretary. ;Each
organization involved with the project implementation is responsible for the
data and for reporting on its own areas of involvement and for supplying these
data to the central Monitoring Unit at regular intervals. The reports?com-
piled by the unit endeavor to highlight important areas, suchl as actual
expenditure against estimates and work completed against time expiration.
For its own management functions, the Project Manager needs additionalland
detailed information, in most cases concerned with individual settlersl This
includes area of cane planted, fertilizer received, rainfall, crop harvests,
production per farm, and number of farm visits by extension workers.
- 58 -
ANNEX IV
Page 12 of 42
The Project Manager reviews extension activities with the Cane
Development Manager on a weekly basis. Harvesting and transport of cane are
subjected to daily review by the Project Manager, Cane Development Manager
and Field Superintendent. Land clearing activity is also subjected to weekly
review by the Project Manager and the Senior Research Officer.
The Monitoring Unit compiles narrative and tabulated reports on
physical progress on quarterly and semiannual bases; a financial progress
report is furnished half-yearly, and a more extensive report is provided
annually. The Project Manager is generally quite aware of the activities
taking place. In many cases, he is the supplier of the basic information
for the systems.
The information obtained through the system is quite adequate,
except for data relating to the settlers performance and debt situation.
The system is designed for the project as a whole. It does not cater to the
monitoring of an individual settler's progress. The system has, however,
been improved to overcome this deficiency.
- 59 -
ANNEX IV
Page 13 of 42
A Case Study on Monitoring and Evaluation in the Second Rural
Development-Land Settlement Project, The Philippines
Josefina P. Nunez
THE PROJECT
The Government of the Philippines has been vigorously pursuing a
policy of land settlement to counter pressures of increasing land scarcity,
of encroachment by the landless on public land, and of resettling farm families
displaced by various infrastructure development programs. This project aims
specifically at accelerating development in the three settlement areas of
Agusan, Bukidnon, and Capiz by way of a village road and a small-scale irriga-
tion program; cooperative development and agricultural credit; an agricultural
extension, demonstration, and farmer training system; and rural health and
family planning services.
The responsibility for implementing the project is with the Central
Project Management Unit (CPMU) in the Ministry of Agrarian Reform, and the
Area Management Units (AMU) coordinate the activities of the Ministry of Agri-
culture, Ministry of Local Government and Community Development, Ministry of
Public Highways, Ministry of Health, Land Bank of the Philippines, National
Irrigation Administration, and others, which provide inputs to the land settle-
ments in the area. The Inter-Agency Project Coordinating Committee (IPCC),
composed of representatives of these participating agencies, forms the policy
making body.
The direct beneficiaries of the project are the 47,600 families in
the three settlements, cultivating a total of 77,000 hectares of land.
The project is for five years, commencing in 1977. The total costs
are estimated to be approximately US$33.0 million, of which about half is to
be financed by World Bank loans.
MONITORING AND EVALUATION
In this project, monitoring and evaluation is conceived as an
integral part of the management organization. It is to provide an effective
management information system where problem areas can be pinpointed promptly,
performance is evaluated, and bottlenecks are reduced to a minimum.
Organizationally, the Monitoring and Evaluation Division is placed
in the Planning and Project Development Office, which is subordinated to the
Office of the Project Manager. The Division is headed by theIMonitoring
and Evaluation Officer, assisted by a Project Evaluation Officer and al
Statistical Officer.
- 60 -
ANNEX IV
Page 14 of 42
The objectives of the monitoring and evaluation system are to
facilitate implementation of the integrated project constituents, supervise
the annual plans and programs, maintain the data bank on field operations,
conduct surveys in the three settlements, and maintain links with other
agencies. The system is to monitor project accomplishments and problems,
prepare reports, and evaluate project performance. Monthly, quarterly,
semiannual, annual, mid-project, and project completion reports are to be
produced for the various government agencies.
For all subprojects within the settlement areas, project work plans
have to be submitted before funds are allocated; progress must be reported
monthly as funding is dependent upon the progress of the project. Subprojects
not directly funded by the CPMU are to be reported monthly by the implementing
agency. All reports are deposited in a project data bank.
Timely data collection proved to be difficult owing to inadequate
access and insufficient enumerators, and some incorrect data have been submitted
occasionally. Furthermore, because of constant revision of annual targets,
the project performance evaluation is delayed and the frequency of reporting
has been reduced to semiannually rather than quarterly as originally intended.
The monitoring and evaluation cost is set at US$406,000 or approxi-
mately 1.7% of the total project cost.
- 61 -
ANNEX IV
Page115 of 42
A Case Study on the Monitoring and Evaluation System of
the Capiz Settlement Project, The Philippines
Pelagio A. Pastor
THE PROJECT
This project is part of the Second Rural Development-Land Settlement
Project involving the land settlements of Agusan del Sur and Bukidnon, in
addition to Capiz. This settlement, like the other two, was selected for
inclusion because it has problems representative of those found in a wider
range of areas, has reasonable productive potential for development and is
of a sufficient size to warrant a more intensive application of technical,
financial, and human resources. It exhibits the mix of more and less productive
upland terrain, which is characteristic of settlements in the Philippines,
and is at present only slightly above the subsistence farming level in terms
of current agronomic practices and standards of living. It would be through
the planning and implementation of projects in such an area that an effective
settlement program would be developed through a number of changes in the
organization and procedures in the Ministry of Agrarian Reform.
The development plan for Capiz Settlement is to be implemented over
a period of five years, starting in 1977. The major components for the area
include road construction, irrigation and drainage, village water supply
and sanitation, land allocation, settlement health services, forestry develop-
ment, agricultural development for coconut, abaca and rubber, agricultural
technical services, and agricultural credit and cooperative development.
To implement the project, the planning units of the Central Project
Management Unit (CPMU), the Area Management Unit and the participating agencies
would draw up an integrated annual work program and budgetary requirements,
which include current operating expenditures and funds for capital outlay.
For implementation by operating units in the settlement area,,detailed project
work plans are prepared for specific projects. These plans are submitted to
CPMU for funding upon approval.
MONITORING AND EVALUATION
The objectives of the Monitoring and Evaluation System are to aid
project implementation, to evaluate the work progress of the field units and
participating agencies and to provide for adjustments in planning, and to
evaluate project accomplishments.
- 62 -
ANNEX IV
Page 16 of 42
Before the inception of the project, baseline surveys were carried
out on the basis of a 10% random sample to obtain information on facilities
and services available in the area and on land resources. An Interim Project
Monitoring System was set up in 1978 to prepare reports on work plans and to
develop a monthly reporting system. This has evolved into the present Moni-
toring and Evaluation System. Data is gathered from the field, summarized,
and reported to the CPMU. Project work plans are approved at the area and
project level before funding is provided, and the project status reports are
submitted by the project supervisor monthly on progress regarding targets, on
problems encountered, and on remedial action taken or requested. Monthly
progress reports from the Monitoring and Evaluation Unit (MEU) are essentially
compiled from the project status reports and provide a complete perspective of
the status of project implementation in all aspects of settlement operations.
Informal data is also gathered by the MEU staff from personnel involved in
the projects, and field inspections are carried out periodically.
Several methods for data and information presentation are used.
These include charts, maps, statistical tables, and written narratives.
Logbooks are kept on the progress of individual projects. The settlement data
bank is a file of settlement information, statistics, reports, relevant
records, survey results, and reference materials on settlement development.
- 63 -
ANNEX IV
Page!17 of 42
A Case Study on the Operation, Monitoring and
Evaluation System of the UPRIIS Project, The Philippines
Wilfredo S. Tiangco
THE PROJECT
The Upper Pampanga River Integrated Irrigation Systems (UPRIIS) is
the largest multi-purpose infrastructure project of the National Irrigation
Administration (NIA). The project is located 170 km north of Manila in the
Province of Nueva Ecija. Construction work started in 1971 and was completed
six years later. The four irrigation districts cover 104,300 hectares, cul-
tivated by about 45,000 farm households.
The main aim of the project is the provision of irrigation for
increasing the cropping intensity to 191% in the service area (96% in the wet
season and 95% in the dry season). Irrigation efficiency is to be increased,
and the water flow of the Pampanga River is to be controlled to reduce
flooding.
The organizational structure integrates the operation and mainte-
nance of all the irrigation systems within the service area. Overall super-
vision and management is the responsibility of the Operations Manager.
The annual operational budget financed by the National Irrigation
Administration is variable, according to requirements for additional and
remaining physical works, rehabilitation and expansion of irrigation systems,
repairs and improvements of farm facilities, erosion control, reforestation,
resettlement and assistance, typhoon damage repairs, and improvements, as well
as operation and maintenance.
MONITORING AND EVALUATION
Irrigation plans are drawn up on the basis of water availability,
canal structures and capacity, cropping patterns, and the need for maintenance
of the irrigation structures. The plan indicates the expected duration of
farming activities, area to be irrigated on a weekly basis, and the target
flow discharges at all irrigation points.
However, because deviations from the expected targe'ts are consider-
able, especially through hydro-meteorological variations, a systematici
information system on measurement, control, monitoring and evaluation is
necessary to be more responsive to the ever changing field coInditions,lwith the
aim of better utilization and conservation of the available irrigation water.
- 64 -
ANNEX IV
Page 18 of 42
During implementation, data are collected daily on the flows of
irrigation water at all flow points, rainfall, and local inflows, and weekly
accountings of farming activities are made. Complete inventory and analysis
of these data and information, with computed corresponding water requirement
for each category of farming activities, allow calculations of target dis-
charges for all irrigation points. Data are reported the day after they are
collected and transmitted by field telephone and radio communication to the
Water Control Coordinating Center (WCCC). Monitoring is performed at the
operational, district, zone, division, and section levels. Evaluation of the
data is performed weekly by the WCCC as an early warning device. The results
are presented in a comprehensive manner and expressed in terms of water duty
and irrigation efficiency.
As a result of the introduction of the monitoring and evaluation
system, overall performance in terms of area irrigated, cropping intensity,
yields and irrigation efficiency, has shown significant improvement but is
not yet comparable with the targets set.
The present scheme of field reporting in UPRIIS is the primary
constraint in the immediate evaluation of the operational status of the
system. There is much still to be desired regarding improvements in com-
munication facilities for transmitting the required data and instructions
from and to the field personnel.
- 65 -
ANNEX IV
Page 19 of 42
The Input-Output Monitoring Program for the Upper Pampanga
River Integrated Irrigation Systems (UPRIIS), The Philippines
Rodrigo N. de Guzman
THE PROJECT
The Upper Pampanga River Integrated Irrigation Systems (UPRIIS) is
a five-year Agricultural Development Plan, implemented by the National Irriga-
tion Administration (NIA) from 1976 to 1981 after the completion of the
physical aspects of the irrigation project. The aim is to increase production
in the service area of 83,000 hectares from 2.3 tons per ha to 4.0 tons per ha
within the five-year time frame. This is to be achieved through integrated
inputs of improved seeds, fertilizers, pesticides, and improved cultivation
techniques. The project beneficiaries are approximately 40,000 farm families.
MONITORING, EVALUATION, AND REPORTING
The objectives of the Input-Output Monitoring Program (IOMP) are to
create an early warning system to monitor the supply of inputs, detecting
shortfalls, informing agencies concerned and devising corrective measures; to
monitor productions and other flows; and to make recommendations for addi-
tional resource requirements during project implementation. The Agricultural
Development Plan is evaluated by comparing the actual flows and the buildup
of inputs in the project areas with the planned provisions.
The IOMP was created by the NIA with the assistance of an SGV con-
sultant group whose services will be progressively phased out. Control of the
IOMP is under a project leader and a team of technical staff composed of an
agricultural specialist, an agricultural engineer, agricultural economists,
an agronomist, and a rural sociologist, supported by field enumerators and
clerical personnel.
The activities of the IOMP include preliminary familiarization of
the project by the team, survey of the project area, review of the five-year
development plan, sample farm surveys, and periodic reports.
Based on the information gathered, two kinds of reports are compiled
regularly. Situational reports were prepared weekly at the beginning of proj-
ect monitoring, but the frequency has subsequently been reduced to a fortnightly
interval. The purpose of these reports is to provide the early warningiof
identified problems, which are to be discussed during regular meetings between
the NIA and the implementing agencies at the central policy making level and
at the Agricultural Development Coordination Council (ADCC), which is the body
responsible for implementing the integrated agricultural development project
in the service areas. Seasonal reports are more in the nature of ongoing
evaluation and are compiled after the completion of different types of surveys
and data collection exercises. Analyses are made in the light of deviations
- 66 -
ANNEX IV
Page 20 of 42
from the program goals and objectives by way of comparing and correlating
delivery of inputs, with resulting crop production measured by output monitor-
ing. Data processing is performed with electronic equipment to capture speed
and timeliness advantages.
The periodicity of reporting depends on the cropping season, which
is largely determined by the irrigation water delivery schedule. At the end
of each cropping season, the actual utilization of production inputs and crop
yields are reported, along with the success of adoption of designed cropping
patterns and recommended farm practices.
Project activities and goals are periodically revised in the light
of the survey information, and feedbacks are provided for the project manage-
ment team. Information is also used by the planners to design the cropping
patterns of the succeeding season and to revise the plans in conjunction with
the other agencies represented on the ADCC. Reports have helped the coordina-
tion at the highest planning level of the National Food and Agriculture Council.
The cost of operating the IOMP for the five-year period is estimated
to be approximately US$1.0 million.
- 67 -
ANNEX IV
Page 21 of 42
A Case Study on the Monitoring and Evaluation System
for the Livestock and Poultry Project
of the Development Bank of the Philippines
Mario M. Songco
THE PROJECT
The Development Bank of the Philippines (DPB) is responsible for
the management of the World Bank-assisted Livestock and Poultry Project,
consisting of 872 farm-level sub-borrowers throughout the country. A
monitoring and evaluation system was instituted by the DPB in;1979 to
provide a continuous flow of project indicators, as well as data for the
eventual ex-post evaluation and impact studies, in order to improve credit
performance and enhance the benefits of the project.
MONITORING AND EVALUATION
The Monitoring and Evaluation System (MES) is primarily concerned
with the effects of credit on outputs, resource utilization, farm income, and
the variance between actual project performance and the planned projections.
Specifically, the MES is interested in the issues of financial and economic
rates of return, technological changes brought about by the project, the
effects of external factors (such as prices) on project performance, employ-
ment, and other benefits, as well as loan repayment performance characteristics.
The methodology of the MES consists of the selection of critical
indicators of project performance, setting up minimum standards of acceptable
performance to provide a basis for evaluating the sub-borrowers, and a com-
parative analysis of the project as a whole. Operationally, the MES is
divided into three components. These are the Technical Module, which keeps
track of the physical inputs, processes and outputs in livestock and poultry
production; the Financial Module, which applies the tools of financial and
business management analysis to the technical data base to monitor the prof-
itability and financial viability of the livestock and poultry enterprises
under different and varying market conditions; and the Economic Module, which
assesses the effect of marketing on productivity, income generation, and loan
repayment, evaluates the effects of credit on technological changes in terms
of farm management, farm employment, resource utilization and farm income
generation, and investigates the relationship between repayment rate accord-
ing to project types, sizes, and other project attributes.
Sample surveys by means of a questionnaire and continuous farm
records are the main instruments of the MES for generating the primary data
for the study. A control group for comparative purposes willl be included
wherever feasible. Sub-borrowers will be stratified for sampling purposes
according to farm type, size of loan, and the nature of partlicipation in
the credit program.
- 68 -
ANNEX IV
Page 22 of 42
The data are processed to produce a detailed set of indicators
covering the technical, financial, marketing and economic aspects. Statis-
tical methods will be employed for data analysis. However, the lack of an
initial baseline survey has hampered a more direct measurement of the impact
of the program through direct comparison of the situation before and after
project intervention.
- 69 -
ANNEX IV
Page 231of 42
A Philippine Experience on an Agricultural Extension
Reporting, Monitoring, and Evaluation System|
Francisco G. Rentutar
THE PROJECT
The agricultural extension service of the Philippines provides a
major link between research institutions and the rural population by dissemi-
nating information on improved practices in crop and livestock production,
home management, and the building of a self-reliant farming community.
In providing this service to the 14.8 million rural families through-
out the country, a number of constraints and problems have been identified
recently. These include inadequate pre-service and in-service training of
the extension staff, lack of subject-matter specialists, lack of mobility of
the field staff, inadequate teaching aids and equipment, confusion brought
about by duplication among numerous government agencies engaging in similar
activities, time taken in organizing farmers credit, the neglect of large
sections of the farming community, and low salaries paid to field workers.
The overall objective of the National Extension Project (NEP) is to
strengthen the extension service and render it more effective in delivery of
its services to the rural population by resolving some of the problems and
constraints confronting it. The project has five major components: (1) orga-
nizational reform through the establishment of a more unified&extension serv-
ice and closer combination between research and extension; (2) provision of
facilities and equipment to increase mobility of the field staff for training
and administration; (3) improvement of extension methodology by introducing
sound extension methods, emphasizing close supervision, regular training of
extension workers, and regular farm visits; (4) improvement of staffing and
training, particularly for additional subject-matter specialists and livestock
extension technicians; and (5) provision for acquiring consultant services for
improvement of management, salary review, and procurement procedures and for
developing more effective monitoring and evaluation procedures.
The project is funded jointly by the Government of the Philippines
and the World Bank in equal proportions for a total amount of US$70.1 million
over a period of four years, beginning in 1979.
MONITORING AND EVALUATION
With the organization of the National Extension Project, a separate
Planning, Monitoring, and Evaluation Unit was created. The unit has a current
staff of five, with the necessary supporting personnel.
- 70 -
ANNEX IV
Page 24 of 42
The monitoring and evaluation system of the NEP seeks to measure the
performance of the field technicians in the various planned programs of the
Bureau of Agricultural Extension, against the planned targets. In agricul-
tural food crop production, the system monitors the hectarage covered by each
extension worker, the number of farmers assisted, and the production. In the
nutrition program, the system follows the growth performance of the targeted
infants, the number of infants covered by each health technician, and the
number of mothers assisted. These data and other information collected by the
system serve as inputs to management planning and control.
For data collection, standardized reporting forms are used. Addi-
tional data are obtained through special sample surveys conducted by Central
Office specialists and program advisers to cross-check data gathered from
the various field reports. Data collected at each level in the organization
are processed and analyzed at the next higher level and all processing is
performed manually at present, though attempts are being made to mechanize
the procedure.
At this initial stage, the unit is still exploring appropriate
indicators for determining the effectiveness of individual extension workers
under extremely diverse conditions of operation. Because of the geographical
distribution of the field offices, the lack of efficient communication facil-
ities is causing some delays in reporting. Some delays are also caused by
staff who are not directly appointed by the Bureau.
- 71 -
ANNEX IV
Page 251 of 42
A Case Study on the Philippine Fourth CB-IBRD Rural Credit Project:
Monitoring and Evaluation Aspects
Leopold J. Magpale
THE PROJECT
This rural credit project is the continuation of a series of similar
ones started in 1965. These projects were designed to assist the Philippines
in raising domestic food production, improving the balance of payments, and
raising rural incomes. Under the current project, commencing in 1977, the
Central Bank of the Philippines will on-lend a total of US$91.3 million of
World Bank loan proceeds and government counterpart funds to 360 Rural Banks
(RBs) and 20 Stock Savings and Loan Associations (SSLAs). These qualified
banks and credit institutions will in turn finance 16,000 farmers, fishermen,
and rural entrepreneurs in investments in farm mechanization, light transport,
cottage and agro-industries, coastal and inland fisheries, and small livestock
development. The World Bank loan would also provide funds for studies to
assess the impact of farm mechanization in the Philippines, for establishment
of an Agricultural Machinery Testing and Evaluation Center, for training
of personnel, and for acquisitions of service vehicles for the central bank
field staff. The project will be completed by the end of 1980.
Implementation of the project is assigned to the Department of Rural
Banks and Savings and Loan Association (DRBSLA) of the Central Bank, which
has a staff strength of about 650.
MONITORING AND EVALUATION
The first task of the unit under the present project was to review
the adequacy of the existing reporting system and to introduce some necessary
revisions in the format, routing, and distribution of the reports to reflect
changes in project design and related loan covenants.
The perceived indicators of success or failure in the implementation
of the project are obtained from the four types of routine reports at monthly
or quarterly intervals. These reports include: the Loan Officers Monthly
Reports, consolidated by Technical System Evaluation Unit to the Project
Director; the Monthly Report of Arrearages on Medium- and Long-term Loans by
RBs and SSLAs; the Monthly Status Report on the CB-IBRD Agro-,industrial
Financing Program to the Monetary Board; and the Quarterly Report of the RBs
and SSLAs on the Rural Credit Projects. On the basis of these reports, the
unit prepares the quarterly and annual reports indicating thej cumulative
performance during the period under review by aggregating thel data contained
in the individual reports of the participating institutions.
- 72 -
ANNEX IV
Page 26 of 42
Developments, achievements, and problems concerning other project
components are based on special ad hoc studies and are reported separately.
Project viability is monitored at the farm level, covering significant changes
in input and output prices. At the farm level evaluation, the TSEU staff
prepares a representative model of the project categories in terms of pro-
forma income and expenditure and cash flow projections. Profitability ratios
are applied to determine the financial viability of ongoing projects. These
occasional field evaluations conducted by the staff complement the more
extensive evaluation component of the project, which is subcontracted to a
professional research firm.
Supplementary information of a specific nature is obtained from both
formal and informal sources regularly or as the need arises. This information
is used to confirm the aggregate data being monitored in the regular system.
Currently the DRBSLA management is attempting to consolidate and
simplify the various reporting formats, so that the rural banks and savings
and loans associations may not be overburdened with numerous and excessive
reports to be completed at frequent intervals.
- 73 -
ANNEX IV
Page 27: of 42
A Proposed Monitoring and Evaluation System for the'Rainfed
Agriculture Project I (Iloilo), The Philippines
Candido S. Dizon
THE PROJECT
The project aims at increasing agricultural production in the
nonirrigated areas of Iloilo Province in the Western Visayas Region of the
Philippines. The five major project components are: (1) Agricultural Pro-
duction Services, with inputs for an agricultural and livestock extension
service, seed distribution and quality control facilities, research into
multiple cropping, and training of extension staff and farmers; (2) farm
credit for dry-land, wet-land and livestock farmers; (3) infrastructure
facilities for an efficient agricultural marketing system and small irriga-
tion schemes; (4) social services with facilities for combatting anemia and
diarrhea in the area; and (5) project management and administrative support
for strengthening of existing institutions at the provincial and national
levels.
Field operation is directed by a project manager. A Provincial
Development Council, assisted by the Regional Council, will formulate project
policy decisions. A Cabinet Coordinating Covncil determines policy for
the National Rainfed Agriculture Program. The secretariat of this council
translates these policy decisions into project operational terms and monitors
the implementation of the project.
The agricultural development and marketing components of the project
will benefit approximately 40,000 small farmers cultivating about 60,000 hec-
tares, including 1,000 farmers who will benefit from the irrigation facilities.
The project will begin in 1980 and will be for five years, with a total esti-
mated cost of US$62.4 million, of which about US$11.5 millionlwill be financed
by a loan from the World Bank.
MONITORING AND EVALUATION
The objectives of the monitoring and evaluation system are to apply
major indicators for measuring the level, pace, and direction of the economic
and social impact of the project; to determine progress achieved in overcoming
constraints to development; to determine the direct distributive effects of
the project; and to provide a data base for inference to the possible effec-
tiveness of the project in other geographical areas.
Routine data and information will be collected on almonthly and
quarterly basis to meet the needs for immediate policy decisions. These data
relate to the farm economy, crop production, credit and loansi, input applica-
tion, crop varieties, damage and problems, training, research', marketing, and
health.
- 74 -
ANNEX IV
Page 28 of 42
Periodic data collection efforts will concentrate on the farm
economy, economic efficiency, and the social impact, with indicators for
housing conditions and facilities, food and nutrition, health, education,
transport, religion, and culture. This information will be gathered before,
during, and after implementation for evaluating the impact of the project.
The monitoring and evaluation will be conducted by an external
agency, but the routine data are to be collected by the project farm techni-
cians. The data are then consolidated at the municipal, district, and pro-
vincial levels and processed manually at the Project Headquarters. At the
central level, the data are analyzed monthly, with the use of computers, to
measure project progress. Results are furnished to the various councils
involved in management and fed back to the project manager and the partici-
pating agencies for follow-up action.
- 75 -
ANNEX IV
Page 29iof 42
A Proposal for the Design of an Evaluation System
for the Second IBRD Rural Development Project in the Philippines
Antonio Perlas
THE PROJECT
The Second Rural Development Project aims to strengthen and support
the technical capacity of the Ministry of Agrarian Reform of the Philippine
Government, and to test the collective impact of government programs for
low-income rural areas. The emphasis of the project is on the better ex-.
ploitation of the remaining public lands in the country suitable for devel-
opment as settlement schemes for small farmers. The project area covers
the three provinces of Capiz in the Vizayas and of Agusan and Bukidnon in
Mindanao.
The project components involve an integrated package of rural
infrastructures, agricultural services and credit cooperatives, public health,
village water supply, family planning, and institution building for strengthen-
ing management capacity and delivery of rural services.
The five-year project will involve costs estimated at US$32.6 million.
THE EVALUATION SYSTEM
The evaluation system aims at assessing the total impact of the
project components on the beneficiaries in the three communities. The system
is designed to address itself not only to the content via quantifiable indi-
cators, but also to the processes of change and adaptation, the interaction
of the people, and the impact of change on the individual and-his institutions.
Both quantitative and qualitative measures are involved in considering external
and internal factors and accounting for formal and informal practices. The
evaluation system is seen as an action-oriented management tool, geared toward
decision making, with provisions for effecting corrections and improvements
during the life of the project. It is recognized that the impact of the
various components may not be totally manifest until years after project
completion, but by stressing the importance of the human factor the eventual
outcome of the project could be assessed.
The structure of the system will be based on the three interrelated
foci of Farmer Development, Crop Development and Land Development. These
factors and the associated indicators will be investigated by way of both a
sample survey and participant observation. The survey will give the study a
breadth of coverage, and the observation will render a depth of analysis.
The combination of these approaches will give a more complete picture of the
situation as it evolves, with changes being instituted by the project.| Norms,
expressed in terms of selected indicators, will be established to reflect the
objectives and goals of the rural development project. The current status of
- 76 -
ANNEX IV
Page 30 of 42
these indicators will be measured before project commencement and will be
compared with the target. These developmental gaps and the necessary course
of action to narrow them progressively will then be the priorities for man-
agement's decision making. Current work on the design of the evaluation
system will determine more precisely the indicators to be used at different
levels of development, the project benefit measures, and the relevant human
factors.
The evaluation system is to be distinct from the monitoring function,
which is a routine in-house activity. Although it was specifically desired
by the project manager that the contracted evaluation team should be inde-
pendent of the project organization to prevent it from being biased, some
overlapping activities and information-gathering of the monitoring unit may
be utilized by the system to prevent costly duplication.
- 77 -
ANNEX IV
Page 311of 42
A Case Study on the Monitoring and Evaluation Sys'tem
for the Agricultural Land Reform Program of Thailand
Suthiporn Chirapanda
THE PROGRAM
The 1975 Agricultural Land Reform Act of Thailand has been designed
to improve a situation of high rate of tenancy (approximately 20% of farm
households), landlessness, and lack of security for those farmers cultivating
public forest reserve land. More specifically, the objectives'of the Act
were to redistribute land, increase agricultural production, improve credit
and marketing facilities, promote farmers- organizations, improve rural'
environment, and promote education, public health and public utilities, while
reducing the income gap between rural and urban population.
Under the provisions of the Act, an Agricultural Land Reform
Executive Committee is established, with the Minister of Agriculture and
Cooperatives as Chairman. In the Ministry, an Agricultural Land Reform Office
(ALRO) administers the program. Areas with serious landlessness, widespread
tenancy, and low productivity are designated as Land Reform Areas (LRAs). In
1979 there are 79 such LRAs being included in the program, involving some
300,000 ha of tenanted land and 650,000 ha of public land. Implementation
is divided into three separate phases: survey and analysis, land distribu-
tion, and development. The ALRO is responsible for land reform preparation
and land redistribution, but for developmental activities, the ALRO performs
the functions of a coordinating agency.
MONITORING AND EVALUATION
The objectives of the monitoring and evaluation system are to
determine the pace of progress of various land reform activities in the
context of the overall program, and with respect to their specific targets,
to identify major elements responsible for the success or failure of the
program, and to make recommendations on implementation.
The activities of the Monitoring and Evaluation Unit include
the conducting of benchmark surveys of socioeconomic conditions by random
sampling in every LRA and a similar survey in the LRA after five years for
comparative study of the changes. Data are also collected from' the various
implementing agencies, specifically for establishing quantifiable annual
targets. The degree of success of the program as a whole is to be measured
by the amount of land allocated to the farmers, number of farmers allocated
land, distribution of land ownership certificates, and levels of farm and
nonfarm incomes.
- 78 -
ANNEX IV
Page 32 of 42
In the field of land reform, political pressures often play an im-
portant role in the decision-making process. Information on land purchase is
sometimes delayed and subsequent adjustments of the land acquisition plan may
not always be possible. Some basic information (such as the state of tenancy)
was occasionally found to be inaccurate and therefore could not be used for
decision-making. Activities and performance of other agencies in conjunction
with various provincial governments are often not available. Despite these
shortcomings, the information system has been able to demonstrate that the
annual targets are in excess of the capacities and provisions, while the
efforts in attempting to achieve the targets have been insufficient and
unbalanced.
The Monitoring and Evaluation Unit has a staff of 20 members and an.
annual operating budget of US$40,000, representing only an extremely small
proportion of the annual program cost of US$21.1 million as of 1978.
- 79 -
ANNEX IV
Page 331of 42
A Report on the Monitoring of the
Phitsanulok Irrigation Project, Thailand
Judha Krishnamra
THE PROJECT
The Phitsanulok Project is a major irrigation development effort
in Thailand. Apart from providing irrigation facilities for 16,000 agri-
cultural holdings, with a total cultivated area of 50,000 hectares, the
project also includes the establishment of five demonstration farms as
focal points for dissemination of modern farming techniques through a
project-based extension service. There will also be a pilot project for
land consolidation.
The implementing agency is the Royal Irrigation Department, which
appoints from its staff the Project Director and the Project Manager.
Project management is responsible for coordinating the activities of irri-
gation, extension, and land consolidation, undertaken by various participating
agencies.
PROJECT MONITORING AND MANAGEMENT ANALYSIS
The monitoring activities are the responsibility of the Economic
Analysis and Land Policy Subdivision of the Land Development Department.
The Monitoring Unit is subordinate to the Project Coordinator and works
closely with the Project Director. The unit has separate sections respon-
sible for data collection, analysis, and information presentation.
The aims of project monitoring are to conduct an agroeconomic base-
line survey, to assess the socioeconomic impact of changes in land tenure and
agricultural activities, to evaluate the progress of the project in relation
to targets set in the appraisal, and to identify problems encountered in the
coordination between the irrigation engineering and agricultural development
components.
A baseline survey was conducted 1976/77, attempting to solicit
farm level information on socioeconomic background of the farm households,
occupational characteristics, marketing of farm produce, loans and credit,
attitude toward the project, and other social characteristics; including
nutrition, health, and exposure to mass media. A multi-stage stratified
random sampling technique was employed. The basis for stratification was
the proximity of the farm households to the operation foci of demonstration
farms and the land consolidation pilot project. The total sample size was
1,980 farm households, with 10% being in the control group residing outside
the project area. Thirty-two people were involved in the baseline study.
- 80 -
ANNEX IV
Page 34 of 42
The bulk of the work was performed by the staff of the Department of Land
Development. Consultants were engaged for providing inputs to planning and
design, data processing and analysis, computer programming, and reporting.
Apart from being responsible for assessing and feeding back the
appropriate information related to project accomplishments, the Monitoring
Unit is also concerned with analyzing administrative problems of coordination
among the participating agencies that might deter efficiency in project per-
formance. Since management analysis is considered a sensitive issue it was
decided that an independent agency outside the project management (the
National Institute of Development Administration) should be invited to
perform the task. When the analysis has become a routine procedure and
accepted by the parties concerned as being constructive, the unit will
handle further follow-up studies.
At present, some staff positions in the unit, particularly in the
vital area of the Information and Presentation Subunit, remain vacant, await-
ing the recruitment of personnel of sufficiently high caliber to perform the
task. Furthermore, a feedback system between the unit and the operational
field units from the Irrigation, Land and Extension Departments and the
Provincial Committees has yet to be established.
- 81 -
ANNEX IVj
Page 351of 42
A Case Study on the Monitoring and Evaluation of the
North Kelantan Rural Development Project, Malaysia
Zaharudin bin Jaafar
THE PROJECT
The North Kelantan Rural Development Project aims at; improving
the productivity and income of the rural population, narrowing the urban-
rural income disparities, and reducing national dependency on imported food-
grains. These objectives are to be achieved through the provision of some
small-scale irrigation facilities, improvement of irrigation service standards
on some existing projects, construction of some rural roads, establishment and
upgrading of a number of Farmer Development Centers from which farmers could
obtain the essential farm inputs on preferential terms, and strengthening
of the agricultural extension service.
Implementation of the project involves several agencies, under the
coordination of the Ministry of Agriculture. The overall development policy
is the responsibility of the Steering Committee, on which the;participating
agencies are represented. The Project Manager has a staff of 155, including
134 agricultural extension workers. Besides being directly responsible for
the agricultural components, he has to coordinate the project activities of
the Drainage and Irrigation Department, the Public Works Department, the
Farmers- Organization and the Kemen Agricultural Development Authority.
The Project Office is composed of the three sectors of Administration,
Publicity, and Monitoring and Evaluation.
The project, commencing in 1977, will cover a five-year period,
with a total cost of US$48.0 million, of which 44% is financed by a World
Bank loan. The target population includes approximately 108,000 poorer .farm
families in six districts.
MONITORING AND EVALUATION
The monitoring and evaluation function is targeted to record the
financial and physical progress of the various components of the project
to ensure that project activities are integrated with and complementary to
each other. Project outputs, effects, and impact are to be measured against
the planned targets. The measurement of changes in farm productivity and
household income will provide the basis for evaluating the su�cess of the
project as a whole.
- 82 -
ANNEX IV
Page 36 of 42
The envisaged activities of the Monitoring and Evaluation Section
include the integration of regular reports from the various implementing
agencies concerning physical and financial progress for the Steering
Committee and the World Bank on a quarterly basis; the conducting of a base-
line survey in all districts within the project area and an annual sample
survey to take stock of the changing farm situation and income status; and
the undertaking of special sample surveys to determine the changing attitude
of the farmers and the performance of the extension scheme. To monitor the
project effects, it is proposed that the crop cutting surveys of the Statistics
Department should be extended to cover the entire project area.
The Monitoring and Evaluation Section was only established in 1978,
two years after the inception of the project. Presently, it has a staff of
six, including field enumerators. The Section has an annual budget allocation
of less than US$10,000, which is considered grossly inadequate for all the
activities it has to perform.
Apart from budgetary and staffing problems, the Monitoring and
Evaluation Section has also encountered difficulties in the uneven supply of
information by the various coordinating agencies, on the basis of which the
reports are compiled. The lack of decision-making authority of the project
management has also detracted from the effectiveness of monitoring and
evaluation.
- 83 -
ANNEX IVI
Page 37 of 42
A Preliminary Proposal for a Monitoring and
Evaluation System for the National
Extension Project in Malaysia
Chan Ah Kiow
THE PROJECT
The National Extension Project (NEP) is the second phase of the
Malaysian Government's strategy in strengthening the agricultural extension
service and has been built upon the foundation of the activities of the fore-
runner Extension Liaison Unit, which bridged the gap between research findings
and known technology, on the one hand, and adoption of improved practices by
farmers, on the other.
The objectives of the NEP include the improvement of agricultural
productivity and increase in income of the poorest segment of Malaysian
society; the reduction in urban-rural and ethnic income disparities; and
the reduction of national dependency on imported food grains.
The project aims at increasing the numbers of agricultural techni-
cians involved in the training and visit system of extension; establishing
four regional Extension Training and Development Centers for in-service train-
ing of field staff; and producing as well as distributing improved seeds for
paddy and minor crops. Construction of new buildings, purchase of vehicles
and equipment, and provision of consultancies and fellowships are also in-
cluded in the project.
The training and seed components of the project would be implemented
by the Extension Branch and the Crop Production Branch in the Department of
Agriculture. Each component would be headed by a project leader administra-
tively responsible to the Project'Director. At the state level, the project
is coordinated through the State Directors of Agriculture.
This five-year project began in 1979 and the total project cost is
estimated at US$46.5 million, of which 41% would be supported by a World Bank
loan.
MONITORING AND EVALUATION'
The monitoring and evaluation system of the project is designed
as a tool for decision making and as a continuous process of problem defi-
nition, measurement, analysis, and assessment. The system would therefore
primarily be concerned with the efficient implementation and evaluation of
project effects and impact.
The project would rely on monitoring to provide time'ly information
concerning the quantity, quality, costs, and timing of inputs and activities,
add the immediate outputs of the project. Ongoing evaluation aims at provid-
ing information for making adjustments in the basic premises, lobjectives and
- 84 -
ANNEX IV
Page 38 of 42
design of the project, while ex-post evaluation will review comprehensively
the experience gained and impact that has been achieved to serve as a basis
for a future policy formulation and project design.
The system applies the methodology of the logical framework as a
tool for monitoring and evaluation. Nine key indicators have been identified
for monitoring the training and visit system of extension, and a further seven
key indicators are to be used for evaluation. Means of verification of the
indicators include the use of personal field observations, extension service
records, government local purchase orders and official receipts of payment,
ad hoc studies, and farm surveys. A baseline survey before the effects of the
project begin to manifest themselves and continuous assessment of changes
brought about during and after the implementation period will also be con-
ducted. The Chief Extension Officer of each state would be required to
furnish a quarterly report in a standard format pertaining to staffing and
to physical and financial progress, as well as problems and key issues. As
approval for the establishment of the evaluation unit is still being sought,
cost estimates remain unavailable.
- 85 -
ANNEX IV
Page 39 of 42
A Case Study on Monitoring and Evaluation in the
Yogyakarta Rural Development Project, Indonesia
Samirin
THE PROJECT
This multi-component rural development project aims at reducing the
extreme poverty among 12 million people in the two poorest districts of Gunung
Kidul and Kulon Progo in Yogyakarta. The project deals with appropriate tech-
nology for increasing agricultural production for hillside farming at various
stages of land degradation. Also included in the project are provision for
small-scale industrial promotion, improvement of credit facilities, rural
water supply, road construction, health care and rural institutional develop-
ment through training and encouragement of local initiatives in developmental
activities.
A major goal of the project is to assist the Government of Indonesia
to decentralize planning by assisting the shift of major responsibility for
planning and executing projects to the provincial level, with the strengthen-
ing of the Provincial Development Planning Board (BAPPEDA). Akmajor long-term
benefit of the project will be the identification of relevant development
packages which could be duplicated throughout the country.
Individual tomponents are to be implemented by specific line-
agencies, and a Project Coordination Unit has been established to ensure that
the project is to be executed in an integrated manner according to design.
The management team of six includes a Monitoring Officer and a Project
Evaluation Officer.
MONITORING AND EVALUATION
Monitoring and evaluation play a vital part in the management pro-
cess. The Monitoring Officer maintains a close liaison with each component
manager and reports to the Project Coordinator on the physical progress of
complementary components undertaken by the different participating agencies.
He also assists the coordinator in preparing the consolidated quarterly
physical progress reports and seeks methods of solving problems encountered
during project implementation. The Evaluation Officer prepares requests for
submission of project evaluation proposals by agencies concerned and reviews
the proposals submitted. He is also responsible for supervising the agencies
contracted to evaluate the project, particularly for the health and drinking
water components. In addition, he makes annual reports on the impact of the
project, based both on the evaluation reports and on personal field observa-
tions. He is to evaluate the physical progress and financial costs of the
various project components and to prepare recommendations to expedite project
schedules and to keep costs within reasonable limits.
- 86 -
ANNEX IV
Page 40 of 42
The consolidated Quarterly Evaluation Reports containing details
on physical progress, budgets, costs, organization problems, and recommenda-
tions are reviewed by the head of the Coordination Unit and approved-by the
Chairman of BAPPEDA. Copies are forwarded to the Governor, the Bupati, the
Dinas of the implementing agencies, the Department of Home Affairs, the
Department of Finance, Bank Rakyat Indonesia, and the World Bank. Through
this process, follow-up activities and management decisions will be formulated.
As the Coordinated Unit started functioning only a few months pre-
viously, the systems and procedures for administration, project management,
finance, monitoring and evaluation are still being formulated.
- 87 -
ANNEX IV!
Page 41 of 42
Aspects of Monitoring and Evaluation of the Southern
Highlands Project, Papua New Guinea
John Wallis
THE PROJECT
The Southern Highlands Province, with a population of 235,000 (90%
in subsistence agriculture), is a backward area in Papua New Guinea. In line
with the National Development Strategy, the general aims in development of
the province include the acquisition of revenue thorugh participation in
business enterprises, the spread of the commercial sector, the provision of
wage-labor employment, the raising of productivity in the subsistence farming
sector, the strengthening of agricultural extension, the improvement of the
health program, the increase in educational opportunities, the promotion of
nonformal education, the use of appropriate technology for basic needs, and
the improvement of the road system.
The Southern Highlands Project has several interrelated components
covering the fields of agricultural development, health facilities, education,
and basic infrastructure. Under this project, research into a relevant type
of agriculture suitable for the area would be promoted. Provisions for a
crop development program involve the establishment of land blocks and export
processing facilities for tea, cardamon, coffee and silk. Accessibility
to health services would be improved by creating more Health Subcenters; the
health services would be strengthened by a program of training to increase
staff quality and motivation. Information on disease patterns, particularly
pneumonia, will be collected and analyzed. Epidemological studies are being
conducted. Through both formal and nonformal education, literacy and ability
to use appropriate technology at the village level would be increased. Elec-
tricity is provided for the tea and coffee processing factories and various
types of roads are being constructed to improve communication.
The project, commenced in 1978, is for six years. The total project
cost is estimated to be approximately US$20.00 million, and the International
Development Association (IDA) will reimburse about 64% of the project costs.
MONITORING AND EVALUATION
In view of the limitation of project funds, compared with the funds
for a diverse number of subprojects and possible alternative approaches among
the project components, a strong selection of subprojects for funding can be
made. The system must be mutually compatible among the threejend-users: the
Provincial and National Governments and the IDA. Some key indicators and
criteria have already been chosen at the national government level.
- 88 -
ANNEX IV
Page 42 of 42
A major objective of the project itself, particularly pertaining to
the Agricultural Field Trials, Studies, Extension and Monitoring Unit (AFTSEMU)
and the Epidemology Unit (EU), is to establish a system of data collection
and monitoring for ongoing use by the,province. It is recognized that a
significant amount of the information required for establishing effective data
and monitoring systems is common to all the three levels of potential users:
project, provincial, and national. Effort is being directed to ensure that
information collected by one user is compatible with the analysis systems of
the others, and costly duplication is to be avoided.
Indicators for the specific needs of monitoring the progress and
impact of each component are being developed. Additional inputs to the
monitoring system include accounting records, loan surveillance reports of
the Development Bank, and a wide range of regular reports to different levels
of government.
- 89 -
ANNEX V
Page 1 of 3
SELECTED REFERENCES
I. General Studies
Barker, David. Some Methodological Issues in the Measurement,iAnalysis,
and Evaluation of Peasant Farmers' Knowledge of Their Environment.
London: Monitoring and Assessment Research Centre, Chelsea College,
University of London, 1977.
Christoffersen, Leif E. "The Bank and Rural Poverty," Finance and
Development, vol. 15, no. 4 (December 1978), pp. 18-22.
Daines, Samuel R. An Overview of Economic and Data Analysis Techniques
for Project Design and Evaluation. Washington, D.C.: Practical Concepts
Inc., for the Agency for International Development, Development Studies
Program, 1977.
Deboeck, Guido. "Monitoring and Evaluation of Rural Development Projects:
An Early Assessment of World Bank Experiences." Paper presented at!a
workshop on Experiences with Information Systems for Rural Development,
Organization for Economic Cooperation and Development, Paris, 1978.
_ Systems for Monitoring and Evaluation of Nutritional
Interventions. Washington, D.C.: Rural Operations Review and Support
Unit, Agriculture Department, World Bank, 1978.
Freedman, Deborah, and Eva Mueller. "A Multi-Purpose Household Question-
naire: Basic Economic and Demographic Modules." Washington, D.C.:
The World Bank for the World Bank and the Agency for International
Development, 1977.
Klatter, Matty. Evaluation of Social Projects: A Selected and Annotated
Bibliography. The Hague: Institute of Social Studies, 1978.
Lorstadt, Mats. "Experience from a Nutritional Survey Bringing Micro-
Computers into the Field." Lund, Sweden: Decision Data Inc.,
1979. Processed.
Lynch, Frank. "Field Data Collection in Developing Countries:
Experiences in Asia." New York: Agriculture Development Council.
Seminar Report, June 1976. Processed.
. "How to Make a Social-survey Interview Schedule--
Instructions for Beginners." Bangkok: The Agricultural Development
Council, Inc., Paper on Survey Research Methodology, 1979.
- 90 -
ANNEX V
Page 2 of 3
Morss, Elliot. "Barriers to the Utilization of Information Systems to
Monitor and Evaluate Rural Development Projects." Washington, D.C.:
Development Alternatives Inc., 1978.
Moser, C. A., and G. Kalton. Survey Methods in Social-Investigations. London:
The English Language Book Society and Heinemann Educational Books Ltd.,
1971.
Oppenheim, A. N. Questionnaire Design and Attitude Measurement (Open
University Set Book). London: Heinemann Educational Books, 1968.
Parel, Cristina, and others. Sampling Design and Procedures (Papers on Survey
Research Methodology). Singapore: The A/D/C Asia Office, Tanglin
P.O. 84, Singapore 10, 1971.
Patton, Michael Q. Utilization-Focused Evaluation. London: Sage
Publications, 1978.
Rossi, Peter H., H.E. Freeman, and Sonia R. Wright. Evaluation: A Systematic
Approach. Beverly Hills, California: Sage Publications, 1979.
Rutman, Leonard S. Evaluation Research Methods: A Basic Guide. London:
Sage Publications, 1977.
Scott, Wolf, with Helen Argalias and D. V. McGranahan. The Measurement
of Real Progress at the Local Level: Examples from the Literature and
a Pilot Study. Report No. 73.3. Geneva: United Nations Research
Institute for Social Development, 1973.
ul Haq, Mahbub. "Changing Emphasis of the Bank's Lending Policies,"
Finance and Development, vol. 15, no. 2 (March 1978), pp. 12-14.
United Nations, Department of Economic and Social Affairs. Systematic
Monitoring and Evaluation of Integrated Development Programmes: A
Source-Book. (ST/ESA/78) New York, 1978.
World Bank. Rural Development: Sector Policy Paper, Washington, D.C., 1975.
. Built-in Project Monitoring and Evaluation: A First Review.
Report No. 1758. Washington, D.C.: Operations Evaluation Department,
1977.
* Monitoring and Evaluation of Rural Development Projects: A
Progress Report." Washington, D.C.: Rural Operations Review and Support
Unit, Agriculture Department, 1978. Processed.
_ Operations Evaluation: World Bank Standards and Procedures.
Washington, D.C., 1979.
- 91 -
ANNEX V
IPage 3 of!3
Yates, Frank. Sampling Methods for Censuses and Surveys. London:
Charles Griffen, 1963.
Yudelman, Montague. "Impact of the Bank's Rural Development Lending,"
Finance and Development, vol. 16, no. 3 (September 1979), pp. 24-28.
Zarkovich, S. S. Sampling Methods and Censuses. Rome: FAO Methodology
Branch, Statistics Division, 1965.
II. Case Studies
Carruthers, I. D., and E. S. Clayton. "Ex-Post Evaluation of Agricultural
Projects: Its Implications for Planning," Journal of Agricultural
Economics, vol. 28, no. 2 (September 1977), pp. 305-318.
Deboeck, Guido. "Monitoring Rural Development with Minicomputers:
The Quinault Indian Experience." Washington, D.C.: Rural Operations
Review and Support Unit, Agriculture Department, World Bank, 1979.
Processed.
and B. Kinsy. "Managing Information for Rural Development:
Lessons from Eastern Africa." World Bank Staff Working Paper, no. 379,
Washington, D.C.: World Bank, March 1980.
Deboeck, Guido, and D. Rubin (eds.). "Selected Case Studies on Monitoring
and Evaluation of Rural Development Projects: Vol. I: Eastern Africa."
Rural Operations Review and Support Unit, Agriculture and Rural Develop-
ment Department, World Bank, 1980.
"Selected Case Studies on Monitoring and Evaluation of Rural
Development Projects: Vol. II: East Asia and the Pacific." Rural
Operations Review and Support Unit, Agriculture and Rural'Development
Department, World Bank, 1980.
Haque, Wahidul, and others. "Micro-level Development: Design'and Evaluation
of Rural Development Projects," Development Dialogue, no.<2. Uppsala,
Sweden: Dag Hammarskjold Foundation, 1977.
Lele, Uma. The Design of Rural Development: Lessons from Africa.
Baltimore: Johns Hopkins University Press, 1975.
School of Oriental and African Studies Team. Land-Use and Socio-Economic
Changes under the Impact of Irrigation in the Lam Pao Project Area,
Thailand. United Kingdom: School of Oriental and African Studies,
University of London, 1978.
I HG3881.5 .W57 W67 no.439 c.3
I Deboeck, Guido.
i Monitoring rural development
'I | in East Asia.
I i
) !I