the international division of Helen Keller Worldwide
Child Survival 15
CA NO. FAO-A-00-99-00015-00
September 30, 1999 – September 29, 2003
Final Evaluation
Judi Aubel, PhD, MPH December 30, 2003
Nutrition-Focused Child Survival Project in Koulikoro, Mali
1
This report was prepared by Judi Aubel and the evaluation team outlined in Annex A. Zeina Sifri, Lina Mahy and Shawn Baker of Helen Keller International were consulted in preparing this report.
Cover pictures : Danielle Hayes
2
Table of Contents
A. B.
Summary…………………………………………………………………..…….1 Assessment of Results and Impact of the Program
1. Summary of baseline and final data for program indicators………..…..4 2. Technical Approach
a. Overview of the project………………………………………..……6 b. Progress by intervention area………………………………..…....6
i. Vitamin A supplementation ii. Iron-folate supplementation iii. Promotion of iodized sal iv. Breastfeeding v. Complementary feeding vi. Consumption of Vitamin-A rich foods vii. Management of diarrhea disease
c. New methodologies and approaches………………………..…..18
i. Regional micronutrient days ii. Qualitative community study of household decision-making and influence on MCH practices iii. Grandmother-focused neonatal health strategy
3. Analysis of cross-cutting approaches and lessons learned
a. Strengthening community/local capacity for health promotion b. Major constraints to accomplishment of DIP BCC objectives c. Progress related to capacity building and sustainability strategy
C. Program Management
1. Planning and follow-up of program activities………………………..…..30 2. Staff training and development………………………...……..…..….,,…32 3. Supervision of program staff……….………………...………………….,,32 4. Human resources and staff management…………….……………….,,33 5. Financial management…………...…………………..………………….,,33 6. Logistics……………...……………………………………………...…….,,34 7. Information management……...…………………………………….…..,,34 8. Administrative and Technical assistance……...……………………….,,36 9. Management lessons learned……………..…………...……………….,,36
D. Conclusions and Recommendations
1. Accomplishment of program objectives……………………………..…..37 2. Main achievements…………………………………………..………..…..38 3. Main constraints……………...…………………………....…………..…..39 4. Main lessons learned………………………………………………….…..40 5. Recommendations for USAID/GH/CSHGP……………..…………..…..42
F. Results Highlight: Involving Grandmothers in promoting maternal and child
nutrition……………………………………………………………………………..…..43
3
Annexes
A. Evaluation team members and their titles B. Assessment methods
C. List of persons interviewed and contacted
4
Abbreviations
ASACO CBD CHV CS CSCOM CSP DHS DIP ESSC IFA JRM M&E MCH MI MIS MOH & MSD MTE MS N/CS PMA PO SIAN
Association de Santé Communautaire (Association of Health Communities)
Community-Based Distribution Community Health Volunteer Child Survival
Centre de Santé Communautaire (Community Health Centers) Child Survival Project
Demographic and Health Survey
Detailed Implementation Plan (Plan Détaillé d’ Exécution)
Equipe Socio-Sanitaire du Cercle (District Health Management Team)
Iron folic acid
Journées Régionales de Micronutriments (Regional Micronutrient Days)
Monitoring and Evaluation Maternal and Child Health Micronutrient Initiative
Management Information System
Ministry of Health & Ministry of Social Development and Solidarity with Older Persons Mid-Term Evaluation
Ministère de la Santé (Minister of Health) Nutrition/Child Survival programs
Paquet Minimum d’Activités (Minimum package of activities) Plan d’Opérations des MOH & MSD (Operational Plan)
Semaine d’Intensification des Activités de Nutrition (Intensified Nutrition Weeks)
5
A. Summary
The CS-15 project was implemented in three districts in Koulikoro Region in Mali from 1999-2003. The total beneficiary population was approximately 175,500 children 0–59 months of age and 210,600 women of reproductive age (WRA). The overall project goal was to assist the Government of Mali to develop programs that promote priority nutrition-related practices at the health facility, community and household levels. Ninety percent of the project effort focused on maternal and child nutrition interventions and ten percent on management of diarrhea disease. The project adopted a capacity-building orientation to
strengthen MOH & MSD, NGO and community structures/practices. The project strategy included activities at the national, regional, district, health facility and community levels. A complementary two-year neo-natal health project, with funding from Save the Children/US, will continue in some project areas through June 2004.
Main accomplishments of the project: There is increased commitment on the part of the Ministry of Health & Social Development (MOH/SD) at the national and regional levels to maternal and child nutrition. At the national level this is reflected in national nutrition policies and tools to which HKI contributed, namely, the national plan of action for nutrition and the first MOH training modules on maternal and child nutrition. An important achievement is the sustained national-level commitment to VA supplementation through NIDs and National Nutrition Weeks (SIAN), for which CS-15 has been the ‘breeding ground’. This is a good example of ‘going to scale’ with an intervention. At the district level there is
increased inclusion of nutrition-related activities in district operational plans (PO), namely: periodic mass distribution of micronutrients; systematic iron
supplementation of pregnant/breastfeeding women; and distribution of VAC to post partum women, well and sick children. At the health facility level, through training and follow-up/supervision, nutrition-related activities have been
strengthened, namely; micronutrient supplementation of women and children; and health education/counseling of pregnant and breast-feeding women
especially related to micronutrients, breastfeeding and complementary feeding. All of these facility levels, activities appear to be sustainable in the post-project period without additional outside resources. The skills of partner NGOs, radio broadcasters and traditional communicators have been strengthened to promote nutrition/child survival in their ongoing programs. There has been strong support from health workers and communities alike for the innovative grandmother-focused strategy as a means of promoting sustained changes in community health/nutrition norms related to maternal and child nutrition/health, and initial results of this strategy are very positive.
Highlights from baseline and final evaluation surveys: Based on baseline and final KPC assessments, the results of the four-year project were somewhat limited. Yet it should be noted that according to the results of the latest NDHS survey (2001), many social and health indicators (including maternal mortality,
1
child mortality, chronic malnutrition) have worsened compared to 1996. Although 61% of the project objectives have improved, only one third of the 21 project objectives (33%) were actually met. Slightly more than one third (38%) were not met and, unfortunately, for almost one third of the DIP indicators (29%) no data was collected at endline making it impossible to assess the level of
accomplishment. The objectives that were met deal with positive increases in: Vitamin A supplementation of post-partum women; consumption of iodized salt; early initiation of breastfeeding; timely introduction of complementary foods; consumption of vitamin A rich foods by both women and young children; and appropriate feeding of children with diarrhea under one year of age. A major constraint to accomplishment of the DIP objectives was the limited development of community activities by NGO partners due both to unrealistic expectations of NGO capacity and to inadequate resources budgeted for NGO activities.
Priority conclusions/lessons learned: 1) DIP objectives and plan: In all CS programs DIP objectives and strategies should be developed with key partners, all implementers should be involved in monitoring progress toward DIP objectives and key strategies should be documented on an ongoing basis. 2) Monitoring and evaluation for program learning: In addition to ongoing quantitative data collection, it is important that CS programs periodically analyze program implementation qualitatively and develop and share lessons with all program implementers and partners. 3) Collaboration with district health teams: In order to strengthen local capacity to implement state-of-the-art nutrition/child survival strategies CS team members must maintain close communication and collaboration with district health/ development team members. It would be
beneficial for CS teams to receive basic training in organizational development strategies to increase their effectiveness in this critical task. 4) Strengthening skills of health/development agents: In addition to “training events” and supervision of health/social development agents other mechanisms should be used to reinforce acquisition of new knowledge/ skills including technical updates in routine coordination meetings and periodic dissemination of short “technical notes.” 5) Collaboration with local NGOs: CS programs should include
collaboration with other NGOs working in the same area and should specifically try to strengthen the capacity of local NGOs involved in health/nutrition. However, future projects should be realistic about what is required to increase NGO effectiveness and should anticipate the financial and technical resources
necessary to do so. A baseline assessment of the NGO capacity can be done in order to determine what is realistic in the particularly situation. 6) Use of
community health volunteers: Over the years CS/community health programs involving community health volunteers (CHV) have repeatedly encountered the same constraints. In future programs an essential first step should be to review lessons learned from past CHV programs in order to anticipate and hopefully avoid these age-old problems. 7) Collaboration with local radio stations: In countries such as Mali, where local radio stations proliferate, it is advantageous for health/development programs to exploit this means of communication.
However, the need to pay for most broadcasts often creates a dependency on
2
externally funded projects, thus decreasing the sustainability of these activities. All projects should carefully analyze these constraints at the outset. 8)
Collaboration with traditional communicators: Where groups of traditional communicators exist it is beneficial for health/development projects to collaborate with them and use their skills to promote priority topics/information. Once again, the challenge is to develop a strategy that is not project-dependent. 9) Simple and sustainable communication tools: Songs and stories are simple, culturally relevant, effective and sustainable tools for communicating key nutrition/health information to communities. 10) Strengthening the role of grandmothers as health promoters: Given the primordial role that grandmothers play in
household decision-making related to health/nutrition in Mali, it is advantageous to develop strategies that involve communication not only with women of
reproductive age but also with senior women, or grandmothers. This intervention is likely to be transferable to many other countries because of the importance given to senior members in most societies.
3
B. Assessment of Results and Impact of the Program
1. Summary of baseline and final data for program indicators
The results from the baseline and final KPC surveys for all program indicators are summarized in Table 1 below.
Table 1: Results from baseline and final KPC surveys
DIP Indicators Baseline Target
Survey F NF
Vitamin A supplementation
Preventive VAC coverage of children 12 to 75% 75% 80% 59 months (% of children who received a VAC in previous six months)
VAC supplementation of sick children (with NC NC 67% chronic diarrhea, measles, xerophthalmia, severe PEM) seen in CSCOMs/functional health centers with minimum package of services
Children with chronic diarrhea seen in NC NC 95% CSCOM with PMA who received VAC
(except for those who had already received it in the previous six months)
Distribution of VAC to women during the first
1
40 days after delivery
7.5% NA 50% • Those who attended CPN in CSCOM
NA 4.5% 30% • Those who delivered in the village/
Home level with CBD
Iron-folate supplementation
Women who took iron folate tablets during 40% 32% 67% pregnancy
Compliance by pregnant women with prescribed iron folate tablets:
NC NC 85% • Those who received tablets from
CSCOMs
NC NC 85% • Those who received tablets from CBD
agent
Prescription of iron-folate to children seen in NA NA 95% CSCOM and diagnosed with severe anemia (% of children)
Final Survey F NF 61% 67%
NC NC
NC NC
88% 12% 80% 20%
49% 58% NC NC
NC NC
NC NC
1
These two indicators are not mutually exclusive and, therefore, it is difficult to formulate clear conclusions based on this data.
4
Promotion of iodized salt
Bi-annual testing of salt sold in local markets NA by CSCOM staff to verify quality
% Of households using iodized salt 56,8% Breastfeeding % of women who initiate breastfeeding 68% within four hours after birth
% of infants exclusively breastfed for 4 14% months Complementary feeding Introduction of complementary foods 53% between 6 and 9 months (% of children) Consumption of vitamin A rich foods Consumption of vitamin A rich foods (based on HKI food frequency score, estimated as ‘times per week’)
3.40 • Children 12 - 71 months of age
3.35 • Women of reproductive age
NA 56,8% 70% 11%
95% 80% 80% 20%
NC NC
98% 94% 85% 86% 8%
10%
51% 60% 90% 75%
2.90 2.85
+1.5 days
F 4.90 5.60 5.10 NF 4.40 F 4.85 5.00 6.00 NF 4.35
Management of diarrhea diseases
% of children < 1 year who receive ORS during diarrhea
5.5% • Children seen in CSCOM
NA • Children treated in NF areas % of children 12 - 71 months who received 8% ORS during diarrhea
% of children < l year old with diarrhea who 63% are given the same or more food than usual % of children 12 – 71 months of age with >1/3 * diarrhea who are given the same or more food than usual
NA 5.5% 9% 67% >1/3 *
50% 40% 25% 75% 75%
45% NA 9% 80% 52%
NA 9% 22% 83% 35%
Two types of health facilities:
F = Functional/officially recognized community health center (CSCOM) NF = Non-functional health center, not yet officially recognized by MOH
Explanation on data not included in the table: NA = Not applicable
NC = Not collected
These results were not presented as precise percentages either in the baseline or final surveys.
5
2. Technical Approach
a. Overview of the project
The project was implemented in the Koulikoro Region in Mali, in three out of
seven districts, Kati, Kolokani and Koulikoro.2 The total beneficiary population is 175,500 children 0–59 months of age and 210,600 women of reproductive age (WRA). The overall CS-15 project goal was to assist the Government of Mali to develop programs that promote priority nutrition-related practices at the health facility, community and household levels. Ninety percent of the project effort focused on maternal and child nutrition interventions and ten percent on management of diarrhea disease. The project adopted a capacity-building
orientation to strengthen MOH & MSD, NGO and community structures/practices. The project strategy included activities at the national, regional, district, health facility and community levels. A complementary two-year neo-natal health project, with funding from Save the Children/US, will continue in some project areas through June 2004.
b. Progress by intervention area
As seen in the summary table above, data was not collected for six of the DIP indicators in the endline survey due to an oversight on the part of project staff. i. Vitamin A supplementation
The project promoted vitamin A supplementation of women after delivery and of well and sick preschool-age children.
Indicator
Preventive VAC coverage of children 12 to 59 months (capsule received in previous six months
VAC supplementation of sick children (with chronic diarrhea, measles, xeropthalmia, severe PEM) seen in CSCOMs/functional health centers with minimum package of services
Children with chronic diarrhea seen in CSCOM with PMA who received VAC (except for those who had already
received it in the previous six months)
Baseline Survey F NF 75% 75%
NC DIP Final Survey objective
F NF 80% 61% 67%
NC 67% NC NC
NC NC 95% NC NC
2
Kolokani district is divided into two zones, Kolokani and Ouéléssébougou, and for this reason, in this document reference is made to “four zones” rather than “three districts/circles.”
6
Distribution of VAC to women during the first 40 days after delivery3
• Those who attended CPN in CSCOM
• Those who delivered in the village/ home level with CBD
7.5% NA NA
4.5%
50% 30% 88% 12% 80% 20%
VAC supplementation of children 6 to 59 months of age
• Preventive supplementation of children 12 to 59 months of age
At baseline, 75% of women in both F and NF areas with children 12 to 59 months reported that their child had received a vitamin A capsule (VAC) in the previous 6 months. The DIP target was 80% coverage. Compared with baseline data, in the final survey a smaller proportion of women reported that their children
received VACs in the past six months (61% in F areas and 67% in NF areas) and thus, the DIP objective was not attained.
The primary activity supported by the project to increase supplementation of
children in this age group was the distribution of VAC during the annual Regional Micronutrient Days (JRnoM) that were established in the region with HKI’s
support. A secondary activity was the training and follow-up of health facility staff in the prescription of vitamin A to both sick and well children during consultations at health facilities. Most children receive the VAC during the JRMs.
The project objective and target was to ensure that at any point during the year 80% of children would be protected with VAC supplementation. The decrease in VAC coverage may be associated with the fact that the recall time between the baseline survey (conducted in Jan/Feb. 2000) and the previous mass distribution (Dec. l999) was somewhat shorter than the time between the final survey (Aug. 2003) and the previous mass distribution (June 2003).
However, it is useful to note that the campaign to deliver VAC in December 1999 was widely publicized, in fact it was endorsed by the President of Mali. The June 2003 campaign was the first SIAN and it involved far less investment and
publicity. This shift in delivery strategy for VACs may partially explain the lower coverage and suggests that more investment in community mobilization and marketing will be needed in the SIANs to obtain high coverage.
Lesson learned: Shifts in program strategy and implementation should be
anticipated over the five-year project. The baseline and endline surveys need to 3
These two indicators are not mutually exclusive and, therefore, it is difficult to formulate clear conclusions based on this data.
7
designed and conducted in ways to ensure that the progress and achievements of the program can be measured.
• VAC supplementation of sick children 6 to 59 months
The project promoted VAC supplementation of sick children (with chronic diarrhea, measles, xeropthalmia, severe PEM) seen in CSCOMs/functional health centers offering the minimum package of services. Two overlapping DIP indicators deal with supplementation of sick children: (1) VAC supplementation of sick children (with chronic diarrhea, measles, xeropthalmia, severe PEM) seen in CSCOMs/functional health centers with minimum package of services; and 2) children with chronic diarrhea seen in CSCOM with PMA who received VAC (except for those who had already received it in the previous six months).
Unfortunately, neither baseline nor endline data was collected on either of these indicators. At baseline it was decided not to do so because secondary data was available on VAC distribution to sick children from a BASICS health facility
assessment conducted in 1997. Since the same survey questionnaire was used for baseline and end survey, the Malian research institution contracted to conduct the survey failed to add this question to the questionnaire. Because of the nature of the evaluation, this indicator should have been more clearly defined and should have been collected at both baseline and endline.
All qualitative reports from regional health officials in Koulikoro and from health facility providers state that “VAC supplementation has become a routine activity during consultations with sick children.” Project-supported training on
micronutrient administration and some project support for supervision visits by district health teams have undoubtedly contributed to promoting this activity in health facilities. Anecdotal information suggests that there has been a significant increase in this activity, however, there is no data to substantiate this claim.
VAC supplementation of post partum women
• Distribution of VAC to women within 40 days after delivery
At baseline VAC coverage for post partum women was very low, only 7.5% among women who attended CPN in CSCOMs, and only 4.5% among women who gave birth in the community/at home and who received services from CBD workers. The target for VAC coverage among those who attended CPN was set at 50% while it was set at 30% for those who received VAC through CBD agents.
At endline, for women who attended CPNs, 88% in F areas reported receiving VAC supplementation, significantly more than the DIP target of 50%. In the NF areas, at baseline there was also a very low rate of VAC post-partum coverage (4.5%) and there was an increase to 20% at endline, though this accomplishment did not meet the DIP objective of 30%. In the project plan it was anticipated that
8
community volunteers would be involved in CBD of VAC, however, this activity was not developed. This is due to the lack of institutional support for free
community based distributions: the Regional Ministry of Health representatives are still in disagreement on free distribution by volunteers of drugs and
supplements. A national dialogue on CBD is urgently needed. While this four-fold increase is positive, it should not be attributed to increased CBD distribution as HKI did not support development of this activity and it does not appear that there has been any other support in the region for CBD of VAC.
Based on qualitative data collected during and at the end of the project, it
appears that most women who deliver in health facilities receive VAC soon after delivery, though only approximately 37% of women in the Koulikoro region do deliver in recognized health facilities (2003). The majority of women give birth at home. A possible explanation for the considerable difference between the two strata of women may be that even though most women do not give birth in health facilities, those who attend CPNs are more likely to come to the health facilities during the post-partum period to receive their VAC than are the women who give birth at home. This also suggests that during the CPN the midwives are encouraging this practice.
VAC supplementation of both well and sick children and of women post-partum appears to have increased considerably. This may be attributable to project-supported activities including: increased knowledge of the importance of VAC supplementation among regional health officers; training of health facility staff ; and some supervision of this activity by district health teams. However, there has not been systematic follow-up of the recording of Vitamin A administration in health facility notebooks (for this purpose) nor is this data routinely analyzed. In addition, for two of the DIP indicators related to supplementation of sick children at endline no data was collected, making it impossible to precisely assess changes in VAC coverage or to determine whether the increases were attributable to CS program activities.
Lessons learned:
• A national consensus on CBD is urgently needed in order not to create discrepancies between regions and de-motivate volunteers
• HKI Mali has actively created demand through its effective IEC activities for post partum VAC supplementation: rural radio, NGOs and theatre groups have been trained and distributed messages. The increase in coverage may be partly attributed to this.
9
ii. Iron-folate supplementation
Indicator
% of women who took iron-folate during their pregnancy4
Compliance by pregnant women with prescribed iron folate tablets:
• Those who received tablets from
CSCOMs
• Those who received tablets from
CBD agents
Prescription of iron-folate to children seen in CSCOM and diagnosed with severe anemia
Baseline Survey F NF 40% 32%
NC NC
DIP objective 67% 85% 85% 95%
Final Survey F 49% NC NC NC
NF 58% NC NC NC
NC NC NA
NA
• Pregnant women who took iron-folate tablets
In the DIP the objective related to the prescription of iron-folate to pregnant
women stipulates that pregnant women “receive 90 iron-folate tablets.” However, in the baseline and endline surveys the questions on iron supplementation did not specifically ask women if they had “received 90 tablets” but only if they had “taken iron when they were pregnant.” At baseline 40% of women in F areas and 32% in NF areas reported that they had taken iron tablets during their pregnancy. Based on these results the DIP objective was set at 67%. The endline survey results show that there has been an increase in the use of iron by pregnant
women, from 40% to 49% in F areas although they are less than hoped for in the DIP target (67%).
The PSE activities reinforced iron-folate supplementation of women attending pre-natal visits in health facilities. Outside of health facilities the CBD activities to distribute iron-folate at the village level were not developed as planned. As
mentioned previously for the CBD of VAC, also for iron-folate supplementation no consensus exists on how, by whom and at what cost these supplements should be provided. Given this situation, it would have been expected that an increase in iron supplementation by pregnant women would be greater in the F areas than in the NF areas. However, the endline survey results suggest that the increase in iron utilization by pregnant women was greater in the NF area. In discussions 4
The DIP indicator refers to “pregnant women who were prescribed 90 tablets of iron-folate” whereas the data collected in the baseline and endline surveys incorrectly assessed “women who took iron during their pregnancy.” Clearly these two indicators deal with two different parameters. It was decided to include in this table the indicator reflecting the question actually asked in the baseline and endline surveys.
10
with project staff and MOH & MSD partners an explanation for this result was not found.
• Compliance of pregnant women with iron-folate supplementation
At baseline it was decided not to collect information retrospectively on women’s compliance with prescribed iron-folate prescribed to them during pregnancy, given the difficulty of doing so. In the DIP, the objective was to reach 85% both among women who received iron tablets at CSCOMs and those who received them from CBD agents. Qualitative information collected during the final
evaluation revealed that many women experienced undesirable side effects while taking the iron tablets but it is unclear what portion of them abandoned the
supplements and which took them regularly during pregnancy. During the project no quantitative data was collected on compliance among pregnant women and in the endline survey no provision was made for collecting this data.
• Prescription of iron folate to children with severe anemia
Another DIP objective concerns iron supplementation of children with severe anemia who are seen as out-patients at CSCOMs. This practice did not exist at baseline and, therefore, it was not assessed at the outset. The DIP objective was set at 95% of all such children seen in health facilities. Project training activities promoted this practice and a system of notebooks was put in place in the CSCOMs in which all micronutrient supplementation activities were to have been recorded. Follow-up of the use of the notebooks was irregular and there was no periodic analysis of the data recorded in the notebooks. At endline no provision was made for collecting this data so it is not possible to draw any conclusions regarding the degree to which this objective was attained.
Lesson learned: There are a number of difficulties in assessing iron+folic acid coverage and compliance. For future work, HKI proposes a new methodology, based on accepted methodology for measuring exclusive breastfeeding rates: Target: 80% of pregnant women in catchment areas and project villages take IFA supplements regularly. Indicator: Percentage of pregnant women in the 2nd and 3rd trimester of pregnancy who took a IFA in the last 24 hours. Target: 80% of postpartum women in catchment areas and project villages take IFA
supplements regularly. Indicator: Percentage of women in the 1st trimester postpartum who took a IFA in the last 24 hours.
iii. Promotion of iodized salt
Indicator
Bi-annual testing of salt sold in local
markets by CSCOM staff to verify quality
Baseline Survey F NA
NF NA
DIP objective
95%
Final Survey F NF NC NC
11
Household use of iodized salt 56,8% 56,8% 80% 98% 94%
• Bi-annual testing of salt sold in local markets
In order to promote the sale of iodized salt by local merchants, one of the project indicators involves bi-annual verification by health center (CSCOM) staff of the quality of salt being sold in local markets using simple testing kits originally provided by UNICEF. At baseline this practice was not assessed as it did not previously exist. The DIP objective was for 95% compliance with this practice. A system for monitoring this indicator was not put in place and neither during the life of the project nor at endline was data collected to measure accomplishments related to this objective. Early in the project some testing of salt was carried out, however, the original salt verification kits expired and in the latter part of the project they were not available to either district or health center staff to continue the testing. For these several reasons, it is not possible to draw any precise conclusions regarding the level of accomplishment of this indicator.
Lessons learned:
• It would have been better to include municipality personnel in the
monitoring of salt in public markets. Health staff are reticent to conduct this exercise.
• A good coordination and communication with the test kit provider (UNICEF) is essential to ensure timely availability of the kits.
• Household use of iodized salt
At baseline 57% of households in both F and NF areas reported using iodized salt. The DIP objective was set at 80%. The final survey showed that in both F and NF areas there has been a significant increase in the use of iodized salt to 98% and 94% respectively. Present levels of reported use of iodized salt greatly exceed the DIP objective. These increases can probably be attributed both to the activities to educate and monitor the quality of salt sold by merchants and to the radio and community level communication activities. It should be noted that IEC messages have been delivered as micronutrient packages including iron, vitamin A and iodine messages.
iv. Breastfeeding
Indicator
Women who initiate breastfeeding within four hours after birth
Infants exclusively breastfed for 4 months 14%
Baseline DIP Final Survey
Survey objective N NF N NF 67,7% 69,5% 80% 85% 86%
11%
20%
8%
10%
12
When the project was designed and the DIP developed in 1999 the MOH policy regarding initiation of breastfeeding stated that newborns should be put to the breast within the first four hours after birth. This explains why this timeframe was used in the DIP indicator. It also explains why both baseline and endline data were collected on this now-outdated indicator. HKI contributed to the subsequent revision of MOH policy that now advises initiation of breastfeeding in the first half hour after birth.
At baseline almost seventy percent of women (68% in F areas and 70% in NF areas) who had given birth in the previous 12 months reported having initiated breastfeeding during the first 4 hours after birth. The DIP objective was set at 80% and endline data show that the objective was attained, with almost the same portion of women in both F and NF areas (85% and 86% respectively) reporting this practice following their last delivery. These increases are probably due to positive changes in the practices of health facility midwives as well as to advice given by midwives to pregnant women during CPN, following the training HKI organized for midwives to sensitize them on the importance of early initiation of breastfeeding.
• Exclusive breastfeeding
Similar to the previous indicator, the DIP indicator on exclusive breastfeeding (EBF) refers to “4 months” rather than 6 for the duration of this important
practice. At the time that the DIP was written in early 1999, MOH policy stated that infants should be breastfed for 4-6 months. Through advocacy efforts and technical support of HKI the MOH’s “thinking” on this matter has shifted and EBF for six months is widely promoted, although the official written MOH policy (les normes et procédures) has not yet been changed accordingly.
At baseline, 14% of women in F areas and 11% in NF areas reported exclusively breastfeeding their infants for four months and the DIP objective was set at 20%. Endline data suggest that there has not been an increase in this practice, with only 8% of women in F areas and 10% in NF areas reporting having practiced EBF with their last child. However, it must be noted that interviewers of the baseline survey did not add the question ‘Haven’t you given anything else,
including water’ to make sure that mothers practice exclusive breastfeeding. The interviewers of the endline survey effectively did ask this question. So, probably the baseline data are a too high an estimate and rates may not have declined during the project. The project was using a phased approach and promotion of EBF was carried out in the second half of the project, including health worker training, NGO community volunteer training and radio broadcasts on this topic. Qualitative interviews during the final evaluation showed that most women still • Early initiation of breastfeeding
13
believe that it is necessary to give infants water in addition to breastmilk. It is generally believed that it is difficult to promote adoption of EBF, however, these results are particularly discouraging given the considerable effort made to promote this practice.
Lesson learned:
• The message on EBF needs to become even simpler: do not give water to children below 6 months, only give breastmilk.
• Also this message needs to be adopted by all partners, and needs to be transversal: EBF is good to reduce diarrhea, is a family planning tool, and improves the infant’s immunity. Yet, the USAID mission has not yet
included a breastfeeding indicator for its next 10-year plan of cooperation.
v. Complementary feeding
Indicator
Introduction of complementary food between 6 and 9 months
Baseline Survey F NF 53% 51%
DIP objective 60%
Final Survey F 90%
NF 75%
• Introduction of complementary foods between 6 and 9 months
At baseline approximately half of all women with children under one year of age in both F and NF areas (53% and 51% respectively) reported starting to give their child complementary foods at between 6 and 9 months. The DIP target was set at 60% and in both areas the endline data suggest that there was significant favorable change in this infant feeding practice, with the target having been widely attained (90% in F areas and 75% in NF areas). The significantly higher endline results in the F areas, where women generally have more access to advice from health workers, suggests that health center staff, who all received training on EBF, have been actively promoting this practice.
There would appear to be a contradiction between these results, which suggest that most women do not introduce foods other than breastmilk before 6 or 9 months, with the results on EBF levels (above) which suggest that by 4 months most women have already starting given other “things” to their infants. The explanation for the apparent contradiction would appear to be that while most women do not introduce additional “foods” until their child is 6 months old, most women have not given up the traditional practice of giving water to their infants, “to quench their thirst,” in the first months of life.
In this regard, one of the lessons formulated by the CS team and MOH partners is that educational efforts to promote changes in practices related to all aspects of infant feeding will be more effective if they focus not only on younger women,
14
but if they also involve older women, or grandmothers, who have a great influence on these practices at the household level.
Leasson learned: The inclusion of older women is a promising activity, which continues through the Save the Children funded SNL project. It is up to HKI Mali to document, disseminate and promote this finding.
vi. Consumption of vitamin A rich foods
Indicator
Consumption of vitamin A rich foods Consumption of vitamin A rich foods (based on HKI food frequency score)
- Children 12 - 71 months of age
- Women of reproductive age
Baseline Survey F NF 3.40 2.90
3.35 2.85
DIP objective
Final Survey F 5.60
NF 5.10 6.00
+1.5 days
F 4.90 NF 4.40 F 4.85 5.00 NF 4.35
• Vitamin A-rich food consumption by children 12 – 71 months
At baseline it was found that the average score for vitamin A–rich food
consumption among children 12-71 months of age was 3.40 in F areas while it was somewhat less (2.90) in NF areas. The DIP objective for F areas was set at 4.90 and at for 4.40 for NF areas. Endline results show that for children in this age group there were positive increases in the consumption of vitamin A-rich foods, to 5.60 in F areas and to 5.10 in NF areas. In both cases, endline results surpassed the DIP objective.
• Vitamin A-rich food consumption by women of reproductive age
For women of reproductive age, at baseline consumption of vitamin A-rich foods was virtually the same as for children. In F areas the consumption score for
women was 3.35 and in NF areas it was 2.85. The DIP objective was set at 4.85 for F areas and 4.35 for NF areas. As with children (previous parameter),
endline data show very positive increases in consumption of vitamin A-rich foods by women of reproductive age, from 3.35 to 5.00 in F areas and from 2.85 to 6.00 in NF areas. In both cases, the DIP objective was surpassed.
Positive increases in the consumption of vitamin-A rich foods both by children and women can probably be attributed to: improved nutritional advice from health workers after receiving training on nutrition/micronutrients; radio messages promoting such foods; and community level interpersonal communication activities.
15
Lesson learned: HKI Mali has made sure to promote locally available vitamin A rich foods, acceptable and affordable by the local population. This has been an important factor for the successful achievement of this indicator.
vii. Management of diarrhea disease
Indicator
Children < 1 year who receive ORS during diarrhea
• Children seen in CSCOMs • Children treated in NF areas Children 12 - 71 months who received ORS during diarrhea
Children < l year old with diarrhea who are given the same or more food than usual Children 12 – 71 months of age with
diarrhea who are given the same or more food than usual
Baseline Survey F NF 5.5% NA NA 5.5% 8% 9% 63% >1/3
67% >1/3
DIP
objective 50% 40% 25% 75% 75%
Endline Survey F NF 45% --NA --NA 9% 9% 22% 80% 52%
83% 35%
At baseline less than six percent of all children with diarrhea received ORS (5.5% in both F and NF areas). The DIP objective was set at 50% for sick children in F areas and 40% for those in NF areas. Endline data show that in F areas there has been considerable increase in the administration of ORS to children with diarrhea (from 5.5% to 45%), though the increase did not quite reach the level of the DIP objective. In NF areas, however, endline results on this indicator show a small increase in ORS use (from 5.5% at baseline to 9% at endline), but much below the DIP objective.
• ORS administration to sick children 12 – 71 months of age
Similar to results regarding ORS use with children under one year of age, at baseline less than one tenth of children between 12 and 71 months received ORS during diarrhea (8% in F areas and 9% in NF areas). The DIP objective for this age group was set at 25%. In F areas there was no increase in ORS use (from 8% to 9%) whereas in NF areas there was considerable increase (from 9% to 22%), though in neither case was the DIP objective attained.
It is unclear why either health providers or household members would be more likely to give ORS to younger children in F areas and less likely to do so in NF areas. A possible explanation could be that mothers with children under one have more frequent contact with health facilities (i.e., vaccination). This would provide the opportunity to obtain ORS if the child simultaneously has diarrhoea.
16
• ORS administration to sick children under l year of age
Once the child is completely vaccinated, the contact with health facilities is
reduced. Therefore the mother of a child 12-71 months might be more inclined to implement other practices like the sugar salt solution. Unfortunately, the KPC results were not yet available when the qualitative analysis of key program implementation issues was carried out during the final evaluation, so it was not possible to investigate these results further or in-depth.
Based on a mid-term evaluation recommendation, in the second half of the
project the training and communication activities focused on promoting the use of home fluids, given constraints to accessing ORS packets. Unfortunately, there was not a DIP objective and target for the use of home fluids to assess possible changes in rehydration practices at the household level.
• Quantity of food given to children under 1 year with diarrhea
At baseline, approximately two-thirds of children < 1 year with diarrhea in both F and NF areas were given the same quantity or more food than usual during their illness (63% in F areas and 67% in NF areas). The DIP objective was set at 75% for both F and NF areas. Endline data from both F and NF areas show that this objective was attained in both areas (80% in F areas and 83% in NF areas.). The HKI team emphasized the nutritional aspect of diarrhoea treatment: health staff have been trained, IEC messages have been developed. Obviously, this has been well implemented by health staff and community workers.
• Quantity of food given to children 12 – 71 months with diarrhea
It is curious that in both the baseline and endline surveys in both F and NF areas, a much smaller proportion of older children (12 – 71 months) were given the same or more food than usual, as compared with the amount of food given to younger children (< 1 year). While at baseline less than one third of women both in F and NF areas reported giving more or the same quantity of food as usual to their sick child, the DIP objective was set at an ambitious level of 75%, at the same level as the objective for this practice with children in the younger age group. In the endline survey, although this feeding practice increased to 52% in F areas and to 35% in NF areas, it falls short of the objective. These levels are considerably lower than those for younger children (previous indicator) at 80% and 83% respectively. Unfortunately, these KPC results were not available during the final evaluation so it was not possible to qualitatively investigate the factors that contribute to these rather unusual results.
Lessons learned: It seems always easier to convince mothers to implement a new behavior for when the child is sick (e.g. diarrhea), whereas when a child is seemingly healthy, it is more difficult to convince mothers into adopting a new behavior (e.g. exclusive breastfeeding). Also mothers need to be encouraged to realize that their milk is a valuable resource that makes their child healthy and
17
helps it to grow. Empowerment of women to have confidence in their bodies needs to be an essential part of future nutrition promotion activities.
18
c. New methodologies and approaches i. Regional micronutrient days (Journées Régionales de
Micronutriments/ JRM) and nutrition weeks (SIAN/Semaine Intensifiée d’Activités Nutritionnelles)
Based on HKI’s experience organizing mass micronutrient distribution activities in other countries in Africa, with support from both the CS and Micronutrient Initiative projects, HKI supported a pioneer experience with JRM and SIAN in
Koulikoro region. An important characteristic of the approach promoted by HKI is that this activity is organized at the regional, rather than the national level, so that ownership and organizational capacity are developed at that level. At national, district and health facility levels, health sector staff express great satisfaction with this activity which enables them to increase coverage of both Vitamin A and iron supplementation to women and children. However, at the national MOH level it has now been decided that the mass distribution strategy will be used only in areas where routine distribution in health centers is weak, while the focus will be on strengthening routine supplementation activities in health facilities.
ii. Qualitative community study of household decision-making and influence on MCH practices
In the context of the complementary neo-natal health project (with funding from Save the Children/USA), an innovative methodology was used to study
household and community patterns of influence and decision-making related to MCH. In contrast to the standard community assessment methodology which narrowly focuses on the knowledge and practices of women of reproductive age (WRA), in this study the methodology was based on a “systems approach” which involved analysis of the influence exerted by different key family and community members on WRA related both to health promotion and to illness management.
The results of the qualitative study clearly revealed that the strategies and practices of WRA related to neonatal health are strongly influenced by various actors at the family and community level, and primarily by male heads of
extended families and by the grandmothers, or muso koroba (which includes both mothers-in-laws and maternal grandmothers). At the family level, the « muso koroba » are the primary resource persons for all issues dealing with women during pregnancy and labour, during the postpartum period and with newborns.
iii. Grandmother-focused neonatal health strategy
Based on the findings of the community study, a community strategy was developed to involve different community actors and traditional and modern health promoters in improving neonatal health practices. A major focus of the strategy is on actively involving grandmothers in participatory communication
19
activities using stories and songs to encourage them to consider incorporating new ideas into their age-old practices. A second component of the strategy involves training both formally-recognized midwives in rural health facilities and TBAs, as well as promoting increased collaboration between them in order to improve the quality of care provided to women during pregnancy and delivery.
The reaction to the grandmother-focused strategy on the part of district MOH officials, health facility staff and community members themselves has been overwhelmingly positive. Many health workers in the districts have expressed their satisfaction with the approach and have stated that the effectiveness of many past MCH programs has been limited by the fact that the influential muso koroba, or grandmothers, were not involved and, therefore, were not exposed to the same information and advice as younger women. At the community level, WRA and younger men alike have stated that involving the GMs very much facilitates adoption of new health-related practices by younger women. The funding for these activities continues until June 2004 at which time a full evaluation of this innovative strategy will be carried out.
3. Analysis of program cross-cutting approaches and lessons learned
The CS project strategy aimed to strengthen capacity to develop and implement nutrition/CS programs at several levels, namely: of the MOH at the national level; of the Regional Health Directorate in Koulikoro; of the district health management teams (Equipes Socio-Sanitaires de Cercles/ESSC) in the three target cercles (districts); of health facility staff in CSCOM and rural maternity
clinics/dispensaries; of community health associations (ASACO) that manage the CSCOMs; of local NGO partners; and of village level structures involved in health promotion. At the family level the project aimed to strengthen the knowledge and practices of household actors related to nutrition/CS. In this section of the report, project-supported strategies and activities at each of these levels are discussed, accomplishments and constraints related to each are described and lessons for future programs are presented.
a. Strengthening community/local capacity for health promotion
Many of the project activities focused on strengthening services provided in health facilities. However, other activities aimed to strengthen the health promotion capacity of local groups/ structures namely: local private radio
stations; traditional communicators; salt merchants; community health volunteers (relais), traditional community leaders and grandmothers.
• Strengthening local radio stations
Working through the district health officers, the project has collaborated with 9 local radio stations in the project area. Radio broadcasters have been trained
20
on the priority N/CS topics and they have all been given taped N/CS messages to broadcast. At the outset contracts were signed with them involving payment for dissemination of N/CS messages, however, it was financially impossible to continue this practice. Given that each station is required to dedicate some time to disseminating health/development messages, in the second half of the project the stations were encouraged to broadcast N/CS messages for free. While some of the stations cooperated on these terms, others did not.
Lesson: Given the potential of local radio stations to promote health and
development, health projects should use this local resource as much as possible. The challenge is to find a strategy for working with the local radio stations which contributes to sustained broadcasting of information on health/development topics. In this regard, externally-supported development projects should work closely with district/regional health institutions to strengthen their collaboration with the radio stations and the stations’ commitment to broadcasting information on health/development topics.
• Working with traditional communicators
Another local resource which exists in many communities are groups of
traditional actors/communicators. The project worked with both existing and newly-formed groups of communicators who were trained to facilitate community performances in which N/CS topics were presented through theatre and music. These sessions were very popular in rural communities and appear to have contributed to the acquisition of new information on N/CS. The groups were contracted to do a certain number of performances, traveling to communities around their home village. When the contracts ended the groups stopped performing. It would be impossible for these groups to self-finance their own transport to other villages though they may continue to perform in their home villages. The “traditional groups” appear to be much more motivated to continue this work on their own than do the newly-created groups.
Lesson: Future collaborative efforts with traditional communicators should focus on existing, traditional groups and efforts should be made at the outset to help them develop a strategy to generate some resources from their
performances so that they can continue with them. District health officials should also discuss with municipal officials the possibility of them providing resources to support these groups to enable the traditional communicators to promote health/ development issues using this much appreciated traditional mode of communication.
• Educating and monitoring salt merchants
In support of government policy encouraging the sale and use of iodized salt, the project developed specific radio messages targeting salt merchants and also encouraging the public to only purchase iodized salt. In the first phase of the
21
project salt being sold in the market was periodically tested, though this practice was, unfortunately, not continued because the project was unable to access additional salt testing kits.
Lesson: In promoting consumption of priority micronutrients/foods it is beneficial not only to educate the consumers but also the merchants on the nutritional
value of the products being sold. It would be beneficial to also educate vegetable and meat sellers on the relative nutritional values of the products they sell.
• Training community health volunteers (relais)
In the DIP it was planned that community volunteers would be used to carry out N/CS promotion activities including CBD of micronutrients and ORS. Some support was provided to NGO partners to train volunteers that were already
working with in the project area. Follow-up of these volunteers has been carried out regularly by only two of the five NGOs and to a limited extent by CS project staff. There is no specific information on activities they have carried out, their effectiveness or impact. In the SNL target areas community volunteers were also chosen and trained but follow-up has been weak there as well. Few communities have developed strategies to motivate these volunteers on an ongoing basis and the sustainability of volunteer activities is uncertain. CBD activities were not developed. In the mid-term evaluation a series of recommendations were developed to reinforce the activities of community volunteers but a systematic strategy was not developed to overcome factors limiting their effectiveness. While both NGO and SNL volunteers were trained, in most cases they were not periodically supervised and they have not enjoyed strong support from their own communities. Constraints associated with volunteer motivation observed in past programs were not sufficiently taken into account in developing the volunteer strategy.
Lesson: Community health volunteers can be a valuable resource in community health promotion activities but their effectiveness and the sustainability of their actions requires not only that they be trained, but also that they be periodically supervised and that they be strongly supported by their own communities.
• Strengthening the knowledge and practices of grandmother (mouso koroba)
In the mid-term evaluation it was recommended that a strategy should be developed to explicitly involve grandmothers in N/CS activities in order to increase their knowledge of priority modern N/CS practices. With
complementary funding from Save the Children/US, such a strategy was developed in which participatory communication activities were used with grandmother groups to encourage them to integrate priority modern practices related to the health/nutrition of pregnant/post-partum women and newborns into their existing practices. Communities have been very receptive to this strategy
22
and grandmothers themselves have shown great interest and openness to the new ideas proposed to them.
Lesson: Given the central role played by grandmothers in health/nutrition in Malian households, it is beneficial to involve them in educational activities in order to encourage them to adopt priority, “modern” maternal and child
health/nutrition practices. Their receptivity to such activities depends greatly on the approach used with them, which must be based on respect, dialogue and negotiation rather than on top-down approaches which dictate behaviors to be adopted. Also, substantively involving traditional community leaders along with the grandmothers greatly increases community support for proposed nutrition/health innovations.
b. Major constraints to accomplishment of DIP behavior change objectives
Many of the DIP objectives deal with individual behavior change and to improved access to basic health services at the village level. A major strategy envisioned in the DIP was development of CBD and behavior change activities at the village level by local NGOs. These activities were developed on a much smaller scale than anticipated. First, there was a structural constraint in the DIP design. The majority of DIP objectives deal with community level activities that were to be carried out primarily by the NGO, however, the geographic areas covered by the 9 NGO partners are less than 25% of the project area. A second major constraint was that resources to support NGO activities were greatly under- budgeted and, therefore, the scope of these activities was much more limited than anticipated. The combined effect of these two factors appears to have directly contributed to the limitations in project accomplishments revealed in the endline survey.
c. Progress related to capacity building and sustainability strategy
The following table summarizes the objectives of the capacity-building
sustainability strategy (left column) included in the DIP and the progress toward accomplishment of each of the objectives supported by the project during the four-year project (right column).
Table 2: Project objectives and accomplishments related to capacity-building and sustainability
Objectives Accomplishments
Regional/district level: -Key personnel in health region and -Training of health & social districts able to plan and implement development staff in nutrition/CS has quality nutrition programs increased their ability to implement quality nutrition programs/activities: for example, regional micronutrient days (JRM) 23
-Regional/district health/social develop-ment structures maintain formal coordination mechanism
District level (cercle): - Districts maintain adequate supervision of community health centers (CSCOM)
-Nutrition activities given greater importance in regional/district plans and budgets
-Regular coordination meetings
between regional and district staff; and between district and health facility staff
-Quarterly supervision visits to each health facility carried out approximately 90% of the time
-Resources for regular supervision visits included in donor support for regional/district operational plans (PO) -Micronutrient/nutrition activities not yet integrated into supervision tools though supervision teams report that they supervise micronutrient activities -HKI obtained a two-year grant from Save The Children/USA for neonatal health project in 48 villages in 4 zones - MOH obtained donor-funding for nutrition activities included in regional POs in 2002 and 2003 for: regional nutrition days; VAC supplementation of women and children
-One NGO (CARD) obtained funding to continue and expand community nutrition activities
HKI/MOH/NGOs define further funding needs and leverage required resources
As indicated in the table above, for each of the four objectives progress was made toward promoting increased capacity of partner organizations/personnel that will contribute to sustained inclusion of nutrition programs and activities in the region. However, the project did not systematically document progress toward accomplishment of these objectives.
i. Strengthening the PVO organization
Through this project, Helen Keller International was able to benefit from its membership with the CORE Group that allowed for extensive information
exchange between PVOs implementing CS projects. HKI was able to access a large number of resources that strengthened the technical and managerial capacity of both headquarters and field offices. The Fall and Spring CORE meetings provided additional information needed for a good implementation of
24
CS projects. The participation of HKI headquarters staff in several of the CORE working groups allowed for easy access to additional information, and a forum for discussing relevant issues. Headquarters administrative staff also benefited from participating in USAID’s Trainings on USAID Grants and Cooperative
Agreements. The CORE membership strengthened partnerships with other PVOs at the headquarters level, that have extended into the field. HKI benefited from the services of a Freedom from Hunger (FFH) staff (Robb Davis) traveling to the region, who undertook a training under this project on Breastfeeding and Infant and Young Child Feeding. The FFH module was freely shared and the experience was a success. This is just one example of the many opportunities this project has opened up. Another partnership developed through this project that is helping strengthen the PVO headquarters is with the FANTA project in Washington DC. HKI is in the process of developing a vitamin A guide for FANTA. HKI Mali staff were also invited to participate in a Roll Back Malaria Workshop organized by the Child Survival Technical Support team that provided technical assistance to HKI in an area that is not traditionally directly covered in the programs. Overall this project has supplied a significant amount of additional experience, technical information and managerial information to the staff directly and indirectly involved in this CS project at the field and headquarters level.
Unfortunately Helen Keller Headquarters were destroyed on September 11, 2001 with no human casualties however. This sad event led to a closing of
headquarters operations for one week and a temporary relocation of the offices in one partner’s building until permanent quarters were found. The event of September 11, 2001 also led to a complete loss of all files, technical, historical and financial records, and equipment. This proved to be a great handicap for the headquarters where staff had to spend an extensive amount of time
reconstituting the lost records and had little possibility to support the field. HKI being a decentralized agency, the Regional Offices supported the field offices. This was also the case for the Child Survival Project in Mali where little support was available from the headquarters office and the country had to manage on its own. This can explain some of the shortcomings of this project where the headquarters was not present to answer country requests for assistance and support.
ii. Strengthening local NGO partners
A major component of the DIP design was to partner with local NGOs working in the project area in order to strengthen their N/CS activities at the community level. This was to include the training of CBD agents to distribute micronutrients and ORS packets.
• Expanding community interventions through NGO partners
It was advantageous for HKI to partner with local NGOs in order to expand community level nutrition/CS activities. NGO-supported community animators (relais) were trained on priority nutrition/CS topics to enable them to promote
25
these topics with community groups. However, follow-up of these animators was weak and it is not clear what the quality or impact of their activities was. It appears that the relais will continue to share their nutrition/CS knowledge with community groups in either formal or informal ways beyond the life of the project.
Lesson: All externally-funded health programs should collaborate with local NGOs already working in the geographical areas targeted by those programs in order to strengthen their capacity to implement effective nutrition/health
interventions. It is important to periodically monitor NGO-supported activities in order to determine their effectiveness and impact.
• Collaboration with the local NGOs
Relative to DIP expectations, project collaboration with local NGOs was much more limited than anticipated due to: the extended period of time required to identify NGO partners; inadequate guidelines provided to NGOs and subsequent submission of NGO proposals far exceeding project resources; long delays in funding revised NGO proposals; suspension of NGO contracts after only 12 months (4 NGOs) and after 18 months (one NGO) due to project budgetary constraints. Planned CBD activities were not developed, to improve access to micronutrient supplements and ORS. The project was unable to carry out most of the mid-term evaluation recommendations intended to strengthen collaboration with the NGOs.
Lesson: In development of the DIP it appears that the complexity of working with local NGOs was underestimated as were the resources required to support efforts to substantively strengthen their capacity. Local NGOs in Mali are
generally very weak and future projects should be realistic about what is required to increase their effectiveness.
iii. Strengthening national nutrition/CS policies and programs
• Influencing national nutrition/CS policy making
Although the focus of the CS program was at the district and regional levels, the project also aimed to promote appropriate nutrition/CS policies and programs at the national level. Along with other key HKI technical staff, the CS project coordinator maintained close contact and communication with national CS/N technical teams and regularly participated in national level policy making and planning forums, contributing technical inputs to these processes. Significant examples of policy changes in the past four years to which HKI contributed include: development of a national strategy on micronutrient supplementation both through health facilities and mass distribution; and revision of the policy on exclusive breastfeeding.
26
Lesson: Although the focus of most CS projects is at the service delivery and community levels, as much as possible they should a maintain periodic
communication with national level policy/program officials and be involved in national level policy making and program development through advocacy efforts and participation in national level planning and policy-setting forums.
iv. Strengthening regional and district nutrition/CS programs
• Integration of nutrition strategies into regional plans
As compared with the pre-project situation, there is a now much greater
awareness of and commitment on the part of regional and district health staff to addressing maternal and child nutrition issues. This is a result of the formal training activities, ongoing formal as well as informal advocacy and consultation on the part of project staff with regional/ district health staff. More specifically, since 2002 nutrition promotion activities have been included in annual regional operational plans (Plans d’Opération) that previously were not part of those plans, for example: the week of intensified nutrition activities (SIAN/Semaine Intensifiée d’Activités de Nutrition); systematic iron supplementation of
pregnant/breastfeeding women; and distribution of VAC to pregnant women, well and sick children.
Lesson: Ongoing formal as well as informal strategies should be used by CS project staff to continuously reinforce the knowledge and commitment of regional/district health officials to new technical priorities and strategies.
• Integration of micronutrient indicators into supervision and MIS systems
Project staff have lobbied for the integration of micronutrient indicators into the supervision and MIS systems used by the region and districts. While there has been progress in terms of integration of nutrition promotion strategies into regional plans, indicators on specific micronutrient supplementation activities have not yet been integrated either into the supervision tools (developed by each region on their own). In the national MIS information is now collected on a monthly basis from the CSCOM/centres de santé on the total number of “VAC distributed” but not separately on distribution to post-partum women, well children and sick children.
District health supervision teams are very enthusiastic about the micronutrient supplementation activities, however, these activities are not yet systematically monitored during their supervision visits. The “micronutrient supplementation notebooks” provided by HKI for each health center are not systematically monitored and the data registered therein is not periodically tallied.
Lesson: In order to ensure the sustainability of micronutrient supplementation activities, training health staff and ensuring a system of supply and distribution of
27
micronutrients are necessary. It is also essential that programs work with national counterparts to integrate supervision of these activities into routine
supervision visits by incorporating specific micronutrient activities/parameters to be monitored into supervision tools and also into the national MIS system.
• Creation of district/zonal project steering committees
Four CS project steering committees were created (Comités de Suivi) to provide ongoing oversight of project implementation. These committees initially met but over time generally ceased to do so, in come cases due to the lack of a “financial incentive.” While the role assigned to these committees was important, to ensure local inputs and ownership, the creation of a new structure was not appropriate, especially given the existence of another multi-sectoral development committee which could have expanded it’s role to include oversight of the CS project.
Lesson: At the district, as well as other levels, it is always preferable for
externally-funded programs to work with existing organizational structures rather than to create new ones. This is advisable in order to demonstrate that these programs are integrated into district programs and to increase ownership and sustained involvement of local actors.
• Micronutrient supplement supply system
For many years both Vitamin A and iron-folate have been on the national list of essential drugs. Before 2000, however, often these micronutrient supplements were not available at the health facility level. Since health facility staff were trained on nutrition/micronutrients, there has been much greater awareness of their importance and now in the CSCOMS these products are generally available. However, in the unaffiliated rural maternity clinics/ dispensaries
(maternités rurales) there are often shortages, particularly of iron-folate tablets. Project staff should have done more to help the staff in these structures to
establish simple systems with revolving funds to ensure continuous availability of these products at the facility level.
While national level MOH officials favor distributing Vitamin A capsules for free, in Koulikoro region the medical officers and regional administrator (Haut
Commissariat) strongly oppose this practice in routine health facility activities. They argue that: a) it is better to make people pay something for health products/ services rather than to provide them for free; and b) free distribution of VAC may not be sustainable should donor support cease and, therefore, should not be carried out.
Lesson: Programs that promote micronutrient supplementation need to give as much attention to ensuring that there is a sustainable supply/distribution system in place, as to promoting prescription of these supplements.
28
v. Strengthening community health centers (CSCOM & maternities)
• Improved quality of services for pregnant and post-partum women
Project training and supervision activities have contributed to improving services provided to pregnant and post-partum women related primarily to nutrition
(advice on diet, iron-folate and vitamin A supplementation) and to complications during and after pregnancy. Training workshops have been organized both on nutrition/micronutrient and neo-natal health topics with various categories of health and social development workers: 168 matrones; 101 nurses and social development agents; 101 nurses aides; 22 doctors; and 491 relais.
Health workers in both CSCOMs and maternités rurales report that they are systematically prescribing iron to pregnant and post-partum women and administering Vitamin A to them after delivery. However, there has not been periodic monitoring of the system for recording information on supplementation activities so it is impossible to know exactly how supplementation activities are being carried out.
Lesson: Attendance at a training workshop is not sufficient to ensure that health workers will adopt new procedures and practices on a regular basis and that they will follow prescribed protocols. After training, health facility staff should be asked to keep simple records on new activities and periodic supervision visits should be carried out to verify and reinforce those activities.
• Pre-natal services to promote compliance with iron-folate supplementation
While health facility staff responsible for conducting pre-natal visits are
enthusiastic about increased iron-folate supplementation of pregnant women, two constraints are identified with this activity. First, mid-wives (matrones) do not always give sufficient advice on secondary effects associated with iron supplementation nor on how to avoid or diminish them. This undoubtedly
contributes to lower compliance rates, though precise data on this is not available as it was not collected in the project, as planned. Second, often when iron tablets are prescribed to women in rural maternity clinics, where the drug cost recovery system is not yet established, women receive a prescription but usually have to travel a considerable distance to purchase the tablets at a pharmacy. This constraint undoubtedly also contributes to decreased compliance. During the CS project no action was taken to facilitate access to iron tablets in the maternity clinics.
Lesson: Strengthening supplementation activities in health facilities requires not only training health workers but also ensuring that there is a sustainable system for women to easily access prescribed supplements. In the SNL project areas this constraint needs to be addressed as soon as possible if compliance rates are to be significantly increased. In addition, the midwives need to be
29
encouraged to focus more on counseling women about side effects both before and after they occur in order to decrease the number of women who stop taking iron supplements when problems arise.
• Improved health center practices with newborns
Project activities, namely training and supervision, have contributed to improving midwives’ practices related to early initiation of breastfeeding and to a lesser extent to promoting exclusive breastfeeding. While it is difficult to verify, all of the midwives state that they now put newborns to the breast immediately after birth. In the SNL target areas many midwives have also adopted the recommended practice of delaying the first bath of the newborn until six hours after birth. The project overlooked the need to organize training or other informational activities on the new practices being promoted in the rural health facilities for district level midwives/supervisors. This oversight has contributed to a gap between the knowledge of these new practices on the part of rural midwives, and on the part of district staff who supervise them.
Lesson: When new practices are to be introduced/encouraged at the health facility level, it is essential that there be a parallel strategy to inform/educate
those responsible for supervising such facilities to avoid a conflict between them. This may or may not need to involve formal training of the supervisors.
vi. Strengthening community health associations (ASACO)
• Increasing the organizational capacity of community health associations
According to the DIP, several types of activities were to be carried out with the ASACOs in order to strengthen their effectiveness, to include: a diagnosis of their organizational capacity; training of ASACO members in planning, management, N/CS, advocacy and health promotion. Relative to DIP plans, project activities with the ASACO were very limited. In one district an assessment of their
organizational capacity was carried out but following the assessment no training was done to address organizational weaknesses. The project did little to
increase the organizational capacity of the ASACOs, much needed in order to increase their effectiveness and prospects of sustainability.
Lesson: Programs that aim to ensure the viability of community participation in health facility management need to address the organizational weaknesses of community committees/associations such as the ASACOs by developing strategies to strengthen their skills in basic planning, management and evaluation of community activities.
30
vii. Training
The overal training stategy has been a TOT approach. All planned trainings have been done. KAP surveys show an increase in nutrition knowledge of health
personnel, community volunteers and theatre group members. This has resulted in a change in certain behaviours like increase in consumption of vitamin A rich foods, increased intake of food of sick children. This continues to be done in the Saving Newborn Lives project where matrons are being trained on Birth
preparation techniques. Although at the close of the project, training might not be continued, it is probable that once staff have adopted and internalized new knowledge, they will continue to put it into practice. Yet, supervision will reinforce this.
viii. Sustainability strategy
A partnership with Koulikoro region has been established. Through the SNL project, activities continue to be implemented; new approaches are tested and frequent liaising with the authorities is taking place. Moreover, the HKI
supervisors have moved back to the field. HKI will continue to provide technical assistance on nutrition which is their national mandate. The fact that the SNL funds have been ensured on itself is a proof of financial sustainability. Radios in the area continue to aire the nutrition messages and the theatre groups continue to perform and are even used by other partners.
C. Program Management
1. Planning and follow-up of program activities
• Planning based on DIP framework
A very beneficial preliminary activity, prior to DIP development, was a series of consultative meetings held with regional and district health-social sector
development staff to elicit their input for DIP development and to increase their sense of ownership of project activities. Subsequently, however, these important partners were not provided with a copy of the DIP in French during the first half of the project. When they did receive the document in French, the translation was poor and this made it difficult for them to understand and follow the proposed workplan.
The DIP should provide guidance to key project staff to orient all project activities and serve as a framework for monitoring and documenting those activities. All HKI staff had access to the DIP and the coordinator used it to some extent,
however, they did not have in depth knowledge of the DIP indicators nor use the DIP document on an ongoing basis to plan and monitor project activities.
31
Planning of project activities was the responsibility of the project coordinator and all plans were discussed in detail with the HKI CS team during regular staff meetings. However, a system of detailed written workplans including specific tasks to be carried out, dates and persons responsible for each task were not developed until the last year of the project.
While a majority of the activities defined in the DIP workplan were carried out several important activities were not, namely: 1) training of the CSP Coordinator and subsequent training of district health teams in IMCI; 2) capacity
building/training of ASACOs (in nutrition, planning, supervision and management) based on the organizational assessment of these organizations; 3) organization of social mobilization campaigns for each JRM et SIAN; 4) development/
implementation of ongoing nutrition advocacy strategy; 5) peer visits/exchange between staff in different districts and NGOs.
• Coordination with Regional and district MOH & MSD partners
Several mechanisms were put in place to ensure ongoing coordination of project activities with MOH & MSD partners.
a. Regional steering committee (Comite de Pilotage)
According to the DIP the regional steering committee was to meet twice a year and to oversee all project activities. The committee was not able to meet twice a year as planned. The final evaluation team concluded that the steering
committee played a beneficial role in guiding project implementation, however, it is difficult to precisely assess its impact on project activities.
b. District Technical Committees (DTC) (Comité Technique de District)
The DIP made provision for the creation of three DTC (one in each district) to provide technical oversight to the project and ensure its coordination with other district programs and staff activities. The quarterly meetings of the DTC did not take place in any of the districts and the motivation of district staff to participate in these forums was generally low. The evaluation team concluded that it would have been preferable to work through one of the existing district committees rather than to have created this new, project-specific structure.
c. Coordination with district health and social development teams
In the first phase of the project there was very close collaboration between HKI project staff and the district health-social development teams. The project
coordinator had an office in Koulikoro and this allowed him to regularly participate in regional and district staff meetings and to be in close contact with staff at that level. In the second half of the project, due to financial constraints, he was relocated to Bamako and contact between him and other CS team staff with
32
district MOH & MSD teams was much less frequent. In the last l8 months of the project, also due to financial constraints, there were fewer project activities and overall, however, complementary SNL resources did allow the project coordinators to make periodic visits to the districts.
2. Staff training and development
CS team members participated in a number of training activities that have increased their skills in the areas of: public health; micronutrients/nutrition; breastfeeding; complementary feeding; qualitative research; participatory evaluation; quality assurance; malaria prevention; and HIV/AIDS prevention.
In the mid-term evaluation an additional skill development need of the CS team members was identified related to group facilitation and adult education. A local resource person was identified to carry out such training but, unfortunately there were not sufficient resources in the budget to organize this training.
3. Supervision of Program Staff
There have been frequent meetings with CS team members, especially during the last l8 months of the project when all were based in Bamako. While such meetings have been frequent there are no minutes/records of issues addressed or decisions made during these meetings.
In the first three years of the project HKI did not have a system for periodic staff assessment. Such a system was established in 2002 and has been used by the CS coordinator with project staff. Two gaps identified in the assessment process are: 1) the deputy project coordinator shares responsibility with the deputy
project coordinator for supervising field supervisors but she does not participate in annual performance assessments of these staff members; 2) the process does not include a mechanism for project staff to assess the coordinator’s work. Certain inadequacies in the performance of certain team members have
apparently not been addressed through this mechanism as these inadequacies persist.
Project activities are carried out in 4 zones though in 2002 it was necessary to reduce zonal staff from four to three. The constraint arising from this situation is that the deputy project coordinator has been required to take direct responsibility for implementing project activities in the fourth zone, thus limiting the time
available to her to supervise field staff in the three other zones. In addition, there is not a clearly-defined system of supervision of field staff.
33
4. Human resources and staff management
1. CS team composition
The CS team is composed of a project director, project deputy director and three zonal supervisors/program officers. All of these staff members have been with the project since the beginning except for the deputy project director who joined the team during the second half of project implementation. In addition, an
administrative assistant was terminated in July 2001 due to financial irregularities and a fourth zonal supervisor was terminated in 2002 due to financial constraints. The considerable continuity in project staff has been very beneficial to project implementation.
2. Team morale and cohesion
There is frequent contact between the team members and morale generally appears to be good. The project coordinators consult frequently with the field supervisors and incorporate their ideas/suggestions into activity planning.
3. Transition to other jobs
With the complementary neo-natal health project continuing until June 2004 no plans have yet been made to assist project staff to obtain other comparable jobs after mid-2004.
5. Financial management
During most of the four year project, financial management of the project was hampered by weaknesses in the administration/ finance support staff in the HKI/Mali country office in Bamako. This was later exacerbated by the lack of records and financial reporting from headquarters due to the loss of all records following the events of September 11, 2001. During the first two years of the
project it was virtually impossible for project staff to get precise information on the status of the various budget line items that they were expected to work with. This made it extremely difficult for them to effectively plan and manage their activities. In mid-2001 the CS project administration-finance officer was discharged from his post and in 2002 the HKI chief of administration-finance was also discharged. Due to overspending in the first phase of the project, in the last l8 months
resources were very limited for critical field level follow-up activities. During this same period the project did benefit from resources from the complementary neo-natal health project, which enabled CS project staff to continue ongoing nutrition-related activities at the health facility and community levels. During the past six months there have been some improvements both in HKI’s financial systems and in efforts to strengthen the financial management skills of the two CS project coordinators. They have both increased their skills in the development of workplans based on available budgetary resources.
34
• Matching funds for project implementation
In addition to USAID and HKI resources several additional sources of funding were obtained to support project implementation. Funds from the Micronutrient Initiative (MI) were used to support health worker and radio broadcaster training activities. A grant from Sight and Life supported the endline KPC survey and final evaluation. A small private foundation provided resources for pilot nutrition-friendly CSCOMs in Koulikoro and Ouéléssébougou and resources from Save the Children, obtained from the Gates Foundation, supported the complementary neo-natal health activities in 48 villages.
• Resources for ongoing activities
Activities to be sustained through June 2004 are the neonatal health activities in 48 villages for which supplemental resources are available from the
complementary funding from SAVE the Children/USA. The majority of the indicators for the neonatal health project are the same as those for the CS-15 project so these additional resources will enable CS team staff to consolidate accomplishments both at the health facility and community levels.
6. Logistics
CS15 project funds did not allow for the purchase of one 4WD vehicle and 4 motorcycles. These were purchased through matching funds and have proven to be essential for the implementation of the project. The HKI supervisors have used the motorcycle, taking their government counterpart on the back of the motorcycle to jointly visit the villages that are difficult to access otherwise.
Therefore we can state that without these the project would not have been able to reach two thirds of its objectives. The 4WD has been used to facilitate regional and district supervisory visits by the project coordinator and deputy coordinator.
7. Information Management
The DIP describes a number of formative monitoring and evaluation (M&E) activities intended to “collect data in order to improve the project.” The DIP reviewers expressed concern regarding how proposed information collection would be used to inform project implementation. Overall data collection and use has been a weak aspect of the project. The several M&E activities presented in the DIP have been developed to a limited extent. In the SNL strategy, however, a more systematic approach to ongoing qualitative M&E has been developed. a. Formative research and other studies:
At the beginning of the project both formative qualitative data and quantitative KPC data were collected as a basis for developing project strategies. Other
35
studies were carried out on ASACO organizational capacity and on micronutrient compliance (through HKI’s funding from the Micronutrient Initiative). In the
complementary neonatal health project (SNL) a qualitative community study was also carried out on the roles and influence of different household members on maternal and child health. Unfortunately, the results of these different studies were not widely shared with project staff, MOH & MSD and NGO partners. In the SNL strategy study results were systematically used, however, the results of the other studies appear to have been used to only a limited extent. b. Continuous monitoring and evaluation throughout the project:
The DIP proposed that there be ongoing collection and analysis of various types of MOH & MSD and project-collected data to monitor project implementation, to identify successes and constraints and to adjust project strategies/activities. A series of indicators were defined in the DIP for monitoring activities at different levels, both quantitatively and qualitatively. While some data was collected there was no mechanism for periodically discussing/ synthesizing data collected from various sources and for developing lessons with project partners to strengthen program implementation. c. Mid-term evaluation
In keeping with HKI’s commitment to M&E processes that contribute to
organizational and team learning, in the 2002 mid-term evaluation, a participatory methodology was used to involve not only HKI staff but also MOH & MSD and NGO partners in a process of self-evaluation. The MOH & MSD staff were very receptive to the participatory methodology employed. Together they identified strengths and weaknesses in project implementation and they formulated a series of approximately 60 lessons learned to strengthen implementation of project strategies. During the final evaluation it was determined that during the second phase of the project, action was taken on most, but not all, of those lessons learned.
d. Strengthening information systems of the MOH & MSD
HKI has been lobbying at the national level for supplementation activities to be integrated into the national MIS and data is now regularly collected on the “distribution of VAC.” However, there were two limitations with the present
system. First, with the data collected on this general indicator it is not possible to determine trends in distribution of VAC specifically to women post partum, to well and sick children. Second, data recorded monthly in health facilities on VAC
distribution sometimes includes only capsules given out during JNV activities and excludes routine distribution at the facility level.
In collaboration with district MOH staff, a system of “notebooks” (cahiers) was established in each health facility to record the prescription/distribution of both
36
micronutrients and ORS. Unfortunately, however, the use of the notebooks has not been regularly followed up either by district or CSP staff and this information has not been analyzed for use in decision-making. e. Filing system for information retrieval:
An essential tool for organizing project documentation and a preliminary step to analysis and use of available information is an efficient filing system. While
numerous project documents are kept in electronic format, no system of files with hard copies of project documents was established to facilitate access and use of this information by project team members.
8. Technical and Administrative support
§ Administrative Support
A major consequence of the loss of the project administration/financial assistant in mid-2001 was that the project coordinator, and later the deputy coordinator, were required to invest a great deal of time and energy to deal with
administrative issues. It would have been preferable for them to have spent more time on issues requiring their technical expertise and less on the administrative aspects of project management. Future projects of this size should have their own administrative-finance officers to ensure appropriate management of all administrative issues.
• Technical assistance from HKI
Project staff were very appreciative of the TA they received, especially at the outset of the project, both from regional HKI staff and from HQ staff. Project staff do, however, identify weaknesses in the TA received for preparation of the qualitative, formative research and for development of the communication
strategy based on data from that research. Valuable TA was also provided on the design and implementation of the KPC survey and on DIP development. Additional technical support for M&E should have been provided.
• External technical assistance
External TA was provided for coordination of the participatory mid-term evaluation, and for the implementation of the final KPC survey.
9. Management lessons learned
Overall, HKI needs to implement a close supervisory/monitoring management approach to monitor all technical and financial aspects of project implementation. Below are some key issues listed:
37
• Regular project review meetings with program staff to identify weaknesses and problems early on and take action to address them.
• Project managers need to ensure that all project staff have in depth knowledge of: project goals, objectives and indicators, action plans/calendars.
• All field/project staff need to be regularly supervised/monitored and given specific feedback on their work on an ongoing basis. • Project managers need to have an excellent knowledge of the project budget at all times: they need to be involved in the approval
process of project expenses and need to be empowered to manage the budget.
• Project managers need to provide periodic information to partner organizations/staff on progress, challenges etc. in program implementation.
• The project should have a system for detailed planning and follow-up of project activities with activities presented in a visual form (table/chart) so that all project staff can monitor progress toward accomplishment of project activities.
• Project staff should participate in development of lessons
learned/recommendations in all evaluations and during technical assistance visits. Once such lessons learned/recommendations are agreed upon it is the responsibility of project managers to make every effort to follow-up/implement the recommendations/lessons learned. • An efficient filing system is an essential tool for organizing project
documentation and a preliminary step to analysis and use of available information: it must be put in place both in hard copy as electronically. • The DIP should be an active, living document that should almost be consulted on a daily basis.
• Staff weaknesses need to be identified and then strengthened. HKI should provide training in essential areas like M&E, group facilitation, adult education, writing skills and financial management.
• The organization should have a closer monitoring of the performance of it staff and effectively relate it to salary increase.
• Financial audits must be effectively done on a yearly basis and the HQ/RO finance department must follow it up.
D. Conclusions and Recommendations
1. Success in meeting program objectives
Based on the baseline and final assessments, the results of the four-year project were rather limited (were not as ood as expected?). One third of the 21 project objectives (33%) were met. Slightly more than one third (38%) were not met and, unfortunately, for almost one third of the DIP indicators (29%) no data was collected at endline making it impossible to assess the level of accomplishment. The objectives met deal with increases in: Vitamin A supplementation of post-
38
partum women; consumption of iodized salt; early initiation of breastfeeding; timely introduction of complementary foods; consumption of vitamin A rich foods; and appropriate feeding of children with diarrhea under 1 year of age.
A major constraint to accomplishment of the DIP objectives relates to the fact that the NGO activities, related to behavior change and provision of basic
services at the community-level, were developed on a much smaller scale than anticipated in the DIP, due both to unrealistic expectations of NGO capacity and to inadequate resources budgeted for NGO activities.
2. Main achievements
Increased commitment of MOH & MSD (Ministry of Health & Ministry of Social Development and Solidarity with Older Persons) at the national and regional levels to maternal and child nutrition: As a result of the various nutrition-focused activities supported by the CS project and to the advocacy role played by HKI staff at both the national and regional levels, there is increased recognition of the importance of nutrition in health and increased commitment to developing nutritional strategies at all levels. This commitment is reflected in national nutrition policies and tools to which HKI has contributed, namely, to the development of the national nutrition plan and to development of first MOH training modules on maternal and child nutrition.
Increased inclusion of nutrition-related activities in district operational plans (PO): As a positive consequence of HKI support to the districts, since 2000 several nutrition-related activities, that were not previously included in the POs, have routinely been incorporated into district operational plans (PO), namely: periodic mass distribution of micronutrients; systematic iron
supplementation of pregnant/breastfeeding women; and distribution of VAC to pregnant women, well and sick children.
Nutrition activities strengthened at the health facility level: MOH & MSD: As a result of training and follow-up of health facility staff, both by MOH & MSD and HKI staff, more attention is now being given to promoting optimal nutrition
practices in the context of both preventive and curative services for women and children. The main nutrition-related activities that have been improved in health facilities include: increased micronutrient supplementation (iron-folate and Vitamin A) of pregnant/post partum women and sick children; early initiation of breastfeeding and promotion of exclusive breastfeeding; and promotion of timely and appropriate complementary feeding. These activities all appear to be sustainable in the post-project period without additional outside resources.
Innovative grandmother-focused strategy to promote maternal and child nutrition/health: There has been strong support from community leaders and health sector staff for innovative grandmother-focused neonatal health/nutrition strategy (with supplemental Save the Children/USA funding) promoting proper
39
health/nutrition of pregnant and post-partum women and newborns. Initial observations show evidence of openness on the part of grandmothers to improved health/nutrition practices and changes in some of the advice they
provide to younger women. (This strategy continues through June 2004 and will be evaluated then.)
3. Major constraints:
Weak administrative support, financial and program management: The major constraint to effective project implementation was weak administrative and financial support to the project from the HKI national office coupled with weak management of human and other resources by project staff. Some of the
manifestations of these weaknesses were: financial resources available for field activities were greatly reduced during the last l8 months of the project; absence of a system for detailed planning and follow-up of project activities; weak periodic monitoring and evaluation of project-supported activities and use of M&E data to improve project implementation.
Limited use of the Detailed Implementation Plan (DIP) as a guide for project implementation: The DIP was intended to serve as a “roadmap” for
implementation of the project, based on the project objectives and key strategies and activities detailed therein. Project coordinators did not use the DIP as a guide to project implementation and this contributed to a number of oversights in project implementation, monitoring and evaluation.
Limited follow-up of supplementation activities in health facilities: While during the first phase of the project periodic follow-up of health facility
supplementation activities was quite regular, during the last l8 months, due to budgetary restrictions, follow-up visits were infrequent. While micronutrient
supplementation activities were recorded by health workers this data was never analyzed, making it impossible to assess the quality of these services. In addition, no concrete progress was made toward integrating micronutrient indicators into guidelines for routine supervision of health centers by district health teams.
Limited development and follow-up of community level behavior change activities by local NGO partners: As discussed above in Section c.ii., NGO community level nutrition/CS activities were developed much less than expected, due to a number of constraints. These limitations in the NGO component undoubtedly contributed to the relatively limited accomplishments on the community level behavior change DIP objectives.
40
4. Main lessons learned
DIP objectives and plan: In all CS programs DIP objectives and strategies should be developed with key partners, followed by all implementers and
progress toward accomplishment of those objectives and strategies should be documented on an ongoing basis.
Monitoring and evaluation for program learning: In addition to ongoing quantitative data collection, it is important that CS programs develop
mechanisms to qualitatively analyze the implementation of program strategies, to identify strengths, weaknesses and lessons to improve these strategies and to periodically share these lessons with all program implementers and partners. To support development of such activities CS teams should be introduced to concepts of organizational learning and to tools for promoting it.
Collaboration with district health teams: In order to strengthen local capacity to implement state-of-the-art nutrition/child survival strategies it is important that CS team members maintain close communication and contact, both formal and informal, with district health/ development team members. In this regard it would be most beneficial for CS teams to receive basic training in organizational development to provide them with basic concepts and tools for promoting organizational change.
Strengthening skills of health/development agents: It should never be
assumed that formal “training events” with health/social development agents are sufficient to bring about skill development. In addition to supervision visits other mechanisms should be identified to reinforce acquisition of new knowledge and skills to include sharing technical updates in routine coordination meetings and periodic dissemination of short “technical notes.”
Collaboration with local NGOs: All CS programs should collaborate with other NGOs working in the same area and should specifically try to strengthen the capacity of local NGOs involved in health/nutrition. However, in addition to organizing “training workshops” for local NGO staff, efforts to strengthen their capacity should be built on an initial assessment of their skills and resources and should include provision of ongoing technical and organizational support to them. Most local NGOs in Mali are quite weak. Future projects should be realistic about what is required to increase their effectiveness and should anticipate the financial and technical resources necessary to carry out these initiatives.
Use of community health volunteers: Most CS programs involve the use of community health volunteers (CHV) of one sort or another. However, CHV strategies seem to repeatedly confront the same constraints related to: the limited support they receive from their own communities; their progressively diminishing motivation to carry out expected activities; and the need for regular supervision/support to motivate them and to increase the quality of their work. In
41
future CS programs an essential first step should be to review lessons learned from past CHV programs in order to anticipate and hopefully avoid these age-old problems.
Collaboration with local radio stations: In countries such as Mali, where local radio stations proliferate, it is most advantageous for health/development programs to exploit this means of communication. However, sustained
broadcasting of information on priority health/development topics requires that broadcasters receive some basic technical training on those topics and that there be ongoing contact, monitoring and feedback to them. In terms of sustainability the need to pay for most broadcasts often creates a dependency on externally-funded projects. In Mali all radio stations are required to broadcast some
health/development material for free and where this exists this resource should be exploited by strengthening the links between district health/development teams and local radio stations.
Collaboration with traditional communicators: Where groups of traditional communicators exist it is beneficial for health/development projects to collaborate with them so that their skills can be used to communicate priority topics/
information. However, the challenge is to develop a strategy which is not project-dependent, i.e. which depends exclusively on project resources. When
developing strategies to work with traditional communicators, this issue should be carefully analyzed in order to design an approach which will be more likely to contribute to sustained promotion of health/development topics by these groups.
Simple and sustainable communication tools: Songs and stories are simple, culturally-relevant and effective tools for communicating key nutrition/health information to communities. An additional advantage of these tools is that
community members can use them themselves over time. Considerable effort is required to develop effective songs and stories but once developed they can be widely used with different community groups including both adults and children.
Strengthening the role of grandmothers as health promoters: Given the primordial role that grandmothers (muso koroba) play in household decision-making related to health/nutrition, it is advantageous to develop strategies that involve communication not only with women of reproductive age but also with senior women, or grandmothers, who greatly influence the practices of these younger women. Community level activities with the grandmothers related to neonatal health have shown that they are very interested in “new ideas” about nutrition/health when the approach used to communicate with them is based on respect and dialogue rather than on one-way communication of “key messages.” Both by MOH & MSD staff and community members state that the grandmother-focused strategy is a culturally-relevant way to promote changes in the practices of younger women.
42
5. Recommendations for USAID/GH/CSHGP
Group facilitation skills to increase CS team effectiveness: The HKI CS project, like most other CS projects, are staffed primarily by individuals with technical backgrounds in health. It is often incorrectly assumed that these individuals have strong group facilitation skills. In order to increase the
effectiveness of CS teams in their work with groups at both the organizational and community levels it would be most beneficial for USAID/GH/CSHGP to encourage the PVOs to provide their CS teams with training in group dynamics, group facilitation and team building. The external consultant has worked with numerous CS projects/teams and in almost all cases has observed weaknesses in this area.
Guidelines for mid-term and final evaluations: The external consultant has coordinated many CS evaluations and has found that the guidelines for doing so have improved over the years. However, in order to further strengthen the
guidelines it is suggested to reorganize the section on “cross-cutting approaches” (which might rather be called strategies/approaches) so that this information is reported by levels (national, regional, district management teams, health
facilities, local/community level, village level and individual level). This would be in keeping with the current thinking on ecological/multi-level assessment and intervention strategies. Most CS programs work at most of these levels and this would be an easier way of organizing information on the various strategies/approaches used.
43
F. Results highlight
Involving Grandmothers in promoting improved maternal and child nutrition practices
One of the recommendations from the mid-term evaluation of the CS-15 project was to develop a strategy to involve grandmothers (GM) in nutrition/child survival activities given their great
influence on the practices of women with young children. A matching grant was obtained from Save the Children/USA to implement a neo-natal health promotion strategy using a GM-focused strategy.
The first step in developing the strategy was to conduct a qualitative community study using a new assessment methodology to analyze the roles of household actors in health/nutrition and decision-making related to health promotion and illness management. Results of the study
showed that, “At the family level, the GMs, or muso koroba, are the primary resource persons for all issues related to women during pregnancy and labour, during the postpartum period and to their infants. Other family members expect them to advise and supervise all activities of younger women related to themselves and their children.“ In addition, the GMs interviewed all stated that they had never before been invited to participate in health/nutrition education activities at the community level and that they would be most interested to do so.
Based on these findings a participatory communication/health education strategy was developed. A set of songs and stories-without-an-ending were developed on priority topics related to
CS/newborn health, for example, women’s work and nutrition during pregnancy, early initiation of breastfeeding and exclusive breastfeeding. In 48 villages, these tools are being used in health education sessions with groups of GMs to involve them in discussing both their “old” practices and recommended “new” practices to improve women and children’s health.
There has been a very positive reaction to the grandmother-focused strategy on the part of both health workers and community members. Many health workers/officials have stated that the effectiveness of many MCH programs is limited by the fact that the influential muso koroba, or GM, are not involved and not exposed to the same information and advice as younger women. At the community level, both younger women and their husbands have stated that involving the GMs greatly facilitates adoption of new health-related practices by younger women.
Community activities with GM groups have been implemented for 8 months and will continue until June 2004. Follow-up community visits reveal that GMs are very interested in the “new practices” presented in the non-formal education activities, that their knowledge of these practices has
increased and that some have started changing their advice to younger women. In June 2004 the GM strategy will be evaluated. Already HKI and MOH staff have stated that the GM-focused strategy is based on culturally-defined roles and patterns in all areas of Mali. It is believed that this approach is equally relevant to other countries where GMs play an influential role in advising and supervising other family members on issues of health and well-being.
44
ANNEX A: CS-15 EVALUATION PARTNERS AND MEMBERS
Team members:
1. Diakalia Koné, HKI project coordinator
2. Assétou Maguiraga, HKI deputy project coordinator 3. Daouda Samaké, HKI field supervisor 4. Nialen Kaba, HKI field supervisor 5. Fama Kondo, HKI field supervisor
6. Aïda Dia, Social Development Chief for Kati District
7. Mamady Sogoré, Social Development Chief Ouélessébougou 8. Baba Diallo, Doctor at the Health and Social Service of Koulikoro 9. Soumaïla Tigana, Social Development Service of Koulikoro 10. Mady Makanéra, Social Development Service of Kolokani
11. Tata Konaté, midwife of the Social Development Service of Kati
12. Namaké Bouaré, Senior Technical Health officer Social Development Service of Kati
Additional member that has participated in the evaluation planning workshop:
13. Bakary Berthé, Senior Technical Health officer Social Development Service of Ouélessébougou
Additional members that have participated in the presentation of results meeting:
14. Moriba Camara, Social Development Chief Koulikoro
15. Fatoumata Sidibé, Doctor at the Health and Social Service Kati
16. Bakary Konaté, Doctor at the Regional Health Directorate of Koulikoro 17. Lamoussa Traoré, Regional Directorate of Social Development and economy of Koulikoro
18. Yacouba Sidibé, Chief medical doctor of the Health and Social services of Kolokani
19. Check Abba Fomba, Social Development Chief Kolokani
45
ANNEX B: ASSESSMENT METHODS
1. Type of survey
A descriptive transversal survey (same as baseline survey)
2. Zone of the survey
The survey was done in the 3 health zones of the Koulikoro Region. The Koulikoro Region comprises of 146 health zones of which 64 are functional. In the 3 health Zones where the project was implemented, 30 out of 74 health zones are functional (40% coverage). In certain non functional health zones, a dispensary or maternity can be found that is implementing some health activities.
Table 1 shows the different survey zones and villages/number of women included in the survey.
Table 1 : Number of villages and number of women interviewed by zone Functional zones Non functional zones Health zones Number of Number of Number Number of villages women of women
interviewed villages interviewed
Kati (n= 100) 5 50 5 50 Kolokani (n= 70) 3 30 4 40 Koulikoro (n= 80) 6 60 2 20 Ouélessébougou=50) 3 30 2 20 Total 17 170 13 130
3. Sampling
As for the baseline survey, 30 villages have been randomly selected in the different intervention zones (functional and non functional health zones). The proportional-to-size probability sample method has been used to select the villages.
In each village, 10 mothers of under one year children and 10 mothers with children between 12 – 71 months have been selected. In total, 300 mothers with under one children and 300 mothers with children 12-71 months have been selected, resulting in a total of 600 women interviewed.
46
4. Information collection tools
Two standard questionnaires, which were elaborated for the baseline survey and have not been amended since, have been used: one for mothers of children under 12 months and one for mothers of children 12-71 months.
Two teams of interviewers (4 interviewers and one supervisor per team) have been trained for 2 days and observed during a field test.
5. Information collection process
For the selection of the households, the team started in the centre of the village and went in a randomly selected direction to meet the first household. The
neighbouring households have been identified as next until all required women were interviewed.
In each household, all women with children less than 12 months or 12-71 months were listed. One woman was selected at random from the list and the questionnaire was administered.
6. Data analysis
After data cleaning, data analysis has been performed using the Epi-info software (version 6.4 fr).
47
ANNEX C. LIST OF PERSONS INTERVIEWED AND CONTACTED FOR THE EVALUATION Team 2: District Team 3 : Team 1 :
Medical Teams Consultations communities
Koulikoro - Chief medical Three matrons
doctor responsable for the - Chief of the Social maternity ward in Development Massala, Domba, Service and Mafeya village - Chief Midwife
Ouélessébougou In the Reference
Health Centre Village of Tinkélé: - 1 Chief Medical meeting with the Doctor Muso Koroba - the Chief Midwife (grandmothers), - 1 obstetrical nurse women of child bearing age, CSCom of village chiefs Sanankoroba: - 1 obstetrical
nurse, responsable for the maternity ward
- 1 Health technician
Kati - 2 doctors Villages of:
- 2 midwives - Donékébougou: - 1 Social meeting with the Development Muso Koroba technician (grandmothers),
women of child bearing age,
husbands, village chiefs
- Siby: meeting with the Muso Koroba
(grandmothers), women of child bearing age
In the Reference Four matrons Kolokani Health Centre responsable for the - 1 Chief Medical maternity ward in Doctor Nonsonbougou, 48
- 1 midwife Massantola, Koumi - 1 obstetrical nurse and Ouolodo.village - the Health
Information System manager d'information sanitaire
- 1 technicien hygiène
assainissement - 1 technicien du développement social
- 1 infirmier
CSCom of Kolokani - 1 medical doctor - 1 midwife
49
因篇幅问题不能全部显示，请点此查看更多更全内容