To correct malnutrition practices, the incoming Duterte administration will need not only to continue this program but also to complement it with other programs.
In 2013, the government’s think tank, the Philippine Institute for Development Studies, was provided extra resources by the Department of Budget and Management (DBM) to carry out impact evaluation studies that seek to measure changes in outcomes that can be attributed to particular projects, programs or policies.
Given my interest in poverty and education over the years, I worked with another statistician, Ana Maria Tabunda of the UP School of Statistics, and a nutritionist, Imelda Angeles-Agdeppa of the Food and Nutrition Research Institute, to carry out an impact evaluation of the government’s School-based Feeding Program (SBFP), implemented by the Department of Education (DepED).
In school year 2013-2014, the SBFP of DepED targeted feeding more than 40,000 “severely wasted” students for a period of 100-120 feeding days using a 20-day cycle of standardized recipes with malunggay. When a child’s weight is below three standard deviations from the median weight-for-height, the child is said to be “severely wasted,” while if the weight-for-height is lower than two standard deviations from the growth standard but higher than three standard deviations, then the child is moderately wasted.
According to DepED, the 2013 implementation of the program aimed to:
1. Rehabilitate at least 70% of severely wasted beneficiaries to normal nutritional status at end of
the feeding days;
2. Ensure 85% to 100% classroom attendance of beneficiaries;
3. Improve children's health and nutrition values and behavior.
The Health and Nutrition Center of DepED suggested that the SBFP’s primary goal is the nutrition goal.
The impact of SBFP for the nutrition goal is the difference in nutrition status when a primary student is a beneficiary of a school feeding program and when s/he is not a beneficiary. The problem here is that we have missing “counterfactuals.” When a student becomes a beneficiary of the program, we can no longer observe her/his outcome as a non-beneficiary. Conversely, when a student is not a beneficiary, we cannot observe her/his outcome as a beneficiary.
Best approach
The best approach for an impact evaluation is to conduct a randomized controlled study, which involves having a targeted set of beneficiaries, and randomly assigned them into the program.
In practice, this is challenging to implement, so other approaches are used, including matching beneficiaries with nonbeneficiaries meant to perform a counterfactual analysis.
Thus the PIDS study to measure the SBFP impact involved mixed methods research, undertaking quantitative surveys of 1,151 program beneficiary pupils and their parents, as well as matched non-program beneficiary children and their parents. These surveys were supplemented with interviews of school heads of sampled beneficiary schools as well as counterpart schools from which matched non-beneficiary children were drawn.
Leaders or members of SBFP core groups in sampled beneficiary schools were also interviewed as well as one teacher of each sampled beneficiary. Focus group discussions (FGDs) were conducted in five schools to probe into program processes. The PIDS released a discussion paper providing the nitty gritty results of this impact evaluation study, as well as the main highlights in a policy note.
Did DepEd carry it out?
Before we undertook the impact evaluation work last year, we also conducted aprocess evaluation of the SBFP in 2014 to investigate whether DepED actually carried out the program as designed.
By and large, the DepED SBFP was carried out rather well, with the program creating a culture of care among stakeholders and fostering camaraderie among parents and teachers involved in the program. Program stakeholders expressed the hope that the SBFP gets continued and better support from government.
We did notice some issues that need improvement:
• Absence of standard weighing protocols and instruments used among schools.
• Difficulties in procurement and liquidation processes, and in filling out SBFP forms.
• The need to re-examine program cost per student (P15 per meal and P1 for administration).
• For 2013-2014, each SBFP meal was also meant to provide at least 300 calories for beneficiaries, which is much lower than (876) calories per meal in similar feeding programs in other developing countries.
• Finally, SBFP gave lower feeding days compared to the 180 feeding-day-average in comparable feeding programs of developing countries.
Challenges in field work
The field work for the impact evaluation study posed many challenges: from difficulties in securing needed complete documentation from schools, to finding matched nonbeneficiaries.
The expanded coverage of the program in school year 2014-2015 (with DepED having been given resources to target all severely wasted students in this schoolyear) introduced a complication since about a third of the matched nonbeneficiary pupils, who were not beneficiaries in schoolyear 2013-2014, were beneficiaries in schoolyear 2014-2015. Further, a number of schools could not provide the study team with copies of nutrition status reports or SBFP forms, or provided incompletely-filled documents. Details of all the study limitations are provided in the PIDS discussion paper.
Impact on nutrition
The field work suggested that there were inconsistencies in verbal descriptors between the nutrition status recorded in SBFP forms and nutrition status reports, and the verified pre-feeding and postfeeding nutrition status computed from birthdates and weight and height measurements.
Working on data with verified pre-feeding and postfeeding nutrition status, we found that, of those children verified to be severely wasted prior to the feeding program, about 62% attained at least normal nutrition status at the end of the feeding program (see Table 1).
At least 62% of parents interviewed said that his/her child attained normal weight for height at the end of the SBFP; 19% said that their child did not, while the remaining 19% could not recall if their child or child did not attain normal weight or body mass index (BMI) at the end of the feeding program.
Table 1. Change in nutrition status of beneficiary children with verified pre- and post-feeding nutrition status, number of children (and row percentage).
Verified pre-feeding nutrition status Verified post-feeding nutrition status TOTAL
Severely Wasted Wasted Normal Overweight
Severely wasted 49
(17.1) 59
(20.6) 178
(62.0) 1
(0.4) 287
(100.0)
Wasted 5
(5.2) 24
(24.7) 68
(70.1) 0
(0.0) 97
(100.0)
Normal 3
(3.8) 5
(6.3) 71
(89.9) 0
(0.0) 79
(100.0)
Overweight 0
(0.0) 1
(100.0) 0
(0.0) 0
(0.0) 1
(100.0)
TOTAL 57
(12.3) 89
(19.2) 317
(68.3) 1
(0.2) 464
(100.0)
While the estimate 62% of reversion to normal status by severely wasted beneficiaries suggests that the program falls slightly short of the SBFP nutrition status target to have at least 70% of the beneficiaries attain normal nutrition status by end of the feeding, there were clearly various factors beyond the control of program implementers. These include characteristics and practices of beneficiary families or parents/guardians and the children themselves (age and severity of wasting at start of feeding program, in particular), that affected the nutrition outcome.
An overall comparison of the nutrition status of sampled beneficiary and nonbeneficiary pupils during the survey showed that more SBFP-fed severely wasted pupils attained and maintained normal nutrition status or better compared to nonbeneficiary counterparts (48% vs 41%).
Thus, we found evidence that SBFP does have an impact on the nutrition status of severely wasted students. A similar counterfactual analysis is found for wasted pupils, with the percentage improvement among the SBFP-fed wasted pupils exceeding that for their nonbeneficiary counterparts by nearly 8 percentage points (53.9% vs 46.1%).
Table 2a. Change in nutrition status of beneficiary children with consistently measured heights
during pre-feeding and survey periods, number of children (and row percentage)
Verified pre-feeding nutrition status Verified post-feeding nutrition status TOTAL
Severely Wasted Wasted Normal
Severely wasted 44
(24.6) 49
(27.4) 86
(48.0) 179
(100.0)
Wasted 8
(12.3) 22
(33.9) 35
(53.9) 65
(100.0)
Normal 8
(11.3) 20
(28.2) 43
(60.6) 71
(100.0)
TOTAL 60
(19.1) 91
(28.9) 164
(52.1) 315
(100.0)
Table 2b. Change in nutrition status of non-beneficiary children with consistently measured heights during initial measurement and and survey periods, number of children (and row percentage)
Verified pre-feeding nutrition status Verified post-feeding nutrition status TOTAL
Severely Wasted Wasted Normal Overweight Obese
Severely wasted 93
(30.0) 128
(27.7) 2
(41.3) 1
(0.65) 1
(0.32) 310
(100.0)
Wasted 27
(23.5) 53
(30.4) 0
(46.1) 0
(0.0) 0
(0.0) 115
(100.0)
Normal 12
(13.5) 62
(16.9) 0
(69.7) 0
(0.0) 0
(0.0) 71
(100.0)
TOTAL 132
(25.7) 243
(26.5) 2
(47.3) 1
(0.4) 1
(0.2) 514
(100.0)
School attendance
Among 200 pupil beneficiaries verified to be severely wasted prior to feeding and who had school attendance data, about 3%, attended school for less than 85% of total school days.
The median percentage attendance for the severely wasted children is 97.5%, which is comparable to school attendance of NB pupils, with 95% of these pupils attending 85% of total school days. Teachers interviewed pointed out that most of the SBFP children enrolled in their classes improved in attentiveness during the feeding program (96%) as well as after (95%).
The children also reportedly became more sociable during the feeding (97%), a development that was sustained after the feeding (96%). Improvement in class attendance was also reported by teachers for 94% of the beneficiary pupils; 92% of the children sustained good attendance.
Sustaining nutrition gains
Nutritional gains of the SBFP do not appear to be sustained in the case of many severely wasted beneficiaries a year or more after the feeding.
In particular, of 179 severely wasted beneficiaries whose nutrition status had improved to normal at end of feeding and who had consistent height measurements for pre-feeding and survey periods, about half (48%) remained normal by the time of the survey. Some regressed to wasted or severely wasted a year or more later.
This suggests the need to continue feeding most of severely wasted beneficiaries beyond the 100-120 day-feeding cycle, while simultaneously introducing government interventions (not necessarily DepED-administered) other than feeding programs to address capacity of disadvantaged families to provide for nutritional needs of their members.
Field interviews and FGDs suggest that the school year 13-14 SBFP program was generally implemented well, with majority of school heads, teachers and parents expressing appreciation for SBFP and with sizeable percentages of heads and teachers expressing a desire to see SBFP continued and expanded.
What children learned
Nearly all beneficiary schools implemented complementary DepED programs, Gulayan sa Paaralan Project (GPP) and Expanded Health Care Program (EHCP). Some parents mentioned that when the school their child was enrolled in sometimes lacked food, the feeding implementers added vegetables from the school garden to make up for the lack.
Children were taught the importance of good grooming, of washing hands before and after meals, of brushing their teeth and of good nutrition. Nearly all children say that they continue to wash their hands before and after eating both at school and at home. But only 69% of field interviewers found the children to be well-groomed at the time of the interview.
The SBFP was found to have positive unintended consequences: helping improve attentiveness and sociability of beneficiary pupils.
Teachers reported most beneficiaries enrolled in their classes improved in attentiveness during feeding (96%) as well as after (95%). The children also became more sociable during the feeding (97%), a development that was sustained after the feeding (96%). Improvement in class attendance was also reported by teachers for 94% of beneficiaries; 92% of the children sustained good attendance.
Ways forward
We reiterated our suggestion in the process evaluation for DepED to provide all schools with recommended weighing and height–measurement scales, rather than leaving procurement of such to the resourcefulness of school heads.
School heads, school nurses and class advisers, if not all teachers, should be trained on the proper use of such scales and on the importance of proper documentation of pre-feeding, feeding, and post-feeding phases of the program to help in the proper selection of beneficiary schools and beneficiary pupils, and in monitoring and evaluating program outcomes.
The basis for the 70% target in the SBFP nutrition goal has to be reviewed since 10% of normal and 30% of wasted children can regress to wasted or severely wasted status, possibly due to severe illness or growth spurt. The SBFP nutrition target has since been increased to 80% in the schoolyear 2015-2016, possibly because wasted children were included in the coverage of the program. But available data indicates that only about 70% of wasted beneficiaries attain normal nutrition status at the end of the feeding program.
Given that the administration component of the budget has been increased after school year 2013-2014, it is also important to increase the food budget allocation and consider inflation-adjusted increases.
While malnutrition is largely an economic issue, poor nutrition persists across socio-economic classes, but there is evidence from FNRI data that poor nutrition is connected with poverty.
Further, FNRI data suggests that malnutrition starts among children under five who carry it over to early childhood, leading to repercussions on learning achievements.
The next government will need not only to continue this program but also to complement it with other programs to correct malnutrition practices for children under 5, especially in the wake of the country’s commitment to the 2030 Agenda for Sustainable Development, and the Sustainable Development Goals to ensure no child gets left behind in both schooling and nutrition. – Rappler.com
Dr. Jose Ramon "Toots" Albert is a professional statistician who has written on poverty measurement, education statistics, agricultural statistics, climate change, macro prudential monitoring, survey design, data mining, and statistical analysis of missing data. He is a Senior Research Fellow of the government’s think tank Philippine Institute for Development Studies.
In 2013, the government’s think tank, the Philippine Institute for Development Studies, was provided extra resources by the Department of Budget and Management (DBM) to carry out impact evaluation studies that seek to measure changes in outcomes that can be attributed to particular projects, programs or policies.
Given my interest in poverty and education over the years, I worked with another statistician, Ana Maria Tabunda of the UP School of Statistics, and a nutritionist, Imelda Angeles-Agdeppa of the Food and Nutrition Research Institute, to carry out an impact evaluation of the government’s School-based Feeding Program (SBFP), implemented by the Department of Education (DepED).
In school year 2013-2014, the SBFP of DepED targeted feeding more than 40,000 “severely wasted” students for a period of 100-120 feeding days using a 20-day cycle of standardized recipes with malunggay. When a child’s weight is below three standard deviations from the median weight-for-height, the child is said to be “severely wasted,” while if the weight-for-height is lower than two standard deviations from the growth standard but higher than three standard deviations, then the child is moderately wasted.
According to DepED, the 2013 implementation of the program aimed to:
1. Rehabilitate at least 70% of severely wasted beneficiaries to normal nutritional status at end of
the feeding days;
2. Ensure 85% to 100% classroom attendance of beneficiaries;
3. Improve children's health and nutrition values and behavior.
The Health and Nutrition Center of DepED suggested that the SBFP’s primary goal is the nutrition goal.
The impact of SBFP for the nutrition goal is the difference in nutrition status when a primary student is a beneficiary of a school feeding program and when s/he is not a beneficiary. The problem here is that we have missing “counterfactuals.” When a student becomes a beneficiary of the program, we can no longer observe her/his outcome as a non-beneficiary. Conversely, when a student is not a beneficiary, we cannot observe her/his outcome as a beneficiary.
Best approach
The best approach for an impact evaluation is to conduct a randomized controlled study, which involves having a targeted set of beneficiaries, and randomly assigned them into the program.
In practice, this is challenging to implement, so other approaches are used, including matching beneficiaries with nonbeneficiaries meant to perform a counterfactual analysis.
Thus the PIDS study to measure the SBFP impact involved mixed methods research, undertaking quantitative surveys of 1,151 program beneficiary pupils and their parents, as well as matched non-program beneficiary children and their parents. These surveys were supplemented with interviews of school heads of sampled beneficiary schools as well as counterpart schools from which matched non-beneficiary children were drawn.
Leaders or members of SBFP core groups in sampled beneficiary schools were also interviewed as well as one teacher of each sampled beneficiary. Focus group discussions (FGDs) were conducted in five schools to probe into program processes. The PIDS released a discussion paper providing the nitty gritty results of this impact evaluation study, as well as the main highlights in a policy note.
Did DepEd carry it out?
Before we undertook the impact evaluation work last year, we also conducted aprocess evaluation of the SBFP in 2014 to investigate whether DepED actually carried out the program as designed.
By and large, the DepED SBFP was carried out rather well, with the program creating a culture of care among stakeholders and fostering camaraderie among parents and teachers involved in the program. Program stakeholders expressed the hope that the SBFP gets continued and better support from government.
We did notice some issues that need improvement:
• Absence of standard weighing protocols and instruments used among schools.
• Difficulties in procurement and liquidation processes, and in filling out SBFP forms.
• The need to re-examine program cost per student (P15 per meal and P1 for administration).
• For 2013-2014, each SBFP meal was also meant to provide at least 300 calories for beneficiaries, which is much lower than (876) calories per meal in similar feeding programs in other developing countries.
• Finally, SBFP gave lower feeding days compared to the 180 feeding-day-average in comparable feeding programs of developing countries.
Challenges in field work
The field work for the impact evaluation study posed many challenges: from difficulties in securing needed complete documentation from schools, to finding matched nonbeneficiaries.
The expanded coverage of the program in school year 2014-2015 (with DepED having been given resources to target all severely wasted students in this schoolyear) introduced a complication since about a third of the matched nonbeneficiary pupils, who were not beneficiaries in schoolyear 2013-2014, were beneficiaries in schoolyear 2014-2015. Further, a number of schools could not provide the study team with copies of nutrition status reports or SBFP forms, or provided incompletely-filled documents. Details of all the study limitations are provided in the PIDS discussion paper.
Impact on nutrition
The field work suggested that there were inconsistencies in verbal descriptors between the nutrition status recorded in SBFP forms and nutrition status reports, and the verified pre-feeding and postfeeding nutrition status computed from birthdates and weight and height measurements.
Working on data with verified pre-feeding and postfeeding nutrition status, we found that, of those children verified to be severely wasted prior to the feeding program, about 62% attained at least normal nutrition status at the end of the feeding program (see Table 1).
At least 62% of parents interviewed said that his/her child attained normal weight for height at the end of the SBFP; 19% said that their child did not, while the remaining 19% could not recall if their child or child did not attain normal weight or body mass index (BMI) at the end of the feeding program.
Table 1. Change in nutrition status of beneficiary children with verified pre- and post-feeding nutrition status, number of children (and row percentage).
Verified pre-feeding nutrition status Verified post-feeding nutrition status TOTAL
Severely Wasted Wasted Normal Overweight
Severely wasted 49
(17.1) 59
(20.6) 178
(62.0) 1
(0.4) 287
(100.0)
Wasted 5
(5.2) 24
(24.7) 68
(70.1) 0
(0.0) 97
(100.0)
Normal 3
(3.8) 5
(6.3) 71
(89.9) 0
(0.0) 79
(100.0)
Overweight 0
(0.0) 1
(100.0) 0
(0.0) 0
(0.0) 1
(100.0)
TOTAL 57
(12.3) 89
(19.2) 317
(68.3) 1
(0.2) 464
(100.0)
While the estimate 62% of reversion to normal status by severely wasted beneficiaries suggests that the program falls slightly short of the SBFP nutrition status target to have at least 70% of the beneficiaries attain normal nutrition status by end of the feeding, there were clearly various factors beyond the control of program implementers. These include characteristics and practices of beneficiary families or parents/guardians and the children themselves (age and severity of wasting at start of feeding program, in particular), that affected the nutrition outcome.
An overall comparison of the nutrition status of sampled beneficiary and nonbeneficiary pupils during the survey showed that more SBFP-fed severely wasted pupils attained and maintained normal nutrition status or better compared to nonbeneficiary counterparts (48% vs 41%).
Thus, we found evidence that SBFP does have an impact on the nutrition status of severely wasted students. A similar counterfactual analysis is found for wasted pupils, with the percentage improvement among the SBFP-fed wasted pupils exceeding that for their nonbeneficiary counterparts by nearly 8 percentage points (53.9% vs 46.1%).
Table 2a. Change in nutrition status of beneficiary children with consistently measured heights
during pre-feeding and survey periods, number of children (and row percentage)
Verified pre-feeding nutrition status Verified post-feeding nutrition status TOTAL
Severely Wasted Wasted Normal
Severely wasted 44
(24.6) 49
(27.4) 86
(48.0) 179
(100.0)
Wasted 8
(12.3) 22
(33.9) 35
(53.9) 65
(100.0)
Normal 8
(11.3) 20
(28.2) 43
(60.6) 71
(100.0)
TOTAL 60
(19.1) 91
(28.9) 164
(52.1) 315
(100.0)
Table 2b. Change in nutrition status of non-beneficiary children with consistently measured heights during initial measurement and and survey periods, number of children (and row percentage)
Verified pre-feeding nutrition status Verified post-feeding nutrition status TOTAL
Severely Wasted Wasted Normal Overweight Obese
Severely wasted 93
(30.0) 128
(27.7) 2
(41.3) 1
(0.65) 1
(0.32) 310
(100.0)
Wasted 27
(23.5) 53
(30.4) 0
(46.1) 0
(0.0) 0
(0.0) 115
(100.0)
Normal 12
(13.5) 62
(16.9) 0
(69.7) 0
(0.0) 0
(0.0) 71
(100.0)
TOTAL 132
(25.7) 243
(26.5) 2
(47.3) 1
(0.4) 1
(0.2) 514
(100.0)
School attendance
Among 200 pupil beneficiaries verified to be severely wasted prior to feeding and who had school attendance data, about 3%, attended school for less than 85% of total school days.
The median percentage attendance for the severely wasted children is 97.5%, which is comparable to school attendance of NB pupils, with 95% of these pupils attending 85% of total school days. Teachers interviewed pointed out that most of the SBFP children enrolled in their classes improved in attentiveness during the feeding program (96%) as well as after (95%).
The children also reportedly became more sociable during the feeding (97%), a development that was sustained after the feeding (96%). Improvement in class attendance was also reported by teachers for 94% of the beneficiary pupils; 92% of the children sustained good attendance.
Sustaining nutrition gains
Nutritional gains of the SBFP do not appear to be sustained in the case of many severely wasted beneficiaries a year or more after the feeding.
In particular, of 179 severely wasted beneficiaries whose nutrition status had improved to normal at end of feeding and who had consistent height measurements for pre-feeding and survey periods, about half (48%) remained normal by the time of the survey. Some regressed to wasted or severely wasted a year or more later.
This suggests the need to continue feeding most of severely wasted beneficiaries beyond the 100-120 day-feeding cycle, while simultaneously introducing government interventions (not necessarily DepED-administered) other than feeding programs to address capacity of disadvantaged families to provide for nutritional needs of their members.
Field interviews and FGDs suggest that the school year 13-14 SBFP program was generally implemented well, with majority of school heads, teachers and parents expressing appreciation for SBFP and with sizeable percentages of heads and teachers expressing a desire to see SBFP continued and expanded.
What children learned
Nearly all beneficiary schools implemented complementary DepED programs, Gulayan sa Paaralan Project (GPP) and Expanded Health Care Program (EHCP). Some parents mentioned that when the school their child was enrolled in sometimes lacked food, the feeding implementers added vegetables from the school garden to make up for the lack.
Children were taught the importance of good grooming, of washing hands before and after meals, of brushing their teeth and of good nutrition. Nearly all children say that they continue to wash their hands before and after eating both at school and at home. But only 69% of field interviewers found the children to be well-groomed at the time of the interview.
The SBFP was found to have positive unintended consequences: helping improve attentiveness and sociability of beneficiary pupils.
Teachers reported most beneficiaries enrolled in their classes improved in attentiveness during feeding (96%) as well as after (95%). The children also became more sociable during the feeding (97%), a development that was sustained after the feeding (96%). Improvement in class attendance was also reported by teachers for 94% of beneficiaries; 92% of the children sustained good attendance.
Ways forward
We reiterated our suggestion in the process evaluation for DepED to provide all schools with recommended weighing and height–measurement scales, rather than leaving procurement of such to the resourcefulness of school heads.
School heads, school nurses and class advisers, if not all teachers, should be trained on the proper use of such scales and on the importance of proper documentation of pre-feeding, feeding, and post-feeding phases of the program to help in the proper selection of beneficiary schools and beneficiary pupils, and in monitoring and evaluating program outcomes.
The basis for the 70% target in the SBFP nutrition goal has to be reviewed since 10% of normal and 30% of wasted children can regress to wasted or severely wasted status, possibly due to severe illness or growth spurt. The SBFP nutrition target has since been increased to 80% in the schoolyear 2015-2016, possibly because wasted children were included in the coverage of the program. But available data indicates that only about 70% of wasted beneficiaries attain normal nutrition status at the end of the feeding program.
Given that the administration component of the budget has been increased after school year 2013-2014, it is also important to increase the food budget allocation and consider inflation-adjusted increases.
While malnutrition is largely an economic issue, poor nutrition persists across socio-economic classes, but there is evidence from FNRI data that poor nutrition is connected with poverty.
Further, FNRI data suggests that malnutrition starts among children under five who carry it over to early childhood, leading to repercussions on learning achievements.
The next government will need not only to continue this program but also to complement it with other programs to correct malnutrition practices for children under 5, especially in the wake of the country’s commitment to the 2030 Agenda for Sustainable Development, and the Sustainable Development Goals to ensure no child gets left behind in both schooling and nutrition. – Rappler.com
Dr. Jose Ramon "Toots" Albert is a professional statistician who has written on poverty measurement, education statistics, agricultural statistics, climate change, macro prudential monitoring, survey design, data mining, and statistical analysis of missing data. He is a Senior Research Fellow of the government’s think tank Philippine Institute for Development Studies.
