Did you hear about the Head Start evaluation?
Head Start is the iconic federal preschool program intended to enhance the health and school readiness of America’s poor children. Begun as a pilot summer program in 1965 as part of Lyndon Johnson’s War on Poverty, it is now budgeted at $7.235 billion annually for programs serving approximately 1 million children.
The children in Head Start are overwhelmingly poor and minority. They are at high risk of starting school far behind their more advantaged peers, and falling further behind over time. They tune-out and drop-out at alarming rates. In a world in which nearly everything we value, from a long lifespan to financial wealth to family stability, is associated with educational attainment, these children’s lives are in danger. They desperately need a good education. Further, the nation needs for them to succeed in school if it is to achieve its goal of social equity and if it is to complete internationally in a knowledge-based economy. Preschool is where it has to start.
There have long been questions about the effectiveness of Head Start. Supporters of the program traditionally point to decades-old research demonstrating long-term positive outcomes for children from participation in two intensive, small-scale early childhood programs. These programs, Perry Preschool and Abecedarian, suggest the potential of preschool intervention, but they are not Head Start or anything close to it. More recent studies of Head Start itself have suffered from a variety of methodological limitations that limit confidence in their findings.
Enter the Head Start Impact Study. Conceptualized during the Clinton administration by an external advisory committee on which I served, it was to be the best that evaluation science could offer to determine the degree to which Head Start worked, for whom, at what points in schooling, and for what outcomes. To the credit of the Clinton and Bush administrations and the professionals overseeing the study in the Department of Health and Human Services (HHS), the study went forward in 2002 on pretty much the same terms recommended by the advisory committee.
It included a nationally representative sample of Head Start programs that had more children seeking admission than available slots. Admission into those programs was determined by lottery. Children who won and lost the lottery were to be followed through the end of third grade, and assessed on a wide variety of outcomes across the domains of cognition, socio-emotional development, health, and parenting. The Head Start Impact Study as designed and implemented is among the strongest evaluations of a government social program ever carried out.
HHS released the Head Start Impact Study Final Report last week.
There are several remarkable things about it:
The federal government through the Office of Management and Budget has, over the years and through different administrations, promulgated a series of directives for federal statistical agencies that go a long way towards protecting the integrity of their products. As a result the public does not have to worry that a government report on number of new unemployment claims has been delayed or subjected to unusual statistical procedures in order to be please those in power. In contrast, no such general protections exist for evaluation studies.
The public has no ready way of knowing what evaluations are under way and when reports are expected. The evaluations are often overseen by the same agencies that are responsible for implementing the programs, as was the case with the Head Start evaluation. This is a structural conflict of interest that often leads to pressures on contractors who carry out the evaluations to please their federal clients by casting findings in a more positive light than they deserve. And there is typically no independent peer review of evaluation reports to serve as a check on the integrity of the analysis and reporting of results.
President Obama has called for doing what works in education, health care, and other fields. His administration includes individuals at the highest levels who understand that doing what works depends on timely information from rigorous evaluations, analyzed and reported with independence and integrity. It is time for an executive order or an OMB directive to foster and protect the integrity of program evaluations.
Although the Head Start evaluation report includes findings derived from relaxed statistical standards and was badly delayed, the quality of the study shines through and the message is clear: Head Start isn’t doing the job the families it serves and the nation need. It must be improved. There are many proposals for doing so. Let’s try them, test them, and do what works.
Topics: advise, budget, children, Children & Families, committee, data, education, emotional. development, equity, goos, Governance, government, Head Start, health care, HHS, learning, lost, Lottery, low income families, minorities, New York Times, news, Office of Management and Budget, parenting, pilot, poor, povery, program, readiness, report, retaining, school, social programs, society, socio, U.S., U.S. Department of Health and Human Services, United States, vocabulary, Wall Street Journal, war, Washington Post, won
Print This Article in Plain Text
|
|