cost effectiveness for nonprofits evaluating a foster care reform projects
Author: baker, debra ratterman
ratterman, d. (n.d.). Cost effectiveness for nonprofits evaluating a foster care reform project.. [online] https://web.b.ebscohost.com/ehost/detail/detail?vid=3&sid=f4634fb7-4db5-4836-9e79-ff1dd0365918%40pdc-v-sessmgr04&bdata=JnNpdGU9ZWhvc3QtbGl2ZQ%3d%3d#AN=2256611&db=aph. Available at: https://web.b.ebscohost.com/ehost/detail/detail?vid=3&sid=f4634fb7-4db5-4836-9e79-ff1dd0365918%40pdc-v-sessmgr04&bdata=JnNpdGU9ZWhvc3QtbGl2ZQ%3d%3d#AN=2256611&db=aph [Accessed 4 Feb. 2020].
A for-profit enterprise seeks to be cost effective to enhance its ability to compete in the market economy. If a company can lower its costs it will increase its profit margin, just as readily as if it increases prices or sells more. This will lead to shareholder wealth maximization, the ultimate goal of a for-profit entity. A company that is not cost effective may be driven out of business by its more efficient competitors.
Market forces do not operate the same way in the nonprofit sector. Individuals or groups who need the services or products of a nonprofit do not typically purchase them directly. Instead governments, foundations, and donors provide funding for projects that will benefit others. There is no supply and demand pricing loop to let nonprofits know if they are meeting the needs of their constituents in a cost-effective manner.
Nevertheless, nonprofits are economic entities: revenues must be raised and managed and costs must be identified and constrained. All funders are, in essence, looking for a return on investment based on social rather financial returns. Being cost effective is essential to the nonprofit's mission of achieving the greater good for the least cost.
Nonprofits face many barriers to assessing the cost effectiveness of their work. Determining the impact of their work is complex. The outcomes are often "soft" and difficult to quantify. It is controversial to translate some outcomes into dollars: how much is saving a human life worth? The ultimate benefits may be far in the future, such as with childhood immunizations. Their specific impact may be lost in a myriad of confounding variables, like the state of the economy. The lack of common measures to compare dissimilar and even similar nonprofits also impedes including cost effectiveness in decision making.
Measures of Cost Effectiveness
One way to measure cost effectiveness for nonprofits is traditional cost-benefit analysis: how many dollars of benefits are produced per dollar of cost? A ratio over one means the project is cost effective, the benefits outweigh the costs. A ratio of less than one means that the benefits are insufficient to justify the cost. The key difference in a nonprofit's use of cost-benefit analysis is the matter of perspective. While a for-profit company determines how its costs benefit itself, a nonprofit determines how its costs benefit others.
Some nonprofits have found cost-benefit analysis to be an inadequate measure of cost effectiveness. The major objection is its failure to include nonfinancial as well as financial benefits. As a result, some groups have developed specialized measures of cost effectiveness to include nonquantifiable benefits.
For example, the Shelter Network in San Mateo County, California, designed a method for measuring the cost effectiveness of their projects to help the homeless. They adapted Maslow's hierarchy of needs to develop a weighted assessment of the impact of their program on:
- Instrumental Needs: food, clothing, shelter, money, health services, and safety.
- Stabilized Needs: own home, life skills, substance abuse treatment, job, placement, childcare, education, and transportation.
- Emotional Needs: self-esteem, control, and support.
Each of these factors is evaluated on a scale of 1 to 5 when the person or family begins the program, finishes the program, and three months after the completion of the program. Costs are allocated to whichever need they address. The program then calculates cost per weighted change in human needs. These ratios can then be used as a baseline to assess changes in program or costs.
Foster Care Reform Project
How can the principles of cost effectiveness be applied to a nonprofit project? I will use my own project on foster care reform to demonstrate. My organization, the ABA Center on Children and the Law, was given a two-year $100,000 federal grant to improve the foster care system in Onondaga County, New York. Our goal was to enhance foster children's sense of stability and permanency by reducing the amount of time they spent in foster care. We focused on speeding the process of freeing foster children for adoption through termination of parental rights.
The project was staffed by an on-site permanency planning specialist, on-site attorney, and myself, a legal expert. We met bimonthly with a multidisciplinary advisory board which included judges, attorneys, caseworkers, service providers, and child advocates. We identified problem areas, developed targeted solutions, created new procedures and trained staff. Our philosophy was that small steps accumulate to make significant changes.
Our cost effectiveness analysis focused on the dollar savings to the government of reducing the amount of time children spent in foster care compared to the costs of the project:
- We began with the New York State Legislature's estimate that one year in foster care costs the state $10,000 to $15,000, and chose the more conservative estimate of $10,000;
- In the first month of the project, we collected time data from foster care records for 50 children who had been freed for adoption up to two years before the start of the project. In the last month, we collected the same data for 50 children freed during the project. Average lengths of stay in foster care were calculated on a Lotus spreadsheet.
- We compared our data on the average time from child placement in foster care to a final order of termination of parental fights before and after the project. In Onondaga County, we reduced the average from 4.6 years to 3.2 years--a difference of 1.4 years.
- We counted the number of children affected by the project: the 137 children for whom a termination petition was filed during the project period.
- To calculate the savings, we assumed that without the project, those 137 children would have stayed in foster care for another 1.4 years. At a cost of $10,000 per year multiplied by 1.4 years and that total multiplied by 137 children, we came up with a savings of $1,918,000
- The cost of the project was calculated based on the total cash grant from the federal government of $100,000 over two years.
- The ratio of savings to costs for the project is 19:1. We saved 19 dollars for every dollar in costs.
We had many choices to make to develop a useful measure of the project's cost effectiveness. A key element of cost effectiveness analysis is identifying the major assumptions made in the study. We looked at 8 issues to develop our cost effectiveness analysis. Here are eight key areas.
1. Should we use a simple or scientifically rigorous analysis?
Any measure of cost effectiveness should be as accurate as possible. However, the time and money we spend on measuring change reduces the resources we can spend on making change. We need to strike an appropriate balance between analytical rigor and time/expense.
A scientifically valid study of Onondaga County's foster care system could have easily used up our entire grant. For example, Girls, Inc., spent $2 million over six years assessing the impact of its teen pregnancy prevention program, a project that served a total of 240 girls.
Not only were our resources limited, but the process we wanted to change (freeing foster children for adoption) already took several years. If we didn't implement changes quickly, we would have been unlikely to see any improvement at the end of two years.
In addition, we hoped that this project would be a model for other counties and states. The simpler analysis we adopted would be easier to teach and duplicate in new jurisdictions. Yet it still had rigor.
2. Should we use objective or anecdotal data?
If you read the newsletters of nonprofit organizations, you will often see profiles of individuals helped. For our foster care reform project, we could have chosen to focus our effectiveness evaluation on documenting the stories of some of the children we helped, like the five black children who were adopted by a childless black couple in Buffalo.
While anecdotal data have a powerful emotional appeal, we chose instead to focus on objective measures of improvement. While all politicians want to "help children," they often have more difficulty once it comes to funding children's programs. We wanted hard data to ensure that the changes we made were institutionalized in Onondaga County. We also wanted data to support replicating the project in other localities.
3. Should we attempt to measure non-quantifiable as well as quantifiable impact?
We wanted objective data that we could quantify and compare. How do you quantify something like a child's sense of permanency and stability? We could have developed a rating system like the Shelter Network's hierarchy of needs scale. This approach, however, posed some problems. First, it requires many subjective judgements by multiple individuals. Second, because Onondaga County had 850 children in foster care when the project began, such a rating system would have overburdened project staff. Its uniqueness also would have limited comparison of our project to other similar reform efforts.
Children who are in foster care are, by definition, in temporary placements and lack "permanency." So to measure time to permanency, we measured length of stay in foster care. These data were also relatively easy to collect and compare.
4. Should we measure participant or systemwide benefits?
The impact of a nonprofit program can often be measured by the benefit the participant receives, like the Shelter Network's human needs assessment. However, these projects also benefit society at large. For example, Girls, Inc., measured the effectiveness of their teen pregnancy program by looking at the savings to society: one teen pregnancy delayed until the mother was in her twenties would save about $8,580 in public assistance costs.
By reducing a child's length of stay in foster care, our foster care reform project benefits not only that child but also the child welfare system in the Onondaga County. These benefits include freeing up resources like foster homes, family services, and caseworker time. We chose to measure these systemwide benefits. We did this by measuring the cost savings of reducing children's stays in foster care.
5. Should we measure the immediate or long-term benefits?
In the longer term, children with more permanency and stability in their lives are more likely to do well in school, have better health, and increase their life satisfaction. They will be less likely to engage in criminal activity, live on the streets, or become teen parents. Looking further into the future, we may also prevent the next generation of children from ending up in foster care.
As important as these benefits are, they are incredibly difficult to measure. In addition, we run into the problem of confounding variables: so many other factors will affect the future lives of these children, it would be almost impossible to separate out the impact of our project.
To keep it simple, we focused only on immediate benefits: the child's reduced amount of time in foster care and its cost savings to the county child welfare system.
6. Should we measure the overall improvement or the net value added of the project?
We want to measure the impact of our project independent of other changes in the Onondaga County foster care system--the "net value added" by the project. To do this, we used a "before and after" comparison. We measured the average stay in foster care two years before our project began and compared it to average stays at the end of our two-year project.
The assumption is, without our intervention, the county would continue to free children for adoption at the same rate as they had done before. We assume that the county would not have improved on its own. Even though the county could have improved (or worsened) without our help, we have no way of accurately measuring "what would have been."
Instead of using a "before and after" equation to measure change, we could have used a "control" group. We could have identified similar children who, during the same time period, would not benefit from our project. This approach poses two significant difficulties. If we chose foster children in another county as our "control," we would have had to overcome issues of comparability of the two counties. If we chose a group of children within Onondaga County as a "control," we would have been denying these children the potential benefits of the project.
Another issue is whether other ongoing or short-term efforts also helped to reduce average stays in foster care in the county during our project period. For example, Onondaga County implemented a new risk assessment program during our project. Did this program make it easier for caseworkers to make decisions about freeing foster children for adoption? Alternatively, did the time caseworkers spent in risk assessment training and filling out new paperwork take away from their time to prepare cases for termination of parental rights?
We chose not to try to tease out the impact of other efforts in the county. Our measurement focuses on the change in average stays in foster care in our project time period.
7. Should we value benefits in monetary units or use alternative comparisons?
Cost-benefit analysis requires that both the benefits and costs be stated in monetary terms. As discussed above, some nonprofits have developed alternative rating systems that allow comparisons without translating benefits into monetary units, such as Shelter Network's hierarchy of needs rating system.
Our foster care reform project measures changes in time: reductions in how long children stay in foster care. Since "Time = Money", we could easily translate time savings into dollar savings.
We used the New York State Legislature's estimate that one year in foster care costs the state $10,000 to $15,000. Our calculation of foster care costs could have been more rigorous. We could have determined the average cost for Onondaga County or collected the actual cost for each child. However, our limited project time and resources made such a calculation impractical.
One advantage of using a statewide average cost was that other counties in New York could look at our calculations in Onondaga County and make rough estimates of what their own savings would be if they duplicated the project. Using a single state rate aided our efforts to disseminate our project across the state.
8. Should we use actual dollars of costs and benefits or calculate present value?
In evaluating the impact of reform projects, nonprofits need to consider the time value of money. Consider two projects: one saves $1 million next year and another saves $1 million 20 years in the future. If the costs are equal for both projects, we would come up with the same cost-benefit ratio if we used actual costs. However, saving $1 million in the next year is better than saving it 20 years from now because that $1 million could be reinvested in new reform efforts. By calculating the present value of each, we would be able to see that the cost-benefit ratio favors the project with short-term savings.
For example, Technoserve fosters the development of community-based agricultural enterprises in developing countries. To measure the financial impact of its efforts, it calculates changes in farmer income, enterprise income, and wages paid annually up to 15 years after their three-year intervention. To come up with comparable financial analyses, they calculate the present value of both the costs and the benefits for the year the project began. Instead of using an investment interest rate for their calculation, they use a Social Discount Rate. The Social Discount Rate estimates the opportunity cost of money to society.
If we had chosen to measure the long-term impact of our foster care project, we would have had to deal with the issue of present vs. future value. However, since our savings were within a two-year period, we chose to use actual costs.
Would using present value (PV) have made a difference in our cost-benefit ratio? I calculated the present value of our costs: $12,500 was paid to us at the end of each quarter for eight quarters by the federal government to cover our project costs. Using the T-bill rate for the start date of the project, 6%, I calculated the PV of our costs to be $93,574. Our savings occurred throughout the project but I used annual figures for the calculation: we saved $602,000 at the end of the first year (43 children) and $1,316,000 at the end of the second year (94 children). Taking the PV of each, we get a total of $1,739,160. The cost benefit ratio comes out to $19 to $1, the same as when we use actual costs.
Conclusion
Our cost effectiveness analysis of the foster care reform project resulted in many benefits. First, it translated the impact of a difficult and diverse project into short and easy-to-understand numbers. Second, it provided strong support for continuing the permanency planning specialist and other project reforms in Onondaga County with local funding after the federal grant ended. Finally, it convinced the New York State Department of Social Services to fund similar projects in ten other counties.
icons on the left to see existing comments.
0 General Document comments
0 Sentence and Paragraph comments
0 Image and Video comments
first four stanzas describe what the problem is
New Conversation
Hide Full Comment
results
New Conversation
Hide Full Comment
stats
New Conversation
Hide Full Comment
Stanzas 1 through 23 discuss how the problem was solved
New Conversation
Hide Full Comment
I believe a simpler analysis would be better because it was the one the ABA center on children used and it worked out great for them also. It would probably be easier to teach other jurisdictions and also easier to duplicate.
New Conversation
Hide Full Comment
anecdotal data could be useful for people to feel for the children in the foster care system. But objective data would be better because hard data ensures changes were made and institutionalized in Onondaga county.
New Conversation
Hide Full Comment
General Document Comments 0
