Key Findings

Four Equity Themes that Demonstrated States’ Capacity to Use Data and Evidence to Achieve Equitable Outcomes

The 2021 Invest in What Works State Standard of Excellence captured evidence-based and data-driven practices to advance equity in state government. The practices included throughout the State Standard of Excellence are centered on four main themes (See them described in Introduction.)

  • Embed equity in setting of strategic goals and outcomes, led by the Governor and executive branch
  • Collect, use, and report disaggregated data, evidence, and results to deepen understanding impacts of policies, programs, and processes
  • Build inclusive processes that provide meaningful decision-making opportunities for communities and stakeholder groups when decisions are being made that affect them
  • Analyze impact of budget, policy, and management decisions through an equity lens to achieve more equitable outcomes and results for residents

In the 2021 State Standard of Excellence, six states (Colorado, Oregon, Utah, Washington, North Carolina, and Rhode Island) described how they embedded equity into their statewide strategic goals, made equity a lens to frame state operations, and used data and performance metrics to hold themselves accountable for progress. 

When developing annual performance goals, Colorado’s state agencies were recommended by the Governor’s Budget Office to consider an equity, diversity, and inclusion framework to set goals, performance measures, and outcomes.

Similarly, in Oregon, the Governor’s Office published a series of strategic initiatives in a 10-point action plan for economic recovery, informed in partnership with the Governor’s Racial Justice Council. Additionally, the state has a strategic direction for guiding Oregon’s recovery from COVID-19 within a framework of racial equity and social justice.

Pennsylvania and Washington described their community engagement strategies through advisory councils and public performance review processes, which allow for direct public engagement in goal setting and performance monitoring.

Five states (Colorado, Virginia, Indiana, Minnesota, and Rhode Island) described how they continued to leverage disaggregated data to manage their COVID-19 responses. States used disaggregated data to inform equitable vaccination distribution strategies.

North Carolina’s executive order directed the North Carolina Department of Health and Human Services to update reporting processes to increase racial and ethnic demographic data; review data to determine impacts on communities of color; and use collected data to allocate resources to mitigate the spread of COVID-19 with a focus on reducing the adverse health impacts on racial and ethnic minorities.

Utah’s “Striving Toward Equity: Utah’s COVID-19 Vaccine Distribution Roadmap,” developed with input from community-based partners, outlined an equitable vaccine distribution based on suggestions for accurate collection and reporting of racial and ethnic data. As the state implemented the plan, missing race/ethnicity data was reduced by 50%, and vaccination rates improved for some communities – the vaccination rate among Hispanic/Latino populations increased by 463% in the same time that rates for white folks increased by 262%.

Minnesota took an evidence-driven approach to boost vaccine access by testing to identify trusted messages and messengers to encourage Black and Hispanic communities to increase their rates of COVID-19 testing and vaccine uptake, including vaccine incentives.

Beyond pandemic responses, states have made efforts to embed equity into statewide data operations. Oregon’s statewide 2021-2023 data strategy outlined a vision for more ethical data use (called data justice) – all rooted in data collection, use, design; community engagement; and transparency. Minnesota’s Evidence Base Demographics tool provides demographic information – race, ethnicity, socioeconomic status, or education, for example – about programs that have been reviewed by national research clearinghouses. It also allows policymakers to apply the best and most relevant evidence base in the programs they implement in serving Minnesota’s residents. This and similar efforts can help policymakers apply the best evidence in their decisions, including identifying and selecting programs that meet residents’ and communities’ needs.

While states engage stakeholders through traditional listening tours, town halls, and forums, some states are reconsidering their strategies to create more authentic engagement opportunities to shift decision-making directly to community members. An exemplary statewide approach is the Results Washington Public Performance Review process (PPR) – recurring monthly meetings with the Governor, leaders, agency experts, and community members – that allows state leaders to hear from residents directly. These reviews inform management and policy decisions. Other states, such as Pennsylvania, have engaged advisory commissions representative of community and stakeholder groups and communities to inform statewide goals.

To inform Minnesota’s COVID-19 response effort, the department of health convened a Community Resiliency and Recovery workgroup composed of external advisors – community leaders representing underrepresented groups including residents identifying as BIPOC, LGBTQIA+, and persons with a disability. The group met monthly to discuss the state’s progress with its COVID-19 response. The discussions included a review of the public data on the Data by Race/Ethnicity section of the state’s COVID-19 dashboard, which led to dashboard improvements, including the addition of congregate care settings data from nursing homes and homeless shelters.

When it comes to government spending, equity should be considered. Minnesota and New Mexico described that agencies and departments may be required to report disaggregated program performance data in budget requests. Colorado made strides in making equity a core element in budgeting. The Colorado Governor directed the Office of State Planning and Budgeting (OSPB) to evaluate the impact of all budget requests on equity, diversity, and inclusion (EDI). OSPB then created an equity, diversity, and inclusion team and worked with state departments to incorporate EDI considerations into the budget process. To further improve government services for diverse populations, all state employees were required to complete coursework on EDI training in FY20-21, as stipulated in the Governor’s executive order.

In Washington, the Department of Children, Youth, and Families (DCYF) is shifting all contracts for client services to performance-based contracts (PBC), a key priority in the agency’s 2021-2026 Racial Equity and Strategic Plan. In 2021, the agency is converting over 70% of its portfolio (more than 1,000 contracts that invest approximately $1 billion every two years) to support nine priority outcome goals. Building on traditional performance-based contracting, the project intentionally focuses on deepening stakeholder engagement, using PBC as a tool to advance racial equity and facilitate continuous improvement.

States Leveraged Evidence-Based Budgeting and Grantmaking to Invest in What Works

The 2021 Invest in What Works State Standard of Excellence captured evidence-based and data-driven practices to advance equity in state government. The practices included throughout the State Standard of Excellence are centered on four main

With state decision-makers facing critical budget choices to support an economic recovery from the COVID-19 pandemic, state budget offices should rely on evidence to ensure taxpayer dollars are invested in what works. In the 2021 State Standard of Excellence, states have demonstrated progress in leveraging evidence and data to enhance the budget process. This has been done by using evidence to make budget decisions and investing in evaluation and other evidence-building activities. 

Evidence-Based Budgeting

In 2021, the Colorado state legislature passed a law that aligned evidence definitions – the five-step evidence continuum – across the state’s executive and legislative branches. This codifies procedures to incorporate evidence-based research into the state budget process and encourages its expansion. The Colorado FY21-22 budget included 53 requests assessed on the evidence continuum, representing 33% of all requests and an increase from FY20-21 in the number and kind of requests assessed on the continuum. The goal of the process is to determine program effectiveness and utilize that information to make program and budget decisions.

 Also in 2021, the New Mexico Legislative Finance Committee launched a new initiative called LegisStat, the first of its kind in the country. These meetings are led by the joint House-Senate committee that dives into root performance challenges (and successes), much like the PerformanceStat model. This effort supports New Mexico’s evidence-based budget. Each year, the Legislative Finance Committee issues budget instructions that provide guidance for agencies for budget expansions limited to committee priorities and evidence-based programs as promulgated by the Legislating for Results framework.

Tennessee’s newly formed Office of Evidence and Impact leads statewide efforts to build and sustain evidence-based budgeting. The office was tasked with identifying what works within the state’s $38 billion annual base budget through research and evaluation technical assistance. Since 2018, the office has partnered with the state Department of Corrections and the Department of Mental Health and Substance Abuse Services to develop pilot program inventories of Tennessee-specific programs based on the state’s tiered evidence definitions. The Office also developed cost-increase and reduction request forms, as part of the annual budget process, to preserve essential, evidence-based programming amid 12% reductions in FY21.

Rhode Island has improved budget request forms to help the state legislature to use evidence in budget processes and to allow budget teams to use an evidence lens when considering budget reductions. The Rhode Island Office of Management and Budget updated its budget requests forms that require agencies to describe a program’s proposed evidence base in the request. The office has also conducted trainings to demonstrate how evidence can be used to create more credibility for budget requests.

Evidence Building and Evidence-Based Grantmaking

Dedicating resources for evaluation can be a tremendous challenge for states. However, a few pioneering states have begun to set aside funds for program evaluation or create statewide evaluations funds to support evidence-building about what works in their state. Some examples include Minnesota’s Opiate Epidemic Response grant program, which included $300,000 (approximately 1% of the agency budget) to conduct experimental and quasi-experimental impact evaluations for opiate epidemic response grant activities. Colorado Department of Public Safety’s Juvenile Diversion Program invests 3% ($120,000) of program funds to conduct evaluations. (Related, Crime Victim Services’ competitive funding opportunities require all applicants to evaluate the effectiveness of their project.)

For the first time ever, the North Carolina FY21-23 budget includes $500,000 in non-recurring funds each year during the fiscal biennium for grants to state agencies for research and evaluation. These grants can be used for partnership with research institutions that will directly inform the agencies’ policy and program decisions and evaluate programs in achieving their intended outcomes. Colorado’s Office of State Planning and Budgeting also administers a $500,000 annual fund for program evaluation and implementation grants, which provides competitive funding for agencies to undertake evaluation and implementation science projects.