Managing for Results

10. Evaluation Policies

Does the state have evaluation policies and a learning agenda to support the building and use of evidence?

The state has evaluation policies and a learning agenda that outline the principles, people, and activities to use evidence to improve results.

Minnesota

Leading Example

In 2018, Minnesota Management and Budget adopted an evaluation policy, which governs its use of evaluations and requires the public release of all completed evaluation reports regardless of findings. The policy outlines key principles for evaluation: rigor, relevance, independence, transparency, and ethics. Part of this requires pre-registration of evaluations and final results. Once complete, evaluations become a part of Minnesota’s Inventory, which shows the evidence base for 730 state interventions, and allows for sorting based on level of evidence, service population, settings, and outcomes.

Issue Areas: Child Welfare, Criminal Justice, Economic Mobility, Education, Equity, Health

Promising Examples

Colorado

The Colorado Governor’s Office of State Planning and Budgeting (OSPB) publishes the results of evaluations funded by the grant program on its website each year. OSPB also publishes annual guidance to state departments requiring the use of data and evidence in the state’s budget process. It also provides definitions for key research and evaluation terms and “encourages agencies to use the evidence continuum outside the budget process as a means to evaluate programming internally, and to guide discussions with OSPB analysts and internal and external stakeholders.”

To align the guidance and definitions across the executive and legislative branches, the Colorado state legislature passed a law in 2021 that establishes a set of evidence-based definitions, aligned with the OSPB evidence continuum, to be used when analyzing a program or practice. The law further requires the joint budget committee staff to “independently analyze and describe the program or practice using the definitions and to include any evidence-based information as part of any recommendation it makes regarding a budget request or budget amendment request.” Beginning in the FY22-23 budget cycle, OSPB requires agencies to complete an evidence inventory, aligned with the state’s tiered evidence continuum, as a part of the annual budget process.

The Colorado Department of Higher Education (CDHE) developed a research agenda to inform education and training policies and programs. Data sharing is required by Colorado law among the Department of Education, Department of Labor, and Department of Economic Development, which collectively generate research covering the education-to-workforce continuum. Additionally, the Colorado Department of Higher Education convenes several stakeholder groups with representation from all types of institutions of higher education ranging from academic affairs staff to data leaders. CDHE works closely with all institutions of higher education to set best practices around evaluation policies and priorities.

Issue Areas: Economic Mobility, Education, Equity, Workforce

Kentucky

The Kentucky Center for Statistics (KYSTATS) has a 2020-22 Research Agenda, which details four primary research areas to identify barriers to education and workforce opportunities: (1) expand data access and data use to inform equity issues; (2) evaluate outcomes and barriers for education and workforce programs over time; (3) connect supply and demand of the state’s future workforce; and (4) measure the impact of out-of-state education and workforce migration. All research results are made publicly available.

Issue Areas: Economic Mobility, Education, Equity, Workforce

Massachusetts

The Massachusetts Department of Elementary and Secondary Education published a research agenda in 2018 that outlines five strategies to prepare all students for success after high school. The department also provides an online Reports Library that showcases recent research and evaluation milestones.

Issue Areas: Education

Maryland

The Maryland Longitudinal Data Systems Center brings together education and workforce data from the Maryland Higher Education Commission, the Maryland State Department of Education, and the Maryland Department of Labor, Licensing, and Regulation. The center’s research agenda is established by a governing board.

Issue Areas: Education, Workforce

Mississippi

The Mississippi Department of Education has a research agenda informed by the State Board of Education’s strategic plan and a focus group of department staff members. The agenda’s associated questions and a dynamic framework shape the activities of the Office of Research and Development, which uses data to drive the department’s evidence-based decision-making.

Issue Areas: Education

Oregon

The Governor’s Racial Justice Council data team is creating a statewide research agenda to help guide evaluation questions and approaches, alongside ethical research toolkits. Agencies such as the Higher Education Coordinating Commission and the Oregon Department of Education are supporting these efforts.

Issue Areas: Equity

Tennessee

The Tennessee Education Research Alliance is a formal research partnership between the Tennessee Department of Education and Vanderbilt University’s Peabody College of Education. Led by full-time staff and guided by a steering committee and advisory council, the Department and the University have co-created a research agenda that builds a body of knowledge to better position the state to make data-driven and evidence-based decisions. The Alliance conducts independent studies and directs external research to provide relevant and timely information to state policymakers across a variety of topic areas, including early reading, professional learning, and school improvement. Similarly, Tennessee Postsecondary Evaluation and Analysis Research Lab, established in 2017, is a joint effort by the Boyd Center for Business and Economic Research, Vanderbilt University’s Peabody College of Education and Human Development, and the Tennessee Higher Education Commission. The lab produces policy-relevant research on higher education and postsecondary.

Issue Areas: Education

Utah

The Utah Data Research Center’s July 2021-June 2022 research agenda investigates themes related to the state’s education and workforce policies and programs in priority order. Previously published research agendas are publicly available. The center’s work is guided by a governance structure that calls for staff engagement, an advisory board, an institutional review board, and a peer-review process.

Issue Areas: Economic Mobility, Education, Health, Workforce

Washington

Since 2012, at the direction of the legislature, the Washington State Institute for Public Policy (WSIPP) has produced a series of inventories of evidence-based, research-based, and promising programs. These inventories are developed with the aim of informing state agencies about effective and cost-beneficial options for service provision. WSIPP has published a technical documentation that details the cost-benefit model and its application. The most recent inventories include reviews of the evidence in the areas of children’s mental health, child welfare, and juvenile justice (2020) and K-12 educational programs for struggling students (2020), which is also featured on the state superintendent’s Menus of Best Practices and Strategies website.

Issue Areas: Child Welfare, Criminal Justice, Economic Mobility, Education, Workforce

Back to all Criteria

Managing for Results

10. Evaluation Policies

Does the state have evaluation policies and a learning agenda to support the building and use of evidence?

Minnesota

Leading Example

Issue Areas: Child Welfare, Criminal Justice, Economic Mobility, Education, Equity, Health

Promising Examples

Colorado

Kentucky

Massachusetts

Maryland

Mississippi

Oregon

Tennessee

Utah

Washington

SIGN UP FOR UPDATES