Colorado
Leading Example
6. Data Policies
The state has data policies that outline the principles, people, and activities that govern its data collection and use.
6. Data Policies
A 2021 Colorado law modified many of the statewide data practices by specifying requirements regarding the creation of a data-sharing and privacy master plan and sharing personal identifying information between state agencies. Moreover, it codified the “share first” expectation (24-37.5-704 C.R.S) for data sharing statewide, specifically that, except where prohibited by state or federal law, agencies are preemptively authorized to share data with other state agencies, legislative and judicial branches, political subdivisions, and non-governmental organizations. Additionally, to enhance data privacy measures, in May 2020, the Governor published guidance on data privacy with respect to state agencies sharing personal identifying information with the federal government or other third parties, some of which was codified into law in the 2021 legislative session.
3. Culture
The state has a culture that supports the sustainable use of data and evidence to deliver results in a transparent, equitable, and ethical manner.
3. Culture
While the 2013 Colorado SMART Act has embedded a culture focused on data and evidence, the Governor has played a key role in communicating its value and importance in recorded remarks as well as the 2021 State of the State address. Within the Governor’s Office, the Office of State Planning and Budgeting continues to lead the development of research and evidence to improve outcomes and to advise the Governor on program effectiveness. The Governor also directed the Office of State Planning and Budgeting (OSPB) to evaluate the impact of all budget requests on equity, diversity, and inclusion (EDI), with a goal of advancing the state’s progress. OSPB has created an EDI team to undertake this work and is working with state departments to incorporate EDI considerations into the state’s budget process.
All state employees were required to complete coursework on equity, diversity, and inclusion (EDI) training in FY 2020-2021, as stipulated in the Governor’s executive order that directs the Department of Personnel and Administration to lead state action on equity, diversity, and inclusion. Examples of questions posed in the training are:
- How have communities been engaged?
- Are there opportunities to expand community engagement?
- What is the data?
- What does the data tell us?
- How will the state ensure accountability, communicate, and evaluate results?
Issue Areas: Economic Mobility, Education, Equity, Health, Workforce
1. Strategic Goals
The state has statewide strategic goals that are public, quantitative, published regularly, customer oriented, aligned, equity oriented, and publicly informed.
1. Strategic Goals
Launched in 2019, Colorado’s Governor’s Dashboard outlines four high-priority strategic goals: tax reform and economic development, energy and renewables, health, and education and the workforce. Progress toward each strategic priority, set by cabinet working groups, is displayed on the publicly available Governor’s Dashboard, established in 2019, with data updated monthly. The strategic priorities are aligned goals, metrics, and actions contained within agency performance plans. The governor’s annual budget request also links these goals to specific agency activities and outcomes. In developing annual performance goals, agencies are advised to consider the impacts of their goals on broader equity, diversity, and inclusion efforts, by both setting performance goals consistent with those efforts, and by collecting and analyzing demographic data to hold themselves accountable for equitable outcomes.
In response to new challenges identified during the COVID-19 pandemic, Colorado launched the Reimagine State Government Initiative in 2020, which identifies six pillars of operational excellence across state agencies. These pillars are tied to specific, quantifiable goals, including: reducing the state’s physical footprint by one million square feet by 2025; improving employee engagement by five percentage points by 2025; increasing access to virtual services by 100,000 transactions by June 2021; and others aimed at making the government more efficient and effective to better serve state residents.
Issue Areas: COVID-19, Economic Mobility, Education, Equity, Health, Workforce
Promising Examples
2. Performance Management
The state has achieved better outcomes through an enterprise performance management system that engages state leaders in using performance data to continuously improve results.
2. Performance Management
A 2013 Colorado law required all state agencies to submit annual performance reports to the Colorado state legislature, which must include: (1) performance measures for the major functions of the department; (2) performance goals for at least the subsequent three years; (3) a description of the strategies necessary to achieve those goals; and (4) a summary of the department’s most recent performance evaluation. The Governor’s Director of Operations and Cabinet Affairs oversees the development and execution of statewide performance goals. The Governor’s Office of Operations maintains the statewide performance management system required, by law, to be published on an annual basis. The office provides guidance on developing annual strategic goals and annual performance plans and identifies opportunities for process improvement (CRS 2-7-204). Consistent with this management system, executive branch agencies continue to identify annual Wildly Important Goals (WIGs) that align with their agency’s mission, the statewide strategic goals, working group goals, and the Reimagine State Government Initiative.
To support the Governor’s strategic priorities, the Governor’s Office of Operations maintains monthly working groups (Economic Development, Environment and Renewables, Health, and Education and Workforce) with cabinet members and relevant leadership, including: deputy directors, division directors, and/or key program staff, as needed. Those working groups set annual performance goals aligned to support the Governor’s priorities and progress toward the goals displayed on the Governor’s Dashboard with data updated monthly. The Office of Operations, Office of State Planning and Budgeting, and Policy Office within the Governor’s Office regularly coordinate to provide feedback on performance goals, budget requests, and legislative items, respectively.
The Governor’s Office of Operations supports innovation by offering free training to all state employees in Lean, Agile, and Change Management. Broadly, within the state, there is a culture of data-driven innovation. One exemplary effort includes that the Colorado Department of Transportation has an Office of Process Improvement that hosts a Lean Everyday Ideas program within the Department and tracks the impact of those projects.
Issue Areas: Child Welfare, Criminal Justice, Economic Mobility, Education, Health, Workforce
5. Data Leadership & Governance
The state has senior leadership and governance structures with the mandate to equitably use data to improve results.
5. Data Leadership & Governance
The State of Colorado’s Office of Information Technology (OIT) includes a Chief Data Officer. The Chief Data Officer is responsible for developing and directing the state’s data strategy, data governance, and data management. The position establishes enterprise data policies, standards, and procedures to maximize the state’s investment in data management systems. The OIT’s statewide data management program leverages data and information as enterprise assets and establishes standards and processes to enable more agile solutions and government services. Additionally, OIT’s Government Data Advisory Board (GDAB), created by a 2009 law, advises the Chief Data Officer on activities and policies necessary to develop the interdepartmental data protocol, made up of members from state and local government, as well as private industry partners. GDAB’s mission is to improve the efficiency and effectiveness of state government, citizen service delivery, and policymaking by providing guidance and recommendations on how the state should govern and manage data and data management systems. For example, in 2021, GDAB was tasked with creating standard templates for interagency data-sharing and data-access agreements through HB 21-12.
7. Data Infrastructure
The state has improved outcomes through technology infrastructure that allows it to efficiently collect, inventory, and share data.
7. Data Infrastructure
The Colorado Information Marketplace is the state’s open data repository with more than 2,000 datasets intended to increase government transparency. Additionally, the 2021 Colorado data law requires each agency to conduct an inventory of its own data assets, with assistance from the Office of Information Technology and the Government Data Advisory Board (GDAB). This inventory is still in development, though some agencies have already started on the effort, and it is being coordinated by the Chief Data Officer.
Beyond state open data, the Colorado Governor’s Office and the Colorado Evaluation and Action Lab co-designed the Linked Information Network of Colorado (LINC) to facilitate data sharing for research and analytics in 2019. The network is designed to share data across state agencies and provide de-identified data to perform robust, academically rigorous research to inform policy. LINC has a three-tier legal structure, which includes: (1) an enterprise memorandum of understanding (eMOU) signed by all data providers; (2) data-sharing agreements to secure, handle, and anonymize data for all LINC projects; and (3) data licenses with roles and responsibilities for users of LINC project data. Additionally, several state agencies – including the departments of higher education, labor and employment, and human services and entities (Colorado Workforce Development Council, and the Colorado Community College System) are working together to build a data trust to share data among state agencies for specific use cases, which will assist in sharing data in a well-governed and highly secure manner.
The Colorado Department of Education received a federal Statewide Longitudinal Data Systems (SLDS) grant in 2019 that is ongoing and involves matching data from the Department’s Student Pathways data with the Colorado Department of Higher Education and Colorado Department of Labor Employment’s data.
Issue Areas: Economic Mobility, Education, Health, Workforce
8. Data Use
The state has improved outcomes by combining and analyzing cross-agency data to inform policy, budget, or management decisions.
8. Data Use
Colorado’s Department of Public Health and Environment extended its geospatial data and resource library to encompass a wide range of COVID-19-related data, including: hospital data, incidence and epidemic curve data, outbreak data, and surveillance data. Daily state-level data keeps policymakers informed with up-to-date information on COVID-19-related indicators. The department’s open data portal also maps operational community testing sites that receive COVID-19 testing resources from the state.
In 2019, the Colorado Governor’s Office and the Colorado Evaluation and Action Lab at the University of Denver co-designed the Linked Information Network of Colorado (LINC) to facilitate data sharing for research and analytics purposes as a way to improve state policies and programs. LINC, through the Colorado Evaluation and Action Lab, produces informative evidence about the impacts of state programs, and those reports are shared with agency leadership and staff during cabinet working group meetings. Using this data, the lab and state agencies have collaborated on projects to improve outcomes in areas such as education, renewable energy, reentry and racial disparities in criminal justice, prenatal health, and the early childhood education workforce.
In addition, the Colorado Department of Higher Education was the first state agency in the nation to partner on a pilot project with the U.S. Census Bureau to match federal unemployment insurance data with postsecondary degree completion data. At the state and county level, the Colorado Department of Human Services’ C-Stat performance management system facilitates data sharing among its 64 counties by providing dashboards to track key metrics and Performance and Partnerships Exchanges to facilitate sharing of best practices.
The Colorado Department of Public Safety worked with The Division of Criminal Justice to gather, analyze, and publish multi-agency data, prompting recommendations improving criminal justice outcomes and equity to be made by the Colorado Commission on Criminal and Juvenile Justice.
Issue Areas: Child Welfare, COVID-19, Criminal Justice, Economic Mobility, Education, Health, Workforce
9. Evaluation Leadership & Governance
The state has a leadership and governance structure with the authority to use evaluations to improve results.
9. Evaluation Leadership & Governance
The Colorado Governor’s Office of State Planning and Budgeting and its director lead the use of data and evidence in the state’s budget process through the review of agency budget requests based upon their level of evidence and evaluation to inform priority policy and budget decisions. This office engages with the University of Denver’s Colorado Evaluation and Action Lab as well the Colorado Evidence-Based Policy Collaborative composed of state agency representatives and community organizations. The office also administers a $500,000 annual fund for program evaluation and implementation grants, which provides competitive funding for agencies to undertake evaluation and implementation science projects.
Issue Areas: Child Welfare, Criminal Justice, Economic Mobility, Education, Health, Workforce
10. Evaluation Policies
The state has evaluation policies and a learning agenda that outline the principles, people, and activities to use evidence to improve results.
10. Evaluation Policies
The Colorado Governor’s Office of State Planning and Budgeting (OSPB) publishes the results of evaluations funded by the grant program on its website each year. OSPB also publishes annual guidance to state departments requiring the use of data and evidence in the state’s budget process. It also provides definitions for key research and evaluation terms and “encourages agencies to use the evidence continuum outside the budget process as a means to evaluate programming internally, and to guide discussions with OSPB analysts and internal and external stakeholders.”
To align the guidance and definitions across the executive and legislative branches, the Colorado state legislature passed a law in 2021 that establishes a set of evidence-based definitions, aligned with the OSPB evidence continuum, to be used when analyzing a program or practice. The law further requires the joint budget committee staff to “independently analyze and describe the program or practice using the definitions and to include any evidence-based information as part of any recommendation it makes regarding a budget request or budget amendment request.” Beginning in the FY22-23 budget cycle, OSPB requires agencies to complete an evidence inventory, aligned with the state’s tiered evidence continuum, as a part of the annual budget process.
The Colorado Department of Higher Education (CDHE) developed a research agenda to inform education and training policies and programs. Data sharing is required by Colorado law among the Department of Education, Department of Labor, and Department of Economic Development, which collectively generate research covering the education-to-workforce continuum. Additionally, the Colorado Department of Higher Education convenes several stakeholder groups with representation from all types of institutions of higher education ranging from academic affairs staff to data leaders. CDHE works closely with all institutions of higher education to set best practices around evaluation policies and priorities.
Issue Areas: Economic Mobility, Education, Equity, Workforce
11. Evaluation Resources
The state has dedicated resources for using evaluations to improve results.
11. Evaluation Resources
The Colorado Department of Public Safety’s Juvenile Diversion Program invests 3% ($120,000) of program funds to conduct evaluations. Crime Victim Services’ competitive funding opportunities require all applicants to evaluate the effectiveness of their project. It also requires those who currently receive funding to answer questions regarding their current effectiveness at meeting their funded projects’ goals and objectives, as well as their adherence to their budget. Additionally, Colorado’s Office of State Planning and Budgeting also administers a $500,000 annual fund for program evaluation and implementation grants, which provides competitive funding to support program implementation or evaluation of outcomes.
Issue Areas: Criminal Justice
12. Evaluation Use
The state has improved results by using evaluation, program inventories, and cost-benefit analysis.
12. Evaluation Use
The Colorado Governor’s Office of State Planning and Budgeting (OSPB) and the Colorado state legislature have aligned evidence definitions for a five-step evidence continuum that informs budget and spending decisions. OSPB requires agencies to identify the level of evidence and the results of any evaluations of programs as a part of the budget proposal process. OSPB also requires agencies to report on cost-benefit analyses performed on agency programs as a part of the annual budget process.
The Department of Higher Education’s Colorado Opportunity Scholarship Initiative (COSI) conducts an annual quasi-experimental design evaluation using propensity score matching to evaluate the success of programs that have received grant funding. Using COSI’s innovative model and the annual reporting requirements, COSI staff can monitor progress of grantees in achieving shared goals.
The Colorado Department of Human Services has been conducting cost-benefit analyses by evaluating at-risk Colorado youth based on their performance under Pay For Success contracts. If outcomes meet criteria of achievement based on the contract, success payments made are detailed in an annual report to the Joint Budget Committee.
Issue Areas: Child Welfare, Criminal Justice, Education, Health
13. Results-Focused Budget Process
The state has improved outcomes by prioritizing data and evidence in its budget process through guidance to agencies and engagement with the legislature.
13. Results-Focused Budget Process
The 2013 Colorado State Measurement for Accountable, Responsive, and Transparent Government (SMART) Act required all Colorado state agencies to submit annual performance reports to the state legislature as part of the state’s budget process. These reports include: (1) performance measures for the major functions of the department; (2) performance goals for at least the following three years; (3) a description of the strategies necessary to achieve those goals; and (4) a summary of the department’s most recent performance evaluation.
The Colorado Governor’s Office of State Planning and Budgeting (OSPB) and the Colorado state legislature passed legislation that aligned evidence definitions, the five-step evidence continuum, that informs budget and spending decisions. OSPB requires agencies to identify the level of evidence and the results of any evaluations of programs as a part of the budget proposal process, “based on the evidence and body of research supporting the program’s effect on desired outcomes and proposed implementation plan.” This guidance is provided in the governor’s budget instructions via an assessment of the data and evidence associated with the requests. The FY21-22 Governor’s budget included 53 requests assessed on the evidence continuum, representing 33% of all requests and an increase from FY20-21 in the number and kind of requests assessed on the continuum.
The Colorado Governor also directed OSPB to evaluate the impact of all budget requests on equity, diversity, and inclusion (EDI) with a goal of advancing equity statewide. In 2021, OSPB has created an EDI team to undertake this work and is working with state departments to incorporate EDI considerations into the state’s budget process.
Issue Areas: Child Welfare, Criminal Justice, Economic Mobility, Education, Equity, Health, Workforce
15. Evidence-Based Grantmaking
The state has improved outcomes by allocating grant funding based on evidence of effectiveness.
15. Evidence-Based Grantmaking
In 2018, the Colorado Department of Education grant program for school improvement, Empowering Action for School Improvement, required schools to use evidence-based practices as defined by the federal Every Student Succeeds Act (ESSA). The application gives points for the use of evidence-based strategies, as well as points based on past performance for applicants who are seeking to expand existing initiatives. Further, the Colorado Reading to Ensure Academic Development (READ) Act, amended in 2019, emphasizes the use of evidence-based instructional practices to support early literacy. It required use of program funds for external program evaluation; created a science-based early literacy block grant program of $2.5 million annually; and required districts receiving funds in the school year 2021-2022 to have K-3 teachers complete evidence-based training in teaching reading.