In order to identify the metrics for the L.A. Compact Charting Progress Dashboard, UNITE-LA led a collaborative process with data and content experts across L.A. Compact organizations. The iterative process, described below, centered on the L.A. Compact Data Workgroup, with support and input from content experts within UNITE-LA and from multiple Compact signing organizations.
We note at the outset that metrics identification and data acquisition are ongoing processes. Our aim is to expand the Charting Progress Dashboard over time—to add high-value metrics as they become available and to include more subgroup disaggregations that will inform and enhance efforts toward equity.
1. Convening Data Experts
UNITE-LA identified institutional researchers and data experts from the range of Compact partnering organizations to form the L.A. Compact Data Workgroup, beginning in January, 2016. The group’s mandate is to identify optimal, available indicators to track progress toward the L.A. Compact’s evolving goals in order to promote continuous systems improvements that benefit youth in the L.A. region. We convened the L.A. Compact Data Workgroup bimonthly in 2016-2018, and periodically—as needed—in the ensuing years. Initial meetings included the full complement of data experts along the cradle-to-career continuum (at the group’s request).
In 2019—as we were finalizing the metrics and grappling with the challenge of measuring socioemotional outcomes—we added subgroup convenings in order to focus on (respectively):
Over the course of the past several years, UNITE-LA staff also engaged multiple one-on-one deep dives with individual workgroup members.
2. Identifying, Reviewing, Prioritizing, and Revising Metrics
The L.A. Compact Data Workgroup initiated our process of metrics identification with a “blue-sky” discussion of ideal, aggregate-level metrics. The group identified and discussed high-value indicators that would best capture progress toward each of the Compact’s goals IF we could wave a magic wand and have any data at our fingertips.
UNITE-LA staff worked with a team of interns for the 2015-16 academic year to support the L.A. Data Workgroup’s kickoff and efforts. We supervised a team of USC Master of Public Policy student interns to produce a comprehensive scan of ECE, higher education, and workforce metrics utilized by other education/workforce efforts around the country. The USC team presented the national scan findings to the L.A. Compact Data Workgroup in Spring 2016 in order to inform the consideration and selection of metrics.
Armed with the national scan and our blue-sky brainstorm, the L.A. Compact Data Workgroup began reviewing and assessing candidate indicators. We rated each metric along both substantive and practical dimensions, using the following criteria: highly valuable (predictive of goals); accessible; and available over time. Our aim was to identify optimal, aggregate indicators that UNITE-LA would be able to track longitudinally. Data had to be publicly available or available for sharing by Compact partners, with a high probability of continued availability into the future.
The workgroup prioritized indicators for K-12 and higher education and reached consensus on a set of proposed metrics. ECE and workforce metrics proved more challenging, given the dearth of appropriate aggregate-level data. The workgroup drafted an initial (albeit limited) set of possible ECE and workforce indicators.
UNITE-LA then engaged content experts in ECE and workforce to vet the draft indicators and to propose additional metrics that partner organizations could share with UNITE-LA. For ECE, this effort included staff from First 5 LA, L.A. County, and UNITE-LA. For workforce, we leveraged the expertise of UNITE-LA staff as well as the L.A. Compact’s Workforce Systems Collaborative, which includes key regional workforce development experts.
UNITE-LA utilized this input to create a revised set of proposed ECE and workforce metrics for consideration by the L.A. Compact Data Workgroup. The workgroup reconsidered all proposed revisions and reached consensus on the full set of proposed metrics in late 2020.
It is worth noting that over the course of obtaining data from partner institutions, we have had to update and make slight modifications to the set of indicators because of changes in data availability. We defer to workgroup members’ recommendations regarding their own institutions’ data, while maintaining as much fidelity as possible to the concerns and decisions of the larger workgroup.
3. Final Vetting and Approval by L.A. Compact Stewardship Group
The Stewardship Group is comprised of each L.A. Compact signing organization’s lead or their representative. Thus, they constitute the ultimate set of content experts. The Stewardship group considered, discussed, and ultimately approved the set of proposed metrics over the course of several years, beginning with a provisional set in 2018. The full set of metrics for the L.A. Compact’s Charting Progress Dashboard received final approval in late 2020.