Justin Roskopf is Search Institute’s Manager of Survey Services and Measurement Strategy. In this spotlight, he discusses the important role data plays in enabling and empowering organizations and practitioners to impact young people's development.

For more than ten years your work has been extremely influential in creating and defining actionable measures in alignment with Search Institute's Developmental Assets and Developmental Relationships frameworks and research. What is the value of aligned measures for organizations to better support young people?

I view our frameworks and measures as complementary tools to equip practitioners with information to understand the needs of the young people they work with and create pathways to meet those needs. Search Institute conducts applied research, with a central goal of translating the research to the practitioners working directly with young people and their families. Our frameworks are one result of that work—they are translational tools that bring positive youth development research into the hands of practitioners, parenting adults, and young people in a simple and easily accessible language.

The measurement tools that we create are an equally important aspect of that translational work. They bring the frameworks to life within the context of how they are being applied. The data provided by these measurement tools facilitates understanding of the current state of each of the assets or developmental relationship elements included in our positive youth development frameworks (Developmental Assets and Developmental Relationships). With insights directly from young people, we are able to better align strategies to meet their needs and build on their strengths. The impact is often felt as a shared and accessible language, now grounded and contextualized with data, that permeates the interactions between staff, leaders, parenting adults, and young people. With approachable data aligned with these frameworks, we are making it easier for practitioners to collaboratively make meaning of the data and use the results to identify practical approaches, promote strengths-based language, improve or develop new programs, create partnerships with young people and families, and strengthen intentionality—all important principles of positive youth development.

How do we approach measure development and what are its benefits?

Search Institute develops measures with practical use in mind. Our approach is centered on the user – practitioners and young people – and grounded in the intended purpose and use of the measures. In order to accomplish this, we use collaborative methods that incorporate the perspectives of young people and others across stages of our measure development journey. That journey contains five key stages.

• The first focuses on the perspectives of the field. What does the literature say about a particular concept? What are the practical needs related to that concept? This broad review of both research and practical insights helps us construct our protocols for stage two.
• In the second stage, we bring in the context of the intended audiences. How do the insights from the broader field align (or not) with their local context and perspectives? How do we contextualize these concepts for use by this group in a manner that aligns with their work and needs?
• The third stage involves us bringing all of these insights together and creating an initial draft of the measures. This draft is the very definition of a draft and used to elicit feedback from the users, including staff, young people, and occasionally parenting adults.
• Stage four is my personal favorite as it provides a mechanism to highlight the experiences and expertise that young people, staff, and parenting adults possess and helps them apply that expertise to the development process. The feedback is so rich, and we typically operate in an iterative manner, allowing us to introduce incremental changes for interviewees to react to. By the end of stage four, we can feel more confident that we have a measurement tool that is accurate in its intent and uses approachable language for young people.
• Stage five is the culmination of this qualitative work where we finalize and pilot version the measurement tool. In this stage, we are focused on understanding how the measures resonate with young people, the types of questions that arise, and the psychometric quality of these pilot measures. After stage five, we will often revisit the prior stages to fill in identified gaps and to ensure that we are in fact meeting the usability needs of our end users.

This process allows us to highlight perspectives of experts who may not be researchers through formalized training, but have the practical and contextual knowledge that is necessary to create a measurement tool that aligns with their work, goals, interests, and use case. We find that building measures around end users’ insights and needs typically leads to easier and more consistent translation between data and action. Ultimately, we want the data to be used to enhance conversations and guide potential avenues for practice change.

We know from our data that young people feel that they aren’t given opportunities to influence decision-making in schools or youth-serving programs. How do our measurements help to address that?

Surveys can certainly be an aspect of youth voice strategy by helping to identify opportunities for young people to participate in and influence the decisions that shape their lives and the lives of their peers. I think the value can be present in two ways—both in terms of the content of the measures and the use and application of results. Most directly, we have developed measures of student voice, belonging, shared power, quality, climate, etc. that can provide organizations with information about the perceived strengths young people demonstrate in these areas.

More indirectly, we see youth surveys as a tool in a larger process of elevating youth voice by using the insights from the results in conversations with young people to not only communicate important findings but also to further shared understanding of the underlying context of a particular finding.

For example, a survey may tell you that young people are struggling with a sense of belonging at your organization, but it does not necessarily tell you the multitude of causes nor the potential ways in which you begin to effect positive change. This combination of quantitative and qualitative data is an example of using youth voice data as a tool and intervention.

Essentially, surveys are an important tool in a mixed methods approach to elevating youth voice. Surveys provide an easy, yet meaningful way for organizations to elicit this information from a census of young people. By standardizing the approach, we are better able to understand how our young people perceive opportunities for youth-level voice, and where these perspectives may differ across participating subgroups of young people. These results help us determine the questions that we still need to ask young people in order to fully understand the ‘so what’ and ‘now what.’ Qualitative methods, such as interviews or focus groups, are effective ways to do this by helping to contextualize the results, determine the meaning behind the insights, and collaborate to determine the goals and plan the actions to meet those goals.

Over the years Search Institute has designed key measurement tools and partnered with organizations by providing valid and reliable data to inform, improve and/or develop their programs, but there is more there to share about our approach than that, correct? 

I am a huge proponent of a well-constructed approach to data collection and use! Most importantly, if we are collecting data, we need to ensure the purpose is clearly communicated and that we use data in a way that honors the perspectives and respects the time of those who participated. To me, that means we need to show participants that we listened and are responding to what they told us. Collecting data for the sake of collecting data can result in  questionable data quality now, and in the future. If young people are not seeing results, it will not feel like a worthwhile activity to them. Data collection on its own simply provides data. The value of data comes from meaning-making activities and responsiveness to the insights provided.

We know that data collection is challenging for numerous reasons (time, logistics, political climate, attrition, etc.), and while we are confident that our measures are valid and reliable, we need a process that translates data into useful and actionable insights.

To help facilitate that process, we are currently developing a platform that combines research and resources with a structured journey to bring data to action. This approach facilitates a collaborative process through which valid and reliable data are collected, shared meaning-making occurs, and data and insights are used to meet identified needs. Like a navigator, this includes four stages: Prepare, Implement, Reflect, and Act. Each stage includes aligned tools and resources intended to build the capacity of organizations to strengthen their approach to data collection and use. I am excited to share more about it as we continue to build out the platform!

What are your favorite outdoor activities? 

I enjoy kayaking, cycling, and tending to my garden in the summer months. Summer is short in the Midwest so I try to make the most of it–even if it is just relaxing in the yard with a book and the grill fired up. If I am outside in the winter, it is likely because I am shoveling (which is NOT a favorite activity). Generally, I use the winter to try out new recipes and plan new travel adventures.

During his more than 10-year tenure at Search Institute, Justin has supported measure development and data utilization efforts across numerous projects including the Relationships for Outcomes Initiative, the Developmental Relationships Enhance Academic Motivation study, Measuring What Matters, Opportunity Reboot, and more. In his role as manager of Survey Services, he works closely with leaders and practitioners at schools, youth programs, coalitions, and other organizations, from the United States and Canada to Singapore, Curacao, Malaysia, and more. He is also actively supporting the adaptation of Search Institute’s measures to fulfill the specific needs of other research projects, both in partnership with academic institutions as well as other organization-driven initiatives.  Learn more about Justin by reading his bio.