Dozens of online tools make it easy to create a survey these days. These tools are great resources. But a good survey platform doesn’t make it easy to create a good survey. A good survey depends on asking really good questions—particularly when you’re asking about abstract or subjective experiences, skills, or priorities.

For the past three years, Search Institute has been working with colleagues at West Virginia University and the University of Rochester to develop new measures of young people’s civic development through a major project supported by the John Templeton Foundation titled the Roots of Engaged Citizenship Project. In this effort, we spent a lot of time focused on developing and testing good questions.

So, what makes a question good?

1. A good question is clear.
This is harder than it sounds. A survey question may mean something completely different to an 11-year-old in Palo Alto, California, than to an 18-year-old in rural West Virginia or a 14-year-old in Malawi.

The best questions are simple yet concrete, and use language that is accessible to all potential participants.

We try to avoid questions that

• are too long,
• are too vague,
• are worded in the opposite direction of the other questions,
• are too similar to other questions,
• are idiomatic to a particular language or culture, and
• include multiple ideas.

To help ensure question clarity, we often conduct one-on-one interviews with youth or other survey respondents during the survey development process. We ask students to “think aloud” as they read and respond to possible survey items. This helps identify possible readability, comprehension, and interpretation issues.

In the Roots of Engaged Citizenship Project, for example, interviews revealed that few middle school students knew that “e.g.” was an abbreviation for “for example.” Interviews have also shown us that middle school students struggle to define the word purpose; prompting us to make small but important revisions to survey questions.

2. A good question reflects young people’s abilities and experiences.
You need to know your audience! How do the phenomena you are interested in measuring play out in their lives? What personal and structural barriers get in the way? What are the descriptive words and examples youth use? How do we best reflect the range of experiences young people have across age, culture, and other differences?

In recent qualitative interviews, we asked youth to articulate definitions of several character strengths. When asked to define respectful, youth offered a range of ideas from using “good manners” to the Golden Rule. The word choices and examples offered by these youth not only illuminated kids’ understanding of these abstract ideas, but they also became the concrete basis of several quantitative survey questions.

Drawing on youths’ own words and experiences makes the questions more authentic, which yields better results in a survey. It also honors the voices of the young people we work with.

3. A good question is actionable.
Good questions do more than produce data; they provide insight and direction for action. This is when data are most powerful.

Questions tend to be most actionable when they examine specific attitudes and behaviors that are changeable or malleable. They are specific expressions of more abstract concepts.

For example, personal responsibility is a character strength that’s seen as very important by educators, parents, employers, and others who work with youth. But it’s abstract and, on the surface, not very actionable. But if you break it down into concrete behaviors, it’s something you can work on. In our study, one of the questions we used to make personal responsibility more concrete was, “If I do something wrong, I take responsibility for my actions.” When we combine young people’s responses to this question and others that pick up other dimensions of being responsible, we approximate the more abstract idea.

Concrete data makes it possible for us to develop strategies and action steps to tap existing strengths and build new strengths that advance our priorities. We can also use the same measures to track progress and improvement over time.

Creating good questions, then, is only the first step. In the end, the effort pays off in the ways data are used to enhance the lives of young people and tap their strengths and commitments to create better schools, communities, and nations.

We’ve developed a measurement toolkit to help researchers ask good questions about some key areas of civic development and character strengths. Our hope is that these questions will stimulate innovative and effective action to nurture and unleash the commitments, skills, and perspectives that young people bring to make our world a better place.

For More Information

• Roots of Engaged Citizenship Project
• Search Institute Survey Services
• CIRCLE Guest Blog by Dr. Amy Syvertsen on the challenges and opportunities inherent in designing developmental civic and character strength measures.