Widgetized Section

Go to Admin » Appearance » Widgets » and move Gabfire Widget: Social into that MastheadOverlay zone

On the Path to Practitioner Relevant Questions and Answers

The views expressed are those of the author and do not necessarily reflect the views of ASPA as an organization.

By The Office of Public Policy Outreach at VCU’s Wilder School
July 13, 2018

In April, only two months removed from the Parkland High School tragedy, our office shared results of our Summer 2017 Wilder School Public Policy Poll with state legislators, public safety officers and educators at a School Safety Forum organized by the Wilder School’s Center for Public Policy and Virginia’s Secretary of Public Safety and Homeland Security, Brian Moran.

Our poll asked a representative sample of Virginians whether they thought schools in their community were “very safe, safe, not very safe or not at all safe.” Seventy-nine percent selected either very safe or safe, a finding that is statistically unchanged over the past five years.

In the context of important national soul-searching, the results were relevant, giving gathered practitioners a baseline that their ongoing efforts to maintain safe schools were judged successful by most of the public. The forum, in turn, produced lively conversation on the best practices that were producing safe contexts and on what further improvements could be made to them.

Conducting the Wilder School Public Policy Poll in Virginia sends us on a twice-yearly search for such relevant questions and ways to share the answers with policy makers, the academy and the general public. In this article we share some of our process and considerations when creating poll questions, hoping that our reflections will be helpful to students, academics and practitioners as they create questions for their own research.

Our takeaway advice is this — relevant questions come from staying focused on relevance as a goal, talking to a range of stakeholders, maintaining good methods and partnering with stakeholders to share the results widely.

A focus on relevance

In their new book Why Research Methods Matter: Essential Skills for Decision Making, ASPA Past President and Wilder School Interim Dean Susan T. Gooden, along with public administration scholar RaJade M. Berry-James of NC State University, suggest some questions to consider when designing a questionnaire, including:

  • “What is it that you need to know?”
  • “Why do I need to know it?”, and
  • “What will happen as a result of this survey?”

In a survey of public opinion, broad policy relevance is our key touchstone that informs our “what” and “why” as well as our dissemination strategy.

Relevance also sometimes means asking the same basic questions, even though they seem old to us as researchers, because they inform recurring debates where trend lines can be important. This allows us to monitor if and how public opinion has changed over time, and identify potential causes and patterns.

Reaching out to stakeholders

Because relevance to policymakers is our touchstone and research indicates that engaging non-academics in the process of survey creation increases the likelihood that our findings will be useful in policy conversations, we engage multiple stakeholders in the process of creating questions.

A typical poll development cycle includes meetings with key state cabinet level officials, and outreach to relevant state legislative committee chairs. Another stakeholder group consists of policy-focused researchers within our university, for example, professors who have been involved in our Translational Research Fellows program. The process is time intensive but gives us insight into what information policymakers are seeking as they create future policies. Our ideal is a set of questions that are useful to key policymakers, to our academic colleagues own research, and to the general community.

Ensuring methodological rigor

As we engage with stakeholders, we are also always clear that final question selection and wording of questions stays within our office. In their book, Gooden and Berry-James stress the importance of knowing who will respond to the survey, which will in turn help to ensure that the results accurately reflect the opinions of the target population. Stakeholders, whether practitioners or academics, may provide us with suggested questions that are too lengthy or need to be rephrased according to the following best practices:

  • avoiding jargon and terms the general public may not know;
  • eliminating leading or double-barreled questions; and,
  • keeping in mind that our target population includes a range of ages and education levels.

To ensure the basic validity of results as well as a sufficient completion rate, holding the line on methodology and succinct phrasing is important. In our case, this also extends to contracting with a polling company that uses industry standard best practices in sampling to make the calls and record responses.

Partnering on dissemination of findings

After analyzing the results the final phase begins – getting the insights in front of people. The first channel of dissemination is the media – we release results in thematic chunks over the course of two to three weeks. When possible we align releases with relevant events, but we often find that some of the same stakeholders mentioned above help identify other channels that are equally impactful for the policy process. For example, education related items from our poll are presented each year to the House Education Committee.

Through this poll and other efforts, we are committed to advancing research that informs public opinion, stimulates conversations, and builds stronger communities. As Gooden and Berry-James write, “research matters.” Our experience is that it matters more, when not only more people hear the answers, but are also involved in developing the questions.

Author: The Office of Public Policy Outreach is part of the Center for Public Policy at Virginia Commonwealth University’s L. Douglas Wilder School of Government and Public Affairs. The office prepares VCU faculty for effective interactions with state and local lawmakers and helps generate and translate academic knowledge into successful public policies for the commonwealth.

Email address: [email protected]

Twitter handle: @OPPOatVCU

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 5.00 out of 5)

Leave a Reply

Your email address will not be published. Required fields are marked *