Characteristics of tools and techniques
This section outlines characteristics of tools and techniques to assist choice of appropriate tools.
The characteristics below refer to the nature of evidence produced by the tools and the outcomes of their processes. They can be thought of as primary qualities that policy makers may seek depending on their policy making context and what they are trying to achieve, and they are not value judgements.
Other characteristics of tools such as timescales, costs and benefits and expertise required are covered in the tool descriptions. Descriptions for depth, breadth, reach and engagement level characteristics can be found below.
Depth concerns the level of exploration of participants’ thinking and thus the likely richness of participant discussion. It considers how the evidence produced by a tool reflects the deeper values and attitudes of participants, and how those relate to participants’ opinions and behaviours. It encompasses the following considerations:
• Level of insight- Does the tool offer top of the mind responses or deeper insight into participants’’ beliefs or values?
• Development of views- Does the evidence give an insight into how views change with new evidence and information?
• Thought processes- Does the tool give an insight into how participants’ views are formed, uncovering assumptions and reasoning?
• Patterns - Does the evidence infer patterns and conceptual linkages amongst participants’ views?
Low depth tools may:
• Identify participants’ opinions
• Provide a snapshot of participants’ views at a given point in time
Medium depth tools may:
• Give an insight into beliefs that inform participants’ views
• Provide an insight into how participants’ views change with new information
• Highlight some assumptions that influence participants’ views
• Produce data that shows trends across participant responses
High depth tools may:
• Give an insight into fundamental values that underpin participants’ views
• Tease out the assumptions that inform participants’ views, and demonstrate how they relate to them
• Produce data that is suitable for thorough thematic analysis
Breadth concerns the extent to which a tool can cover a range of issues. Whilst depth focuses on the level of exploration of participants’ views, breadth concerns the diversity of discussion and the range of angles and interpretations considered during them. The breadth of views covered may be influenced by the ability of participants to consider and reflect upon others’ views, and by provision of information. It encompasses the following considerations:
• Provision of information- Does the tool allow for discussion with a range of independent topic specialists and/or provide information that could be used to develop the participants’ views?
• Participant discussion – To what extent can participants reflect and build upon on their views through discussion with others?
• Perspective - Does the method challenge assumptions and encourage fresh thinking and innovation?
Low breadth tools may:
• Consult participants in isolation without participant interaction
• Engage participants without provision of information
Medium breadth tools may:
• Focus on a number of pre-determined issues or policy proposals or solutions
• Allow participants to exchange of ideas and views with each other
High breadth tools may:
• Re-frame policy-maker interpretations
• Be cross-cutting across a range of disciplines, policy areas, themes, issues
• Allow policy makers and participants to develop their views through provision of subject information or discussion with specialists
Reach concerns the number of participants that can be engaged by a tool. Higher reach can help to engage a range of participant groups and ensure robustness of findings. It encompasses:
• Scale – How many public views can be understood?
Low reach tools:
• Engage 1-50 participants
Medium reach tools:
• Engage 50-1000 participants
High Reach tools:
• Engage 1000+ participants
Engagement level concerns the extent to which the various participants can influence the process and/or outcomes. Linked to this is the relationship a tool establishes between policy makers and participants, and the skills, knowledge and trust developed as a result. It encompasses the following considerations:
• Influence on process: Does the tool allow participants to influence the issues discussed, agreement on the conclusions from discussions, and the reporting of the results?
• Capacity building: To what extent does the tool allow participants to develop their own views and skills?
• Influence on policy: What influence do the outputs of the process have on resultant policy?
Low engagement level tools may:
• Increase participants’ understanding of issues
• Consult with participants to provide one-way feedback to policy makers
• Be framed by policy makers, with limited scope for participants to guide discussions
Medium engagement level tools may:
• Create a participant community to facilitate exchange of views and skills between participants
• Have a mixed format, with policy makers framing some elements and participants’ others
High engagement level tools may:
• Allow policy makers to form a partnership with participants, characterised by a continued and collaborative engagement
• Delegate some decision-making power to participants
• Enable participants to record and report the results of the discussions to decision makers
• Have an open format, allowing participants to frame discussions and guide the direction of conversation