Collaborate

A framework for thinking about the social nature of evidence use

Efforts to improve the use of evidence in practice and policy internationally have involved different approaches, one of which has been ‘relationship models’ (Best & Holmes, 2010). Relationship models focus on knowledge sharing through partnerships and networks among individuals who have common interests. Through these approaches, the effective use of evidence is facilitated by relationships that are underpinned by a clear commitment to collaboration and shared learning.

One way to think about collaboration between professionals including educators is in terms of a continuum (Himmelman, 2002). At one end of the continuum sits ‘networking’ or simple information exchange, while at the other end sits ‘collaboration’, involving not just information sharing but also practice change, resource sharing, capacity building and shared purpose. In between these two extremes are the intermediary processes of ‘co-ordinating’ and ‘co-operating’ (see Figure 2). Where the work of a group of educators might sit on such a continuum is shaped by a variety of factors, such as the extent to which they share goals, are committed to shared tasks, identify with the team and feel the need to be included, are dependent on each other to deliver shared tasks and goals, and perceive their achievements as shared accomplishments (e.g., Little, 1990; Vangrieken et al., 2015).

This way of thinking about collaboration is helpful because, as will become clear in the following sections, the use of evidence by educators seems to involve movement along the continuum, depending on the task at hand. For example, ‘finding’ evidence seems to involve more light-touch, informal, networking-type processes, whereas ‘analysing’ evidence and then ‘implementing’ evidence seems to require more involved and structured co-operating and collaborating-type processes. Thinking about collaboration as a continuum therefore can help leaders craft targeted interventions to support educators’ improved use of evidence in practice.

Finding, locating, identifying

For educators to use relevant and appropriate evidence in practice, they first need to be able to find and access it. Two thirds (65%, n=318) of educators involved in Q Project’s survey indicated that, ‘when confronted with a new problem or decision, they look for research that may be relevant’. The same proportion (65%, n=318) indicated they ‘know where to find relevant research that may help to inform their teaching practices’.

“I [find and] read research and often cross-examine it with other experts. I reflect on [the research and] my practice and consider what I can change or develop… often sharing [views] with other colleagues.” – Middle leader, Catholic Primary School, Queensland

When describing the myriad of ways in which they sourced evidence, Q Project respondents used language such as “networking”, “sharing different sources with others”, “seeking advice”, “relying on others’ previous use and experience”, and “taking recommendations from colleagues”. These different approaches were reflected in educators’ responses when they were asked what had influenced them to use certain information over the past 12 months. Overall, one third of educators ranked ‘word of mouth or recommendation from others’ in their top three influences (34%, n=120) (see Figure 3). Social processes also influenced the use of research in particular, with more than one quarter of educators ranking ‘the research was recommended by colleagues and/or school leaders’ as one of their top three reasons for using it (27%, n=93).

Q Project respondents described their social processes of finding and accessing evidence in ways that would seem to locate these at the left of the collaboration continuum described earlier (see Figure 2). Their interactions seemed largely motivated by information and resource exchange, sharing of experiences and ideas, and fulfilment of others’ requests or needs.

This finding represents an important insight for school and system level leaders. Educators rely on opportunities to talk or connect with colleagues to find and access evidence. They often prefer to be recommended evidence by their colleagues, underpinned by the understanding that their colleagues would only recommend evidence they found to be useful or relevant to practice themselves (Williams & Coles, 2007).

Using the idea of a collaboration continuum, the Q Project’s findings suggest that these social interactions do not need to be overly formal or strongly organised for educators to gain value from them (Bush, 2000). To find and access relevant evidence, teachers and school leaders need to have access to different social, professional and school networks, both internal and external to their own school environment, as well as formal and informal.

“We visited [or looked to] other schools to see their programs…[always] connecting research to [our] context [and] engaging with the research in discussions with [our] colleagues and other leaders.” – Senior Leader, Catholic Secondary School, Victoria

We came across the research through “networking with other schools [and seeking out] a variety of sources [including] university and professional recommendations [and] external agencies who are elite in their niche.” – Teacher, Government Secondary School, New South Wales

Attending professional development events is a form of networking. These opportunities allow educators to engage with new evidence, including evidence that has already been successfully trialled or implemented in similar contexts. Within their own school context, educators can socialise this evidence with their colleagues to establish a school-wide understanding of it and how it could be implemented in their context.

Research was identified through “professional development for all teachers, [with] collaborative discussion [then with] middle leaders, [whose] feedback [was] sought [about the research] and implemented with their agreement.” – Senior Leader, Independent P-12 School, New South Wales

“Teachers refer to notes and research from professional learning, [as well as] the theories that underpin their work as they work together … When I hear my teachers talking about their work using the [research-based] language and sharing what they have been looking at, I know they are using research evidence well.” – Senior Leader, Government Primary School, South Australia

Establishing school-university partnerships can also provide ways for educators to locate and engage with relevant evidence (e.g., Godfrey, 2016). University partnerships can help to supplement and enhance educators’ evidence-related skills and capacities (Judkins, Oliver, McCrone, & Inniss, 2014) via teacher education opportunities, professional development, ‘critical friend’ advice and information sharing, as well as internships (OECD, 2013). Close to two thirds of the educators (62%, n=303) involved in the Q Project’s survey agreed they would like ‘opportunities to work with researchers to help with their own learning’.

Our school has “critical friend access via [a] university [that supports us] develop[ing] an evidence-based approach to change [including finding and] implementing the research with enough fidelity so that the same results can be expected, but with enough flexibility so that it meets the needs of [our] context.” – Middle Leader, Catholic Secondary School, Victoria

Social media networks can play an important role in enabling educators to find and share evidence (see Professional Learning Networks below). They can facilitate equitable access to evidence for an increasing number of educators who, for example, live in regional and remote areas and cannot easily access professional learning experiences or incidental networking opportunities, which often act as sources of new and different evidence. Educators around the world are able to connect with each other through these platforms to exchange insights and information.

However, these platforms can present challenges for educators when sourcing evidence. Whilst Q Project educators acknowledged their value, they indicated preferences for accessing more immediate, personal and known networks when sourcing evidence. Their responses highlighted the risks of engaging with content shared online without consideration of its relevance and reliability.

Poor research use means using “research that ‘fits’ a particular trend or fad that doesn’t have any evidence of improving student learning. Something [found] on Twitter that has not been looked at critically. There is a lot of self-promotion on social networking platforms like this [although] some good things as well.” – Senior Leader, Government Primary School, New South Wales

Analysing, interpreting, deciding

Just as finding and identifying relevant evidence involves social processes, interpreting and making sense of it can also involve collaboration. Working with colleagues can help educators decide whether the evidence they have found is robust and reliable and whether it is relevant to their school’s context.

We assess research for use by “making evidence-informed decisions based on quality data sources and engaging in dialogue with colleagues to make informed choices.” – Senior Leader, Government Primary School, New South Wales

As part of the Q Project’s survey, educators were asked how they assessed the quality of evidence when deciding on approaches to improve student outcomes. Overall, educators cited ‘information sources being backed by academic research’ (64%, n=267) as one of their top three ways of assessing quality. Highlighting the importance of social relationships, almost two in five educators (38%, n=141) indicated that they relied on ‘critique of the information with school colleagues’, with a similar proportion selecting ‘word of mouth or recommendations from others’ (35%, n=123) as one of their top three assessment methods.

With regards to research specifically, teachers reported lower levels of confidence in their research-related skills and capacities when compared with school leaders. Approximately two in five teachers indicated they did not feel ‘confident in analysing and interpreting research for my own teaching context’ (36%, n=102), with approximately half expressing a lack of ‘confidence in how to judge the quality of research’ (51%, n=143). These findings suggest a role for leaders to facilitate more consultative processes so as to help teachers unpack the meaning and usefulness of evidence in their own context, alongside their colleagues and peers. These collaborative processes can also encourage information sharing and help to address emerging questions.

When deciding on research for use, “a range of teachers [are] involved, if not the whole school, and teachers and students [are] empowered to give feedback on the [research decision-making] process.” – Middle Leader, Catholic Secondary School, Victoria

On the whole, educators involved in the Q Project’s survey expressed positive perceptions of the opportunities to engage with evidence within their schools. More than two thirds (69%, n=341) indicated that their school has ‘informal processes’ to help staff engage critically with information sources, while a slightly smaller proportion indicated their school has ‘formal processes’ to support this (64%, n=315). When describing their school environments, Q Project respondents emphasised the importance of social processes when assessing and interpreting evidence for contextual relevance. They used language such as “being consulted”, “teachers having an equal say”, “everyone being involved”, “sharing” and “having a whole-of-school approach and understanding”.

“Whole-of-school improvements are introduced [at our school] using research as a basis of evidence as to ‘why’ things are changing. This [consultation process] is very collaborative with many people involved.” – Senior Leader, Catholic Secondary School, Victoria

“Research evidence from [both] internal and external sources should be studied and examined carefully in a collaborative way so that it is not one person’s interpretation. Colleagues work together to critically review research…seeking external validation [about the research] to avoid ‘tunnel vision’.” – Senior Leader, Independent Primary School, Queensland }

They also appeared to value professional learning communities (PLC) and collaborative learning as ways in which to come together to critically engage with evidence, with the majority of Q Project participating teachers (84%, n=236) and an even greater proportion of school leaders (93%, n=148) believing their school facilitated these structured opportunities.

PLCs call for collaboration, sharing and ongoing critical interrogation of teaching practices in line with professional standards. They are often learning-oriented and can help to facilitate teacher growth. PLCs also provide educators with the opportunity to regroup following the implementation of evidence-informed initiatives to discuss any benefits that have occurred as well as any issues. PLCs are a significant resource as teachers are more likely to collect and use evidence systematically when working as a group (e.g., Dimmock, 2019; Supovitz, 2015). Professional collaborative activities between educators, including PLCs, have also been found to contribute to improved student outcomes (Schleifer, Rinehart, & Yanisch, 2017).

Educators involved in the Q Project’s survey also highlighted the pitfalls of assessing, interpreting and making decisions about evidence within their schools without proper collaboration and collective buy-in from staff – effectively illustrating what not to do. Their experiences highlight the risks associated with centralised, non-consultative decision-making and the use of evidence without a proper shared understanding of the ‘why’ motivating these decisions. These comments also underscore the importance of properly exploring the relevance of evidence-informed practices to the school’s specific context and only adapting and implementing relevant practices.

Poor research decision-making means “one person/group implementing research that they thought was correct without consultation, collaboration or without dissecting its reliability and suitability to our college.” – Teacher, Independent P-12 School, New South Wales

Poor research use involves “the introduction of a [research-based] initiative where staff are told to do it. [There is] no room for adaption…no reflection or collation of [different views] to ascertain the impact. [There is] lots of jargon and no collaboration.” – Middle Leader, Government Secondary School, Victoria

Q Project respondents described their social processes of assessing, interpreting and deciding on specific evidence for use in ways that would seem to locate them further along the collaboration continuum described earlier. Their descriptions suggested a need for collaboration (see Figure 2) including shared and common understandings of the evidence, inclusivity and joint decision-making. These meaningful interactions can foster greater trust among educators and openness to change (Moolenaar, Sleegers, & Daly, 2012).

Experimenting, applying, reflecting

Once educators have together made sense of the evidence and decided what it means for their learning environment, they can then begin using it in practice. Collaborative processes again can play a role, with teachers and leaders working together to experiment with different evidence, implement and monitor its effect on their practice and student learning, and then discuss and reflect on its impact.

Quality research use involves “using the research carefully to make your own practices better, [but] only after discussion with colleagues and school leaders.” – Teacher Aide, Catholic Primary School, Victoria

The majority of educators involved in the Q Project’s survey (70%, n=342) indicated they had ‘used research to inform their practice in the last 12 months’. These educators indicated they had used research directly and indirectly in their practice, both by themselves and in groups. ‘Discussions of best practice with colleagues’ was the strongest reason for using research (76%, n=260), highlighting the importance of collaboration and information sharing between colleagues in the integration of research into practice.

Once educators have developed a shared understanding of what the evidence means and how it could be applied in their context, the processes of experimenting, trialling and implementation begin. Again, collaboration is key to the effectiveness of these processes in practice. A good starting point is to establish shared and well-defined objectives that articulate what school teams are aiming to gain from the implementation of evidence. Co-constructing problem resolution approaches are also important for group buy-in and uniform troubleshooting of issues as they arise. Structured, ongoing opportunities to collaboratively reflect on impacts post-implementation also help educators to decide together how best to proceed and sustain improved practice (Sharples, Albers, & Fraser, 2019).

In a research-supported culture “there would be ongoing opportunities for professional discussions, collaborative practice, [collective] feedback and reflection…[a] common language across the school [regarding research use].” – Teacher, Government Primary School, Queensland

Using research well means “staff would be unpacking the research together to gain a common understanding. [They] would be working together to plan and implement the research evidence effectively in the classrooms.” – Senior Leader, Government Primary School, Victoria

When describing the ways in which they used or implemented evidence, Q Project respondents used language such as “collegial discussion”, “group debate”, “team implementation”, “going on one improvement journey together”, “colleagues working together”, “openly sharing thoughts and ideas for improvement with others”, and “collaborative practice, feedback and reflection”. Their responses suggested a need for collaborating-type processes when implementing and using evidence in practice, locating them at the far right of the continuum described earlier (see Figure 2).

Effective research implementation involves “subsequent debriefing along with colleagues after a suitable trial period and trial of other methods to gather evidence of what works or not.” – Teacher, Catholic Secondary School, New South Wales

“Teachers were choosing research [applicable to] their own classrooms, [then] framing their own theories of action and investigating their own problems [before] then collaborating with others to explore those problems through practice.” – Senior Leader, Government Secondary School, Victoria

Collaborative teacher inquiry is a helpful example of a collaborating-type process whereby educators work together, sometimes as members of a PLC, to identify common challenges, interrogate evidence and test practice applications. This approach is based on the understanding that inquiry is a cyclical process that can help to foster ongoing collaboration between educators. In a collaborative inquiry cycle, educators:

• commit to a common goal or focus,
• develop a plan for action,
• carry out the plan while collecting and analysing research, evidence and data, and
• determine the implications of their findings as they relate to their individual practice and learning environment (Butler & Schnellert, 2012).

This process allows educators to incorporate different evidence types into their own practice and context, creating opportunities for collective dialogue, observation and reflection. This type of collaboration empowers educators to jointly trial evidence in their practice and builds shared commitment to and responsibility for improved evidence use (Timperley, Kaser, & Halbert, 2014). This meaningful and sustained collaboration is crucial to affecting sustainable change within learning environments (Levine & Marcus, 2010). Q Project findings suggest that organisational supports including trustworthy leadership, time and resource investment, and defined group roles and aims are crucial to facilitating this level of collaboration in a sustained way (Rickinson et al., 2020).

Poor research use means “only limited [or] one-sided research [was sourced}, no consultation [and] no debate, [resulting in] no positive impact on student outcomes.” – Senior Leader, Catholic Secondary School, Victoria}

Research use is poor when “only a few staff, or pockets of staff, [are] talking and using [the] research-informed practices in their classroom, but [even they are] not really able to articulate what they [are] doing or why they [are] doing it.” – Senior Leader, Government Secondary School, South Australia

Overall, collaborative social processes can help educators determine how evidence can best be implemented within their own contexts. Ongoing monitoring and joint reflection can help teachers and school leaders understand what has and has not worked in their school, and together they can work towards improving student outcomes.

What do these findings mean?

How can teachers and school leaders work better together, as well as with others, to improve evidence use?

Social processes can help educators to locate evidence, understand what it means for their early learning centre or school, and make decisions regarding implementation. At the heart of these processes are strong and meaningful relationships between educators, leaders and the broader learning environment community.

Interactions between educators often go beyond just sharing of resources, anecdotes and experiences. The Q Project’s findings demonstrate that educators are engaging with evidence in different ways, depending on the task at hand, and have different needs and expectations regarding the depth of collaboration required. Effective collaboration requires strong leadership support and facilitation, especially when it comes to analysing, experimenting with and implementing evidence in practice. Schools and system leaders can support effective collaboration by providing opportunities for educators to come together and share and interrogate evidence. By focusing on understanding evidence and the impact it can have on learners, educators can develop a shared understanding of their context-specific nuances and how the evidence can be tailored for implementation in their own context.

Thinking about evidence use as a stepped, social process can help educators engage with appropriate evidence that can strengthen their practice. Collaborating with colleagues to locate, interrogate and implement evidence represents a great opportunity for educators to learn from and with each other.

References

Best, A., & Holmes, B. (2010). Systems thinking, knowledge and action: Towards better models and methods. Evidence & Policy, 6(2), 145-159.

Brown, C., Daly, A., & Liou, Y-H. (2016). Improving trust, improving schools: Findings from a social network analysis of 43 primary schools in England. Journal of Professional Capital and Community, 1(1), 69-91.

Brown, C., Zhang, D., Xu, N., & Corbett, S. (2018). Exploring the impact of social relationships on teachers’ use of research: A regression analysis of 389 teachers in England. International Journal of Educational Research, 89, 36-46.

Bush, J. (2000). "Here's what I did:" making "sharing" research meaningful for teaching. English Education, 32(2), 86-106.

Butler, D. L., & Schnellert, L. (2012). Collaborative inquiry in teacher professional development. Teaching and Teacher Education, 28(8), 1206-1220.

Coldwell, M., Greany, T., Higgins, S., Brown, C., Maxwell, B., Stiell, B., . . . Burns, H. (2017). Evidence-informed teaching: an evaluation of progress in England. London: Department for Education. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/625007/Evidence-informed_teaching_-_an_evaluation_of_progress_in_England.pdf.

Daly, A. J., Moolenaar, N. M., Bolivar, J. M., & Burke, P. (2010). Relationships in reform: The role of teachers' social networks. Journal of Educational Administration, 48(3), 359-391.

Darling-Hammond, L., Burns, D., Campbell, C., Goodwin, A. L., Hammerness, E. L., McIntyre, A., Sato, M., & Zeichner, K. (2017). Empowered educators: How high-performing systems shape teaching quality around the world. San Francisco: Jossey-Bass.

Dimmock, C. (2019). Leading research-informed practice in schools. In D. Godfrey, & C. Brown (Eds.), An eco-system for research-engaged schools: Reforming education through research, (pp. 56-72). London: Routledge.

Godfrey, D. (2016). Leadership of schools as research-led organisations in the English educational environment: Cultivating a research-engaged school culture. Educational Management Administration & Leadership, 44(2), 301-321.

Godfrey, D., & Brown, C. (Eds.). (2019). An ecosystem for research-engaged schools: Reforming education through research. London: Routledge.

Henrick, E. C., Cobb, P., Penuel, W. R., Jackson, K., & Clark, T. (2017). Assessing research-practice partnerships: Five dimensions of effectiveness. New York, NY: William T. Grant Foundation.

Himmelman, A. T. (2002). Collaboration for a change: Definitions, decision‐making models, roles, and collaboration process guide. Minneapolis, MN: Himmelman Consulting.

Judkins, M., Oliver, S., McCrone, T., & Inniss, M. (2014). Teachers' use of research evidence: a case study of United Learning schools. Slough: National Foundation for Educational Research. Retrieved from https://www.nfer.ac.uk/publications/IMUL01/IMUL01.pdf.

Levine, A. H., & Marcus, A. S. (2010). How the structure and focus of teachers' collaborative activities facilitate and constrain teacher learning. Teaching and Teacher Education, 26, 389-398.

Little, J. W. (1990). The persistence of privacy: Autonomy and initiative in teachers' professional relations. Teachers College Record, 91, 509-536.

Masters, G. (2018). The role of evidence in teaching and learning. Melbourne: Australian Council for Educational Research. Retrieved from https://research.acer.edu.au/cgi/viewcontent.cgi?article=1335&context=research_conference#:~:text=First%2C%20effective%20teachers%20use%20quality,an%20appropriate%20level%20of%20challenge.

Moolenaar, N. M., Sleegers, P. J. C., & Daly, A. J. (2012). Teaming up: Linking collaboration networks, collective efficacy, and student achievement. Teaching and Teacher Education, 28, 251-262.

OECD. (2013). Innovative learning environments. Educational Research and Innovation. OECD Publishing. Retrieved from https://read.oecd-ilibrary.org/education/innovative-learning-environments_9789264203488-en#page1.

Rickinson, M., Cirkony, C., Walsh, L., Gleeson, J., & Salisbury, M. (2020). Towards quality use of research evidence in education: Discussion paper. Melbourne: Monash University. Retrieved from https://www.monash.edu/education/research/projects/qproject/publications.

Rickinson, M., & Edwards, A. (2021). The relational features of evidence use. Cambridge Journal of Education, doi: 10.1080/0305764X.2020.1865877.

Schleifer, D., Rinehart, C., & Yanisch, T. (2017). Teacher collaboration in perspective: a guide to research. Brooklyn: Public Agenda.

Sharples, J., Albers, B., & Fraser, S. (2019). Putting evidence to work: A school's guide to implementation. Sydney: Evidence for Learning. Retrieved from https://e4l.org.au/assets/Guidance-Reports/Implementation/Guidance-Report-Putting-evidence-to-work-a-schools-guide-to-implementation.pdf.

Supovitz, J. (2015). Teacher data use for improving teaching and learning. In C. D. Brown (Ed.), Leading the use of research & evidence in schools, (pp. 117-125). London: Institute of Education Press.

Timperley, H., Kaser, L., & Halbert, J. (2014). A framework for transforming learning in schools: innovation and the spiral of inquiry. Melbourne: Centre for Strategic Education.

Trust, T. (2012). Professional learning networks designed for teacher learning. Journal of Digital Learning in Teacher Education, 28(4), 133-138.

Vangrieken, K., Dochy, F., Raes, E., & Kyndt, E. (2015). Teacher collaboration: a systematic review. Educational Research Review, 15, 17-40.

Williams, D., & Coles, L. (2007). Evidence-based practice in teaching: An information perspective. Journal of Documentation, 63(6), 812-835.