WHAT IS THE PRIORITY III STUDY?
The Priority III study identified research priorities about how to improve how we plan, do and share the results of rapid reviews in the context of healthcare.
The study used a priority setting partnership (PSP) based on the methods of the James Lind Alliance (JLA). The JLA (UK) brings patients, carers and healthcare professionals together in Priority Setting Partnerships. These partnerships identify and prioritise unanswered questions about healthcare that the public, carers and professionals jointly agree are the most important.
The PRioRiTy I and PRioRiTy II PSPs were focussed on uncertainties about how we do research and used a modified JLA approach. As Priority III project focuses on uncertainties of rapid reviews, we also use this modified approach.
We encourage research funders to incorporate the priorities that have been identified into their research strategy. We encourage researchers and research teams to use the priorities as an agenda for their work to ensure that work is being carried out to answer the questions that have been agreed by the public, carers and professionals as being most important.
The Priority III study has been funded by the Health Research Board (HRB, Ireland) and the Health and Social Care division of the Public Health Agency (PHA) of Northern Ireland within Evidence Synthesis Ireland (ESI).
A rapid review is a type of evidence synthesis that brings together and summarises information from lots of different research studies to produce evidence for people such as the public, researchers, policy makers and funders in a systematic, resource-efficient manner. This is done by speeding up the ways we plan, do and/or share the results of conventional structured (systematic) reviews, by simplifying or omitting a variety of methods that should be clearly defined by the authors.
The rapid reviews that we refer to in this study are done in relation to healthcare.
For more information on what evidence synthesis, rapid reviews and this project, please watch this short video by clicking HERE.
The study is governed by a Steering Group that represents the public, researchers, health professionals and experts in rapid reviews.
|Beccy Maeso||James Lind Alliance (JLA)||Beccy leads the James Lind Alliance (JLA) based at the Wessex Institute, University of Southampton. The JLA brings patients, carers and clinicians together in Priority Setting Partnerships (PSPs) to identify and prioritise the Top 10 uncertainties, or unanswered questions for research. We provide advice and guidance to groups wanting to initiate a PSP, and working with our JLA Advisers, we oversee the PSP method as set out in the JLA Guidebook, as well as undertaking a range of engagement activities with the research community. The JLA represents a distinctive and influential model, capturing the voices of patients and carers, enabling them to work as equal partners with healthcare professionals to identify and prioritise evidence uncertainties and influence the future research agenda.|
|Caroline Whiting||James Lind Alliance (JLA)||Caroline works in the team that coordinates the James Lind Alliance (JLA) at the Wessex Institute, based at the University of Southampton. Caroline supports people in starting new JLA Priority Setting Partnerships (PSPs) and works closely with the rest of the JLA team to keep on improving what they do.|
|Medical Research Council National Institutes of Health Research (MRC NIHR) Trials Methodology Research Partnership (TMRP)||A former teacher, Derek was treated successfully for cancer in 1995. He is an advocate for involving and engaging patients and the public in research. Derek is PPIE Lead for the MRC NIHR Trials Methodology Research Partnership and a member of the Advisory Group for Evidence Synthesis Ireland|
|Andrew Worrall is a retired teacher and educationalist. He has been involved with the NIHR as a lay member of a local research network board and a funding advisory board. He is a co-applicant on a current surgical trial. He is a writer and gardener and involved in music education, chairing a local charity and regional music education partnership board.|
|Jim Elliott||NIHR (National Institutes of Health Research)||Jim provides leadership to the Health Research Authority on the involvement of patients and the public in health research [part time] as part of his work as an advocate for patients in research. He has been a carer for close family with cancer and other health conditions for many years.|
|Maureen Smith||Cochrane Consumers Executive||Maureen Smith is the Chair of the Cochrane Consumer Executive and is involved as a patient partner in several research projects in Canada and internationally. Her diagnosis with a rare, chronic endocrine disease in childhood and many healthcare experiences have motivated her to serve on various committees and advisory groups in areas such as health technology assessment, rare diseases, patient involvement in the development of core outcome sets, and innovative clinical trials for rare diseases.|
|Theresa Tierney||Primary Care PPI Group, HRB Primary Care Clinical Trials Network Ireland||Theresa Tierney is a website developer, ecommerce manager, project manager, natural health blog editor and digital marketing consultant. She works with small to medium-sized businesses and organisations to most effectively communicate and market through a wide range of online mediums. She has been the project manager responsible for coordinating teams of diverse stakeholders from programmers and designers to business owners through to final launch. She has particular interest in the areas of health and environment.|
|Declan Devane||NUI Galway/Evidence Synthesis Ireland/Cochrane Ireland||Professor Declan Devane is the Director of Evidence Synthesis Ireland and Director of Cochrane Ireland. He holds the Chair in Midwifery and is Deputy Dean of the College of Medicine, Nursing and Health Sciences at NUI Galway. He is also Scientific Director of the HRB-Trials Methodology Research Network, and Principal Investigator with the INFANT – Irish Centre for Fetal and Neonatal Translational Research.
Declan trained as a nurse and a midwife, meandered (with the help of opportunity, interest and luck) his way into trial methodology and evidence synthesis and his work now focusses on a blend across midwifery (and broader maternity care), randomised trials and how they are done and synthesising evidence.
|Claire Beecher||NUI Galway/Evidence Synthesis Ireland||Dr Claire Beecher is a postdoctoral researcher at Evidence Synthesis Ireland (ESI) and the Health Research Board Trials Methodology Research Network (HRB TMRN). She holds Bachelor’s degrees in both Business Studies & Event Management (Limerick Institute of Technology) and Midwifery Science (National University of Ireland, Galway). NUI Galway awarded Claire’s PhD thesis.|
|Elaine Toomey||University of Limerick||Dr Elaine Toomey is a Lecturer in the School of Allied Health in the University of Limerick. She is a Research Associate of Evidence Synthesis Ireland and Cochrane Ireland and a member of the Health Behaviour Change Research Group (National University of Ireland Galway). Until April 2020, Elaine was Associate Director of Cochrane Ireland within Evidence Synthesis Ireland and led the implementation of the Evidence Synthesis Ireland Fellowship Scheme. Elaine’s research primarily focuses on methods used in the development, evaluation and implementation of health behaviour change interventions, particularly about chronic disease prevention and management. She has specific expertise in evidence synthesis, rapid reviews, implementation science/knowledge translation, process evaluation and exploring the fidelity/adaptation of behaviour change interventions.|
|Bronagh Blackwood||Queen’s University Belfast||Dr Bronagh Blackwood is a Professor in Critical Care at the Wellcome-Wolfson Institute for Experimental Medicine, Queen’s University Belfast. She is the President of the European federation of Critical Care Nursing associations and a Content Editor for the Cochrane Emergency and Critical Care Group.|
|Teresa Maguire||Health Research Board||Teresa has recently returned to the Health Research Board following five years as Head of Research Services and Policy at the Department of Health in Ireland. Building on her prior experience of supporting Cochrane in Ireland, clinical trials and evidence synthesis for guideline development, she managed an annual programme where rapid evidence reviews were prioritised and conducted to support decision making in key policy areas.|
|Melissa Kampman||Health Canada||Dr. Melissa Kampman is a senior epidemiologist at Health Canada. She holds an MSc in Chemical and Environmental Toxicology and a PhD in Population Health. Her training is in pharmacoepidemiology and pharmacovigilance. Her main areas of interest are population health, study design methodology for pharmacoepidemiologic research, drug safety and effectiveness, and regulatory policy and decision-making|
|Benny Ling||Health Canada||Benny Ling is a scientific reviewer for Health Canada. He has over fifteen years of regulatory toxicology experience in conducting human health risk assessments on a wide variety of substances including post-market reviews of pharmaceuticals, traffic-related air pollutants, industrial chemicals, natural health products and pesticides. He has been involved with several initiatives applying more systematic review principles into risk assessments.|
|Sandra Aitcheson||Public Health Agency Northern Ireland||Sandra Aitcheson is Assistant Director of Nursing Adult and Older People’s Services for the Public Health Agency in Northern Ireland. She has dedicated her career to specialising in the care of Older People holding a number of clinical, managerial and strategic posts including Nurse Consultant for Older People in the Public Health Agency, Nurse Advisor for Older People at the Department of Health and Director of Stroke Strategy Implementation Eastern Health and Social Care Board. Her current role involves leading from a nursing perspective the promotion of improved health and well-being for the adult and older population across Northern Ireland with an emphasis on formulation of strategies, commissioning, service improvement and professional development. Crucial in this role is creating the conditions which ensures both value and contribution of nursing and midwifery is maximised. In addition she is the professional lead for Frailty across the Public Health Agency.|
|Catherine Gill||Health Research Board (Ireland)||Catherine is a Programme Manager in Post Award and Evaluation at the Health Research Board (HRB), overseeing monitoring and evaluation of Projects, Programmes and Career awards. She has a particular interest in Research Integrity (RI), with responsiblity for implementation of HRB RI policy, and participates in the National Forum on RI, and European RI initiatives.|
|Patricia Healy||NUI Galway||Dr Patricia Healy is a registered nurse and midwife currently working as a lecturer in the School of Nursing and Midwifery at National University of Ireland Galway (NUIG). Patricia’s professional interests focus on neonatal and maternity care, quality, safety and risk management in healthcare, research methodology, clinical trials, systematic reviews, meta-analyses and evidence synthesis.|
|Sheffield University/ School of Health And Related Research (SCHARR)||Dr Andrew Booth is a review methodologist specialising in qualitative evidence syntheses, realist syntheses and rapid reviews at the School of Health and Related Research (ScHARR), University of Sheffield. His particular interest is in the use of theories, logic models and frameworks to enhance the conduct and interpretation of reviews. Andrew is a co-convenor of the Cochrane Qualitative and Implementation Methods Group and an ordinary member of their Information Retrieval and Rapid Review Groups. He has published extensively on review methods and has contributed to methodological guidelines for Cochrane and GRADE-CERQual.|
|James Thomas||The Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) & University College London||James Thomas is Professor of Social Research & Policy at the EPPI-Centre, UCL in London. His research covers substantive disciplinary fields – such as public health and education – and also computer and information science. He has written extensively on methodology for research synthesis, including methods for combining qualitative and quantitative research in reviews, and leads the Evidence Synthesis Facility for the Department of Health, England.|
|Public Health Agency of Canada||Chantelle Garritty is a Senior Epidemiologist with the Public Health Agency of Canada. Prior to this, she was the Senior Research Manager and Scientific Lead of the Rapid Reviews Program at the Ottawa Hospital Research Institute. For over 10 years she’s been involved in conducting rapid reviews and related methods research to support timely decision-making. She is currently a Co-convenor of the Cochrane Rapid Reviews Methods Group, and an Advisor to Cochrane Response.|
|Andrea Tricco||Unity Health Toronto||Andrea Tricco holds a MSc in Epidemiology and PhD in Population Health. She is a Scientist and Director of the Knowledge Synthesis Team in the Knowledge Translation Program of the Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto. She is an Associate Professor at the University of Toronto in the Dalla Lana School of Public Health & Institute of Health Policy, Management, and Evaluation. She is also a Co-Director and Adjunct Associate Professor of the Queen’s Collaboration for Health Care Quality Joanna Briggs Institute (JBI) Centre of Excellence at Queen’s University.|
|Nikita Burke||NUI Galway/Evidence Synthesis Ireland||Dr Nikita Burke is the Programme Manager for Evidence Synthesis Ireland, which includes Cochrane Ireland. Prior to joining Evidence Synthesis Ireland, Nikita was the Development Manager for the HRB Primary Care Clinical Trials Network Ireland where she co-ordinated network activities, dissemination, and public and patient involvement in research. She has a PhD in Neuroscience from NUI Galway.|
|Ciara Keenan||Campbell UK & Ireland||Ciara is a Research Fellow in Campbell UK & Ireland. In this role, she is a Methods Editor and Information Retrieval specialist for the Campbell Collaboration and Co-Convenor of a global Information Scientist Network and a working group for overviews.
Ciara’s methodological interest is evidence syntheses and she is motivated by capacity building which has led to her work as founder and Editor of the meta-evidence blog and Twitter’s @evidencerobot and @COVID_Evidence. She has earned an established international reputation in evidence synthesis methodology and systematic review projects have demonstrated her expertise in the intersections of health, social welfare, disability, and education.
|Christopher Gravel*||Health Canada||*Until Jan 2021|
|Matthew Westmore*||NIHR (National Institute of Health Research) UK||*Until Jan 2021|
Priority III Top 10 Questions
Q1What are the best approaches to identify people or groups who will use the results of a rapid review (e.g., stakeholders such as patients and the public, clinicians, policy makers), and how can they have meaningful (i.e., purposeful, relevant) involvement in planning and doing a rapid review, and in reporting and sharing the findings?
- “It’s important to have a defined end user for the work that can help focus down the review onto the factors that matter most to them”
- “It is critical to determine, at the planning stage, who the results of a rapid review is intended and how they can use it”
- “If a rapid review was commissioned by a decision maker it would be helpful to know prior to carrying out the review what impact is the evidence from the review going to have. What is expected?”
- “At which points are involving people who will use the review particularly helpful in shaping the review?”
“How can researchers and patients engage effectively to co-design and co-produce a rapid review?”
Q2Do rapid reviews generate similar findings to full systematic reviews, and should the findings from all rapid reviews be considered at lower certainty compared to full systematic reviews?
- “Should the findings from a rapid review be considered at lower certainty to begin with – i.e., before we assess the certainty of the evidence?”
- “How do I measure the effectiveness of rapid review methodology vs a systematic review approach? What measurements or indicators tell me this is a better approach?”
- “Are rapid reviews considered scholarly work equivalent to a systematic review”
Q3How best can underserved stakeholder groups (e.g., ethnic minorities, socio-economically disadvantaged) and stakeholders from under represented countries (e.g. countries of different income levels) be identified and have meaningful (i.e., purposeful, relevant) involvement in planning and doing rapid reviews, and in sharing the results?
- “Please try to involve researchers and clinicians from the developing countries and LMIC. Most of the systematic reviews including the rapid reviews are planned, done, and published in the high-income countries and this evidence has very little relevance to the issues and constraints being faced by patients and clinicians in the LMIC”
- “Include patient partners in the early stages, and be creative in finding voices that are not always at the table”
Q4When deciding if a research question would benefit from being the focus of a rapid review, rather than a full systematic review, what criteria are helpful?
- “I am always interested in knowing if the questions that are asked in the context of a rapid review are relevant e.g., are they focused enough or is it just the time associated completion that guides the ‘rapidness’? What criteria are used to suggest a question lends itself to a rapid response?”
- “What criteria should be used to decide whether we need a rapid or a standard review?”
- “When do you know you should do a systematic review not a rapid review?”
Q5What simplified or omitted methods of a systematic review (e.g., single versus dual screening of citations for inclusion, restrictions on types of studies included) are appropriate to apply in a rapid review, and what are the effects of these simplifications or omissions (e.g., effect on the methods, conclusions, funding available)?
- “How much can be the methods be simplified/ omitted before relevant/ important evidence starts to be missed?”
- “What do we lose (if anything) if we don’t do anything in duplicate?”
- “Is it appropriate in the context of a rapid reviews to include only one researcher at every stage or how is the rigour of the review addressed?”
- “What components/review tasks can be omitted from rapid reviews comparative to full effectiveness reviews and still maintain the integrity of the review (and confidence in the findings)”
- “What elements are absolutely essential for a RR comparative to a systematic review – e.g., summary narrative results or meta-analysis?”
Q6What are the best approaches to assess the quality of studies included in a rapid review, and if a quality assessment is either limited or excluded, how should the findings be interpreted?
- “I would like to see more guidance on how to assess the quality of included studies”
- “Is it ever justifiable to omit the quality appraisal step?”
- “Should quality assessment be included in a rapid review? Especially for complex questions that may include qualitative or mixed methods primary studies?”
- “Could abridged versions of quality appraisal be used/developed”
- “I would like some clear standards on the quality review Process in a RR”
Q7How best can information on ongoing and completed rapid reviews be shared in a way that minimises research waste?
- “How can we organise rapid reviews in a place where the decision makers know to look?”
- “How can we get better at sharing data to avoid duplicated effort extracting data in overlapping rapid reviews?”
- “Is there an open access RR repository for emerging health systems or for low- and middle-income countries that do not yet have a team capable of generating their own RR?”
Q8What are the best approaches for developing a search strategy for use in a rapid review, and what is the impact of applying restrictions (e.g., years of inclusion, language, phase of study)?
- “How to optimise the search process for rapid reviews for the best combination of specificity and sensitivity?”
- “Are there guidelines available about how extensive the search process should be?”
- “What evidence exists to suggest that only including studies from a specific language range is appropriate to the wider claims of a rapid review?”
- “How can we ensure, with a more limited search, that important studies and research aren’t omitted?”
Q9What are the best approaches for reporting the findings of a rapid review in a clear, succinct way without limiting information on the complete methods, findings and strength of the evidence?
- “How do we communicate evidence in a concise and informative way while also getting across the limitations of the evidence? Both to general audience but also to scientific audience who are bombarded with information and may not have time to read the full paper”
- “How can the review report which are typically long, be condensed to reduce time in writing/reviewing of reports”
- “If reports are made shorter, how do we ensure the evidence is fairly evaluated and this process recorded?”
Q10What are the most useful processes to use when developing a rapid review research question?
- “How will question identification and refining be different – how long is allowed for this and can you go back and refine it further if needed” “How are the research topics chosen?”
- “Is there room for a mnemonic to describe the elements of a good rapid review question”
Priority III Questions 11-17
Q11How broad or focused should the scope of a research question for a rapid review be?
- “Smaller focused questions are a good idea”
- “I think answering a focussed question is the key to reducing the scope of a systematic review”
- “Questions should not become so narrow that they miss the real question or become uninformative”
- “Small question can become too simplistic. It can fail to include the wider determinants of a health behaviour or condition, which may be critical in implementation of a practice”
Q12What are the best approaches for determining the inclusion and exclusion criteria for a rapid review, and what is the impact of the various restrictions that may be applied (e.g., requiring ethical approval and any restrictions which may limit the inclusion of underserved groups)?
- “The clearer one is on the exclusion/inclusion criteria, the smoother this phase goes”
- “As a part of study inclusion criteria, reviewers should ensure each data source confirms receipt of study ethical approval. Ethical approval is often not sought or obtained in medical research performed world-wide, and the results of rapid reviews should be based on studies which had ethical approval”
- “Decisions on inclusion and exclusion criteria should be required to report on domains of the Equal Status Act – so to show how the review has taken into account issues of gender, ethnicity etc. For example, while gender may not be an explicit exclusion criterion, most clinical studies include male healthy subjects”
Q13How best can a definition of ‘rapid reviews’ be agreed?
- “How do you define rapid? Is it related to the time taken to undertake the review – and in which case when does a rapid review become ‘not rapid’ or is it about using a slimmed down set of methods? If a review is undertaken quickly but without compromising methods, is it still ‘rapid’?”
- “What is the definition of a rapid review? Does the term need to reflect a compromise in methodological rigour or is it only about the speed of conduct?”
- “It would help to define what one means by a rapid review, as it can mean different things to different people”
Q14What training or supports are needed to help people to plan, do and share the findings of rapid reviews?
- “Rapid reviews are supposed to be rapid for a reason (pressing decisions). Thus, concise write up of results without loss of critical information are critical. What can aid researcher in achieving this, as in my experience reports are often lengthy and more difficult to digest by users than need be?”
- “Access to teams with experience in systematic reviews”
- “Easy access to relevant networks”
Q15What is the value of doing a rapid review before a full systematic review or as an update of a systematic review, and should the plans for a rapid review include a commitment to return to the question and carry out a fuller review when more time and resources are available?
- “How can you make rapid reviews useful if they need to be upgraded to a full systematic review?”
- “Should the plans for a rapid review include a commitment to return to the question and conduct a fuller review when there are more time/resources available?”
Q16What is the best composition of a rapid review team (e.g., expertise)?
- “How many people need to be on a team to address all methods in what is considered to be a rapid manner?”
- “Have we evidence on the optimum number of reviewers to conduct a rapid review?”
- “Who is best to develop a dissemination strategy, should marketing or communication expertise be included in rapid review teams?”
- “You should involve librarians and other information specialists in all aspects of the review, but they are especially useful for disseminating findings.”
- “Bilingual researchers might be recruited, in case research studies exist in certain languages other than English”
Q17How can a scoping exercise of the literature inform the planning of rapid reviews?
- “What scoping activities should be done prior to deciding the research question”
- “Scoping work (including a scoping search) can be undertaken to assess the scale of the review work and identify areas for making the review more specific to ensure that meaningful and useable results are delivered within the specified timeframe”
Evidence Synthesis – Evidence synthesis uses specific, rigorous methods to bring together information from multiple studies that have looked at the same topic and provide an account of all that is known about the topic. This definition has been adapted from the Evidence Synthesis International website, available HERE and the Evidence Synthesis Ireland video, available HERE.
Systematic reviews – A systematic review is a type of evidence synthesis that brings together information from multiple studies to help answer a clear question. It uses systematic and specific methods to identify, select and quality assess included studies, followed by the collection and analysis of information. Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies. This definition has been adapted from the Cochrane glossary, available HERE.
Healthcare – We define as being related to the treatment, control or prevention of disease, illness, injury or disability, and the care or aftercare of a person with these needs (whether or not the tasks involved have to be carried out by a health professional). This definition has been adapted from the UK Department of Health and Social Care guidance document titled ‘National Framework for NHS Continuing Healthcare and NHS-funded Nursing Care’ available HERE.