How a review develops in a collaborative review group...
The Review Author contacts the Cochrane Review Group relevant to his or her area of interest and/or the systematic review they wish to undertake. Following discussion with the Coordinating Editors and Managing Editors, he or she decides upon an appropriate title. The review author can check the titles that have already been registered by authors interested in developing a review and also determine the reviews that have recently been completed here.
The 7 steps of preparing and maintaining a systematic review:
(Outlined in the Cochrane Handbook)
1. Formulating a problem
2. Locating and selecting studies
3. Critical appraisal of studies
4. Collecting data
5. Analyzing and presenting results
6. Interpreting results
7. Improving and updating reviews
The review author completes and returns the Review Proposal Form which asks for basic information, including: the population, the intervention, the proposed comparison(s) and the main outcomes of the proposed review, as well as the composition of the team. The proposal is circulated to members of Cochrane (Fields, Networks, international centers and other collaborative review groups) to guard against possibility of overlap. If the proposal is successful, then review proposal becomes a registered title.
It is the policy of the Cochrane Musculoskeletal Group to ask for an outline of your proposed topic.
For a complete information package to aid you in developing your topic and submitting a review proposal, click here. The review proposal form can be downloaded here. The form can be sent to Renea.Johnston@med.monash.edu.au
All Cochrane Musculoskeletal reviews should adhere to the MECIR standards.
Following title registration, the review author begins work on a protocol. The protocol is probably the most important aspect of a review. The completed protocol will provide the template that will guide the completion of the review and subsequent updates. It is the protocol that ensures transparency and accountability. Feedback from the editorial board of the review group is supplied at each draft, with specialist statistical advice brought in where necessary. Publication of the protocol on the Cochrane Library follows approval by the Group's editorial board, feedback from two External Referees and final approval by the Coordinating Editor.
For an example of a CMSG Cochrane protocol, click here.
For a complete information package to aid you to develop and write a protocol, click here.
Protocol development workshops are held around the world and most are free. For more information, click here.
A Cochrane Protocol Contains...
Criteria for considering studies for this review
Types of studies
Types of participants
Types of interventions
Types of outcome measures (Click here to see a list of proposed outcomes)
Search strategy for identification of studies
Methods of the review
Selection of trials
Assessment of risk of bias
Data management and extraction
Getting Started on Your Protocol
The editorial base of the Cochrane Musculoskeletal Group consists of the Coordinating Editors, the Managing Editors (MEs) and a Trial Search Coordinator (TSC). In addition, there is an editorial board of editors selected from around the world on the criteria of specialist content expertise in the areas covered by the scope of the Group, and special statistical advisers.
All submitted protocols are peer reviewed by at least two internal members of the editorial team and at least two external peer referees. The biostatistical editor and a consumer also review all protocols prior to submission to the Cochrane Library. The peer review process is conducted using a standard checklist designed by the editorial team. In addition, the CMSG consumer members have developed a consumer version of this checklist, which is currently being used for all protocols. The coordinating office compiles the comments received from the various peer reviewers and a summary is sent to the author. The final copy-editing prior to submission to the Cochrane Library is done at the coordinating office. Once a protocol is submitted, the author has up to two years to submit a review, depending on the extent of the review, and on whether the author is awaiting feedback regarding publication in a paper-based journal.
CMSG Search Strategy
A literature search should be conducted to identify the maximum number of trials relevant to your review. The first step is to search the Central Register of Controlled Trials (CENTRAL) on the Cochrane Library. This is the major source of information for randomized controlled trials (RCTs). Remember when you search that not all records are indexed with MeSH terms (for more information on MeSH terms, see http://www.nlm.nih.gov/mesh/meshhome.html). Be sure to select appropriate MeSH terms but also include all relevant textwords when doing your search. When choosing your textwords, think broadly about which terms to use and include synonyms, alternative spellings, singular/plural forms, etc.
Additional search strategies
Generally, the updated highly sensitive search strategies developed by members of the Cochrane are recommended for use. These strategies can be found in the Cochrane Handbook. To ensure that as many relevant studies as possible are identified, authors need to conduct additional searches to those described for the specialized register. These strategies include personal contact with experts in the area, citation searches (using SCISEARCH or Science Citation Index), and use of other relevant bibliographic databases. It is recommended that EMBASE is searched because it is a major source of biomedical and pharmacological literature, much of which is not on MEDLINE. There are also a number of web sites that contain information about recent or ongoing trials such as ClinicalTrials.gov or International Clinical Trials Registry Platform (ICTRP). It is also important to follow up on the references of relevant papers that you have found including those of any existing reviews. In addition, you could contact the authors of papers and/or other experts in your particular field. Additional search strategies are described in the methods section.
Review Development and Review Publication and Updates
After submission and publication of the review protocol, the authors begin carrying out the review according to the methods established in the protocol. There are many resources available to CMSG review authors to help them with all steps involved, such as assessing the risk of bias of included studies and presenting the findings effectively.
Completed reviews are put through an internal and external review process to ensure quality and rigor prior to submission for publication. Following approval and publication, reviews are ideally updated at least every 2 years in order to keep the review up-to-date with the most recent evidence.
For a complete information package describing the process of writing a review, click here.
The primary contact person for the review should liaise with the managing editor while developing their review. The managing editor will be able to provide details of the support and resources available from the CMSG (such as technical assistance and methodological advice including help with developing search strategies and the use of CMSG data collection checklists, which describe the inclusion criteria for studies, quality issues, the collection of relevant data, etc).
For all CMSG reviews, 'a priori' peer reviewed protocols are produced prior to submission of the review to the Cochrane Library. The inclusion and exclusion criteria are established and at least two authors carry out the selection process for included studies. If consensus is not reached, a third review author is assigned.
Assessment of risk of bias
The CMSG requires an assessment of the risk of bias of included studies.
The CMSG editorial team encourages review authors to use the Cochrane Collaboration's Risk of Bias tool, which is neither a scale nor a checklist. This tool is a domain-based evaluation, in which critical assessments are made separately for six different domains: sequence generation; allocation concealment; blinding of participants, personnel and outcome assessors; incomplete outcome data; selective outcome reporting; and 'other sources of bias'. The Risk of Bias tool is described in more detail in chapter 8 of The Cochrane Handbook.
Grading of Evidence
Methods for grading the scientific evidence have evolved over the past decade as evidence-based health care has become increasingly important in clinical practice. All Cochrane Musculoskeletal reviews use the 'Grading of Recommendation Assessment Development and Evaluation' (GRADE) approach (GRADE Working Group 2004, Schunemann et all 2008) to summarize the overall strength, or grading, of the evidence for each major outcome included in a review. This grading system specifies four levels of quality: high, moderate, low, and very low. The highest quality rating is for a body of evidence based on randomized trials without important limitations.
Review authors can downgrade randomized trial evidence depending on the presence of five factors (study limitations, inconsistency of results, indirectness of evidence, imprecision and publication bias) and upgrade the quality of evidence of observational studies depending on three factors (large magnitude of effect, plausible confounding, which would reduce a demonstrated effect, and dose-response gradient).
The GRADE approach is described in more detail in chapter 12 of The Cochrane Handbook.
Summary of Findings Tables
The CM editorial team now recommends the use of Summary of Findings tables to present results in reviews. These replace clinical relevance tables that were previously used. The summary of findings table provides key information concerning the quality of evidence, the magnitude of effect of the interventions examined, and the sum of available data on all important outcomes for a given comparison. The absolute risk reduction and the relative risk reduction should be provided and for those outcomes that are statistically significant, the number needed to treat should be calculated. A maximum of seven important outcomes including benefit and harm outcomes should be identified early in the review development process; if possible, during the review proposal stage. When review authors do not include information on these important outcomes in the review they should say so.
CMSG summary of findings table guidelines are available at the editorial office.
It is the Musculoskeletal Group's policy to recommend that at least two authors independently select the studies, assess the methodological quality and extract the data. Data collection forms should be used on all CMSG reviews. At the editorial office we have developed a number of data collection forms that can be modified for each of the reviews.
CM also has statistical advisers who will work closely with all CM authors to provide ongoing statistical guidance for completion of their Cochrane reviews. For statistical guidelines used by the Musculoskeletal Group, please refer to the Cochrane Handbook or contact the editorial office.
Reporting of reviews
The CM editorial team is supportive of authors who wish to submit their review to a paper-based journal prior to or simultaneous to submission to the Cochrane Library. Agreements vary depending on the number of modifications and timing for publication in those journals. Please contact the managing editor if you plan to do this. Indeed, the CM has an agreement with the Journal of Rheumatology and selected CMSG reviews will be considered for co-publication in this journal. When publishing in the Journal of Rheumatology, it is the authors' responsibility to reformat their review accordingly and cover the expenses requested by the journal.
Reviews and the Peer-review process
All submitted reviews are peer reviewed by at least two internal members of the editorial team and two external peer referees. At least one external referee is a consumer. The CMSG consumer members have developed a consumer checklist with which the reviews are assessed. Again, the coordinating office compiles the comments received from the various peer reviewers and a summary is sent to the author. The final copy-editing prior to submission to the Cochrane Library is done at the coordinating office. Once a review is submitted the author has up to two years to submit an updated review, depending on the extent of the review.
For a complete information package describing the process of publishing and maintaining a review, click here.
Reviews should be ideally updated at least every two years and more frequently where possible. Failure to update a review can result in the review being assigned to a new author. The CMSG editorial team encourages all new authors who have not conducted a Cochrane review to be assigned to assist with updating a particular review. This gives new authors the opportunity to learn the process prior to taking on the task of completing a full Cochrane systematic review.
For a complete information package describing the process of updating a review, click here.
Once the updated review has been submitted, it will again be subject to the editorial process and copy-editing as outlined in the Review Development section. If one of the following events occurs during the update of a review, the coordinating editors should be consulted to determine whether the updated review should be externally peer reviewed:
1. More than three new studies are identified.
2. One of the outcomes in the Summary of Findings table changes in terms of which group (treatment or placebo) is favoured. For example, if a dichotomous outcome was on one side of the "1" in the forest plot in the original review and changes to the other side in the update.
3. One of the outcomes in the Summary of Findings table changes in terms of its statistical significance. For example, if an outcome changes from being statistically significant in the original review to being non-significant in the update.