Education Coordinating Group

Information Retrieval Methods Group Systematic Review Checklist*

*Please note this checklist has not yet been approved by the C2 Steering Group

Instructions : This checklist is designed to aid you in organizing the evaluation of the information retrieval activities for a review and to make explicit the criteria to be use during the evaluation. Each section of the checklist requires the evaluation of a specific activity. For each question in the checklist, you will make the following assessments:

For each question in the checklist, the reader will make the following assessments:

  1. Was the checklist question or criterion addressed in the PROTOCOL (PR)?
    • Y (yes)
    • N (no)
    • U (unsure)
    • or NA (not applicable)
    The above response choice should be entered under the PR column heading in the checklist.
  2. Was the question answered or criteria addressed appropriately in the REVIEW (RW)?
    • Y (yes)
    • N (no)
    • U (unsure)
    • or NA (not applicable)
    The above response choice should be entered under the RW column heading in the checklist.

For checklist questions or criteria that require further comment, please use the comment section at the end of the checklist.

Search Tools and Strategies
1. Database Selection and Strategies
 PR RW
A. Were the full (original) search strategies listed for the main databases searched?    
If so, did the search strategies include keywords or descriptors appropriate for the topic and the database?
   
B. Were databases for related fields consulted?    
If so, were the full (complete) search strategy listed for each database searched?
    
Did the search strategy include keywords appropriate for the topic and the database?
   
C. Were the time periods for each source indicated?    
D. Did the authors consult the list of databases in the IRMG Brief?    
E. Did the authors search for dissertations separately?    
F. Did the electronic searches appear sufficiently comprehensive for the topic?    
2. Campbell Databases
    
A. Did the authors include C2-SPECTR in their search?    
If so, was the date of the database included?
    
B. Did the authors include C2- PROT in their search?    
C. Did the authors consult any C2 Coordinating Group specialized register (if available) in their search?    
3. Print-based Indices
    
Were any print-based subject indices (e.g. Education Index) used?    
If so, did the authors state why (or why not) they were (not) used?
    
Did the authors include the subject headings that were used?
    
If so, did the search strategy include subject headings appropriate for the topic and the field?
    
4. Web Search Tools
    
Did the authors use the main search engines (e.g. Google, AlltheWeb, Yahoo Search!, Teoma etc.) to locate web-based material?    
If so, were the search strategies included?
    
5. Locating Other Material including Grey Literature
    
A. Did the authors search for conference proceedings separately?    
B. Did the authors search for government documents separately?    
C. Did the author's document their grey literature (e.g. unpublished research reports) searches?    
If not, did the authors provide their rationale for not doing so?
    
6. Hand searches
    
A. Did the authors conduct a hand search of journals relevant to the topic?    
If not, did the authors provide their rationale for not doing so?
    
B. Did the authors consult the reference lists of reviews and/or previous trials?    
7. Non U.S. Literature
    
A. Were Canadian studies included?    
If so, were the search strategies included for each of the retrieval tools used?
    
If not, did the author provide their rationale for not doing so?
    
B. Were British and/or Australian studies included?    
If so, were the search strategies included for each of the retrieval tools used?
    
If not, did the author provide their rationale for not doing so?
    
C. Were non-English studies included?    
If so, were the search strategies included for each of the retrieval tools used?
    
If not, did the author provide their rationale for not doing so?
    
8. Personal Contacts
    
Did the authors contact experts in the field?    
If so, were the contact strategies documented?
    
Was a list of names and contact information included in the documentation?
    
9. Currency
    
Will (or have) the searches be(en) updated to ensure retrieving the most current information?    

Other
 
PR
RW
Was the information retrieval stage of this review sufficiently informed by the use of an information specialist?    
Was the information retrieval stage described in sufficient detail for replication?    
Was documentation provided for steps taken to assess the need for an update at the end of three years?    
Was bibliographic reference manager software used to store the citations?    
If so, what package was used? Name:
    


General Comments

 

 

 

 

 

 

 

 

What is your overall assessment of this protocol (PR) or review (RW)?

•  Accept without revision

•  Accept with revisions. Authors will revise the protocol or review in consultation with the Coordinator.

•  Substantial revisions required. Authors should revise the protocol or review and resubmit it to the Coordinator who will send it to external readers.

 

Reader _________________________________________________ Date:___/___/__

First Name Middle Initial Last Name

Reader: ____________________________________________ Date:______________