Clinical practice guideline: Difference between revisions

From Citizendium
Jump to navigation Jump to search
imported>Robert Badgett
imported>Robert Badgett
Line 81: Line 81:


Because of concerns about the quality and validity of some practice guidelines<ref name="pmid10349893">{{cite journal |author=Shaneyfelt TM, Mayo-Smith MF, Rothwangl J |title=Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature |journal=JAMA |volume=281 |issue=20 |pages=1900–5 |year=1999 |pmid=10349893 |doi=|url=http://jama.ama-assn.org/cgi/content/full/281/20/1900}}</ref><ref name="pmid10675167">{{cite journal |author=Grilli R, Magrini N, Penna A, Mura G, Liberati A |title=Practice guidelines developed by specialty societies: the need for a critical appraisal |journal=Lancet |volume=355 |issue=9198 |pages=103–6 |year=2000 |pmid=10675167 |doi=10.1016/S0140-6736(99)02171-6 }}</ref>, standards have been developed in both their reporting<ref name="pmid13679327">{{cite journal |author=Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM |title=Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization |journal=Ann. Intern. Med. |volume=139 |issue=6 |pages=493–8 |year=2003 |pmid=13679327 |doi=|url=http://annals.org/cgi/content/full/139/6/493}}</ref><ref name="pmid8460861">{{cite journal |author=Hayward RS, Wilson MC, Tunis SR, Bass EB, Rubin HR, Haynes RB |title=More informative abstracts of articles describing clinical practice guidelines |journal=Ann. Intern. Med. |volume=118 |issue=9 |pages=731–7 |year=1993 |pmid=8460861 |doi=|url=http://www.annals.org/cgi/content/full/118/9/731}}</ref> and assessment.<ref name="pmid12571340">{{cite journal |author=AGREE Collaboration |title=Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project |journal=Quality & safety in health care |volume=12 |issue=1 |pages=18–23 |year=2003 |pmid=12571340 |doi=|url=http://qshc.bmj.com/cgi/content/full/12/1/18}}</ref><ref name="pmid15205295">{{cite journal |author=Atkins D, Best D, Briss PA, ''et al'' for the GRADE Working Group. |title=Grading quality of evidence and strength of recommendations |journal=BMJ |volume=328 |issue=7454 |pages=1490 |year=2004 |pmid=15205295 |doi=10.1136/bmj.328.7454.1490|url=http://www.bmj.com/cgi/content/full/328/7454/1490/}} [http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=428525 PubMed Central]</ref>
Because of concerns about the quality and validity of some practice guidelines<ref name="pmid10349893">{{cite journal |author=Shaneyfelt TM, Mayo-Smith MF, Rothwangl J |title=Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature |journal=JAMA |volume=281 |issue=20 |pages=1900–5 |year=1999 |pmid=10349893 |doi=|url=http://jama.ama-assn.org/cgi/content/full/281/20/1900}}</ref><ref name="pmid10675167">{{cite journal |author=Grilli R, Magrini N, Penna A, Mura G, Liberati A |title=Practice guidelines developed by specialty societies: the need for a critical appraisal |journal=Lancet |volume=355 |issue=9198 |pages=103–6 |year=2000 |pmid=10675167 |doi=10.1016/S0140-6736(99)02171-6 }}</ref>, standards have been developed in both their reporting<ref name="pmid13679327">{{cite journal |author=Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM |title=Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization |journal=Ann. Intern. Med. |volume=139 |issue=6 |pages=493–8 |year=2003 |pmid=13679327 |doi=|url=http://annals.org/cgi/content/full/139/6/493}}</ref><ref name="pmid8460861">{{cite journal |author=Hayward RS, Wilson MC, Tunis SR, Bass EB, Rubin HR, Haynes RB |title=More informative abstracts of articles describing clinical practice guidelines |journal=Ann. Intern. Med. |volume=118 |issue=9 |pages=731–7 |year=1993 |pmid=8460861 |doi=|url=http://www.annals.org/cgi/content/full/118/9/731}}</ref> and assessment.<ref name="pmid12571340">{{cite journal |author=AGREE Collaboration |title=Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project |journal=Quality & safety in health care |volume=12 |issue=1 |pages=18–23 |year=2003 |pmid=12571340 |doi=|url=http://qshc.bmj.com/cgi/content/full/12/1/18}}</ref><ref name="pmid15205295">{{cite journal |author=Atkins D, Best D, Briss PA, ''et al'' for the GRADE Working Group. |title=Grading quality of evidence and strength of recommendations |journal=BMJ |volume=328 |issue=7454 |pages=1490 |year=2004 |pmid=15205295 |doi=10.1136/bmj.328.7454.1490|url=http://www.bmj.com/cgi/content/full/328/7454/1490/}} [http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=428525 PubMed Central]</ref>
Oncology guidelines may have poor methods.<ref name="pmid23752105">{{cite journal| author=Reames BN, Krell RW, Ponto SN, Wong SL| title=Critical evaluation of oncology clinical practice guidelines. | journal=J Clin Oncol | year= 2013 | volume= 31 | issue= 20 | pages= 2563-8 | pmid=23752105 | doi=10.1200/JCO.2012.46.8371 | pmc= | url=http://www.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&tool=sumsearch.org/cite&retmode=ref&cmd=prlinks&id=23752105 }} </ref>


====Conflict of interest====
====Conflict of interest====
[[Conflict of interest]] is a common problem with guidelines.<ref name="pmid11829700">{{cite journal |author=Choudhry NK, Stelfox HT, Detsky AS |title=Relationships between authors of clinical practice guidelines and the pharmaceutical industry |journal=JAMA |volume=287 |issue=5 |pages=612–7 |year=2002 |month=February |pmid=11829700 |doi= |url=http://jama.ama-assn.org/cgi/pmidlookup?view=long&pmid=11829700 |issn=}}</ref><ref name="pmid21990257">{{cite journal| author=Neuman J, Korenstein D, Ross JS, Keyhani S| title=Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study. | journal=BMJ | year= 2011 | volume= 343 | issue=  | pages= d5621 | pmid=21990257 | doi=10.1136/bmj.d5621 | pmc=PMC3191201 | url= }} </ref><ref>http://dx.doi.org/10.1001/2013.jamainternmed.56</ref>
[[Conflict of interest]] is a common problem with guidelines.<ref name="pmid11829700">{{cite journal |author=Choudhry NK, Stelfox HT, Detsky AS |title=Relationships between authors of clinical practice guidelines and the pharmaceutical industry |journal=JAMA |volume=287 |issue=5 |pages=612–7 |year=2002 |month=February |pmid=11829700 |doi= |url=http://jama.ama-assn.org/cgi/pmidlookup?view=long&pmid=11829700 |issn=}}</ref><ref name="pmid21990257">{{cite journal| author=Neuman J, Korenstein D, Ross JS, Keyhani S| title=Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study. | journal=BMJ | year= 2011 | volume= 343 | issue=  | pages= d5621 | pmid=21990257 | doi=10.1136/bmj.d5621 | pmc=PMC3191201 | url= }} </ref><ref>http://dx.doi.org/10.1001/2013.jamainternmed.56</ref>
Oncology guidelines may have conflict-of-interest.<ref name="pmid23752105">{{cite journal| author=Reames BN, Krell RW, Ponto SN, Wong SL| title=Critical evaluation of oncology clinical practice guidelines. | journal=J Clin Oncol | year= 2013 | volume= 31 | issue= 20 | pages= 2563-8 | pmid=23752105 | doi=10.1200/JCO.2012.46.8371 | pmc= | url=http://www.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&tool=sumsearch.org/cite&retmode=ref&cmd=prlinks&id=23752105 }} </ref>


===Applicability to individual patients===
===Applicability to individual patients===

Revision as of 15:55, 10 October 2013

This article is developing and not approved.
Main Article
Discussion
Related Articles  [?]
Bibliography  [?]
External Links  [?]
Citable Version  [?]
 
This editable Main Article is under development and subject to a disclaimer.

Clinical practice guidelines are defined as "Directions or principles presenting current or future rules of policy for assisting health care practitioners in patient care decisions regarding diagnosis, therapy, or related clinical circumstances. The guidelines may be developed by government agencies at any level, institutions, professional societies, governing boards, or by the convening of expert panels. The guidelines form a basis for the evaluation of all aspects of health care and delivery."[1]

Clinical practice guidelines were initially encouraged by the Institute of Medicine in 1990.[2]

They are becoming increasingly important in healthcare economics, as a means of avoiding waste on ineffective treatment approaches. At the same time, there are concerns that overemphasizing them may limit legitimate physician decisionmaking about the needs of specific patients. Some insurers emphasize their use,[3] needed to go outside them.

Methods

Standards for the conduct of clinical practice guidelines are available.[4]

Part of creating a clinical practice guideline is to gather and assess all of the relevant research. This task can be done using a framework such as the GRADE system.[5] Techniques such as decision analysis [6] and nominal group technique[6] may improve the validity of clinical practice guidelines.

Benefits and disadvantages of guidelines

Medical care guided by evidence-based guidelines may be associated with better outcomes among patients with pneumonia.[7][8]

Disadvantages of guidelines

Guidelines may have trouble with both validity and readability.[9] This trouble may be increasing.[10] Even guidelines that are technically well-done may not be acceptable to clinicians.[11]

A structured comparison of clinical practice guidelines for diabetes mellitus type 2 found variation in guideline quality with the best guideline being produced by the United Kingdom National Institute for Health and Clinical Excellence.[12]

There are proposals to reform the production of guidelines.[13]

Validity

Guidelines may not always be based on high quality evidence [14][15] and the quality of evidence used may be decreasing[10].

Two important attributes of a practice guideline are:[16]

  1. Credibility to a range of stakeholders including clinicians, patients, and policy makers.
  2. The developers are accountable for their conclusions by using a process that is open, documented, and reproducible.

Bayesian interpretation of guidelines may help in their interpretation.[17]

Less rigorously developed guidelines may have more aggressive conclusions.[18]

Because of concerns about the quality and validity of some practice guidelines[19][20], standards have been developed in both their reporting[21][22] and assessment.[23][24]

Oncology guidelines may have poor methods.[25]

Conflict of interest

Conflict of interest is a common problem with guidelines.[26][27][28]

Oncology guidelines may have conflict-of-interest.[25]

Applicability to individual patients

Guidelines are usually developed for the care of a single disease; consequently, guidelines may direct care that is contrary to the overall well being of a patient with multiple comorbid diseases.[29]

Practice guidelines may not direct the health of a population as efficiently as customizing medical care based on the risk factors that each individual has in a population.[30][31][32]

Readability

Guidelines can be very difficult to read which may contribute to lack of impact.[33] For this reason, some are now written with the inverse pyramid structure.[34][35]

Durability

Guidelines may loose their clinical relevance as they age and newer research emerges.[36]

Effectiveness

The effect of publication

The publication of clinical practice guidelines do not always influence health care[37][24] or take several years to do so.[38] Guidelines published with supporting material may be effective.[39]Balanced against this is that guidelines may be out-of-date after an average of three years.[36]

Other measures to increase effect

Improving outpatient therapy for patients with low risk pneumonia[40]
Intervention(s) Proportion of low-risk patients treated as outpatients Proportion of outpatients receiving all recommended processes of care
Audit and feedback Instruction on using the pneumonia severity index Real-time reminders
Yes No No 38% 25%
Yes Yes No 61% 28%
Yes Yes Yes 62% 61%
Improving the Glycosylated hemoglobin A (Hb A1c)[41][42]
Intervention(s) Mean reduction in Hb A1c
Physicians given list of patients who are uncontrolled Letters sent to patients who are uncontrolled Physicians given time to discuss list with care team
O'Connor PJ et al, 2009[42]
No No No -0.19
Yes No No -0.07
No Yes No -0.12
Yes Yes No -0.11
Stroebel RJ et al, 2002[41]
Yes No No -0.14
Yes No Yes -0.13
Yes Yes Yes -0.25

The more modalities used to implement a guideline, the more impact the guideline will have.[40][43][41]

In addition, a systematic review of organizational interventions to improve the quality of care of diabetes mellitus type 2 suggests that interventions based on complexity science will be more successful.[44]

Finding published clinical practice guidelines

For guidelines published in the United States the National Guidelines Clearinghouse republishes many guidelines in a standardized format.

Guidelines can also be located at PubMed by searching for the publication type 'practice guideline'.[45] In addition, the Hedges Team has published a MEDLINE search strategy.[46]

References

  1. National Library of Medicine. Clinical practice guidelines. Retrieved on 2007-10-19.
  2. Lohr, Kathleen N.; Grossman, Jerome H.; Field, Marilyn J. (1990). Clinical practice guidelines: directions for a new program. Washington, D.C: National Academy Press. ISBN 0-309-04346-8. 
  3. Clinical Practice Guidelines in Health Plans: A Model Process, America's Health Insurance Plans. Retrieved on 7 November 2010
  4. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P et al. (2012). "Guidelines International Network: toward international standards for clinical practice guidelines.". Ann Intern Med 156 (7): 525-31. DOI:10.1059/0003-4819-156-7-201204030-00009. PMID 22473437. Research Blogging.
  5. Kavanagh, Brian P. (2009). "The GRADE System for Rating Clinical Guidelines". PLoS Med 6 (9): e1000094. DOI:10.1371/journal.pmed.1000094. Retrieved on 2009-09-16. Research Blogging.
  6. 6.0 6.1 Kuntz KM, Tsevat J, Weinstein MC, Goldman L (1999). "Expert panel vs decision-analysis recommendations for postdischarge coronary angiography after myocardial infarction.". JAMA 282 (23): 2246-51. PMID 10605977.
  7. McCabe, Caitlin; Cheryl Kirchner, Huiling Zhang, Jennifer Daley, David N. Fisman (2009-09-14). "Guideline-Concordant Therapy and Reduced Mortality and Length of Stay in Adults With Community-Acquired Pneumonia: Playing by the Rules". Arch Intern Med 169 (16): 1525-1531. DOI:10.1001/archinternmed.2009.259. Retrieved on 2009-09-15. Research Blogging.
  8. Arnold, Forest W.; A. Scott LaJoie, Guy N. Brock, Paula Peyrani, Jordi Rello, Rosario Menendez, Gustavo Lopardo, Antoni Torres, Paolo Rossi, Julio A. Ramirez, for the Community-Acquired Pneumonia Organization (CAPO) Investigators (2009-09-14). "Improving Outcomes in Elderly Patients With Community-Acquired Pneumonia by Adhering to National Guidelines: Community-Acquired Pneumonia Organization International Cohort Study Results". Arch Intern Med 169 (16): 1515-1524. DOI:10.1001/archinternmed.2009.265. Retrieved on 2009-09-15. Research Blogging.
  9. Dartnell J, Hemming M, Collier J, Ollenschlaeger G (2007). "Putting evidence into context: some advice for guideline writers". Evidence-based medicine 12 (5): 130–2. DOI:10.1136/ebm.12.5.130-a. PMID 17909225. Research Blogging.
  10. 10.0 10.1 Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC (February 2009). "Scientific evidence underlying the ACC/AHA clinical practice guidelines". JAMA 301 (8): 831–41. DOI:10.1001/jama.2009.205. PMID 19244190. Research Blogging. Cite error: Invalid <ref> tag; name "pmid19244190" defined multiple times with different content
  11. Nuckols TK, Lim YW, Wynn BO, et al (2007). "Rigorous Development does not Ensure that Guidelines are Acceptable to a Panel of Knowledgeable Providers". J Gen Intern Med. DOI:10.1007/s11606-007-0440-9. PMID 18030541. Research Blogging.
  12. Qaseem A, Vijan S, Snow V, Cross JT, Weiss KB, Owens DK (September 2007). "Glycemic control and type 2 diabetes mellitus: the optimal hemoglobin A1c targets. A guidance statement from the American College of Physicians". Annals of internal medicine 147 (6): 417–22. PMID 17876024[e]
  13. Sniderman AD, Furberg CD (January 2009). "Why guideline-making requires reform". JAMA 301 (4): 429–31. DOI:10.1001/jama.2009.15. PMID 19176446. Research Blogging.
  14. Hayward RA, Hofer TP, Vijan S (2006). "Narrative review: lack of evidence for recommended low-density lipoprotein treatment targets: a solvable problem.". Ann Intern Med 145 (7): 520-30. PMID 17015870[e]
  15. Lee DH, Vielemeyer O (2011). "Analysis of overall level of evidence behind infectious diseases society of america practice guidelines.". Arch Intern Med 171 (1): 18-22. DOI:10.1001/archinternmed.2010.482. PMID 21220656. Research Blogging.
  16. Mulrow CD, Lohr KN (2001). "Proof and policy from medical research evidence". Journal of health politics, policy and law 26 (2): 249–66. PMID 11330080[e]
  17. Diamond GA, Kaul S (2009). "Bayesian classification of clinical practice guidelines.". Arch Intern Med 169 (15): 1431-5. DOI:10.1001/archinternmed.2009.235. PMID 19667308. Research Blogging.
  18. Fretheim A, Williams JW, Oxman AD, Herrin J (2002). "The relation between methods and recommendations in clinical practice guidelines for hypertension and hyperlipidemia.". J Fam Pract 51 (11): 963-8. PMID 12485552.
  19. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J (1999). "Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature". JAMA 281 (20): 1900–5. PMID 10349893[e]
  20. Grilli R, Magrini N, Penna A, Mura G, Liberati A (2000). "Practice guidelines developed by specialty societies: the need for a critical appraisal". Lancet 355 (9198): 103–6. DOI:10.1016/S0140-6736(99)02171-6. PMID 10675167. Research Blogging.
  21. Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM (2003). "Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization". Ann. Intern. Med. 139 (6): 493–8. PMID 13679327[e]
  22. Hayward RS, Wilson MC, Tunis SR, Bass EB, Rubin HR, Haynes RB (1993). "More informative abstracts of articles describing clinical practice guidelines". Ann. Intern. Med. 118 (9): 731–7. PMID 8460861[e]
  23. AGREE Collaboration (2003). "Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project". Quality & safety in health care 12 (1): 18–23. PMID 12571340[e]
  24. 24.0 24.1 Atkins D, Best D, Briss PA, et al for the GRADE Working Group. (2004). "Grading quality of evidence and strength of recommendations". BMJ 328 (7454): 1490. DOI:10.1136/bmj.328.7454.1490. PMID 15205295. Research Blogging. PubMed Central Cite error: Invalid <ref> tag; name "pmid15205295" defined multiple times with different content
  25. 25.0 25.1 Reames BN, Krell RW, Ponto SN, Wong SL (2013). "Critical evaluation of oncology clinical practice guidelines.". J Clin Oncol 31 (20): 2563-8. DOI:10.1200/JCO.2012.46.8371. PMID 23752105. Research Blogging.
  26. Choudhry NK, Stelfox HT, Detsky AS (February 2002). "Relationships between authors of clinical practice guidelines and the pharmaceutical industry". JAMA 287 (5): 612–7. PMID 11829700[e]
  27. Neuman J, Korenstein D, Ross JS, Keyhani S (2011). "Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study.". BMJ 343: d5621. DOI:10.1136/bmj.d5621. PMID 21990257. PMC PMC3191201. Research Blogging.
  28. http://dx.doi.org/10.1001/2013.jamainternmed.56
  29. Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW (2005). "Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance". JAMA 294 (6): 716–24. DOI:10.1001/jama.294.6.716. PMID 16091574. Research Blogging.
  30. Timbie JW, Hayward RA, Vijan S (2010). "Variation in the net benefit of aggressive cardiovascular risk factor control across the US population of patients with diabetes mellitus.". Arch Intern Med 170 (12): 1037-44. DOI:10.1001/archinternmed.2010.150. PMID 20585069. PMC PMC2897053. Research Blogging.
  31. Eddy DM, Adler J, Patterson B, Lucas D, Smith KA, Morris M (2011). "Individualized guidelines: the potential for increasing quality and reducing costs.". Ann Intern Med 154 (9): 627-34. DOI:10.1059/0003-4819-154-9-201105030-00008. PMID 21536939. Research Blogging.
  32. Hayward RA, Krumholz HM, Zulman DM, Timbie JW, Vijan S (2010). "Optimizing statin treatment for primary prevention of coronary artery disease.". Ann Intern Med 152 (2): 69-77. DOI:10.1059/0003-4819-152-2-201001190-00004. PMID 20083825. Research Blogging.
  33. Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H (1998). "Attributes of clinical guidelines that influence use of guidelines in general practice: observational study". BMJ 317 (7162): 858–61. PMID 9748183[e]
  34. Nielsen J. Writing Inverted Pyramids in Cyberspace (Alertbox). Retrieved on 2007-10-19.
  35. Barton MB, Miller T, Wolff T, et al (2007). "How to read the new recommendation statement: methods update from the U.S. Preventive Services Task Force". Ann. Intern. Med. 147 (2): 123–7. PMID 17576997[e]
  36. 36.0 36.1 Shekelle PG, Ortiz E, Rhodes S, et al (2001). "Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated?". JAMA 286 (12): 1461–7. PMID 11572738[e]
  37. Cabana MD, Rand CS, Powe NR, et al (1999). "Why don't physicians follow clinical practice guidelines? A framework for improvement". JAMA 282 (15): 1458–65. PMID 10535437[e]
  38. Lomas J, Sisk JE, Stocking B (1993). "From evidence to practice in the United States, the United Kingdom, and Canada". The Milbank quarterly 71 (3): 405–10. PMID 8413068[e]
  39. Thornhill, M. H.; M. J. Dayer, J. M. Forde, G. R. Corey, V. H. Chu, D. J. Couper, P. B. Lockhart (2011). "Impact of the NICE guideline recommending cessation of antibiotic prophylaxis for prevention of infective endocarditis: before and after study". BMJ 342 (may03 1): d2392-d2392. DOI:10.1136/bmj.d2392. ISSN 0959-8138. Retrieved on 2011-05-05. Research Blogging.
  40. 40.0 40.1 Yealy DM, Auble TE, Stone RA, Lave JR, Meehan TP, Graff LG et al. (2005). "Effect of increasing the intensity of implementing pneumonia guidelines: a randomized, controlled trial.". Ann Intern Med 143 (12): 881-94. PMID 16365469[e] Review in: ACP J Club. 2006 May-Jun;144(3):80 Cite error: Invalid <ref> tag; name "pmid16365469" defined multiple times with different content
  41. 41.0 41.1 41.2 Stroebel RJ, Scheitel SM, Fitz JS, et al. (August 2002). "A randomized trial of three diabetes registry implementation strategies in a community internal medicine practice". Jt Comm J Qual Improv 28 (8): 441–50. PMID 12174408[e]
  42. 42.0 42.1 O'Connor PJ, Sperl-Hillen J, Johnson PE, Rush WA, Crain AL (July 2009). "Customized feedback to patients and providers failed to improve safety or quality of diabetes care: a randomized trial". Diabetes Care 32 (7): 1158–63. DOI:10.2337/dc08-2247. PMID 19366977. Research Blogging.
  43. Roumie CL, Elasy TA, Greevy R, et al (August 2006). "Improving blood pressure control through provider education, provider alerts, and patient education: a cluster randomized trial". Ann. Intern. Med. 145 (3): 165–75. PMID 16880458[e]
  44. Leykum LK et al. (2007). "Organizational interventions employing principles of complexity science have improved outcomes for patients with Type II diabetes". Implementation Science : IS 2: 28. DOI:10.1186/1748-5908-2-28. PMID 17725834. Research Blogging.
  45. Anonymous (Nov–Dec 2007). MEDLINE Data Changes - 2008. NLM Technical Bulletin. National Library of Medicine. Retrieved on 2008-03-19.
  46. Wilczynski NL, Haynes RB, Lavis JN, Ramkissoonsingh R, Arnold-Oatley AE (2004). "Optimal search strategies for detecting health services research studies in MEDLINE". CMAJ 171 (10): 1179-85. DOI:10.1503/cmaj.1040512. PMID 15534310. Research Blogging.