Clinical scenario
You are a young general surgeon in a community practice. A new consultation is scheduled in your clinic: a 60-year-old woman presenting with chronic venous ulcers on both legs. She has no other notable medical history. Over the previous 5 years, she had a number of debridements and skin grafts performed by a surgeon who recently retired. She is very frustrated by her odorous oozing ulcers and is embarrassed to visit her family. You review the records at your hospital but cannot find much useful information. You do not know why her previous surgeries and nonsurgical treatments have failed. You know that venous ulcers have different etiologies, such as persisting edema, superinfections or concomitant arterial insufficiency. You decide to review the literature guidelines that give specific recommendations on the management of venous ulcers to ensure an option on the treatment algorithm has not been overlooked.
Literature search
As described in a previous article in the “users’ guide to the surgical literature” series,1 you begin with a Medline search. The terms “venous ulcer” and “guideline” are entered separately. Based on medical subject heading (MESH) terms, Medline prompts inclusion of the terms “leg ulcer/” or “varicose ulcer/” and “guideline” or “practice guideline,” respectively. These terms are combined and results limited to the English language, yielding 10 articles.2–11 Three of them do not relate to venous ulcers,2,3,6 and 3 were published before the year 2000.8–10 Two are nonspecific for ulcers,5,7 and 1 focuses on prevention.4 You select the article entitled, “Guidelines for the treatment of venous ulcers,”11 which appears to address your question. You print the guideline and review it before your next visit with the patient.
Introduction
Clinical practice guidelines (CPGs) are defined in the literature as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.”12–15 They distill a large body of literature on a topic into a format that is high-yield and easy for physicians to use.
Worldwide, surgeons perform 200 million procedures annually.16 There is constant effort to optimize this complex and expensive health care facet.16 Surgeons are faced with difficult management decisions while balancing evidence-based recommendations.17 When trial evidence exists, it often cannot be perfectly applied to specific patient presentations. It is difficult to independently condense primary research for each patient. Moreover, health care providers and insurers are increasingly concerned with quality improvement and cost effectiveness. Guidelines aim to balance these factors,12,15 and direct consistent and reliable care.17 The number of surgical guidelines available in the literature is increasing. However, CPGs vary in their quality and sometimes deviate from high methodological rigor.16 It is necessary for surgeons to be able to appraise CPGs before deciding to adopt their recommendations.
Since 1990, CPGs have been an increasingly popular tool influencing physician practice.18,19 More than 20 tools to interpret and appraise CPGs have been published;14 the latest is the AGREE-II instrument (appraisal of guidelines for research and evaluation).13 It was originally released in 2003 to address guideline development, reporting and evaluation. Two further studies,20,21 have refined the instrument, now recognized as the methodological standard in guideline evaluation.14
In this article, we discuss a practical approach to the appraisal of a CPG; Box 1 contains the key items readers should consider when using a CPG in surgery. As in previous users’ guide to the surgical literature articles,22 we use a condensed framework to approach a guideline from a surgical perspective. This will provide surgeons with a practical approach to interpreting and applying recommendations in a CPG, using the guideline by Robson and colleagues11 as an example.
Box 1. Users’ guides for an article on clinical practice guidelines
Are the recommendations valid?
1. Is there a clear statement of a clinical problem?
2. Who was involved in guideline development (i.e., authors, reviewers, patients, readers)?
3. How is the guideline reviewed?
4. What literature are recommendations based on?
What recommendations are made?
5. Are useful recommendations presented?
6. How do authors move from evidence to recommendations?
Will the results help me in caring for my patients?
7. Were all outcomes considered (surgical outcomes versus natural course of disease)?
8. Will I be able to implement these recommendations?
Are the recommendations valid?
Is there a clear statement of a clinical problem?
Like other publications, CPGs address a defined problem in a specific group of patients. Surgeons must always consider whether the CPG recommendations can be applied to their own patients.15,23 The PIPOH items (patient population, intevention(s), professionals/patients, outcomes to be considered, health care setting) are suggested in the ADAPTE process (www.adapte.org) to frame the content and clinical question in a guideline.24 Readers should use these categories to decide if the recommendations presented are representative of their patient and treatment goals. Surgeons are cautioned in applying CPGs not designed for their patient populations.25 Subtle differences in any category can alter the CPG’s applicability.
Robson and colleagues11 summarize the management of venous ulcers in 8 categories: diagnosis, compression, infection control, wound bed preparation, dressings, surgery, adjuvant agents and long-term maintenance. However, the guideline does not include specific PIPOH criteria. For example, the authors need to be more specific in Recommendation #6.3: “Less extensive surgery on the venous system, such as superficial venous ablation, endovenous laser ablation, or valvuloplasty, especially when combined with compression therapy, can be useful in decreasing the recurrence of venous ulcers (Level I).” The reader must carefully consider patient population and health care setting in this recommendation. Venous ulcers are associated with comorbidity. If our hypothetical patient had diabetes or an inflammatory disorder, interventions would differ. Further, procedures such as endovenous laser ablation may not be available in every health care setting.
Who was involved in guideline development? (ie. authors, reviewers, patients, readers)
No guideline is developed in isolation, free from potential bias.26,27 These biases may be subconscious and difficult to detect.28 Surgeons must critically consider how and why the guideline has been created. What inherent biases may the authors have? Organizing committees and professional organizations beyond those listed in the authorship may have reviewed the guideline. While reviewing evidence and providing recommendations, each group will have their own influence.29 Who are the people in these roles? Who are they representing? What is their expertise? Guideline panels and authorship are often sponsored by the pharmaceutical industry in some capacity.28
The authorship should be analyzed for surgical input. Review by a surgical association or publishing in a surgical journal demonstrate evaluation with surgical familiarity. Even when presented with the same research evidence, professional groups can differ in their recommendations.30 Shaneyfelt and colleagues15 identify examples in breast and prostate cancer: a cancer interest group may support adoption of new, costly population screening interventions despite limited effectiveness, whereas public health groups may not view the intervention as a cost-effective strategy in the general population.
Robson and colleagues’ work11 is developed by the Wound Healing Society, with grant support from its educational/charitable arm, the Wound Healing Foundation.31 The guideline is published in Wound Repair and Regeneration by the Wound Healing Society. The guideline has been developed and published by the same association, indicating a potential conflict of interest. Further, CPGs authored by research/care societies may be of lower quality than those published by guideline societies.16 The CPG by Robson and colleagues listis its authors and their affiliations and positions. The group is composed of academicians, private practice physicians, podiatrists, nurse clinicians, research nurses, industrial scientists and an epidemiologist. The diverse author group reflects the multidisciplinary approach to chronic wounds and helps to reduce professional bias.32 However, little detail is given to the roles of each professional, and there is no mention of surgeons.
How is the guideline reviewed?
Surgeons should scrutinize the review and revision process of CPGs. Like any other publication, CPGs are subject to peer review. Beyond the guideline’s sponsoring association and authors, surgeons should be sure that independent experts are involved.33 This includes experts in medical and research methodology and possibly patient groups. The process should be transparent. Commentary and editing from the review panel should be included or available in a supplement. Prior to dissemination, guidelines may be pilot tested on small patient samples to ensure applicability. Authors should describe the process of reviewing and updating the guideline on an ongoing basis.26 Some groups establish a team monitoring for new evidence, whereas others provide a predetermined schedule of updates to their guidelines.32 Guidelines with extensive readership and consistent new research findings (e.g., ACCP34 and ACLS35 guidelines) often schedule new releases.
In Robson and colleagues’ work,11 specific revision methodology is lacking. There is no indication the Wound Healing Society has reviewed the guideline, despite their sponsorship. Details of expert review, review scales, specialties/disciplines of reviewers and edits suggested during the review process would all be pertinent to the surgeon. Without insight into who approved the CPG, it is difficult to discern the potential biases that would impact surgical decision making.33 No procedure is defined for revision. Given that leaders in the ever changing field of wound management provided this CPG, surgeons should expect a schedule of updates.
Editorial independence and funding should be declared with all forms of research. Surgeons should be critical in assessing the interests of governing bodies or pharmaceutical/equipment sponsors. The translation of primary literature to clinical recommendations requires judgment.15 Surgeons must ensure this judgment is not biased.36 For example, does a company marketing dressings have any stake in these recommendations? Do involved professional organizations have a monetary or public interest? The interests of authors, Wound Repair and Regeneration and the Wound Healing Society are not discussed in the CPG by Robson and colleagues. It is difficult to interpret the biases authors may impose on the CPG outside of credentials listed. No information is available for financial or research support of members. Robson and colleagues succeed in not emphasizing the use of brand name products. Instead it indicates the evidence-based properties of a dressing that improve wound care.
What is the evidence base?
Authors should use appropriate methodology to support their recommendations. A transparent and structured methodology reflects rigorous development. This is evaluated stepwise, beginning with the search strategy, appraising evidence and grading recommendations. Similar to the rigor of a systematic review, a good CPG will reflect a body of high-quality research with coherent results.17 Quality of referenced studies should be clear. Issues with blinding, allocation concealment and equal expertise among groups are unique challenges in surgical RCTs.37 Unfortunately, CPGs in surgery can rarely depend solely on high-level evidence (systematic reviews, randomized controlled trials [RCTs]).38 In some surgical areas, observational studies and case reports may be the only evidence available,39 and these must be analyzed for confounding and bias. Often the available evidence is not of high quality.38 However, guidelines addressing questions without available high-quality evidence are still important in guiding physician decision making.40 Moreover, these complex situations require a transparent and rigorous methodology.40
Guidelines should include, either in the text or supporting documentation, a statement detailing the development process. The availability of this process is a good predictor of the CPG’s overall rigor.41 Search strategies should incorporate multiple databases and a search of grey literature (unpublished sources, such as conferences and thesis work). Explicit inclusion and exclusion criteria should be defined, and the assessment of the validity of the evidence should be reproducible and consistent among studies.36 A lag time exists between guideline development and publication. The CPG by Robson and colleagues11 was developed in October 2005 and published in the November/December 2006 issue of Wound Repair and Regeneration. In some rapidly changing specialties, it is possible that new evidence becomes available within this lag time.
The CPG by Robson and colleagues11 includes a methods section. While databases are listed, no search terms are specified to ensure a reproducible search methodology for the references cited. Without search terms, transparency is difficult to establish. Robson and colleagues specify that their methodology differs from that of previous publications, including laboratory/animal studies and findings extrapolating from treatment of other ulcers. Beyond this, their process for selecting evidence is vague.
Robson and colleagues11 succeed in defining the level of evidence for each recommendation. For example, they cite the following literature for Recommendation #2.1: “Cullum N, Nelson EA, Fletcher AW, Sheldon TA. Compression for venous leg ulcers. The Cochrane Database of Systematic Reviews. (2001 Issue 2) The Cochrane Collaboration. John Wiley & Sons Ltd. [STAT, 23 RCT].” This illustrates that the recommendation is based on a meta-analysis of 23 RCTs and provides readers with a reference to the original data. Recommendations are followed by contributing references, each marked with 1 of 8 levels of evidence: STAT (Statistical analysis, meta-analysis, consensus statement by commissioned panel of experts), RCT, LIT REV (literature review), CLIN S (clinical case series), RETRO S (retrospective series review), EXP (experimental laboratory or animal study), TECH (technique or methodology description) or PATH S (pathological series review). Of the 41 grouped recommendations made, 5 do not reference RCT or higher levels of evidence: 1.3, 1.4, 4.3, 5.4 and 6.4. While RCTs, systematic reviews and meta-analyses represent the top of the level of evidence hierarchy, there is no discussion of the merits of each reference. Preferably, the RCTs should each be evaluated for individual methodological quality, especially given the unique issues in surgical trials.
What recommendations are made?
Are useful recommendations presented?
Surgeons use guidelines for specific and practical evidence-based advice to direct patient care. An RCT measuring physician practice finds specific recommendations leading to more appropriate and fewer inappropriate clinical tests when compared with unspecific recommendations.32 For surgical CPGs and decision making, choices for patient care can often be reduced to a decision tree (e.g., nonoperative v. procedure X v. procedure Y). From a surgical standpoint, attention to this paradigm is critical. Given a patient presentation, readers will turn to CPGs to illustrate both when a procedure should be performed and which procedure should be performed if different options exist.
In Robson and colleagues’ work,11 recommendations are specific in most cases. For example, their Recommendation #1.4 states, “Apparent venous ulcers that have been open continuously without signs of healing for 3 months or that do not demonstrate any response to treatment after 6 weeks should be biopsied for histological diagnosis (Level III),” and Recommendation #1.1 states, “Gross arterial disease should be ruled out by establishing that pedal pulses are present on physical examination and/or that the ankle: brachial index (ABI) is > 0.8. (Any ABI < 1.0 suggests a degree of vascular disease and compression therapy is usually considered to be contraindicated with an ABI < 0.7) […] (Level I).” These 2 examples reflect objective recommendations for venous ulcers.
The clarity of other recommendations could be improved. For example, Recommendation #4.1 states, “Examination of the patient as a whole is important to evaluate and correct causes of tissue damage. This includes factors such as (A) systemic diseases and medications, (B) nutrition, and (C) tissue perfusion and oxygenation (Level II).” What specific diseases and medications are most important for venous ulcers? What components of a nutrition workup are relevant? What typically needs to be supplemented? This information should be provided from the primary literature. The guideline succeeds in presenting surgical, nonsurgical and preventative surgical options for venous ulcers where applicable.
How do authors move from evidence to recommendations?
Arriving at a guideline recommendation is complex, combining best evidence, clinical decision making and patient preferences.42 Good CPGs will provide simple, straightforward care recommendations despite the complexities behind them. When authors use a systematic method to arrive at a judgment, recommendations are more clear and accurate in guiding practice.43 Using this methodology, CPG authors should provide a strength or grade for each recommendation.43 This provides surgeons an indication of the confidence authors have in the literature, level of evidence and real-world effectiveness behind each of their recommendations.17 While CPG authors use a variety of methods to grade recommendations,17 use of a consistent and transparent methodology allows CPGs to be compared across different fields and specialties.43 The GRADE39 methodology is used widely, including the Cochrane Collaboration and UpToDate.39 The GRADE methodology uses a simple system to categorize the quality of evidence into 4 levels (high, moderate, low and very low) and strength of recommendations (strong or weak). Authors interpret methodology, heterogeneity, directness, precision and publication bias of each primary paper.44 For example, the Society for Vascular Surgery adopts the GRADE framework and has a transparent methodology in forming their rigorous, patient-important guideline recommendations.45
Robson and colleagues11 do not describe the strengths and limitations in the body of evidence for each recommendation. There is no formal tool used to illustrate the quality of each paper cited. A classification is used to indicate the strength of each recommendation. This helps illustrate the judgment process for each recommendation. However, the authors do not include patient values in their judgment:
“Level I: Meta-analysis of multiple RCTs or at least 2 RCTs support the intervention of the guideline. Another route would be multiple laboratory or animal experiments with at least 2 clinical series supporting the laboratory results.”
“Level II: Less than Level I, but at least 1 RCT and at least 2 significant clinical series or expert opinion papers with literature reviews support the intervention. Experimental evidence that is quite convincing, but not yet supported by adequate human experience, is included.”
“Level III: Suggestive data of proof of principle, but lacking sufficient data, such as meta-analysis, RCT or multiple clinical series.”
The suggestion in the guideline can be positive or negative at the proposed level (e.g., meta-analysis and 2 RCTs stating intervention is not of use in treating venous ulcers).”
A high level of evidence may not lead to a strong recommendation. For example, Recommendation #7b.4 states, “Negative pressure wound therapy may be useful prior to a skin graft/flap by helping promote the development of granulation tissue in the wound base, or postoperatively by preventing shearing and removing exudates. However, its reported experience in venous ulcers is limited (Level II).” Despite high-level evidence there has not been an illustration of clinical effectiveness, and the impact of therapy may outweigh its potential benefits to patients.
A classification of each recommendation’s strength is missing. Grading recommendations based on this system would allow for comparisons among recommendations in this guideline. No consensus methodology (e.g., Delphi method46) is included. Without explicit methodology, it is difficult to ascertain how the CPG committee arrived at their recommendations. Insight on how decisions were made is necessary for surgeons to apply findings in their own decision making. The aforementioned GRADE methodology provides structure to the review process and limits the bias of “expert opinion” where evidence is unclear.
Will the results help me care for my patients?
Are all outcomes considered?
The process used to select the relevant outcomes and importance of these outcomes must be explicit and sensible. The importance of a certain outcome is directly related to what a patient cares about most. Therefore, CPG authors need to describe the methods with which the outcomes were chosen and a description of the process used to decide on the importance of each outcome. Information on who was involved in outcome choice as well as how values were assigned to outcomes should be apparent in the guideline.
Surgical decision making, like other recommendations, can often be reduced to analysis of benefit versus risk and harm.44 Guidelines should identify not only the interventions of interest, but also sensible alternatives. Surgeons must consider whether the benefits of the treatment discussed outweigh not only the side effects and risks of treatment, but also the implications of another treatment or no treatment. For example, under what circumstances does the benefit of diagnostic laparoscopy outweigh the risk? In a CPG for basal cell carcinoma,43 authors weigh surgical excision against curettage and desiccation, cryotherapy, radiation, chemotherapy and carbon dioxide laser. Considerations include the clinical situation, availability of equipment and patient values/risk profiles among other variables.
Robson and colleagues11 provide a thorough approach to workup and treatment of venous ulcers. Surgical CPGs are sometimes guilty of focusing on the surgical aspects of care while ignoring other aspects of patient management. The nonsurgical multidisciplinary approach is well defined for workup, allowing readers a guide to the workup and preoperative preparation of a venous ulcer. Operative interventions should be compared more directly. For example, Recommendation #6.3 states, “Less extensive surgery on the venous system, such as superficial venous ablation, endovenous laser ablation, or valvuloplasty, especially when combined with compression therapy, can be useful in decreasing the recurrence of venous ulcers (Level I).” This recommendation should be scrutinized because authors can expand on the specific indication of each procedure compared with traditional deep ligation of multiple perforating veins and previously mentioned subfascial endoscopic perforator surgery. This approach to surgical decision making would be helpful to readers.
Will I be able to implement these recommendations?
Moving from primary evidence to CPGs, authors consider the potential barriers in offering these procedures to patients.32 When using a guideline, surgeons interpret recommendations in their own setting. An academic tertiary care centre and community hospital have different patient populations, resources and support personnel.36 Applicability and assessment of barriers is often overlooked,47 especially in surgery. Guidelines are expected to illustrate how recommendations can be applied in the settings the authors intended.41
Robson and colleagues11 describe the necessary components for proper management of venous ulcers. While the CPG touches on the multidisciplinary care required in the preoperative workup, operative/postoperative management and follow-up, the barriers and difficulties in this process are not specifically discussed. The CPG focuses solely on interventions. Practically, surgeons are most often limited by the resources available to them. Using a CPG, surgeons must consider if their own resources would support recommendations. Are new, expensive dressings more effective in treatment? Questions of cost-effectiveness and economic analysis are increasingly important to answer.18
Resolution
Although Robson and colleagues’ work11 is not specific to a particular population or any comorbid conditions, you have no reason to believe that it is not applicable to your patient. You consider the recommendations in this guideline in a stepwise manner. A biopsy of the ulcer first rules out malignancy. A quantitative biopsy rules out clincially important bacterial contamination. You proceed to debride the ulcer in your clinic to minimize the bacterial medium. Home care services are used for daily moist dressing in addition to compression to minimize edema. Two weeks later, the ulcer has a clean base, and you perform a split thickness skin graft. With weekly outpatient follow-up, to the patient’s surprise the ulcer proceeds to heal for the first time in 5 years. This is not the end of the story though. You recommend that the patient should continue applying the compression dressings for life to avoid recurrence.
Footnotes
Competing interests: None declared.
Contributors: C. Coroneos, S. Voineskos, T. Ignacy and A. Thoma designed the study. All authors acquired and interpreted the data. C. Coroneos, T. Ignacy and A. Thoma wrote the article, which all authors reviewed and approved for publication.
- Accepted October 21, 2013.