Users’ guide to evidence-based surgery: how to use an article evaluating surgical interventions
=================================================================================================

* John D. Urschel
* Charles H. Goldsmith
* Ved R. Tandan
* John D. Miller

Surgeons have traditionally made therapeutic decisions based on existing surgical dogma, personal experience, recommendations of surgical authorities and thoughtful application of surgical basic sciences. Although this approach has served surgeons and their patients fairly well, ineffective or even harmful treatments can be erroneously accepted as the surgical treatment of choice. Evidence-based surgery emphasizes the need to evaluate properly the efficacy of diagnostic and therapeutic interventions before accepting them as standard surgical practice. Evidence in clinical surgery varies in its quality, and this is reflected in several commonly used grading systems for medical evidence. 1–4 Single case reports represent the lowest level of published evidence. They can be valuable as a stimulus for more formal research or as an important observation about a very rare condition. Retrospective case series are a common form of surgical publication. 5 Comparisons of 2 or more therapies are often attempted in these studies, but the retrospective nature of data collection and difficulties comparing heterogeneous patient populations (often in different treatment eras) limit their usefulness. Prospective nonrandomized studies, such as comparisons of a concurrent cohort of patients, overcome some of the limitations of retrospective data collection and comparison to historical “controls.” However, they are prone to bias. Bias is defined as any factor or process that tends to deviate the results or conclusions of a trial systematically (not randomly) away from the truth.6 A properly designed and conducted randomized controlled trial provides a very high level of evidence on which to base surgical decisions.1–3 Finally, when several randomized controlled trials exist, a meta-analysis of these trials gives the highest level of evidence to support a specific form of treatment.3,4 Ideally, surgeons should critically examine published evidence and then adjust their practices accordingly. 6 The purpose of this article is to assist surgeons with this process.

## Clinical scenario

You are a surgeon performing an esophagectomy for squamous cell carcinoma of the mid-esophagus. You have selected a laparotomy and right thoracotomy for your operative approach. The operation is going well and you are ready to fashion an esophagogastric anastomosis at the apex of the right hemithorax. You are about to do a hand-sewn anastomosis when your assistant (chief resident) suggests using a stapler. He makes reference to a paper that supposedly shows a reduction in anastomotic leaks with stapled esophagogastric anastomoses. You are convinced by his argument and proceed with a stapled anastomosis using an end-to-end anastomotic stapler.

The patient does well postoperatively apart from some pulmonary atelectasis. Contrast esophagography on postoperative day 7 is satisfactory so oral feeding is slowly commenced. The patient is discharged home on postoperative day 14. Unfortunately, early postoperative dysphagia develops and 3 endoscopic dilatations are needed over the ensuing 3 months. You are frustrated by the anastomotic stricture and wonder if a hand-sewn anastomosis would have been better. You decide to review the literature and base your future surgical interventions on the best possible evidence.

## The search

Your search is designed to include a large number of relevant citations initially, but ultimately to permit a very focused review of key articles.7 You search MEDLINE and seek to create one large set of articles dealing with esophagectomy surgery and another large set dealing with anastomoses. You search the Medical Subject Headings (MeSH) of “esophagectomy,” and “esophagus/ surgery” (“esophagus/su”). These citations are combined together using the Boolean “OR” function. Similarly, searching the textword “anastomosis” (“anastomosis.tw”) creates a large set of articles related to anastomoses. Using the text word instead of the MeSH keyword for anastomosis ensures inclusion of abstracts containing this text word, even if the indexers neglected to index the article for the subject of anastomoses. The 2 large sets (“esophagectomy” OR “esophagus/ su,” “anastomosis.tw”) are then combined using the Boolean “AND” function. For practical reasons the combined set is now limited to the English-language, although this common strategy does eliminate good studies and can introduce bias into systematic reviews.8 The resulting set is large and inclusive. It is useful for browsing but it is too large for your current purpose. Finally, the set is limited by publication type. You select “randomized controlled trial” in the publication type menu and find less than 10 articles. Four of these compare hand-sewn and stapled anastomoses.9–12 Although your search successfully yields 4 randomized controlled trials for review, it is important to realize that some trials may have escaped detection during the search process. The indexing of trials is less than perfect so use of additional publication types (“clinical trial, phase III” or “controlled clinical trial”) in the publication types menu may increase sensitivity.5,7 In the library you retrieve the 4 randomized controlled trials and some other articles that caught your eye during MEDLINE browsing.

A review of the various prospective cohort and retrospective comparative studies suggests that stapled esophagogastric anastomoses have a lower rate of leakage than hand-sewn anastomoses, but they may be associated with a higher incidence of anastomotic strictures.13–20 However, you note that many of these studies compared contemporary stapled anastomotic experience with earlier hand-sewn experience. These papers exemplify the problem of using historical controls in surgical research.21 You review the 4 randomized controlled trials carefully. 9–12 One is a multicentre study that permitted a wide variety of approaches to esophagectomy,9 another is a single institution study using a left thoracoabdominal esophagectomy,10 and the third is a small study using a left cervicotomy for anastomosis.11 The last paper is a single institution study using the Lewis (laparotomy and right thoracotomy) esophagectomy.12 Since the Lewis esophagectomy is typically the operative approach that you use, this last paper, by Law and associates, is most applicable to your practice. Your goal is to determine whether this paper should influence the care of your patients. You critique it with this goal in mind.

## Critique of an article

Three questions should be answered when critiquing an article comparing surgical interventions: Are the results valid? What are the results? Are the results applicable to my patients?22–24 (Table 1). These 3 questions will be discussed using the article of Law and associates (Table 2) as an example.

View this table:
[Table 1](http://canjsurg.ca/content/44/2/95/T1)

Table 1 
How to Critique an Article Evaluating Surgical Interventions

View this table:
[Table 2](http://canjsurg.ca/content/44/2/95/T2)

Table 2 
Data Used in This Review, Adapted From the Article by Law and Associates12

## Are the results valid?

If a study comparing surgical interventions is not methodologically sound, we should be cautious about using its recommendations in our practices. The most basic (but not the only) issue of study methodology is the randomization of study participants. 6 The surgical literature is replete with reports of successful operations that have subsequently proven to be ineffective or even harmful when examined with randomized controlled trials. The extracranial–intracranial bypass trial is just one example of this phenomenon.25 The bypass operation made sense from anatomic and physiologic perspectives, and nonrandomized observational studies suggested some benefit for patients with cerebral ischemia. However, the randomized controlled trial showed that the operation was actually harmful to patients. This article highlights the importance of an evidence-based approach to surgical treatments.

Surgical outcomes are influenced not only by the specific surgical intervention but also by patient selection, comorbidities, known prognostic factors and a host of unknown factors. Comparisons of transhiatal and transthoracic esophagectomies, for example, are commonly hampered by an imbalance of chronic lung disease (a comorbidity) in the treatment arms. Randomization is the best way to evenly distribute these determinants of outcome between the treatment and control groups. In addition, the randomization process must be “concealed” from investigators (i.e., the investigator cannot anticipate the direction of randomization for any given patient) until after the patient is entered into the study.26 If an investigator knows a patient’s random allocation destiny before entering the patient on the trial, bias will likely be introduced and the benefits of randomization will be negated. Law’s study used intraoperative randomization by a closed envelope method. The closed envelope suggests that the randomization process was concealed, but the authors could have stated this more explicitly in the article.

Readers should be concerned if all patients entered in a trial are not accounted for at the conclusion. Trials may have some patients drop out, and a few patients may be excluded after randomization because of protocol violations. However, these patients must be detailed in the report so the reader can be assured that this was a random, as opposed to a systematic, occurrence. Otherwise we should be concerned that patients had dropped out because of treatment morbidity. Similarly, if patients are not evaluated at a consistent time after treatment or duration of followup is inadequate we have an incomplete view of all possible outcomes. Patients may manifest complications at various times after treatment so follow-up must be rigorous and consistent. In Law’s article every patient was accounted for and follow-up was complete. The mean follow-up was approximately 20 months. This is adequate for the outcomes of interest (leaks, strictures, operative death).

The “intention-to-treat” concept is important in randomized trials. It is not unusual for patients to be randomized to a given treatment, and then either have no treatment at all or have treatment in the other arm of the study. For example, if a randomized trial compared surgery and radiotherapy for a given cancer, some of the patients randomized to surgery would have their surgery cancelled because of deteriorating fitness or a change of attitude regarding the invasive treatment. Similarly, some patients randomized to radiotherapy may decide that surgery is “better” and undergo an operation outside the study protocol. At first glance it seems appropriate to assess patients based on the treatment that they actually received. However, this can introduce bias. For example, if the surgical arm of a trial is purged of the patients who ultimately did not undergo surgery, the surgical treatment may appear to be superior simply because patients destined to do poorly (surgery cancelled) were excluded. If a trial has too many patients who fail to receive their randomly allocated treatment, the conclusions can become nonsensical. For the most part, however, the intention-to-treat strategy is sound; it is based on the notion that confounding variables will balance out and the randomization will provide a valid comparison of the groups. In the study of Law and associates, the “treatment” (anastomotic technique) was delivered within minutes of randomization so we can assume that patients underwent the treatment they were randomized to receive. However, the article makes no mention of incidents of intraoperative stapler malfunction and resulting hand-sewn anastomotic correction. If this happened, the patient should have remained within the stapled group based on the intention-to-treat principle. Since the article does not discuss this issue we are left wondering about an occasional intention-to-treat violation.

In drug trials patients, clinicians, outcome assessors and investigators are usually “blind” to the treatment that individual patients receive. This is done to prevent expectations of treatment outcome from influencing actual outcomes or outcome evaluations. Although blinding is simple in drug trials, it is usually not possible to blind everyone involved in surgical trials. For example, in Law’s study patients with stapled anastomoses would be identifiable by visible staples on radiographs or at endoscopy. To minimize bias when blinding is not possible, post-surgical treatment evaluation can be done by a separate group of clinicians; in theory these clinicians have no interest in the trial outcome. In Law’s study, for example, a separate team of clinicians could have done follow-up clinic evaluations (assessment of symptomatic dysphagia) and endoscopies (assessment of possible strictures) to reduce the possibility of bias. This does not appear to be the case and therefore bias in the non-blinded evaluation is possible.

If a study is sufficiently large, and randomized, the distribution of patients with various prognostic factors should be “balanced” in the treatment and control groups. However, randomized trials with small patient numbers (less than 30 per group) run the risk of comparing, by chance, 2 fairly dissimilar groups of patients. Readers can be reassured that chance imbalance of the 2 groups does not occur if the known prognostic variables are detailed and the issue of chance imbalance is discussed. In Law’s article data are given on age, sex, anatomic location of tumour, stage of tumour and size of esophagus. The patient groups appear very similar before treatment. Similarly, the reader should be wary of differences in general treatment of the 2 study groups. These differences are called “cointerventions.” If, for example, minimally invasive surgery were to be compared to open surgery, provision of epidural analgesia in only 1 arm of the study would be an important cointervention that could bias the results. In Law’s study the patients were treated identically except for the experimental intervention itself.

## What are the results?

Surgical randomized trials commonly measure dichotomous outcomes or events such as death, cancer recurrence and surgical complications. These dichotomous events either happen or they don’t, so the article usually reports the proportion of patients having the event of interest. Analysis and presentation of results is therefore a process of comparing proportions. These comparisons can be made and expressed in various ways; readers should be familiar with the various methods and terms (Table 3). In the article of Law and associates, for example, strictures developed in 9% of the patients with hand-sewn anastomoses and 40% of patients with stapled anastomoses (Table 2). The absolute risk reduction for the hand-sewn anastomotic technique is 0.31 (0.40 – 0.09 = 0.31). In other words, the handsewn technique resulted in a 31% absolute reduction in anastomotic strictures. This is one way of expressing the results. Another way is to give the relative risk: the risk of events in one group relative to that in the other group. In Law’s article the relative risk of anastomotic stricturing for the hand-sewn technique is 0.22 (0.09/0.40 = 0.22). In other words, the occurrence of strictures with the hand-sewn technique was about one-fifth (22%) of that seen with the stapled technique. The complement of relative risk, termed relative risk reduction, is a common way to express results of dichotomous outcomes. Since it is the complement of relative risk, it is calculated by subtracting relative risk from 1. By convention it is expressed as a percentage. The relative risk reduction for anastomotic stricturing with the hand-sewn technique is 78% (1 – 0.22 = 0.78 × 100 = 78%). One could say that using the hand-sewn technique reduced the risk of strictures by 78%.

View this table:
[Table 3](http://canjsurg.ca/content/44/2/95/T3)

Table 3 
Terms Used to Show the Magnitude and Precision of the Treatment Effect (Decreased Risk of Anastomotic Stricture With the Hand-Sewn Technique)*

Randomized surgical trials study a treatment in a small group of patients and then provide an estimate of treatment effect that is applicable to the larger population of patients we treat. That raises an important question: How precise is the estimate of treatment effect? Investigators can indicate the precision of their treatment estimate in several ways. One approach is to give the traditional *p* value. Law and associates used this approach when giving the anastomotic stricture results. The *p* value for the comparison of stricture incidence in the 2 treatment groups was 0.0003 (χ2 test). This means that the chance of the observed difference in outcome being a random event (when there is no true difference in the 2 procedures) is 3 in 10 000. Of note, the conventional criterion for *p* values for “positive” studies in medicine is 0.05 (chance of the observed difference being a random event is 5 in 100). This particular threshold for statistical significance is completely arbitrary, so investigators should report specific *p* values. Statements such as “*p* < 0.05, significant” or “*p* > 0.05, not significant” should not be used. Finally, readers should be aware that 2 treatments may be significantly different statistically without this difference being clinically important.

Although Law’s provision of the *p* value reassures us that the reported difference is in fact statistically significant, there are other ways of communicating this information. The confidence interval is another, and often preferable way, of showing the precision of the treatment effect estimate. Traditionally, a 95% confidence interval is reported. These confidence intervals define an interval of values that include the true value 95% of the time. Law and associates did not provide confidence intervals, but we are able to calculate them ourselves since the raw data were provided.27 The relative risk reduction for anastomotic stricturing with the hand-sewn technique is 78%, with a 95% confidence interval from 47% to 90%. We can see that a relative risk reduction anywhere within this interval is clinically important. We are therefore confident that the estimate of treatment effect (relative risk reduction for stricturing of 78% with the hand-sewn technique) is sufficiently precise to accept it. Having determined the magnitude and precision of a treatment effect, the next question involves the issue of general applicability of the results to a larger patient population.

## Are the results applicable to my patients?

Surgeons should ask this question before incorporating trial recommendations into their clinical practices. If a randomized trial had very rigid inclusion criteria for enrolment, the results may not be applicable to the larger population of surgical patients28,29 The Veterans Affairs Gastroesophageal Reflux Study Group, for example, showed that fundoplication was superior to medical management for complicated reflux disease in male veterans.30 Caution should be exercised in generalizing these results to other populations (women, non-veterans and patients with uncomplicated reflux) or to patients being treated with more effective medications. Patient age and fitness are other trial inclusion criteria that pose problems when generalizing results. For example, if a randomized trial only involved fit patients its recommendations for an aggressive treatment strategy would not necessarily be applicable to old or frail patients. Surgeons should examine the inclusion criteria to see if the study results are applicable to their patients. In the article of Law and associates, the enrolment criterion was defined as any patient undergoing Lewis esophagectomy for squamous carcinoma of the esophagus. The results should therefore be applicable to the patient in our clinical scenario.

Before accepting one treatment as superior to another, surgeons should question the clinical relevance of the main outcome measures. For example, randomized trials comparing laparoscopic and open surgery often use length of hospital stay as a major outcome measure. Although length of stay is important, it may not be the most clinically relevant outcome of interest to surgeons or patients. In Law’s article the time needed to construct a hand-sewn and a stapled anastomosis was compared. This outcome is not as important as the other outcomes measured, such as operative mortality, anastomotic leakage and anastomotic stricturing. Most surgeons and patients would agree that these are 3 clinically important anastomotic outcome issues.20,31

Finally, surgeons should consider their own skill and experience before applying research findings to their practices. Surgeons must ask themselves if they are as technically proficient with the reported operative procedures as the investigating team of surgeons. Alternatively, the investigating surgeons’ proficiency with a given operation may be open to question.32,33 Proficiency with a particular operation (on the part of an individual surgeon or a team of surgical investigators) is obviously a very important component of its effectiveness. 34 If published evidence conclusively favours an operation that a surgeon does not perform, or does not perform well, that surgeon is faced with 3 choices: proceed with another operation, refer the patient to a colleague or seek additional training to master the operation.35,36 This issue of surgical proficiency creates a difficult dilemma for practising surgeons and highlights a fundamental difference between medical and surgical trials.6,37 A lack of surgical proficiency will bias a randomized controlled trial in favour of the simpler of 2 operative procedures. On the other hand, a randomized controlled trial involving a very complex operation performed by exceptionally skilled surgeons may not be applicable to the larger surgical community.

## Resolution of the clinical scenario

A careful critique of the article by Law and associates leads one to conclude that both stapled and handsewn esophagogastric anastomoses are acceptable in terms of the most critical immediate outcomes (death and anastomotic leaks). However, anastomotic stricturing causes dysphagia and impairs quality of life.38,39 Since palliation of dysphagia is one of the goals of esophagectomy for cancer, postoperative anastomotic stricturing is a major concern. Hand-sewn anastomoses carry a lower risk of anastomotic stricture. As the surgeon in our clinical scenario, you decide to revert to your previous practice of constructing esophagogastric anastomoses with a hand-sewn technique.

## Acknowledgements

The Evidence-Based Surgery Working Group members are as follows: Stuart Archibald, MD;*†‡ Mohit Bhandari, MD;† Charles H. Goldsmith, PhD;‡§ Dennis Hong, MD;† John D. Miller, MD;*†‡ Marko Simunovic, MD, MPH;†‡ Ved Tandan, MD, MSc;*†‡§ Achilleas Thoma, MD;*†‡ John D. Urschel, MD;*†‡ Susan Dimitry, BA.‡ *Department of Surgery, St. Joseph’s Hospital, †Department of Surgery, McMaster University, ‡Surgical Outcomes Research Centre, McMaster University, and §Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ont.

## Footnotes

*   * See acknowledgements for a listing of the members of the Evidence-Based Surgery Working Group.

*   Accepted August 17, 2000.

## References

1.  Howes N, Chaga L, Thorpe M, McCulloch P. Surgical practice is evidence based. Br J Surg 1997;84: 1220–3.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1002/bjs.1800840907&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9313697&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1997XX60300006&link_type=ISI) 

2.  Lee JS, Urschel DM, Urschel JD. Is general thoracic surgical practice evidence based? Ann Thorac Surg 2000; 70:429–31.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0003-4975(00)01483-1&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10969657&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000088886300022&link_type=ISI) 

3.  Cook DJ, Guyatt GH, Laupacis A, Sackett DL, Goldberg RJ. Clinical recommendations using levels of evidence for antithrombotic agents. Chest 1995;108227S–230S.
    
    

4.  Antes G, Galandi D, Bouillon B. What is evidence-based medicine? Langenbecks Arch Surg 1999;384: 409–16.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1007/s004230050223&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10552284&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 

5.  Solomon MJ, Laxamana A, Devore L, McLeod RS. Randomized controlled trials in surgery. Surgery 1994; 115:707–12.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8197563&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1994NP75500009&link_type=ISI) 

6.  McLeod RS. Issues in surgical randomized controlled trials. World J Surg 1999;23: 1210–4.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1007/s002689900649&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10552108&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000083534100003&link_type=ISI) 

7.  Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ 1994;309: 1286–91.
    
    [Abstract/FREE Full Text](http://canjsurg.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMDkvNjk2NC8xMjg2IjtzOjQ6ImF0b20iO3M6MTc6Ii9janMvNDQvMi85NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

8.  Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet 1997; 350:326–9.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0140-6736(97)02419-7&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9251637&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1997XP24200010&link_type=ISI) 

9.  Valverde A, Hay J-M, Fingerhut A, Elhadad A. Manual versus mechanical esophagogastric anastomosis after resection for carcinoma: a controlled trial. Surgery 1996;120:476–83.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0039-6060(96)80066-3&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8784400&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1996VK88800004&link_type=ISI) 

10. Craig SR, Walker WS, Cameron EW, Wightman AJ. A prospective randomised study comparing stapled with handsewn oesophagogastric anastomoses. J R Coll Surg Edinb 1996;41:17–9.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8930036&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1996UL03400004&link_type=ISI) 

11. Laterza E, de’Manzoni G, Veraldi GF, Guglielmi A, Tedesco P, Cordiano C. Manual compared with mechanical cervical oesophagogastric anastomosis: a randomised trial. Eur J Surg 1999;165:1051–4.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1080/110241599750007883&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10595609&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 

12. Law S, Fok M, Chu K-M, Wong J. Comparison of hand-sewn and stapled esophagogastric anastomosis after esophageal resection for cancer: a prospective randomized controlled trial. Ann Surg 1997;226:169–73.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1097/00000658-199708000-00008&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9296510&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1997XV48200008&link_type=ISI) 

13. Fok M, Ah-Chong AK, Cheng SW, Wong J. Comparison of a single layer continuous hand-sewn method and circular stapling in 580 oesophageal anastomoses. Br J Surg 1991;78:342–5.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1002/bjs.1800780323&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=2021852&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 

14. Guili R, Sancho-Garnier H. Diagnostic, therapeutic, and prognostic features of cancers of the esophagus: results of the international prospective study conducted by the OESO Group (790 patients). Surgery 1986; 99:614–22.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=3704918&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1986C190300015&link_type=ISI) 

15. Honkoop P, Siersema PD, Tilanus HW, Stassen LP, Hop WC, van Blankenstein M. Benign anastomotic strictures after transhiatal esophagectomy and cervical esophagogastrosomy: risk factors and management. J Thorac Cardiovasc Surg 1996;111: 1141–8.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0022-5223(96)70215-5&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8642814&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1996UQ38300005&link_type=ISI) 

16. Hopkins RA, Alexander JC, Postlethwait RW. Stapled esophagogastric anastomosis. Am J Surg 1984;147:283–7.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/0002-9610(84)90108-9&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=6364862&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 

17. Lee Y, Fujita H, Yamana H, Kakegawa T. Factors affecting leakage following esophageal anastomosis. Jpn J Surg 1994; 24:24–9.
    
    

18. McManus KG, Ritchie AJ, McGuigan J, Stevenson HM, Gibbons JR. Sutures, staplers, leaks and strictures: a review of anastomoses in oesophageal resection at Royal Victoria Hospital, Belfast 1977–1986. Eur J Cardiothorac Surg 1990;4:97–100.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/1010-7940(90)90222-L&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=2184859&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 

19. Peracchia A, Bardini R, Ruol A, Asolati M, Scibetta D. Esophagovisceral anastomotic leak: a prospective statistical study of predisposing factors. J Thorac Cardiovasc Surg 1988; 95:685–91.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=3280882&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1988M851800020&link_type=ISI) 

20. Beitler AL, Urschel JD. Comparison of stapled and hand-sewn esophagogastric anastomoses. Am J Surg 1998;175:337–40.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0002-9610(98)00002-6&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9568667&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000073118700022&link_type=ISI) 

21. Sacks HS, Chalmers TC, Smith H Jr.. Randomized versus historical assignment in controlled clinical trials. Am J Med 1983; 309:1353–61.
    
    

22. Jadad AR. Randomised controlled trials: a user’s guide. London (UK): BMJ Books; 1998. p. 45–77.
    
    

23. Guyatt GH, Sackett DL, Cook DJ. Users’ guides to the medical literature. II. How to use an article about therapy and prevention. A. Are the results valid? JAMA 1993;270:2598–601.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.1993.03510210084032&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8230645&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1993MJ26600028&link_type=ISI) 

24. Guyatt GH, Sackett DL, Cook DJ. Users’ guides to the medical literature. II. How to use an article about therapy and prevention. B. What were the results and will they help me in caring for my patients? JAMA 1994;271:59–63.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.1994.03510250075039&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8258890&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1994MN72500033&link_type=ISI) 

25. Haynes RB, Mukherjee J, Sackett DL, Taylor DW, Barnett HJ, Peerless SJ. Functional status changes following medical or surgical treatment for cerebral ischemia: results of the extracranial-intracranial bypass study. JAMA 1987: 257:2043–6.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.1987.03390150059034&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=3550162&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 

26. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408–12.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.1995.03520290060030&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=7823387&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1995QD20300026&link_type=ISI) 

27. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. 2nd ed. Boston: Little Brown; 1991. p. 217–20.
    
    

28. Mant D. Can randomised trials inform clinical decisions about individual patients? Lancet 1999;353:743–6.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0140-6736(98)09102-8&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10073531&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000078966400045&link_type=ISI) 

29. Solomon MJ, McLeod RS. Should we be performing more randomized controlled trials evaluating surgical operations? Surgery 1995;118:459–67.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0039-6060(05)80359-9&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=7652679&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1995RT05900001&link_type=ISI) 

30. Spechler SJ. Comparison of medical and surgical therapy for complicated gastroesophageal reflux disease in veterans. N Engl J Med 1992;326:786–92.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJM199203193261202&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=1538721&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1992HJ59200002&link_type=ISI) 

31. Urschel JD. Esophagogastrostomy anastomotic leaks complicating esophagectomy — a review [review]. Am J Surg 1995; 169:634–40.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S0002-9610(99)80238-4&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=7771633&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1995RB02500020&link_type=ISI) 

32. Bonenkamp JJ, Hermans J, Sasako M, van de Velde CJ. Extended lymph-node dissection for gastric cancer. N Engl J Med 1999;340:908–14.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJM199903253401202&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10089184&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000079362600002&link_type=ISI) 

33. Brennan MF. Lymph-node dissection for gastric cancer. N Engl J Med 1999;340: 956–8.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1056/NEJM199903253401210&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10089191&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000079362600010&link_type=ISI) 

34. van der Linden W. Pitfalls in randomized surgical trials. Surgery 1980;87: 258–62.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=6987765&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1980JH77400005&link_type=ISI) 

35. Miller JD, Jain MK, de Gara CJ, Morgan D, Urschel JD. The effect of surgical experience on results of esophagectomy for esophageal carcinoma. J Surg Oncol 1997; 65:20–1.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1002/(SICI)1096-9098(199705)65:1<20::AID-JSO4>3.0.CO;2-Q&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9179262&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=A1997XC32000004&link_type=ISI) 

36. Begg CB, Cramer LD, Hoskins WJ, Brennan MF. Impact of hospital volume on operative mortality for major cancer surgery. JAMA 1998;280: 1747–51.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1001/jama.280.20.1747&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=9842949&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000077081400029&link_type=ISI) 

37. Law S, Wong J. Use of controlled randomized trials to evaluate new technologies and new operative procedures in surgery. J Gastrointest Surg 1998;2:496–503.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1016/S1091-255X(98)80048-6&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10457308&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 

38. Brenneman FD, Wright JG, Kennedy ED, McLeod RS. Outcomes research in surgery. World J Surg 1999;23: 1220–3.
    
    [CrossRef](http://canjsurg.ca/lookup/external-ref?access_num=10.1007/s002689900651&link_type=DOI) 
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=10552110&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom) 
    
    [Web of Science](http://canjsurg.ca/lookup/external-ref?access_num=000083534100005&link_type=ISI) 

39. Gelfand GA, Finley RJ. Quality of life with carcinoma of the esophagus. World J Surg 1994;18:399–405.
    
    [PubMed](http://canjsurg.ca/lookup/external-ref?access_num=8091781&link_type=MED&atom=%2Fcjs%2F44%2F2%2F95.atom)