Chad Ball 00:14
We’d like to welcome you to a new program we’re trying out at Cold Steel. We call it the Surgical Companion series because it’s meant to be a more conversational format that outlines and discusses current media events, and recent publications in a novel interesting manner. Our standing members of the companion are Ameer Farooq, Kelly Vogt, Morad Hameed and myself with guests to come and go, depending on the show. We hope to stimulate respectful and thoughtful conversations and initiatives. And we’re really looking forward to developing it with you.
Ameer Farooq 00:51
So welcome everyone once again to another companion episode. I’m really excited to do this particular episode, because it dwells on a lot of different topics, not least of which is the little pandemic that we’re all going through. And I’ll just start by summarizing the paper that I wanted to get into. And this is the Danish masks that many people by this point, probably are aware of this study. The lead author on this study was Henning Bundgard and I apologize if I’m pronouncing that incorrectly. And the senior authors were Christian Torre Peterson, and Casper Iverson. And this was published in Annals of Internal Medicine on November 18, 2020. So this was a randomized control trial and really neat kind of topic, a very obviously important topic, which is, they were trying to see… the objective of this study was to see whether getting people to wear a mask in the setting of an area where there was not much mask wearing prevalence, if that made a difference in terms of developing COVID. So just to briefly go over this trial. This was a randomized control trial and participants who were included were those over the age of 18, and who spent at least three hours outside of their house. And so they were randomized to either wearing no mask versus wearing a mask whenever they went outside of their home. And it’s important to note that this study was conducted at a time when masks were not mandated. And less than 5% of people according to their estimates were wearing them within Denmark. And they did have other measures in place like social distancing, and hand hygiene, but masks were not mandated. So they randomized people to either wearing a mask or not wearing a mask, and they were randomized in a 1:1 ratio and stratified in the different regions within Denmark. And they tested everyone with both PCR and nasal pharyngeal swab testing at both baseline and at one month. And their primary outcome was looking for either one of these three outcomes: either positive results on a nasal swab test, development of positive antibodies, or a hospital based diagnosis of COVID-19. Their results are quite interesting. So they had a total of 17,258 citizens respond to their initial call, which is amazing. 6024 of which completed the baseline survey. And they randomized 2392 to the mass group, and 2470 into the control group. Now it’s worth noting that 46% of those in the mask group wore a mask as recommended, 47% wore it predominantly and 7%, as not recommended. So clearly, you know, this was truly a study of what people did, sort of, quote unquote, out in the wild, and not really like a test tube type conditions where things were very heavily regulated. This was testing what people actually would do and what people actually do in real life. And their results: the bottom line of what the results showed is that the facemask group had a 1.8% positive testing rate or they were positive for their primary composite outcome. And the control group was 2.1% with an odds ratio of 0.82 in favor of the mask but a chronic confidence interval that crossed one. And they did a number of sensitivity analyses to exclude those participants that didn’t actually wear a mask even though they were randomized to the mask group and they did not still find any difference. So, obviously, there was a whole media fallout from this, but I first want to just delve into the actual study. And my first question, and maybe we’ll start with Dr. Vogt is, I wonder what you think sort of at face value as a researcher and someone who’s interested in doing research and does lots of research…what do you think about just the simple idea of doing this study? Is this a study that you would do? Why or why not?
Kelly Vogt 05:33
Thanks, Ameer. I’ve also found this study tremendously fascinating. And I would say, I think everybody wanted to do this study, because at a time where the data around masks was so controversial, and went from the beginning of the pandemic, being told by our public health experts that they weren’t useful at all, to now being recommended across the world. Everybody wants to be the scientist that proves that using high quality methodology. So would I do it? Yes. Is it impossible to do? In my opinion? I think so. It’s such a tough subject to tackle with the degree of rigor that’s needed to really answer the question that you’re looking to answer. And I think, from my perspective, you know, one of the key things when I think about this study, and I know we’re gonna get into a lot of the methodology, but one of the key things for me is, does this really represent who we want to look at and what else is happening at the same time? And so looking at the inclusion criteria, maybe this is a bit too far down the line for you, but looking for me at who was included and how they found them. What I found most interesting was that their recruitment strategy, they got a lot of people, but they had to have internet access. They had to be out of their home for three hours a day. So what the first thing I thought when I read this study was, I’m not sure that this is exactly the population we want to target. Because I suspect that many of these people were the ones who were following the other public health measures reasonably well.
Ameer Farooq 07:04
Yeah, I think that is a very important point about how they actually selected for these people. And you wonder about having a response bias, for sure. Even though they got 17,000 people, you certainly still wonder about a response bias in terms of people who were more proactive or interested in pursuing measures that would protect them from COVID, were the ones that actually responded.
Kelly Vogt 07:32
Yeah they got 17,000 responses, and then they only randomize 6000. And then that study was only finished in just over, like 4000. So all 17,000 didn’t complete the study.
Ameer Farooq 07:48
Totally. So you wonder like whether people, when they actually were faced with the implications, or had to, you know, sit down and actually think about doing the study actually backed off? And what happened to that other whatever, you know, 11,000 people that didn’t randomize, you know, when they actually read the study? You know, it would be interesting to find out their perspective. But the challenge here, of course, is that this is just not a simple easy question to answer, as you pointed out. Dr. Ball, you know, in your experience, how hard is it to do a study on a topic that already has very passionate supporters on each side? You know, in the surgery world, I think about, you know, for example, antibiotics versus surgery for appendicitis or something that you’ve written about, which is selective, non-operative management for penetrating trauma. You know, like, these are controversial subjects. How hard is it as a researcher to tackle a topic that already has very passionate supporters for both sides of an argument?
Chad Ball 08:52
Yeah that’s a neat question, Ameer. I don’t have a good answer for you. I mean, it can be very hard, or I think it can be very easy in some ways. I think the more controversial a topic is, a research topic, the tighter your methodology has to be. And at the end of the day, the simpler your question has to be. So, you know, I think back to much of the work that myself and Manny Kirkpatrick did about a (insert surgey), just as one example, that ended up being about 15 different peer reviewed publications over 15 years. But the initial question was, what do we do with (insert surgey) in this scenario? But it was too complex, there was too many covariates, it was just unmanageable to try and answer that with an RCT upfront. So we had to work through that concept. And I think at the end of the day, in that particular example, we did a nice job, you know, to be fair, of answering all of the elements that go into that disease process. The problem probably with you know, COVID-19, and the mass question is that, the urgency to that response is so pressured and so hot, maybe to use your word, that it becomes impossible, I think to try and do it in a methodologically rigorous and sequential way.
Ameer Farooq 10:20
Well as an editor, you must be seeing like a huge influx. And we talked to Dr. Luamo about this on the podcast as well, that there was just an overwhelming amount of research about COVID. And so, you know, it becomes very difficult. You probably even as an editor, and maybe you comment on that, it becomes very difficult as an editor to, you know, slow things down enough to actually check and make sure that this is high quality science.
Morad Hameed 10:47
Yeah, I think all the journals, all the peer reviewed journals anyway, have noticed the same trend, which is you’re exactly right, there was an explosion of people with commentaries and sort of really basic, well maybe “low-level” is unfair, but relatively low level quality or methodology science that’s rounded up. And then that slowed down a little bit. And it’s following now the more predictable path and more logical well thought out path, research wise going forward, you know, as you would expect as per many quote, unquote, “hot topics” that come and go. What was interesting, though, is that, we found that, you know, I know, again, it wasn’t just us at the Canadian Journal of Surgery, I think it was really all journalism and talking to other editors. We found that people were particularly over upset or angry if their COVID paper was rejected, compared to really every other topic. That’s not to say that, you know, authors, more or less are not more impatient in 2020, than they probably were a decade ago and more aggressive and more irritated in those scenarios. But in particular, this topic, you could tell people were very passionate about it, and wanted to be heard about it.
Ameer Farooq 12:05
Yeah, it’s so interesting, although this study really had a hard time actually finding a home. Apparently, according to the authors. They submitted it to, we don’t know who but some major journals, and were rejected. Dr. Hameed, my question to you, is, as someone who sort of thinks about the policy side of whatever research you’re doing, because it directly has implications for your job as a division head, do you ever think about policy implications of a study prior to doing it? You know, like, do you think that you can just do research from a pure scientific curiosity perspective? Or do you think like, well, what impact is this research going to have, you know, depending on what the results are?
Morad Hameed 12:58
Well, I mean, thanks for asking that question. One of my research mentors and partners used to say that the job of an academic is to ask questions. You know, think broadly, and leave hypothesis generating answers and move on. But I could never do that. Every research program I’ve ever been involved with, I always wonder how it’s going to shape policy and what its effect will be in the real world. So I think that there is certainly an absolutely essential role for blue sky thinking and the formulation and validation of theories. But I think, personally, one of the roles of the clinician scientists would be to ask questions that have real world immediacy and application.
Ameer Farooq 14:00
So I guess the question really here… I guess what I’m trying to sort of think about even before we delve into the study is: is this the type of study that one really should do in the sense that no matter what the results are, one could see a scenario where proponents of mask wearing will continue to want to wear a mask and encourage others to do so. And those who don’t wear a mask will use this gleefully as their evidence for not wearing a mask. So Dr. Vogt, putting the same question to you. Do you think of policy implications before you do research or is that not what you think about? You just think about trying to do good science?
Kelly Vogt 14:47
I’d like to think I think about those things. I mean, I think as you’re designing research programs, it’s key to design a program that is going to have impact beyond you know, the publication that you’re aiming for. I’m not sure I always achieve that. But I think it’s certainly important. But that being said, think the methodology and as you’re thinking about a new study, designing a study that actually answers the question that you’re asking as conclusively as possible, I think is probably even more important. Because when the science is there, and the answers are there, you hope that it’s going to be easier for the policy to follow. Not always the case, because as we’ve seen in other countries, and sometimes even in ours, the politics get in the way. But you have a much better leg to stand on if the science is sound.
Ameer Farooq 15:45
Right. So I think that’s a good time to transition to actually digging in a little bit more into the study. And Dr. Vogt, it sounds like you had some other ideas about sort of the strengths and limitations of the study in your mind, what were the strengths and limitations of the study itself?
Kelly Vogt 16:08
Well I think the study was what we all, or certainly I would hope to achieve, which is a relatively pragmatic study – what’s actually happening in the real world. And, you know, I’m always a proponent when the question is right, for randomized methodology, so that you’re balancing both the known and unknown confounding factors, which really is something that can’t be done with any of the other study methodology. So I certainly commend the authors on undertaking a randomized control trial to try to answer this question. And I think, you know, they did a lot of the parts of that very well. I already touched a bit on sort of the participants who were in the study. And I think there is certainly a bias in the way that they identified participants for the study, and also the participants who ultimately undertook the study. I think there’s definitely a risk there that we are under representing many of the patients at risk for marginalization, who make up so much of the burden of what we’re seeing with COVID these days. And so I do think that is a limitation. And then, you know, in terms of the other limitation, really, it’s how well did they measure? And how well could they control for the things that might have impacted on this relationship? And to me, that’s really where we’re talking about the use of the other public health measures. So if you have, at least from a proof of concept standpoint, if you have a group of people who are perfectly following social distancing, and hand washing and all the other public health measures, well, maybe you’re not going to see a difference in those who are wearing masks and those who aren’t. That would be where my simple mind takes me as we’re thinking about this study.
Morad Hameed 17:49
Ameer, could I ask you like, it’s easy to criticize some ambitious clinical trials. And I recognize that. But I think beyond sort of the prestige of completing an RCT and accomplishing a publication, when investigators have a very important responsibility to protect the public. A trial like this, David Urbach once compared quality improvement measures to searching for your dropped keys under the streetlight, you know, where you can see them, and ignoring every place they could be that’s outside the spotlight. And a trial like this, as Dr. Vogt says, is confined by its own methods, and its own environment. And if it’s underpowered, or if it’s flawed in a major way, that could be a great disservice. That might create a semblance of evidence base where there isn’t any, or where there’s no generalizability. And it made me wonder, like, if you have something that seems sensible, and has some theoretical validity to it, you have to be very careful, I think, to challenge it. An example would be like seatbelts. If you randomize people to wearing seatbelts or not wearing seatbelts and you study them for a month in a town where there’s not too many cars, and where there’s a lot of other safety measures in place, and you find that people who wear seatbelts have a slightly better survival, would you conclude that seatbelts are useless? I think it’s kind of a concern that if you do this trial, you have to do it right. And probably scale it better.
Ameer Farooq 19:49
Yeah, I think that was a fundamental issue. I mean, I quite like the idea of doing trials that actually examine what people do in the real world. I really like the idea of pragmatic trials, because I think, you know, the strict conditions of mask-wearing versus not mask-wearing, if you were 100% adherent, or you have very strict criteria, that wouldn’t really capture what people actually do. But I completely agree about the fact that this is probably underpowered for the question that they were trying to answer, especially given the fact that, you know, a considerable percentage of their participants didn’t really wear the mask as instructed. You know, 46% were predominantly. It’s not clear to me exactly what that means. And 7% didn’t wear it as recommended at all. So yeah, I think you’re completely right, that they’re probably underpowered given, you know, that the overall positive composite outcome was only about 2%, or 1.8%. You know, they’re probably underpowered to detect a difference at that range. Dr. Ball, what were your thoughts about this? Any other strengths or limitations of this paper?
Morad Hameed 21:10
I think you guys really hit most of the nails on the head. Morad’s comment about seatbelts is fantastic. And it echoes so many other things, whether it’s helmets or parachutes. And, you know, for me, I come back to two things when I think about this. The first is sort of what I touched on before, you know, if you really wanted to look at this, you should probably go to a Walmart in Alberta and do the trial under direct inspection. I think it would be that kind of environment. I have nothing against Walmart, but you know, that’s a ripe environment with lots of the milieu you’re looking for. So I think it’s almost, as Kelly said, initially, you know, it’s almost impossible to do this properly in the way that they’ve done it. The second thing is that, you know, just to sort of come out from the details of it. Maybe you’re gonna go here, but it does make me think back to, you know, all of the discussion that’s gone on from the very beginning. We all had local and national and, of course, international leaders from various different backgrounds, who presented this data initially. So with such certainty and such intensity, that it was so easy to believe, and it kept changing, and it kept changing. And, you know, I think probably the readers and I don’t just mean the lay public, I mean, physicians are probably a little bit fatigued at this as well. Because I could be very ignorant, but I still don’t get the sense we have a handle on a lot of the issues that surround this whole experience.
Ameer Farooq 22:48
I think that’s such an important point. And I’ll just make two comments on that. One is that if anything, I think that we need many more rigorous trials to try to answer some of these fundamental questions, right? Like, what is the impact of, for example, closing schools on the rate of COVID positivity and COVID spread. And I’m not the first one to propose this, you know? Vinay Prasad, for example, is one person who’s been a big proponent of cluster randomized trials to try to answer these types of questions. So I think that is critical, that we actually get better studies. And it’s kind of surprising that in some ways, despite the huge deluge of all this COVID research, there’s been very little, you know, randomized or high quality or rigorous science around any of this. And so that’s sort of my first comment. And the second comment, I think, Dr. Ball, what you’re getting at, and that’s so critical, is the messaging around this piece. You know, there is a nuance to how the messaging is around this whole piece. With any topic in science. But it becomes so much more important when every issue is under the microscope, and the whole world literally is watching for these studies. And is it even going to preprints and reading all this stuff. You know, and I think one of the fundamental issues here is: how do we convey uncertainty which is inherent in science, to a public that’s looking for definite answers? So maybe I’ll turn to you Dr. Vogt. Do you think that, you know, scientists or public health officials should continue to sort of present things as absolutes? Like, you should wear a mask? It is definitely helpful. Or do you think we don’t give people enough credit and there needs to be more nuance in the way that we present issues?
Kelly Vogt 24:49
I think that’s a fascinating question. And I definitely don’t pretend to be a psychologist but early in the pandemic, I don’t know who to credit for this, but I read something on social media about individuals and the public looking at the health care establishment and whether that was public health or whomever. And saying you don’t know what you’re saying, because things have changed in your recommendations. But the counter to that in the statement was, and I completely agree with this, it’s actually exactly what the scientific method is designed to do. And what makes this pandemic different from what we usually do in medicine, is that the lay public is watching what we do all the time, which is a mass evidence over time and refine our recommendations based on additional evidence emerging. We just don’t do a good job of explaining that process, because we’ve never had to really before this. At least not on such a big stage. So I think if we could do a better job of explaining to individuals, that this is not a sign of weakness, in fact, it’s a sign of strength in our healthcare infrastructure: that we’re able to say to people, we’re just building on what we’re learning. I think we might do a better job. But that’s such a hard thing to explain. When usually, we’re so certain.
Ameer Farooq 26:17
You know, Dr. Hameed, there’s been this mantra that has gone around on social media about, quote unquote, “listen to the science”. Again, do you think… is that the kind of message that we should be promulgating? Or are we just not giving people you know, the nuance that they need? Or maybe, you know, to play the devil’s advocate or to play the other side, how do you do policy changes or ensure sort of adherence to a policy if you sort of present all this uncertainty, especially in a world – and we’re going to get to this in a second – but in a world where there’s so much misinformation, there’s so many conspiracy theories, there’s so many avenues for false information to make their way around. How do you navigate that balance?
Morad Hameed 27:08
Ameer, in BC we have Dr. Bonnie Henry, who as you know, is a rockstar in public health, and I read about her that she… and this, I think really underscores what Kelly just said… she had respect for the public. And she had this level of transparency. And I think empathy and that respect, and empathy and transparency, I think, really captured the trust of the population, and sort of brought people in for the long haul. She anticipated, this should be a long struggle, and that we would be learning along the way. And I think to me, that seems to be a great approach to have faith that people are smart. And they understand that we’re trying to make decisions under conditions of uncertainty, and that the science evolves. And I think if you try to project an aura of certainty, it’ll backfire. And it’ll undermine both science and public policy. It’s kind of like, when you run a code, or when you run a trauma, you’re in a bad situation in the OR, I find it’s sometimes okay to say, here’s what I’m thinking, this could be wrong. But let’s see what happens and adjust along the way. And I think that’s a style of leadership that really does bring engagement, and it maximizes the contributions of everybody.
Ameer Farooq 28:42
I really do like your analogy to the operating room and this idea that, you know, asking questions, or inviting people in to be part of your thought processes is so valuable. But, you know, when you have a huge public that, you know, is very different than sort of a group of people that are committed to the same aims and same sort of overall goal, which is to take care of the patient, you know, it does become sort of a different beast. Dr. Ball, we’ve had lots of discussions about this sort of topic, but how do you see us walking that line about conveying the nuance of science while also trying to give people some recommendations that they can hold on to?
Morad Hameed 29:32
Yeah, I don’t know Ameer. I think I’m increasingly overwhelmed by the direction – maybe it’s right and maybe it’s wrong – of social media, and sort of how we communicate in 2020 and potentially in many folks’ eyes and certainly large swaths, large segments of society. The lack of recognition of expertise, and I would argue of the scientific method, and even if you don’t understand or don’t care to understand the scientific method, the conclusion of experts within the scientific method. And I don’t know how to necessarily navigate that reality for a large part of society now. So I actually don’t know how you deliver a message that is not overbearing, that continues as more access to be humble. But to be accurate, and granular enough, from person to person.
Ameer Farooq 30:42
Of course, this gets really complicated, as you point out because of social media, and just the way that can all play out. It’s worth noting that one of the important aspects to this story with the Danish mask study is that two of the scientists that wrote a piece on this, not involved with the trial. But two scientists separately wrote a piece on this that was shared on Facebook and Facebook actually censored it and essentially said that this article was misleading, or that the facts within this study were not true. Which is, you know, it’s kind of ironic in the sense that Twitter and Facebook have put up these warnings to try to get people to think more critically about the information that’s being spread. And my sense, is that Facebook has now incorrectly labeled this as being false or misinformation. You know, without belaboring this too much, you know, we do have another really big issue coming up, which is administering vaccines. And there’s already this overwhelming sense of a lot of misinformation and a lot of confusion around vaccinations. So maybe the last thing that I’ll ask the three of you is: going forward, how do you think that we can increase scientific literacy, both in the medical community and beyond? Going forward, so that I think people can better handle the uncertainty that comes with a changing scientific literature? And maybe I’ll start with you, Dr. Ball.
Chad Ball 32:32
Yeah I mean, that’s probably three different parts to that answer. The first is the person that delivers the message or the group that delivers that message. They have to be eloquent, as we’ve said, and as Morad initially pointed out. You have to come across as humble, measured, calm, and trustworthy, so that the deliverer of the message I think, is critical. And that doesn’t necessarily mean all politicians, or all physicians, or all celebrities, or all pro athletes are able to do that. It has to be, you know, the best voices amongst us all. The second component would be content. Again, you know, as Kelly pointed out so beautifully, the scientific method has been playing out with this particular issue at this time, as it should, as it’s supposed to. And many people have talked about that, at a very high level. And it’s been interesting to listen to those discussions. So we have to be as accurate as we can be, and as granular as we can be in terms of content of the message. And then I think the third thing is a combination of all of it. It’s the packaging of it, it’s the methodology of how we deliver these messages. So understanding probably how social media works, probably how, you know, backhaul conversations work, both, you know, locally, nationally, and of course, internationally. So I think you need a broad perspective on the world. I know that sounds kitchy, but you know, that’s really what would drive those three things in my mind: to try and deliver whatever the message is going to be about, you know? Vaccines now or the next pandemic or or whatever it is, that is a critical public health issue that we don’t necessarily see coming or maybe aren’t prepared for.
Ameer Farooq 34:33
Yeah, you’re speaking to the need to really build a big team that can really think about all those different aspects and how to deliver it. Dr. Hameed, what are your thoughts?
Morad Hameed 34:45
Thanks Ameer. I think in some ways, you’re really doing it, you know? With these podcasts. These types of formats are a start. You’re bringing so much thoughtfulness and imagination to public discourse. Cold Steel is for a surgical audience. But I think this type of format seems to be catching on. And so I know that social media can be used to over simplify messages or mislead or support special interests. But I think it can also be used for thoughtful exploration of ideas. And I think that method, that strategy could prevail in the long term.
Ameer Farooq 35:35
Dr. Vogt, we’ll leave the last word to you.
Kelly Vogt 35:39
It’s hard to have anything to add after those two great commentaries. The only thing I would say is, in addition to everything that’s been said, I think we have to lead by example. We’re very privileged to be in a position to critically appraise literature as it’s coming out, to critically appraise the information or at least the scientific information that’s being provided to us. And so once we’ve done that, and interpreted it, I think being bold in the decisions we’re making for ourselves and those around us and being willing to share that, I think, at least we’ll capitalize on the small portion of the world that we touch and can potentially influence.
Ameer Farooq 36:26
You’ve been listening to Cold Steel, the official podcast of the Canadian Journal of Surgery. If you’ve liked what you’ve been listening to, please leave us a review on iTunes. We’d love to hear your comments and feedback. So feel free to email us at firstname.lastname@example.org, or connect with us on Twitter @CanJSurg. Thanks again.