Hard Science | Circulation: Cardiovascular Quality and Outcomes

In this issue of Circulation: Cardiovascular Quality and Outcomes, we highlight 4 pieces debating the scientific validity of an original research article we published last year. The article in question was a systematic review and meta-analysis of 10 observational studies examining the association between dog ownership and outcomes authored by Caroline Kramer et al.² It reported a 24% lower overall mortality risk linked to dog ownership and was published alongside a companion article on the same topic (that also suggested benefits of dog ownership after major cardiovascular events in a large cohort study) and an accompanying editorial.^3,4

The first 2 of these 4 pieces are Letters to the Editor by Owen et al⁵ and Simonato et al.⁶ Both raise methodological issues that challenge findings from the original meta-analysis. In a detailed response, Kramer et al⁷ justify their analytical approach and suggest their results were robust to these concerns in a sensitivity analysis. The final piece is a Cardiovascular Perspective by Adrian Bauman et al (including Katherine Owen) that argues for broader reappraisal of evidence on health benefits of dog ownership and overall greater caution in interpreting meta-analyses of observational studies.⁸

A reasonable question at this point may be: Why are we spending so many pages on this topic (including this editor’s note)? Assessing the association between dog ownership and survival may feel like an especially small problem to be focusing on at this big moment in history when we have an ongoing viral pandemic of extraordinary scale and a tumultuous reckoning of long-standing racial disparities in healthcare. Debating nuances of statistical modeling for a privileged lifestyle behavior like dog ownership seems like peak academic boondoggle.

Yet while this debate may appear to be admittedly trivial on the surface, it isn’t. Specifics of dog ownership aside, many issues pointed out in these 4 pieces are at the core of what can be challenging for how best to communicate and debate ideas within contemporary science as well as its relationship to the general public. Reviewing the path of the original meta-analysis toward publication in CQO and the concerns raised in the Letters to the Editor provide an opportunity to examine forces that shape how we produce, report, and consume modern research in medicine.

First, I like dogs—and it turns out so do lots of other people. The feel-good nature of the topic of dog ownership is appealing and this allure can shape how research is assessed during the review process. Scientists are humans first and foremost. We bring our biases to every question with our beliefs about how the world works (for better or for worse). And this topic’s popular message made it especially attractive because we want to believe in the truth of such findings—especially the lay media and general public. As Bauman points out in his Viewpoint, this article (along with its companion article and accompanying editorial) was picked up by news outlets worldwide and resulted in the highest Altmetric scores in CQO’s history (https://ahajournals.altmetric.com/details/68007068).

Of course, there are strong underlying ideas that drove our interest in the science beyond the topic’s sparkle factor. Understanding the health benefits of dog ownership is arguably a high-impact, patient-centered question that affects millions of people on a daily basis. This is a question that many of us would find important to better understand (and not just as a researchers). I am proud that we published the meta-analysis by Kramer et al (as well as the additional articles). The American Heart Association previously published a Scientific Statement on this topic given its value to the research and patient communities.⁹ The large cohort study we published in the same issue by Mubanga et al³ linked a national registry with individual-level data on dog ownership to provide even more unique insights. And Kazi’s⁴ spectacular editorial discussed multiple mechanistic pathways that could explain these findings in earlier studies as well as these new reports. That said, it also is easy to see how this desire to believe in the story may have led us to downplay limitations. Confirmation bias—the tendency to interpret new evidence as confirmation of existing belief—can always play a role in evaluating trendy topics.

But to me, there is a second and more unsettling reality raised by these 4 pieces that is frequently underappreciated: science is hard. It’s hard because—as in these studies—the answers we come across are most often complicated and messy. For instance, the major concern raised in the Letters to the Editor was that the original meta-analysis relied on unadjusted data from observational studies. This is a problem given that the relationship between dog ownership and survival is surely confounded. Kramer et al re-analyzed a subset of 5 studies that provided adjusted data and found that the association was still present under assumptions of a fixed-effects model, which was appropriate now that the sample size of studies was smaller (ie, 5 rather than 10).¹⁰ However, Bauman et al⁸ note in their Perspective that the association between dog ownership and survival disappears when analyzing this subset of 5 studies using a random-effects model. This is largely due to the outsized influence of a single study driving the entire association. They conclude by urging caution given substantial heterogeneity in results across the included studies.

When we first received the Letters to the Editor, we struggled with how best to share this complex discussion with readers. Quietly publishing the letters and response by Kramer et al would have been the easiest thing to do. Yet it seemed insufficient to capture the nuances described above. When authors of an original article have the last word in a debate made up of short exchanges, it is challenging to have real dialog. Ideally, our peer review process should have raised these issues before publication and let Kramer et al respond fully in their final version. This did not happen. And while I’m disappointed that this opportunity slipped by us, I am also sensible enough to know that in the process of scientific inquiry the publication of a single article after peer review should never be viewed as reaching an illusory endzone of truth. It is merely a step in a longer journey driven importantly by postpublication peer review. Thus, the need for us to share these letters but also some thoughts on the entire process.

What lessons has this debate on dog ownership had for me beyond this moment? “Science is self-correcting” is a commonly shared aphorism that recognizes its inherent messiness.¹¹ In research, we all recognize that moving an idea from point A to point B is rarely a straight line. Yet given that much of the enormous volume of scientific articles published each year is likely ignored, it is not certain that self-correction always occurs on its own—at least not as often as we’d like. Despite the growth of postpublication peer review in social media platforms like Twitter, it also is unclear who is responsible for the process or ultimately how it should happen. In these articles on dog ownership, we wanted to present arguments from both sides of this debate fully and fairly. In the end, we felt that Owen et al and Simonato et al raised important concerns that needed to be voiced. We also believed that Kramer et al addressed these issues transparently and to the best of their ability. Finally, we thought there was additional value in allowing Bauman et al to put these issues into a larger context around the role of meta-analyses when underlying data are limited.

We hope this discussion will be valuable for readers. It has been a reminder to me that many ideas in research just can’t be packaged neatly into easy answers. This is a fundamental and underlying reality of the discovery process—and it doesn’t always look pretty. It is, therefore, the role of journals to be transparent about such challenges and to encourage venues for discourse rather than settling for oversimplified truths. This is crucial because it turns out science is hard—“really (expletive) hard.”

Acknowledgments

I would like to thank Drs Mike Ho, Jim Burke, and Bill Borden for their thoughts on earlier drafts of this article.

Footnotes