Invited Commentary 

Pediatrics

July 20, 2020

Leveraging Tweets, Citations, and Social Networks to Improve Bibliometrics

N. Seth Trueger, Yusuf Yilmaz, Teresa M. Chan

JAMA Netw Open. 2020;3(7):e2010911. doi:10.1001/jamanetworkopen.2020.10911

Citations have long been the leading metric for articles, with the citation-based journal impact factor being the primary journal metric. As the world becomes increasingly digital, altmetrics (“alternative metrics,” including article page views and measures of social media sharing, such as the proprietary Altmetric attention score) have become more important in assessing the spread or potential value of a given scientific article.¹ Giustini and colleagues² compare citations with altmetrics in the pediatric literature. This study of pediatric articles highlights that while an article’s altmetrics are associated with future citations, most articles have very low metrics across all measures; only a very small fraction of articles account for most citations, page views, and other altmetrics.²

Medicine has traditionally used article citations and the impact factor of the journal in which an article is published as a surrogate measure of quality. Altmetrics may be better described as a marker of disseminative impact as opposed to the scholarly impact of citation-based measures.¹ As the digital community becomes more ubiquitous, the relationship between early social media attention around an article and later citations has become unclear.³

All metrics inherently have limitations; citations do not measure the impact of a well-read but rarely cited article that influences clinical care, and altmetrics can measure initial interest and social media–based sharing but cannot guarantee a truly engaged readership (or even reading). As with impact factors, altmetrics vary across fields, as conventions around citations and social media sharing, as well as other community factors, vary substantially across different areas of the broader scientific community. Some fields or subfields are small or tight knit and scholars may know and cite each other more, whereas broader fields may have a wide range of scholars who participate at varying levels, diminishing the frequency of citations.¹

The study by Giustini and colleagues² highlights the variation across medical subspecialties regarding their altmetrics. Each discipline within medicine has varying presence and participation in social media,⁴ which may help explain the variations in the association between citations and altmetrics. Specialties with a community less based on social media and, relatedly, with less scholarly discourse about literature on social media can be expected to have lower altmetrics and less of a relationship between social-media sharing and citations.

Not surprisingly, journal editors and others have explored the use of social media as an intervention to propel metrics and spur web traffic. In cardiology, Fox and colleagues⁵ performed a randomized trial that found no statistical difference in 30-day page views between 74 articles repeatedly tweeted and posted to Facebook and 78 control articles.⁵ The intervention, however, may have been too weak to show an effect, as the articles were simple text-based messages from the journal’s account with a link to the studies. An emergency medicine journal used a team of active members of the emergency medicine online community to share new articles from their own accounts with their own interpretations and reactions to the studies; across more than 500 articles, the median number of page views from Twitter increased from 33 to 130 per article.⁶ Hawkins and colleagues randomized previously published articles from a radiology journal; when journal editors shared articles from their personal accounts, the median number of page views per article more than doubled compared with the control. A third arm with basic tweets from the journal account did not statistically increase traffic.⁷

The differences across studies may be prompted by the nature of the intervention (individuals with an online presence in their online community vs routine tweets from an impersonal journal account). However, differences in the specialty communities themselves may mediate some of the differences; emergency medicine, for example, has a notoriously active online community. Other factors such as visual appeal (eg, including figures or other graphics from the article) and endorsement of key influencers contribute to attracting visibility for scientific publications within the noisy digital world. The study by Giustini and colleagues² contributes to the understanding of the complex relationship between scientific reports, social media sharing, and citations; study authors, journals, and publishers will all be better off if valid metrics are standardized to better describe the disseminative impact of new studies. Such important work can be appropriately valued and rewarded.

We should learn the lessons from other attempts to measure both patients’ clinical parameters and broader health care quality; that something is measurable does not necessarily make it a useful measure nor a target that can be successfully (or appropriately) manipulated.