How accurate are Twitter and Facebook altmetrics data? A comparative content analysis

Data accuracy is essential for reliable and valid altmetrics analysis. Although Twitter and Facebook altmetrics data are widely used for scholarly communication and scientific evaluation, few studies have tapped into their accuracy issue. Based on content analysis of random sample records over two phases, this study has investigated and compared the accuracy of Twitter and Facebook altmetrics data. Major conclusions are drawn as follows. (1) Three error types were identified from the altmetric data provider and six error types were identified from the altmetric data aggregator. Twitter and Facebook have shared most of the error types except for minor differences in the sub-categories. (2) The overall error rate is substantially high, being 17% and 32% for Twitter and Facebook respectively in April, 2019. However, except for publication date error and posting date error, the percentage of the other error types is relatively low (being around 3%). (3) The percentage of error types related to the dynamic nature of Twitter and Facebook is increasing over time, while percentage of error types concerning the bibliographic data is decreasing over time. (4) The error types are either high seriousness low percentage or low seriousness high percentage, therefore, they would probably not bring significant negative influence. (5) Underlying reasons of these error types are various. They could be attributable to the Twitter (or Facebook) user, Twitter (or Facebook) platform, altmetric database, as well as the third-party data provider. These results suggest that Twitter and Facebook altmetrics data in the Altmetric database are reliable on the whole, although there is still space for further improvement.

Automated summary

This study compared the accuracy of Twitter and Facebook altmetrics data through content analysis of random sample records, revealing different error types related to dynamics and bibliographic data. The overall error rate was high, but most error types were not considered to have significant negative impact.