bdc: A toolkit for standardizing, integrating and cleaning biodiversity data

The increase in online and openly accessible biodiversity databases provides a vast and invaluable resource to support research and policy. However, without scrutiny, errors in primary species occurrence data can lead to erroneous results and misleading information. Here, we introduce the Biodiversity Data Cleaning (bdc), an R package to address quality issues and improve the fitness-for-use of biodiversity datasets. The bdc package brings together several aspects of biodiversity data cleaning in one place. It is organized in thematic modules related to different biodiversity dimensions, including (a) Merge datasets: standardization and integration of different datasets; (b) Pre-filter: flagging and removal of invalid or non-interpretable information, followed by data amendments; (c) Taxonomy: cleaning, parsing and harmonization of scientific names from several taxonomic groups against taxonomic databases locally stored through the application of exact and partial matching algorithms; (d) Space: flagging of erroneous, suspect and low-precision geographic coordinates; and (e) Time: flagging and, whenever possible, correction of inconsistent collection date. In addition, the package contains features to visualize, document and report data quality-which is essential for making data quality assessment transparent and reproducible. The modules illustrated, and functions within, were linked to form a proposed reproducible workflow that can also integrate functions from other R packages. We demonstrated the bdc package's applicability in cleaning more than 30 million occurrence records for terrestrial plant species in Brazil. We found that around one-fifth of the original datasets hold the standard quality requirements. Compared to other available R packages, the main strengths of the bdc package are that it brings together available tools-and a series of new ones-to assess the quality of different dimensions of biodiversity data into a single and flexible toolkit. The functions can be applied to many taxonomic groups, datasets (including regional or local repositories), countries, or world-wide. We hope the bdc package can facilitate the data cleaning process and catalyse improvements to allow the wise and efficient use of primary biodiversity data.

Automated summary

The increase in online and openly accessible biodiversity databases provides a valuable resource for research and policy, but errors in primary species occurrence data can lead to misleading information. This study introduces an R package, bdc, that addresses quality issues and improves the fitness-for-use of biodiversity datasets by integrating several aspects of data cleaning.