Evaluation of freely available data profiling tools for health data research application: a functional evaluation review

Objectives To objectively evaluate freely available data profiling software tools using healthcare data. Design Data profiling tools were evaluated for their capabilities using publicly available information and data sheets. From initial assessment, several underwent further detailed evaluation for application on healthcare data using a synthetic dataset of 1000 patients and associated data using a common health data model, and tools scored based on their functionality with this dataset. Setting Improving the quality of healthcare data for research use is a priority. Profiling tools can assist by evaluating datasets across a range of quality dimensions. Several freely available software packages with profiling capabilities are available but healthcare organisations often have limited data engineering capability and expertise. Participants 28 profiling tools, 8 undergoing evaluation on synthetic dataset of 1000 patients. Results Of 28 potential profiling tools initially identified, 8 showed high potential for applicability with healthcare datasets based on available documentation, of which two performed consistently well for these purposes across multiple tasks including determination of completeness, consistency, uniqueness, validity, accuracy and provision of distribution metrics. Conclusions Numerous freely available profiling tools are serviceable for potential use with health datasets, of which at least two demonstrated high performance across a range of technical data quality dimensions based on testing with synthetic health dataset and common data model. The appropriate tool choice depends on factors including underlying organisational infrastructure, level of data engineering and coding expertise, but there are freely available tools helping profile health datasets for research use and inform curation activity.

Automated summary

This study objectively evaluated freely available data profiling software tools for their applicability in healthcare data. Several tools showed high potential and functionality for use with healthcare datasets. In a synthetic dataset of 1000 patients, two tools consistently performed well across multiple tasks including completeness, consistency, uniqueness, validity, accuracy, and distribution metrics.