Evaluation of health information systems-problems and challenges

Objectives: Information technology (IT) is emerging in health care. A rigorous evaluation of this technology is recommended and of high importance for decision makers and users. However, many authors report problems during the evaluation of information technology in health care. In this paper, we discuss some of these problems, and propose possible solutions for these problems. Methods: Based on own experience and backed up by a literature review, some important problems during IT evaluation in health care together with their reasons, consequences and possible solutions are presented and structured. Results and conclusions: We define three main problem areas-the complexity of the evaluation object, the complexity of an evaluation project, and the motivation for evaluation. Many evaluation problems can be subsumed under those three problem areas. A broadly accepted framework for evaluation of IT in healthcare seems desirable to address those problems. Such a framework should help to formulate relevant questions, to find adequate methods and tools, and to apply them in a sensible way. (C) 2003 Published by Elsevier Ireland Ltd.