Inter-coder agreement for ICD-9-CM coding of stroke

The objective was to evaluate agreement among neurologists for coding stroke, using the International Classification of Diseases 9th Revision - Clinical Modification (ICD-9-CM). Clinical records of 53 consecutive patients (27 stroke or TIA and 26 other diseases) discharged from our general neurology department were coded by four neurology residents, four general neurologists and four neurologists trained in ICD-coding (experts). Inter-coder agreement was evaluated by the kappa statistic. The overall kappa value for coding 430-438 was 0.77 (95% confidence interval 0.70-0.84) for primary (PDx) and 0.82 (0.77-0.88) for PDx+secondary diagnoses (SDx). It was 0.73-0.86 for the residents, 0.78-0.90 for the neurologists and 0.67-0.88 for the experts. The overall kappa values (PDx, SDx) for specific codes were 0.83-0.88 for 431-intracerebral haemorrhage (residents: 0.75-0.89; neurologists: 0.91-0.96; experts: 0.78-0.88) and 0.48-0.51 for 434-occlusion of cerebral arteries) (residents: 0.26-0.26; neurologists: 0.61-0.66; experts: 0.60-0.65). Agreement was substantial for the whole code group 430-438 and higher for code 431 than 434. Reliability was generally improved when both PDx and SDx were considered. Specific training did not increase inter-coders' agreement.