Feature Selection and Classification of Microarray Data for Cancer Prediction Using MapReduce Implementation of Random Forest Algorithm

Cancer is an invasive disease if it detects at a later stage. We strongly believe that the early detection of cancer can increase the efficiency of treatment and decreases the mortality rate. The microarray is a technique where we can study thousands of genes in a short amount of time when we compared to any other traditional methods. The main drawback of microarray data is its curse of dimensionality problem. Since it has a very large number of genes (features) as compared to a number of samples, it creates computational instability for a single system to give an effective result. Hadoop processes big data in divide and conquer manner in its master-slave architecture to project results in a short amount of time. Random forest is an ensemble technique for feature selection and classification. To select a relevant feature from all feature set of microarray data, we need to randomly permute the value of features and check for misclassification rate. If the misclassification rate changes then that particular feature is important. With the proposed method, the accuracy level for detecting cancer at the early stage is effectively improved.