Algorithm Recommendation and Performance Prediction Using Meta-Learning

In the last years, the number of machine learning algorithms and their parameters has increased significantly. On the one hand, this increases the chances of finding better models. On the other hand, it increases the complexity of the task of training a model, as the search space expands significantly. As the size of datasets also grows, traditional approaches based on extensive search start to become prohibitively expensive in terms of computational resources and time, especially in data streaming scenarios. This paper describes an approach based on meta-learning that tackles two main challenges. The first is to predict key performance indicators of machine learning models. The second is to recommend the best algorithm/configuration for training a model for a given machine learning problem. When compared to a state-of-the-art method (AutoML), the proposed approach is up to 130x faster and only 4% worse in terms of average model quality. Hence, it is especially suited for scenarios in which models need to be updated regularly, such as in streaming scenarios with big data, in which some accuracy can be traded for a much shorter model training time.

Automated summary

In recent years, there has been a significant increase in the number of machine learning algorithms and their parameters. This presents both opportunities and challenges in training models. Traditional search-based methods become computationally expensive and time-consuming as datasets grow, especially in data streaming scenarios. This paper proposes a meta-learning approach that can predict performance indicators and recommend the best algorithm/configuration for training models. The proposed approach is up to 130 times faster than a state-of-the-art method and only slightly worse in terms of model quality, making it suitable for scenarios that require regular model updates with shorter training time.