Can Machines Learn Creativity Needs? An Approach Based on Matrix Completion

Technological progress has been recently associated with a crowding-out of cognitive-skill intensive jobs in favour of jobs requiring soft skills, such as ones related to social intelligence, flexibility and creativity. The nature of soft skills makes them hardly replaceable by machine work and among subsets of soft skills, creativity is one of the hardest to define and codify. Therefore, creativity-intensive occupations have been shielded from automation. Given this framework, our study contributes to a nascent field on interdisciplinary research to predict the impact of artificial intelligence on work activities and future jobs using machine learning. In our work, we focus on creativity, starting from its possible definitions, then we get significant insights on creativity patterns and dynamics in the Italian labour market, using a machine learning approach. We make use of the INAPP-ISTAT Survey on Occupations (ICP), where we identify 25 skills associated with creativity. Then, we apply matrix completion-a machine learning technique which is often used by recommender systems-to predict the average importance levels of various creative skills for each profession, showing its excellent prediction capability for the specific problem. We also find that matrix completion typically underestimates the average importance levels of soft skills associated with creativity, especially in the case of professions belonging to the major group of legislators, senior officials and managers, as well as intellectual professionals. Conversely, overestimates are typically obtained for other professions, which may be associated with a higher risk of being automated.

Automated summary

Technological progress has led to a shift from cognitive-skill intensive jobs to jobs requiring soft skills, such as social intelligence and creativity. Creativity is particularly difficult to define and codify, making it less replaceable by machines. This study uses machine learning to examine creativity patterns and dynamics in the Italian labor market, and predicts the importance of creative skills for different professions. The findings highlight the underestimation of soft skills associated with creativity, suggesting a potential risk of automation for some professions.