Web pages from mockup design based on convolutional neural network and class activation mapping

The objective of this study is to validate the use of Deep Neural Networks (DNNs) to segment and classify web elements. To achieve this, a dataset of 2200 images was created through screenshots of real web pages, with 10 distinct classes to represent the most common web elements. The contributions of this study encompass the validation of classification-only Convolutional Neural Networks (CNNs) with the support of Class Activation Mapping (CAM), a weakly-supervised semantic segmentation technique that requires no in-image annotation, significantly simplifying the dataset creation process when compared to traditional segmentation models. Multiple networks with distinct hyper-parameter combinations were cross-validated with 10 folds, with a final accuracy rating of 95.71% on the best-performing model. Although the final CNN showed promising results, further improvements on the dataset and architecture are still required for it to be employed as the centerpiece of a real-time dynamic web page building solution, with clear improvements needed on the clarity of the segmentation heatmap.

Automated summary

The objective of this study is to validate the use of deep neural networks (DNNs) for segmenting and classifying web elements. A dataset of 2200 images representing 10 distinct classes was created using screenshots of real web pages. The study contributes by validating classification-only convolutional neural networks (CNNs) with the support of Class Activation Mapping (CAM), a weakly-supervised semantic segmentation technique. The best-performing model achieved a final accuracy rating of 95.71%, but improvements are still needed on the dataset and architecture for real-time dynamic web page building.